makepst 0.2.0__tar.gz → 0.3.0__tar.gz

{makepst-0.2.0 → makepst-0.3.0}/CHANGELOG.md

@@ -1,5 +1,21 @@
 # Changelog
 
+## 0.3.0 (2026-09-21)
+
+- `log`: ledger of every manifest under a folder — when, which command, which output from
+  which sources, dimensions, version; `--file` filters by file name, path or sha256 prefix
+  ("which runs used that version of the workbook?"), `--command`, `--last`, `--check`
+  verifies every recorded hash
+- `bundle`: zip a run for reproduction or review — control file, manifest, templates,
+  instruction files, model inputs, command-line files; `--sources`, `--outputs`, `--extra`,
+  `--list`, `--strict`; the zip gets its own manifest
+- `provenance`: compare the hashes recorded in a manifest with the files on disk
+  (`unchanged` / `changed` / `missing`; exit 1 unless all unchanged)
+- `update --dry_run`: preview matched names, target columns and skipped cells without writing
+- `build` warns when a referenced sheet has formulas without cached values (a workbook saved
+  by openpyxl and not yet recalculated in Excel)
+- `hpstart`: a PEST_HP `.hp` file with observation values from residuals or an IES realization
+
 ## 0.2.0 (2026-09-19)
 
 - `update`: the xlwings backend reads only headers, name and target columns, writes contiguous

{makepst-0.2.0 → makepst-0.3.0}/CITATION.cff

@@ -6,8 +6,8 @@ authors:
   - family-names: Ou
     given-names: Michael
     email: ougengxin@gmail.com
-version: 0.2.0
-date-released: 2026-09-19
+version: 0.3.0
+date-released: 2026-09-21
 repository-code: "https://github.com/ougx/makePst"
 license: MIT
 keywords:

{makepst-0.2.0 → makepst-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: makepst
-Version: 0.2.0
+Version: 0.3.0
 Summary: Build PEST / PEST++ / PEST_HP control files from spreadsheet tables, read them back, and push results into the workbook
 Author-email: Michael Ou <ougengxin@gmail.com>
 License: MIT
@@ -45,8 +45,11 @@ reads control files back into editable workbooks, and returns calibration result
 Excel / CSV tables                     ->  .pst                   makepst build
 .pst                                   ->  editable workbook      makepst dump
 .par / .res / .rei / IES ensemble      ->  existing workbook      makepst update
-.par or IES realization (+ tweaks)     ->  new .pst               makepst parrep
+    .par or IES realization (+ tweaks)     ->  new .pst               makepst parrep
+    .res/.rei or IES obs + existing .hp     ->  new .hp                makepst hpstart
 two .pst / workbooks                   ->  what changed           makepst diff
+manifests in a folder                  ->  run ledger             makepst log
+.pst + everything it references        ->  zip for review         makepst bundle
 ```
 
 See [What round-tripping preserves](#what-round-tripping-preserves) for the exact guarantees
@@ -223,6 +226,7 @@ makepst update book.xlsm --res run.res                          # adds MODELLED
 makepst update book.xlsm --pst run.pst --par_cols PARVAL1,PARTRANS,PARLBND,PARUBND
 makepst update book.xlsm --par run.3.par.csv --real best        # PESTPP-IES ensemble
 makepst update book.xlsm --obs_csv run.3.obs.csv --pst run.pst  # IES simulated values
+makepst update book.xlsm --par run.par --dry_run               # preview; write nothing
     [--sheet "PAR_*"] [--group hk,sy] [--overwrite_formulas] [--out copy.xlsm] [--backend openpyxl|xlwings]
 ```
 
@@ -260,8 +264,13 @@ Both sheets are rewritten on each run (`--no_phi` to skip).
 
 Two backends: **xlwings** (used when installed; drives Excel invisibly, so everything in the
 workbook is preserved and formulas recalculate; Windows/macOS with Excel) or **openpyxl**
-(no Excel needed; keeps VBA macros; drops charts and images; formula results are stale until
-Excel next opens the file). The backend used is printed.
+(no Excel needed; keeps VBA macros; drops charts and images; does not recalculate formulas).
+Before building from a workbook updated with openpyxl, recalculate and save it in Excel;
+otherwise the build may read missing or stale cached formula values. The backend used is printed.
+`--dry_run` uses openpyxl to preview matched names, target columns, skipped formulas and
+unmatched names without saving a workbook or manifest. It does not test whether Excel will
+refuse individual writes. During `build`, missing cached formula results in referenced
+tables produce a warning; existing cached values may still be stale and cannot be verified.
 
 ### parrep — .par values → new .pst
 
@@ -283,6 +292,26 @@ to that workbook; other relative names resolve against the workbook's folder. No
 route rebuilds from the workbook's *current* state, so any edits made since the original
 control file was written are picked up too.
 
+### hpstart — modeled observations → new PEST_HP .hp file
+
+```
+makepst hpstart case.pst start.hp --template case.hp --res case.res
+makepst hpstart case.pst start.hp --template case.hp --obs_csv case.3.obs.csv --real best
+makepst hpstart case.pst start.hp --template case.hp --res case.rei --par case.par
+```
+
+By default, parameter values are copied unchanged from the template `.hp`, **not** from
+`PARVAL1` in the `.pst`. Use `--par` to replace them with values from a PEST `.par` file
+or a selected IES parameter realization. The `.pst` supplies parameter and observation
+names and order; the template supplies PEST_HP instruction metadata that a residual or
+IES observation file cannot provide. `hpstart` replaces the template's reference
+modeled-observation array in `.pst` observation order. Every required observation must
+be present with a finite value; extra result rows are ignored. The template must come
+from the same control-file layout and
+observation/parameter order. The binary format stores counts but not names, so matching
+counts alone cannot establish that the template has the right order. The original `.hp`
+is never modified, and the new file receives a provenance manifest by default.
+
 ### tempchek — templates → model input files
 
 ```
@@ -333,6 +362,39 @@ tolerance, so `49.7377` vs `49.73775` shows up at the default `1e-9` and disappe
 `--rtol 1e-5`. Prints a table per section and a one-line summary; `--xlsx` writes the same
 tables to a workbook for review. Exits 1 when there are differences, like `diff`.
 
+### log — the run ledger
+
+```
+makepst log [FOLDER ...] [--file tr13.xlsm] [--command build] [--last 10] [--check] [--no_recursive]
+```
+
+Every manifest under the folders (default: the current one, recursively), oldest first, one
+line each: when, which command, which output, from which sources, the dimensions, the
+makePst version. `--file` keeps the entries whose output or sources include a file — by name,
+path or **sha256 prefix**, so `--file b43fc509` answers "which runs used *that* version of
+the workbook?" even after the file was renamed or overwritten. `--check` adds a column from
+`provenance`: `unchanged` / `changed` / `missing`.
+
+```
+2026-09-19 21:14 build      pest/tr13.pst <- tr13.xlsm  [1068 par 41747 obs 159 prior]  makepst 0.3.0
+2026-09-21 08:10 update     tr13-xlwings.xlsm <- tr13.xlsm, tr13.par  [1068 rows in 8 sheets]  makepst 0.3.0
+```
+
+### bundle — a run, zipped for review
+
+```
+makepst bundle model.pst [run.zip] [--sources] [--outputs] [--extra "bin/**"] [--list] [--strict]
+```
+
+Zips the control file with its manifest, every template and instruction file, the model
+input files the templates write, and any file named on the model command line, with paths
+relative to the run directory so the archive unzips runnable. `--sources` adds the manifest's
+sources (the workbook, the `.par` ...), `--outputs` the model output files, `--extra` any
+glob relative to the run directory (the model binaries, say). Files the control file
+references but that don't exist are listed as `MISSING` (exit 1 with `--strict`); `--list`
+shows what would be bundled without writing. The zip gets its own manifest, so `log` shows
+who bundled what and when.
+
 ## What round-tripping preserves
 
 `dump` → `build` and `read_pst` → `write_pst` are **semantically lossless for supported
@@ -439,7 +501,7 @@ from?" after the fact:
 
 ```json
 {
-  "makepst": "0.2.0",
+  "makepst": "0.3.0",
   "command": "build",
   "argv": ["build", "tr13.pst", "regul", "--set_ctl_xls", "tr13.xlsm,CONTROL", "..."],
   "created": "2026-09-19T13:51:39-04:00",
@@ -451,7 +513,7 @@ from?" after the fact:
      "sheets": ["CONTROL", "PARGP", "PAR_HK", "PAR_VK", "...", "PPglm"]}
   ],
   "output": {"path": "D:\\...\\tr13.pst", "sha256": "d93f317d...", "size": 2163331,
-             "npar": 1068, "nobs": 41747, "npargp": 12, "nprior": 185, "nobsgp": 12,
+             "npar": 1068, "nobs": 41747, "npargp": 12, "nprior": 159, "nobsgp": 12,
              "ntplfle": 18, "ninsfle": 7, "pestmode": "regularisation"}
 }
 ```
@@ -463,6 +525,15 @@ realization and `--set` values (and, from a workbook, the resolved build argumen
 left alone, and the backend. A hash mismatch between a manifest and the workbook on disk is
 the signal that the control file no longer corresponds to the spreadsheet.
 
+Run `makepst provenance model.pst` (or pass `model.pst.manifest.json`) to compare every
+recorded input and output hash with the files at their recorded paths. It reports
+`unchanged`, `changed`, `missing`, or `not recorded` and exits with status 1 unless all
+records are unchanged. This checks file identity, not the validity of PEST tables; use
+`makepst validate` for the latter. Moving files requires updating their recorded paths.
+`makepst log` lists the manifests of a whole project folder as a ledger (with `--check`,
+each one verified), and `makepst bundle` zips a run with everything its control file
+references — see the two commands above.
+
 ## Relationship to pyEMU
 
 [pyEMU](https://github.com/pypest/pyemu) is the broader toolkit: it constructs PEST++
@@ -532,14 +603,14 @@ makepst/
   writer.py         Pst -> .pst text (+ dump.tpl)
   reader.py         .pst text -> Pst
   excel.py          sheets -> Pst; Pst -> workbook; results -> existing workbook
-  cli.py            init / validate / build / dump / update / parrep / diff / tempchek / inschek
+  cli.py            init / validate / build / dump / update / parrep / diff / tempchek / inschek / log / bundle
   diff.py           semantic comparison of two Pst objects
   phi.py            objective function by group from residuals; IES realization phis
   pyemu_bridge.py   to_pyemu / from_pyemu
   checks.py         the validate report: table checks, template / instruction cross-checks, instruction interpreter
   rules.py          pestchek's rules (parameters, groups, observations, prior information, control variables)
   model_files.py    PEST's number writer, template filling, .obf files (tempchek / inschek)
-  provenance.py     the <output>.manifest.json sidecar
+  provenance.py     the <output>.manifest.json sidecar; provenance / log / bundle
   starter.py        the `init` workbook: control-variable descriptions, example rows, drop-downs
 examples/minimal/   the tutorial inputs
 tests/              suite + fixture generator
@@ -552,7 +623,7 @@ stored anywhere. To release: bump the version in `pyproject.toml` and the fallba
 `makepst/__init__.py` (a test keeps them equal), add a `CHANGELOG.md` entry, commit, then
 
 ```
-git tag v0.2.0 && git push origin main v0.2.0
+git tag v0.3.0 && git push origin main v0.3.0
 ```
 
 The `publish` workflow checks that the tag matches the package version, runs the tests, builds

{makepst-0.2.0 → makepst-0.3.0}/README.md

@@ -14,8 +14,11 @@ reads control files back into editable workbooks, and returns calibration result
 Excel / CSV tables                     ->  .pst                   makepst build
 .pst                                   ->  editable workbook      makepst dump
 .par / .res / .rei / IES ensemble      ->  existing workbook      makepst update
-.par or IES realization (+ tweaks)     ->  new .pst               makepst parrep
+    .par or IES realization (+ tweaks)     ->  new .pst               makepst parrep
+    .res/.rei or IES obs + existing .hp     ->  new .hp                makepst hpstart
 two .pst / workbooks                   ->  what changed           makepst diff
+manifests in a folder                  ->  run ledger             makepst log
+.pst + everything it references        ->  zip for review         makepst bundle
 ```
 
 See [What round-tripping preserves](#what-round-tripping-preserves) for the exact guarantees
@@ -192,6 +195,7 @@ makepst update book.xlsm --res run.res                          # adds MODELLED
 makepst update book.xlsm --pst run.pst --par_cols PARVAL1,PARTRANS,PARLBND,PARUBND
 makepst update book.xlsm --par run.3.par.csv --real best        # PESTPP-IES ensemble
 makepst update book.xlsm --obs_csv run.3.obs.csv --pst run.pst  # IES simulated values
+makepst update book.xlsm --par run.par --dry_run               # preview; write nothing
     [--sheet "PAR_*"] [--group hk,sy] [--overwrite_formulas] [--out copy.xlsm] [--backend openpyxl|xlwings]
 ```
 
@@ -229,8 +233,13 @@ Both sheets are rewritten on each run (`--no_phi` to skip).
 
 Two backends: **xlwings** (used when installed; drives Excel invisibly, so everything in the
 workbook is preserved and formulas recalculate; Windows/macOS with Excel) or **openpyxl**
-(no Excel needed; keeps VBA macros; drops charts and images; formula results are stale until
-Excel next opens the file). The backend used is printed.
+(no Excel needed; keeps VBA macros; drops charts and images; does not recalculate formulas).
+Before building from a workbook updated with openpyxl, recalculate and save it in Excel;
+otherwise the build may read missing or stale cached formula values. The backend used is printed.
+`--dry_run` uses openpyxl to preview matched names, target columns, skipped formulas and
+unmatched names without saving a workbook or manifest. It does not test whether Excel will
+refuse individual writes. During `build`, missing cached formula results in referenced
+tables produce a warning; existing cached values may still be stale and cannot be verified.
 
 ### parrep — .par values → new .pst
 
@@ -252,6 +261,26 @@ to that workbook; other relative names resolve against the workbook's folder. No
 route rebuilds from the workbook's *current* state, so any edits made since the original
 control file was written are picked up too.
 
+### hpstart — modeled observations → new PEST_HP .hp file
+
+```
+makepst hpstart case.pst start.hp --template case.hp --res case.res
+makepst hpstart case.pst start.hp --template case.hp --obs_csv case.3.obs.csv --real best
+makepst hpstart case.pst start.hp --template case.hp --res case.rei --par case.par
+```
+
+By default, parameter values are copied unchanged from the template `.hp`, **not** from
+`PARVAL1` in the `.pst`. Use `--par` to replace them with values from a PEST `.par` file
+or a selected IES parameter realization. The `.pst` supplies parameter and observation
+names and order; the template supplies PEST_HP instruction metadata that a residual or
+IES observation file cannot provide. `hpstart` replaces the template's reference
+modeled-observation array in `.pst` observation order. Every required observation must
+be present with a finite value; extra result rows are ignored. The template must come
+from the same control-file layout and
+observation/parameter order. The binary format stores counts but not names, so matching
+counts alone cannot establish that the template has the right order. The original `.hp`
+is never modified, and the new file receives a provenance manifest by default.
+
 ### tempchek — templates → model input files
 
 ```
@@ -302,6 +331,39 @@ tolerance, so `49.7377` vs `49.73775` shows up at the default `1e-9` and disappe
 `--rtol 1e-5`. Prints a table per section and a one-line summary; `--xlsx` writes the same
 tables to a workbook for review. Exits 1 when there are differences, like `diff`.
 
+### log — the run ledger
+
+```
+makepst log [FOLDER ...] [--file tr13.xlsm] [--command build] [--last 10] [--check] [--no_recursive]
+```
+
+Every manifest under the folders (default: the current one, recursively), oldest first, one
+line each: when, which command, which output, from which sources, the dimensions, the
+makePst version. `--file` keeps the entries whose output or sources include a file — by name,
+path or **sha256 prefix**, so `--file b43fc509` answers "which runs used *that* version of
+the workbook?" even after the file was renamed or overwritten. `--check` adds a column from
+`provenance`: `unchanged` / `changed` / `missing`.
+
+```
+2026-09-19 21:14 build      pest/tr13.pst <- tr13.xlsm  [1068 par 41747 obs 159 prior]  makepst 0.3.0
+2026-09-21 08:10 update     tr13-xlwings.xlsm <- tr13.xlsm, tr13.par  [1068 rows in 8 sheets]  makepst 0.3.0
+```
+
+### bundle — a run, zipped for review
+
+```
+makepst bundle model.pst [run.zip] [--sources] [--outputs] [--extra "bin/**"] [--list] [--strict]
+```
+
+Zips the control file with its manifest, every template and instruction file, the model
+input files the templates write, and any file named on the model command line, with paths
+relative to the run directory so the archive unzips runnable. `--sources` adds the manifest's
+sources (the workbook, the `.par` ...), `--outputs` the model output files, `--extra` any
+glob relative to the run directory (the model binaries, say). Files the control file
+references but that don't exist are listed as `MISSING` (exit 1 with `--strict`); `--list`
+shows what would be bundled without writing. The zip gets its own manifest, so `log` shows
+who bundled what and when.
+
 ## What round-tripping preserves
 
 `dump` → `build` and `read_pst` → `write_pst` are **semantically lossless for supported
@@ -408,7 +470,7 @@ from?" after the fact:
 
 ```json
 {
-  "makepst": "0.2.0",
+  "makepst": "0.3.0",
   "command": "build",
   "argv": ["build", "tr13.pst", "regul", "--set_ctl_xls", "tr13.xlsm,CONTROL", "..."],
   "created": "2026-09-19T13:51:39-04:00",
@@ -420,7 +482,7 @@ from?" after the fact:
      "sheets": ["CONTROL", "PARGP", "PAR_HK", "PAR_VK", "...", "PPglm"]}
   ],
   "output": {"path": "D:\\...\\tr13.pst", "sha256": "d93f317d...", "size": 2163331,
-             "npar": 1068, "nobs": 41747, "npargp": 12, "nprior": 185, "nobsgp": 12,
+             "npar": 1068, "nobs": 41747, "npargp": 12, "nprior": 159, "nobsgp": 12,
              "ntplfle": 18, "ninsfle": 7, "pestmode": "regularisation"}
 }
 ```
@@ -432,6 +494,15 @@ realization and `--set` values (and, from a workbook, the resolved build argumen
 left alone, and the backend. A hash mismatch between a manifest and the workbook on disk is
 the signal that the control file no longer corresponds to the spreadsheet.
 
+Run `makepst provenance model.pst` (or pass `model.pst.manifest.json`) to compare every
+recorded input and output hash with the files at their recorded paths. It reports
+`unchanged`, `changed`, `missing`, or `not recorded` and exits with status 1 unless all
+records are unchanged. This checks file identity, not the validity of PEST tables; use
+`makepst validate` for the latter. Moving files requires updating their recorded paths.
+`makepst log` lists the manifests of a whole project folder as a ledger (with `--check`,
+each one verified), and `makepst bundle` zips a run with everything its control file
+references — see the two commands above.
+
 ## Relationship to pyEMU
 
 [pyEMU](https://github.com/pypest/pyemu) is the broader toolkit: it constructs PEST++
@@ -501,14 +572,14 @@ makepst/
   writer.py         Pst -> .pst text (+ dump.tpl)
   reader.py         .pst text -> Pst
   excel.py          sheets -> Pst; Pst -> workbook; results -> existing workbook
-  cli.py            init / validate / build / dump / update / parrep / diff / tempchek / inschek
+  cli.py            init / validate / build / dump / update / parrep / diff / tempchek / inschek / log / bundle
   diff.py           semantic comparison of two Pst objects
   phi.py            objective function by group from residuals; IES realization phis
   pyemu_bridge.py   to_pyemu / from_pyemu
   checks.py         the validate report: table checks, template / instruction cross-checks, instruction interpreter
   rules.py          pestchek's rules (parameters, groups, observations, prior information, control variables)
   model_files.py    PEST's number writer, template filling, .obf files (tempchek / inschek)
-  provenance.py     the <output>.manifest.json sidecar
+  provenance.py     the <output>.manifest.json sidecar; provenance / log / bundle
   starter.py        the `init` workbook: control-variable descriptions, example rows, drop-downs
 examples/minimal/   the tutorial inputs
 tests/              suite + fixture generator
@@ -521,7 +592,7 @@ stored anywhere. To release: bump the version in `pyproject.toml` and the fallba
 `makepst/__init__.py` (a test keeps them equal), add a `CHANGELOG.md` entry, commit, then
 
 ```
-git tag v0.2.0 && git push origin main v0.2.0
+git tag v0.3.0 && git push origin main v0.3.0
 ```
 
 The `publish` workflow checks that the tag matches the package version, runs the tests, builds

{makepst-0.2.0 → makepst-0.3.0}/makepst/__init__.py

@@ -3,9 +3,10 @@ try:                                        # the installed package's metadata (
     from importlib.metadata import version as _dist_version
     __version__ = _dist_version('makepst')
 except Exception:                           # a checkout run without installing
-    __version__ = '0.2.0'
+    __version__ = '0.3.0'
 
 from .excel import load_table, to_workbook, update_workbook
+from .hpstart import write_hpstart
 from .model_files import fill_template, write_number
 from .provenance import Manifest
 from .pst import Pst, read_ensemble, read_obs_ensemble, read_par, read_res
@@ -24,4 +25,4 @@ def from_pyemu(ppst):
 
 __all__ = ['Pst', 'read_pst', 'from_text', 'write_pst', 'to_text',
            'load_table', 'to_workbook', 'update_workbook', 'read_par', 'read_res', 'read_ensemble', 'read_obs_ensemble',
-           'fill_template', 'write_number', 'Manifest', '__version__', 'to_pyemu', 'from_pyemu']
+           'fill_template', 'write_number', 'write_hpstart', 'Manifest', '__version__', 'to_pyemu', 'from_pyemu']

{makepst-0.2.0 → makepst-0.3.0}/makepst/cli.py

@@ -1,6 +1,7 @@
 """Command line: build (Excel/CSV -> pst), dump (pst -> Excel), update (values -> existing workbook),
 parrep (.par values -> pst), init (starter workbook), validate (pestchek-style checks), diff (compare two),
-tempchek (templates -> model input files), inschek (model output files -> observation values)."""
+tempchek (templates -> model input files), inschek (model output files -> observation values),
+provenance / log / bundle (manifests: verify, list as a ledger, zip a run)."""
 import argparse
 import os
 import shlex
@@ -8,7 +9,7 @@ import sys
 
 from . import __version__
 from .excel import expand_spec, load_table, to_workbook, update_workbook
-from .provenance import Manifest
+from .provenance import Manifest, check_manifest
 from .pst import Pst, read_par
 from .reader import read_pst
 from .writer import write_pst
@@ -445,7 +446,7 @@ def update(args):
     def flat(items):   # repeated flags, each possibly a comma list
         return [x.strip() for s in items for x in s.split(',') if x.strip()] or None
 
-    manifest = None if args.no_manifest else Manifest('update', args._argv)
+    manifest = None if args.no_manifest or args.dry_run else Manifest('update', args._argv)
     if manifest is not None:
         manifest.add_source(args.workbook, role='workbook')
         for path, role in ((args.par, 'parameter values'), (args.pst, 'control file'),
@@ -456,7 +457,8 @@ def update(args):
                              real=args.real, pst=pst, out=args.out, backend=args.backend,
                              par_cols=args.par_cols.upper().split(','), obs_cols=args.obs_cols.upper().split(','),
                              overwrite_formulas=args.overwrite_formulas,
-                             sheets=flat(args.sheet), groups=flat(args.group), phi=not args.no_phi)
+                             sheets=flat(args.sheet), groups=flat(args.group), phi=not args.no_phi,
+                             dry_run=args.dry_run)
     if manifest is not None:
         manifest.set_output(args.out or args.workbook, sheets=result['sheets'], backend=result['backend'])
         manifest.add(realization=args.real, par_cols=args.par_cols, obs_cols=args.obs_cols,
@@ -464,11 +466,87 @@ def update(args):
         print(f'manifest written to {manifest.write()}')
 
 
+def provenance(args):
+    path = args.target if args.target.endswith('.manifest.json') else args.target + '.manifest.json'
+    for status, name in check_manifest(path):
+        print(f'{status:12s} {name}')
+        if status != 'unchanged':
+            args._failed = True
+    if getattr(args, '_failed', False):
+        raise SystemExit(1)
+
+
+def log(args):
+    """The ledger: every manifest under the folders, oldest first — what produced which file from what."""
+    from .provenance import find_manifests, log_line, mentions
+    rows = find_manifests(args.folder or ['.'], recursive=not args.no_recursive)
+    if args.file:
+        rows = [r for r in rows if any(mentions(r, f) for f in args.file)]
+    if args.command:
+        rows = [r for r in rows if r.get('command') in args.command]
+    if args.last:
+        rows = rows[-args.last:]
+    for r in rows:
+        print(log_line(r, check=args.check))
+    if not rows:
+        print('no manifests found')
+
+
+def bundle(args):
+    """Zip a run so it can be reproduced or reviewed elsewhere (control file, manifest, templates,
+    instruction files, model inputs, command files; --sources / --outputs / --extra for more)."""
+    from .provenance import bundle_run
+    pst = read_pst(args.pstfile)
+    base = args.base_dir or guess_base_dir(pst, args.pstfile)
+    manifest = None if (args.no_manifest or args.list) else Manifest('bundle', args._argv)
+    out = None if args.list else (args.out or os.path.splitext(args.pstfile)[0] + '.zip')
+    included, missing = bundle_run(pst, args.pstfile, out, base_dir=base, sources=args.sources,
+                                   outputs=args.outputs, extra=args.extra, manifest=manifest)
+
+    def show(path):
+        rel = os.path.relpath(path, base)
+        return path if rel.startswith('..') else rel
+
+    for role, path in included:
+        print(f'{"would add" if args.list else "added":10s} {role:18s} {show(path)}')
+    for role, path in missing:
+        print(f'MISSING    {role:18s} {show(path)}')
+    if args.list:
+        print(f'{len(included)} files, {len(missing)} missing (nothing written)')
+        return
+    size = os.path.getsize(out) / 1e6
+    print(f'written {out}: {len(included)} files, {size:.1f} MB' + (f', {len(missing)} missing' if missing else ''))
+    if manifest is not None:
+        manifest.set_output(out, files=len(included), missing=[p for _, p in missing])
+        print(f'manifest written to {manifest.write()}')
+    if missing and args.strict:
+        raise SystemExit(1)
+
+
+def hpstart(args):
+    from .hpstart import write_hpstart
+    pst = read_pst(args.pstfile)
+    manifest = None if args.no_manifest else Manifest('hpstart', args._argv)
+    if manifest is not None:
+        for path, role in ((args.pstfile, 'control file'), (args.template, 'hp template'),
+                           (args.res, 'residuals'), (args.obs_csv, 'observation ensemble'),
+                           (args.par, 'parameter values')):
+            if path:
+                manifest.add_source(path, role=role)
+    counts = write_hpstart(pst, args.template, args.out, res=args.res,
+                           obs_csv=args.obs_csv, real=args.real, par=args.par)
+    print(f'written {args.out}: {counts["npar"]} parameters, {counts["nobs"]} observations')
+    if manifest is not None:
+        manifest.set_output(args.out, **counts)
+        manifest.add(realization=args.real)
+        print(f'manifest written to {manifest.write()}')
+
+
 def main(argv=None):
     argv = list(sys.argv[1:] if argv is None else argv)
     # legacy form: makePst.py out.pst mode --add_par_xls ...  (no subcommand)
-    if argv and argv[0] not in ('build', 'dump', 'update', 'parrep', 'init', 'validate', 'diff', 'tempchek', 'inschek',
-                                '-h', '--help', '-v', '--version'):
+    if argv and argv[0] not in ('build', 'dump', 'update', 'parrep', 'init', 'validate', 'provenance', 'log', 'bundle',
+                                'diff', 'tempchek', 'inschek', 'hpstart', '-h', '--help', '-v', '--version'):
         argv.insert(0, 'build')
 
     ap = argparse.ArgumentParser(prog='makepst', description=__doc__)
@@ -500,6 +578,17 @@ def main(argv=None):
     rf.add_argument('--v2', action='store_true', help='write a PEST++ version-2 file')
     rf.add_argument('--v1', action='store_true', help='write the classic format (default: same as the input)')
 
+    h = sub.add_parser('hpstart', help='update a PEST_HP .hp file from residual or IES observation values')
+    h.add_argument('pstfile', help='control file defining observation and parameter order')
+    h.add_argument('out', help='new .hp file (must differ from the template)')
+    h.add_argument('--template', required=True, help='existing .hp with matching order; instruction metadata is preserved')
+    hs = h.add_mutually_exclusive_group(required=True)
+    hs.add_argument('--res', help='PEST .res/.rei file providing MODELLED values')
+    hs.add_argument('--obs_csv', help='PESTPP-IES case.N.obs.csv providing modeled values')
+    h.add_argument('--real', help='IES realization name, or best (default base)')
+    h.add_argument('--par', help='optional .par or IES parameter file to replace template parameter values')
+    h.add_argument('--no_manifest', action='store_true', help="don't write <out>.manifest.json")
+
     v = sub.add_parser('validate', help='pestchek-style checks on a control file or a workbook')
     v.add_argument('target', help='control file, or a workbook with a BUILD sheet')
     v.add_argument('--outputs', action='store_true',
@@ -509,6 +598,31 @@ def main(argv=None):
     v.add_argument('--strict', action='store_true', help='exit 1 on warnings as well as errors')
     v.add_argument('--quiet', action='store_true', help='hide informational lines')
 
+    q = sub.add_parser('provenance', help='compare files against a saved manifest')
+    q.add_argument('target', help='manifest path, or output path with a .manifest.json sidecar')
+
+    g = sub.add_parser('log', help='ledger of every manifest under a folder: what produced which file from what')
+    g.add_argument('folder', nargs='*', help='folders (or manifest files) to search; default the current one')
+    g.add_argument('--file', action='append', default=[], metavar='NAME',
+                   help='only entries whose output or sources include this file name, path or sha256 prefix')
+    g.add_argument('--command', action='append', default=[], metavar='CMD', help='only this command (repeatable)')
+    g.add_argument('--last', type=int, metavar='N', help='only the N most recent entries')
+    g.add_argument('--check', action='store_true', help='also compare every recorded hash with the file on disk')
+    g.add_argument('--no_recursive', action='store_true', help='do not descend into subfolders')
+
+    b = sub.add_parser('bundle', help='zip a run for reproduction or review: control file, manifest, templates, '
+                                      'instruction files, model inputs, command files')
+    b.add_argument('pstfile', help='control file')
+    b.add_argument('out', nargs='?', help='zip file to write (default: the control file name with .zip)')
+    b.add_argument('--sources', action='store_true', help="also the manifest's sources (the workbook, .par, ...)")
+    b.add_argument('--outputs', action='store_true', help='also the model output files the instruction files read')
+    b.add_argument('--extra', action='append', default=[], metavar='GLOB',
+                   help='more files, relative to the run directory (repeatable; ** allowed)')
+    b.add_argument('--base_dir', metavar='DIR', help='the directory PEST runs in (default: guessed as validate does)')
+    b.add_argument('--list', action='store_true', help='show what would be bundled, write nothing')
+    b.add_argument('--strict', action='store_true', help='exit 1 when a referenced file is missing')
+    b.add_argument('--no_manifest', action='store_true', help="don't write <zip>.manifest.json")
+
     t = sub.add_parser('tempchek', help='check a template file, or write model input files from parameter values (like TEMPCHEK)')
     t.add_argument('target', help='template file, or a control file / workbook (all of its templates)')
     t.add_argument('modfile', nargs='?', help='with a template: model input file to write')
@@ -552,6 +666,7 @@ def main(argv=None):
     u.add_argument('--overwrite_formulas', action='store_true',
                    help='also write into cells that currently hold formulas (default: leave them alone)')
     u.add_argument('--out', help='save to this file instead of in place')
+    u.add_argument('--dry_run', action='store_true', help='preview matches and skipped cells without writing files')
     u.add_argument('--backend', choices=['xlwings', 'openpyxl'], help='default: xlwings if installed')
     u.add_argument('--no_manifest', action='store_true', help="don't write <workbook>.manifest.json")
     u.add_argument('--no_phi', action='store_true', help="don't write the PHI / PHI_IES sheets with --res / --obs_csv")
@@ -559,7 +674,9 @@ def main(argv=None):
     args = ap.parse_args(argv)
     args._argv = argv                 # the command line as given, for the manifest
     {'build': build, 'dump': dump, 'update': update, 'parrep': parrep, 'init': init,
-     'validate': validate, 'diff': diff, 'tempchek': tempchek, 'inschek': inschek}[args.cmd](args)
+     'validate': validate, 'provenance': provenance, 'log': log, 'bundle': bundle, 'diff': diff,
+     'tempchek': tempchek, 'inschek': inschek,
+     'hpstart': hpstart}[args.cmd](args)
 
 
 if __name__ == '__main__':