mapsmith 0.7.0__tar.gz → 0.8.0__tar.gz

{mapsmith-0.7.0 → mapsmith-0.8.0}/.gitignore

@@ -33,6 +33,7 @@ data/
 tests/data/*
 !tests/data/manifest_spec_validator.py
 !tests/data/manifest-v1.schema.json
+!tests/data/manifest_spec_multi_file_digest.py
 # The discovery benchmark: 95 queries written by a second model from work
 # scenarios, never from this catalog. It is tracked BECAUSE it is a dataset — an
 # evaluation set nobody can inspect is a claim, not a measurement.

{mapsmith-0.7.0 → mapsmith-0.8.0}/.zenodo.json

@@ -1,7 +1,7 @@
 {
   "upload_type": "software",
   "title": "MapSmith: geospatial processing over MCP, with verifiable provenance",
-  "version": "0.7.0",
+  "version": "0.8.0",
   "creators": [
     {"name": "The MapSmith project"}
   ],

{mapsmith-0.7.0 → mapsmith-0.8.0}/CHANGELOG.md

@@ -6,7 +6,198 @@ All notable changes to MapSmith are documented here, in the format of
 
 ## [Unreleased]
 
-Nothing yet.
+## [0.8.0] - 2026-10-05
+
+Records declare `spec_version` `1.0.0-draft.10`. `draft.9` defines the digest
+of a dataset made of several files, which is what the first fix below
+implements; `draft.10` gives each input a role and a configuration of its own.
+**One key moves**: a second input's own georeferencing facts,
+`inputs[].x-mapsmith:environment` on 0.7.x, are now `inputs[].environment`.
+
+### Added
+
+- **`cluster_points_by_distance`: group points chained together within a
+  distance.** "Group everything within half a kilometre" is single linkage, not
+  thinning: two points share a cluster when a chain of points joins them with
+  every link at most the distance long, so chains merge, and every point is kept
+  with its `cluster_id` and `cluster_size`. `thin_points` keeps one point per
+  crowd, and its count is close to the number of groups but not equal to it. A
+  check re-reads the written file and re-derives the chains by a second
+  algorithm. A geographic CRS is refused, and so are lines, polygons and
+  multipart points, which a point cluster has no single position for.
+- **`list_operations` takes a `geometry`: what the caller's layer holds**, as
+  `point`, `line` or `polygon`, or a list of them for a mixed layer. The
+  catalogue now has 31 operations taking one vector layer to another, one more
+  than the size it hands over whole, so a caller declaring everything it knew
+  got a shortlist back for the most common request there is; with the geometry
+  declared, the right operation is in the answer for 116 of the 118 benchmark
+  requests again. An entry declares the families it accepts only where its code
+  refuses the others (fifteen do), an entry without a declaration is kept for
+  every family, and a test calls each declared entry with every family it
+  leaves out and requires the refusal.
+- **`benchmarks/choice_at_scale.py`**, with every choice it scores in
+  `benchmarks/choice_at_scale.json`: each of the 118 requests handed to Claude
+  Opus 5.5 in sets of 30, 40 and 50 candidates and compared with itself. From
+  40 it chose right 6.9 points less often than from 30, from 50 5.1 points
+  less, so the size `list_operations` hands over whole stays 30, and a test
+  holds it at or below the largest size measured without loss.
+- **Each input of an operation that reads more than one dataset says which
+  argument read it** (`inputs[].argument`, spec `draft.10`). The order of
+  `inputs` is not significant, so until now a weighted zonal statistic with
+  values and weights swapped produced a record that differed only in file
+  names. Eighteen operations name their inputs (twenty code paths:
+  `spatial_join` has three engines), with the names the input checks
+  already use in `verification[].argument`, so the two can be joined; a
+  single-input record is unchanged.
+
+- **`get_lineage` takes a `layer`.** The history of one layer of a file
+  geodatabase now starts from the geodatabase itself: its record sits beside
+  the container (spec section 3.1), and the walk matches the layer at every hop
+  into a container. A layer given for a path that is not a container is
+  refused rather than ignored.
+
+### Changed
+
+- **`get_lineage` steps carry `inputs[].argument`, `inputs[].environment` and
+  `output.layer`** when the records hold them.
+- **A shapefile output whose member files are ambiguous** (`out.dbf` beside
+  `out.DBF`, possible only on a case-sensitive filesystem) is written with a
+  manifest that has no `output` digest and a note saying why, instead of a
+  digest that would choose one of the two.
+- **`inputs[].x-mapsmith:environment` is `inputs[].environment`**, the field
+  spec `draft.10` defines for the configuration that changed how one input was
+  read. Same content; only the name moves, and a reader of 0.7.x records reads
+  the old one.
+- **With more than one input, no file's facts sit on the record's
+  `environment`, the first input's included.** A sidecar beside the first
+  raster was recorded there, which the format reads as true of the whole run,
+  since the order of `inputs` means nothing: in a `clip_raster` it read as true
+  of the mask. It is now on that input's own `environment`, and only process
+  settings stay on the record. Seven operations can meet it (clip_raster,
+  sample_raster_at_points, elevation_profile, least_cost_path, viewshed,
+  watershed, zonal_statistics), and only when a sidecar is present. A
+  single-input record is unchanged.
+
+### Fixed
+
+- **`cluster_points_by_distance` is bounded.** The pairs of points within the
+  distance grow with the square of the points when the distance is large, and
+  6,000 points at a distance of 1e9 took two minutes and 2.3 GB. The pairs are
+  now found in blocks and the call is refused once they pass
+  `MAPSMITH_MAX_SAMPLES`, the operator's limit, before memory is spent.
+  `hot_spots` with `weights="distance_band"` uses the same query: its weights
+  were a double loop over every pair of features in Python, quadratic at any
+  band, and a band that is not a finite positive number is now refused.
+- **`get_lineage` no longer hashes every shapefile in the workspace.** Since
+  the draft.9 digest rule it computed, while indexing, the other rule's digest
+  of every shapefile record's output, whatever the chain asked about. A
+  shapefile is now hashed only when a hop misses on it.
+- **A Windows junction inside a `.gdb` is not followed.** The digest of a
+  directory container walked into junctions, which `is_symlink()` does not
+  report, so files outside the container could enter its digest. Section 3.3 of
+  the specification says links are neither members nor followed. The size used
+  for the re-hash cap now counts the same files as the digest.
+- **A single-layer `.gdb` read by any operation records its layer.** Every
+  operation now reads a single-layer file geodatabase, not only
+  `extract_layer`, and the input records the layer it read: without it, a
+  lineage walk meeting the record of that layer stopped with a false "records
+  claim this container's digest, but for other layers".
+- **`get_lineage` refuses a `layer` holding `:` or a control character.** On
+  Windows `:` in a file name opens an alternate data stream of another file.
+- **`thin_points` refuses a distance that is not a finite positive number**,
+  like `cluster_points_by_distance`; and `least_cost_path` accepts a start or
+  end given as a single-part MultiPoint, which it used to count as no point.
+- **`zonal_statistics` refuses zones that are not polygons.** Line zones were
+  accepted and returned the line's length in pixels as the `count`, with a
+  mean over the cells it crossed: a plausible table, and not a zonal
+  statistic. `count_in_polygons` and `summarize_points_in_polygons` refuse a
+  polygon layer holding anything else the same way. Found by the test that
+  holds the new geometry declarations to the code.
+- **`least_cost_path` is found by a caller holding points.** It reads a cost
+  raster and two point layers but declared its input as raster only, so a
+  caller who said, correctly, that it held a vector layer never saw it. A start
+  or end layer holding a line or a polygon beside its point is now refused; it
+  used to pass with the extra feature ignored.
+- **`snap_layer` snaps to a polygon or multipart reference.** Both crashed with
+  a raw `NotImplementedError` from shapely; every vertex of every part, holes
+  included, is now a target.
+- **A feature with no geometry is refused, with a count, by `thin_points`,
+  `sample_raster_at_points`, `points_along_lines` and `line_intersections`.**
+  They crashed on it with an `AttributeError`, "cannot convert float NaN to
+  integer" and a failed sort.
+- **A shapefile's digest covers its attributes and its CRS.** An input or
+  output shapefile was digested as its `.shp` alone, so a record kept matching
+  a dataset whose `.dbf` (attributes) or `.prj` (CRS) had been rewritten, while
+  it said `sha256` is of the bytes. The digest is now the listing of the
+  `.shp`, `.shx`, `.dbf`, `.prj` and `.cpg` files, as section 3.3 of the
+  specification defines it since draft.9; a file geodatabase input is digested
+  the same way, lock files excluded. `get_lineage` recomputes digests with the
+  same function, so a chain written by this release still resolves. A record
+  written by an earlier release for a shapefile carries the old digest:
+  `get_lineage` compares it under the rule it was written with, so an
+  unchanged shapefile is not reported as edited, and a chain that crosses
+  from a record written by 0.7.1 to one written now still resolves. A record
+  written before this release can only vouch for the `.shp`: an edit to the
+  attributes or the CRS made after it was written stays invisible to it, so
+  such a hop is reported as `reverified_shp_only`, not `reverified`, and a
+  history that contains one is not `verified`.
+- **`extract_layer` works on a file geodatabase.** The catalog listed `.gdb`
+  among its containers, and the input digest opened the path as a file: a
+  geodatabase is a directory, so the operation died before writing anything.
+- **`get_lineage` matches the layer when a digest belongs to a container.**
+  Two records for two layers of one geodatabase share the container's digest
+  (spec section 6, draft.9); a hop into a container now resolves to the record
+  for the layer it names, and stops with `other_layer` rather than attribute
+  one layer to the operation that wrote another. It re-checks a layer's record
+  against the container it sits beside (`<container>.<layer>.provenance.json`,
+  spec section 3.1), and two layers of one container reached on two branches
+  are two histories.
+- **A URI in a manifest path keeps its `//`, so its credentials are masked.**
+  The path normaliser collapsed `https://user:pw@host/x` to `https:/user:pw@…`,
+  and the redaction of URI credentials looks for `://`: the password would have
+  stayed in the record. Not reachable while remote paths are refused, which is
+  the default; found in the 0.7.0 audit. The weights raster of
+  `zonal_statistics` is recorded through the same normaliser now.
+- **A redaction in the environment of a secondary input is declared.** Masked
+  values there did not raise `parameters_redacted`, so the record changed a
+  value without saying so.
+- **The package metadata names the Copernicus NOTICE** beside the AGPL licence,
+  since the source distribution ships the two Copernicus clips the notebook
+  reads.
+- **The links on the PyPI page resolve.** PyPI renders the README with nothing
+  to resolve a relative link against, so every link to a document of the repo
+  and the map-panel screenshot answered 404 there. The build now rewrites them
+  to GitHub URLs pinned to the release tag; on GitHub the README is unchanged.
+
+## [0.7.1] - 2026-09-26
+
+Two fixes from the pre-release audit of 0.7.0, both present in earlier releases,
+and the Argleton run scored against trap 024's corrected truth, which arrived
+after the 0.7.0 tag: that release's PyPI page still quotes the recount of the
+old run. No answer 0.7.0 got right changes, and no manifest key moves: records
+declare `spec_version` `1.0.0-draft.8`, as in 0.7.0.
+
+### Fixed
+
+- **An output that is one of the inputs, or has an extension its writer cannot
+  honour, is refused before anything is written**
+  ([#31](https://github.com/mapsmith-ai/MapSmith/issues/31)). `zonal_statistics`
+  with `output_path` equal to its weights raster replaced the raster with a
+  directory of shapefiles — a vector writer given `.tif` falls back to the
+  Shapefile driver — and failed before writing a manifest. The dedicated tools
+  now refuse both at the boundary, as `run_operation` already refused the
+  collision; and the plan validator turns an extension no writer can honour from
+  a warning into an error (`OUTPUT_EXTENSION_REFUSED`), keeping the warning for
+  writable but non-canonical ones such as `.geojson`.
+
+- **A spacing could ask for any number of points.** `elevation_profile` and
+  `points_along_lines` materialised every point before looking at how many
+  there were: a spacing of `1e-6` on a 90 m line is ninety million. Both now
+  count the points from the line lengths and refuse above
+  `MAPSMITH_MAX_SAMPLES` (default 1 000 000), an environment variable of the
+  server that no tool argument can move; `SECURITY.md` says what is and is not
+  limited. `points_along_lines` also refuses a NaN or infinite spacing, which
+  `spacing <= 0` let through.
 
 ## [0.7.0] - 2026-09-26
 

{mapsmith-0.7.0 → mapsmith-0.8.0}/CITATION.cff

@@ -12,8 +12,8 @@ identifiers:
     value: "10.5281/zenodo.22213091"
     description: "Concept DOI — always resolves to the latest release"
 license: AGPL-3.0-or-later
-version: "0.7.0"
-date-released: "2026-09-26"
+version: "0.8.0"
+date-released: "2026-10-05"
 keywords:
   - GIS
   - geospatial

{mapsmith-0.7.0 → mapsmith-0.8.0}/CLAUDE.md

@@ -5,7 +5,7 @@ MapSmith is professional-grade geoprocessing with **verifiable provenance**, rea
 ## Non-negotiable invariants
 
 1. **Deterministic outputs only**: geometry and numbers always come from engine executions, never from a model.
-2. **Provenance on every writer**: any function that writes a dataset emits `<output>.provenance.json` via `ProvenanceRecord` (inputs with sha256, exact parameters, motivated CRS decisions, engine+version).
+2. **Provenance on every writer**: any function that writes a dataset emits `<output>.provenance.json` via `ProvenanceRecord` (inputs with sha256, exact parameters, motivated CRS decisions, engine+version). The digest is `provenance.dataset_sha256` -- for a shapefile or a `.gdb`, the listing digest of spec section 3.3 -- never `sha256_of` on a dataset path, which hashes one file and missed a shapefile's attributes and CRS until 0.7.1.
 3. **Deterministic verification**: the order is the invariant — verify, repair what is mechanically repairable, write the manifest, and *only then* enforce, so the audit trail survives the error instead of being lost with it. `verify.audited(...)` does all four and is how a new writer is written. A minority still open-code the same sequence inline; that is a migration in progress rather than a second sanctioned shape, and two tests hold the line so this sentence does not have to carry a number that rots: one fails if any hand-written writer skips `verify.enforce`, the other ratchets their count downwards only. (It used to say writers go through `audited` "so the invariant cannot be bypassed", which was false for seventeen of fifty-seven — and one of them enforced nothing at all.) Preconditions: `verify.verify_loaded_inputs(...)` (per input: CRS presence is CRITICAL, emptiness is a warning) **before** any CRS alignment, and `verify.verify_input_pairs(...)` (extent overlap) **after** — get that order wrong and a raw pyproj error beats our own message. Never re-read data just to check it. `verify.audit_on_failure(...)` keeps the diagnosis on disk when the engine crashes. Mechanical repair (`make_valid`) is bounded (2 rounds), writes to a temp file and swaps atomically, uses `gdf.geometry.name` (never assumes "geometry"), refuses multi-layer containers and multi-file formats, and lands in `repairs` — in the manifest *and* in the tool result. Anything needing judgement (empty results, CRS choices) is a hinted check in `warnings`, never silently fixed. `on_empty` is `fail` only where emptiness is provably a bug (buffer, reproject), `warn` where it is legitimate but suspicious (clip, join).
 4. **Explicit CRS discipline**: reject inputs without a CRS (helpful message, naming the declaration when the file has one MapSmith will not resolve); never run metric ops on geographic CRS without a recorded reprojection decision. Resolving a CRS and labelling one are different jobs and live apart: `readers.native_crs` returns coordinate systems, `verify.crs_label` returns strings for manifests, and comparisons go through `verify.same_crs` — a label fed back in as a CRS is how a declared CRS got dropped (#28). The keys of `crs_decisions` are the specification's (`analysis_crs`, `source_crs`, `target_crs`, `transformation`, `reason`) or they carry the `x-mapsmith:` prefix that says they are ours — stricter than section 3.5, which only recommends a prefix, and a house rule because a SHOULD with nothing reading it let two operations invent synonyms of `analysis_crs` for a week. `tests/test_verify.py` derives the recommended set from the vendored schema and sweeps every writing site in the source; `tests/test_showcase.py` does the same for every manifest shown on the README or in the gallery, which the source sweep cannot see.
 5. **GPL boundary**: QGIS/GRASS only via subprocess (CLI/files/JSON), never in-process imports.