mapsmith 0.2.1__tar.gz → 0.2.2__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (72) hide show
  1. mapsmith-0.2.2/.dockerignore +35 -0
  2. mapsmith-0.2.2/CHANGELOG.md +336 -0
  3. {mapsmith-0.2.1 → mapsmith-0.2.2}/CONTRIBUTING.md +1 -1
  4. mapsmith-0.2.2/Dockerfile +32 -0
  5. {mapsmith-0.2.1 → mapsmith-0.2.2}/PKG-INFO +82 -26
  6. {mapsmith-0.2.1 → mapsmith-0.2.2}/README.md +430 -374
  7. mapsmith-0.2.2/SECURITY.md +159 -0
  8. mapsmith-0.2.2/docs/benchmarks.md +473 -0
  9. {mapsmith-0.2.1 → mapsmith-0.2.2}/examples/03_validated_plans.ipynb +23 -23
  10. mapsmith-0.2.2/funding.json +71 -0
  11. {mapsmith-0.2.1 → mapsmith-0.2.2}/pyproject.toml +87 -66
  12. {mapsmith-0.2.1 → mapsmith-0.2.2}/server.json +49 -49
  13. mapsmith-0.2.2/src/mapsmith/__init__.py +12 -0
  14. {mapsmith-0.2.1 → mapsmith-0.2.2}/src/mapsmith/engines/duckdb_engine.py +32 -3
  15. {mapsmith-0.2.1 → mapsmith-0.2.2}/src/mapsmith/engines/vector.py +36 -1
  16. {mapsmith-0.2.1 → mapsmith-0.2.2}/src/mapsmith/engines/whitebox_engine.py +21 -8
  17. mapsmith-0.2.2/src/mapsmith/gdal_policy.py +128 -0
  18. {mapsmith-0.2.1 → mapsmith-0.2.2}/src/mapsmith/jobs.py +95 -87
  19. {mapsmith-0.2.1 → mapsmith-0.2.2}/src/mapsmith/plans/executor.py +169 -158
  20. mapsmith-0.2.2/src/mapsmith/provenance.py +278 -0
  21. mapsmith-0.2.2/src/mapsmith/sql_policy.py +105 -0
  22. {mapsmith-0.2.1 → mapsmith-0.2.2}/src/mapsmith/verify.py +702 -630
  23. mapsmith-0.2.2/src/mapsmith/workspace.py +201 -0
  24. {mapsmith-0.2.1 → mapsmith-0.2.2}/tests/test_duckdb_sandbox.py +43 -14
  25. mapsmith-0.2.2/tests/test_gdal_policy.py +313 -0
  26. mapsmith-0.2.2/tests/test_geoparquet2.py +177 -0
  27. mapsmith-0.2.2/tests/test_redaction.py +207 -0
  28. {mapsmith-0.2.1 → mapsmith-0.2.2}/tests/test_showcase.py +75 -0
  29. mapsmith-0.2.2/tests/test_sql_credentials.py +117 -0
  30. {mapsmith-0.2.1 → mapsmith-0.2.2}/tests/test_workspace.py +52 -9
  31. mapsmith-0.2.1/CHANGELOG.md +0 -141
  32. mapsmith-0.2.1/Dockerfile +0 -17
  33. mapsmith-0.2.1/SECURITY.md +0 -83
  34. mapsmith-0.2.1/docs/benchmarks.md +0 -173
  35. mapsmith-0.2.1/src/mapsmith/__init__.py +0 -3
  36. mapsmith-0.2.1/src/mapsmith/provenance.py +0 -90
  37. mapsmith-0.2.1/src/mapsmith/workspace.py +0 -106
  38. {mapsmith-0.2.1 → mapsmith-0.2.2}/.gitignore +0 -0
  39. {mapsmith-0.2.1 → mapsmith-0.2.2}/CLAUDE.md +0 -0
  40. {mapsmith-0.2.1 → mapsmith-0.2.2}/LICENSE +0 -0
  41. {mapsmith-0.2.1 → mapsmith-0.2.2}/MANIFESTO.md +0 -0
  42. {mapsmith-0.2.1 → mapsmith-0.2.2}/TRADEMARKS.md +0 -0
  43. {mapsmith-0.2.1 → mapsmith-0.2.2}/docker-compose.yml +0 -0
  44. {mapsmith-0.2.1 → mapsmith-0.2.2}/examples/01_verified_geoprocessing.ipynb +0 -0
  45. {mapsmith-0.2.1 → mapsmith-0.2.2}/examples/02_terrain_hydrology.ipynb +0 -0
  46. {mapsmith-0.2.1 → mapsmith-0.2.2}/examples/README.md +0 -0
  47. {mapsmith-0.2.1 → mapsmith-0.2.2}/examples/fixtures/mount_st_helens_dem.tif +0 -0
  48. {mapsmith-0.2.1 → mapsmith-0.2.2}/src/mapsmith/catalog.py +0 -0
  49. {mapsmith-0.2.1 → mapsmith-0.2.2}/src/mapsmith/engines/__init__.py +0 -0
  50. {mapsmith-0.2.1 → mapsmith-0.2.2}/src/mapsmith/engines/dispatch.py +0 -0
  51. {mapsmith-0.2.1 → mapsmith-0.2.2}/src/mapsmith/engines/raster.py +0 -0
  52. {mapsmith-0.2.1 → mapsmith-0.2.2}/src/mapsmith/engines/sedona_engine.py +0 -0
  53. {mapsmith-0.2.1 → mapsmith-0.2.2}/src/mapsmith/plans/__init__.py +0 -0
  54. {mapsmith-0.2.1 → mapsmith-0.2.2}/src/mapsmith/plans/models.py +0 -0
  55. {mapsmith-0.2.1 → mapsmith-0.2.2}/src/mapsmith/plans/registry.py +0 -0
  56. {mapsmith-0.2.1 → mapsmith-0.2.2}/src/mapsmith/plans/validator.py +0 -0
  57. {mapsmith-0.2.1 → mapsmith-0.2.2}/src/mapsmith/preview.py +0 -0
  58. {mapsmith-0.2.1 → mapsmith-0.2.2}/src/mapsmith/server.py +0 -0
  59. {mapsmith-0.2.1 → mapsmith-0.2.2}/src/mapsmith/ui.py +0 -0
  60. {mapsmith-0.2.1 → mapsmith-0.2.2}/tests/test_catalog.py +0 -0
  61. {mapsmith-0.2.1 → mapsmith-0.2.2}/tests/test_catalog_retrieval.py +0 -0
  62. {mapsmith-0.2.1 → mapsmith-0.2.2}/tests/test_engines.py +0 -0
  63. {mapsmith-0.2.1 → mapsmith-0.2.2}/tests/test_plans.py +0 -0
  64. {mapsmith-0.2.1 → mapsmith-0.2.2}/tests/test_preview.py +0 -0
  65. {mapsmith-0.2.1 → mapsmith-0.2.2}/tests/test_raster.py +0 -0
  66. {mapsmith-0.2.1 → mapsmith-0.2.2}/tests/test_smoke.py +0 -0
  67. {mapsmith-0.2.1 → mapsmith-0.2.2}/tests/test_ui.py +0 -0
  68. {mapsmith-0.2.1 → mapsmith-0.2.2}/tests/test_verification_status.py +0 -0
  69. {mapsmith-0.2.1 → mapsmith-0.2.2}/tests/test_verify.py +0 -0
  70. {mapsmith-0.2.1 → mapsmith-0.2.2}/tests/test_verify_repair.py +0 -0
  71. {mapsmith-0.2.1 → mapsmith-0.2.2}/tests/test_whitebox.py +0 -0
  72. {mapsmith-0.2.1 → mapsmith-0.2.2}/tests/test_whitebox_encoding.py +0 -0
@@ -0,0 +1,35 @@
1
+ # The build only needs pyproject/README/LICENSE/src, but everything else in the
2
+ # directory is still sent to the daemon without this file — including a local
3
+ # .venv, built wheels and any .env sitting next to them. CI builds from a clean
4
+ # checkout, so nothing private has ever reached a published layer; this is here
5
+ # so a `docker build .` on a working machine cannot be the first time.
6
+ .git/
7
+ .github/
8
+ .venv/
9
+ venv/
10
+ env/
11
+ dist/
12
+ build/
13
+ *.egg-info/
14
+ __pycache__/
15
+ *.py[cod]
16
+ .pytest_cache/
17
+ .ruff_cache/
18
+ .mypy_cache/
19
+ .ipynb_checkpoints/
20
+
21
+ # secrets and local configuration
22
+ .env
23
+ .env.*
24
+ *.pem
25
+ *.key
26
+
27
+ # large or irrelevant to the image
28
+ benchmarks/
29
+ docs/
30
+ examples/
31
+ tests/
32
+ deploy/
33
+ *.md
34
+ !README.md
35
+ uv.lock
@@ -0,0 +1,336 @@
1
+ # Changelog
2
+
3
+ All notable changes to MapSmith are documented here, in the format of
4
+ [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). This project follows
5
+ [semantic versioning](https://semver.org/).
6
+
7
+ ## [Unreleased]
8
+
9
+ ## [0.2.2] — 2026-08-22
10
+
11
+ A security release. The fixes below close holes present in 0.2.1, which is
12
+ published: **if you run MapSmith on data or paths an agent can influence,
13
+ upgrade.** The headline is that a plain local file could make GDAL fetch a URL
14
+ or read a dataset from outside the workspace, and that credentials could reach
15
+ a manifest.
16
+
17
+ Worth saying plainly, because it shaped the release: each fix here was found by
18
+ auditing the previous one. The remote opt-in left `.vrt` able to reach the
19
+ network; closing `.vrt` left `GDALG` and `MRF` doing the same thing; refusing
20
+ credential SQL wrote the credential into the refusal message. The pattern is
21
+ not bad luck — it is what happens when a guard is written against an instance
22
+ instead of a class, and the response in each case was to move the check one
23
+ level down rather than add another name to a list.
24
+
25
+ ### Added
26
+
27
+ - **`funding.json`** at the repository root ([FLOSS/fund
28
+ manifest](https://fundingjson.org/)), stating what the project would use
29
+ funding for: the correctness suite and the provenance specification. No
30
+ payment provider is published — arrangements are made in writing.
31
+
32
+ ### Changed
33
+
34
+ - **Remote and virtual paths are refused by default** (`MAPSMITH_ALLOW_REMOTE=1`
35
+ to allow them). GDAL `/vsi*` and `https://` forms used to be accepted whenever
36
+ no workspace was set, justified as being the user's own responsibility — and
37
+ the user is not who decides: the path is written by the model, from whatever it
38
+ read, so a third-party dataset carrying "the updated layer lives at
39
+ https://evil.tld/x.gpkg" was enough to have GDAL parse attacker-chosen bytes
40
+ in-process. The refusal now covers both path arguments and `run_sql` text,
41
+ which closes the SSRF that came with it (raw SQL could read any endpoint the
42
+ host can reach — internal services, cloud metadata — and return the content in
43
+ the tool result) as well as the `INSTALL ... FROM '<url>'` fetch. Cloud-native
44
+ data stays a supported use case: the capability is gated, not removed. A
45
+ workspace refuses remote forms regardless, and validated plans stay strict
46
+ whatever the setting.
47
+ - **The container is unprivileged and confined by default.** The published image
48
+ runs as uid 1000 instead of root and sets `MAPSMITH_WORKSPACE=/data` itself, so
49
+ `docker run -v your/data:/data ghcr.io/mapsmith-ai/mapsmith` gets the path jail
50
+ and the sandboxed SQL engine without the operator having to remember `-e` — the
51
+ wrong way round for a default. Two consequences before you upgrade: a bind
52
+ mount owned by another user is no longer writable (pass
53
+ `--user $(id -u):$(id -g)`), and everything outside `/data` is refused,
54
+ including remote paths, which a workspace refuses whatever
55
+ `MAPSMITH_ALLOW_REMOTE` says. The Kubernetes example states the same posture at
56
+ pod level (`runAsNonRoot`, `readOnlyRootFilesystem`, dropped capabilities).
57
+
58
+ ### Fixed
59
+
60
+ - **GDAL indirection was closed as an instance, not as a class.** Deregistering
61
+ the drivers behind `.vrt` left `GDALG` and `MRF` doing the same job. GDALG
62
+ (*GDAL Streamed Algorithm*, GDAL 3.11) reads a JSON document holding a `gdal`
63
+ command line and runs it when the dataset is opened, and it is recognised by
64
+ **content, not by extension** — so the filename tells you nothing, and the
65
+ command line can name a local path as readily as a URL. Measured with remote
66
+ reads off **and** a workspace set: a file called `roads.geojson` inside the
67
+ workspace issued a GET, and another read a dataset from *outside* the
68
+ workspace and handed back its rows. That is containment broken, not only
69
+ egress. MRF is narrower — extension-gated, fetched on the first pixel read.
70
+ Both are now skipped, along with the remaining drivers GDAL's own security
71
+ page names as opening other datasets internally.
72
+
73
+ The list is not the fix. It was correct when written and became incomplete
74
+ because GDAL shipped a new driver, which will happen again. A test now
75
+ enumerates the drivers registered in a clean subprocess and fails on any name
76
+ nobody has reviewed — the only version of this check that keeps working
77
+ across upstream releases.
78
+ - **A workspace no longer loses to the opt-in.** `MAPSMITH_ALLOW_REMOTE=1`
79
+ together with `MAPSMITH_WORKSPACE` re-registered the indirection drivers,
80
+ because the predicate answering "is remote allowed" read only the environment
81
+ variable while its own documentation said a workspace overrides it. Two
82
+ callers compensated for that and one did not. The predicate answers the whole
83
+ question now: a check each caller has to remember to add is a check that will
84
+ be missing somewhere.
85
+ - **Plan manifests are redacted, and the credential refusal no longer quotes the
86
+ credential.** `<output>.plan.json` is written as a plain dict, so the redaction
87
+ applied to every provenance record never reached it — while `goal`, each
88
+ step's `comment` and a failed step's `error` all carry text written by the
89
+ model or the user. The sharp case: refusing `ATTACH 'postgres://user:pw@…'`
90
+ produced a message quoting the fragment, password included, and that message
91
+ is what the manifest recorded. The mechanism that exists to keep credentials
92
+ out of manifests was putting one in. Redaction now runs on the whole manifest
93
+ at the point it becomes a file.
94
+ - **A credential written with a quoted identifier is refused.**
95
+ `SET "s3_secret_access_key" = '…'` and `PRAGMA "…"='…'` defeated *both* layers:
96
+ the refusal pattern required the credential word to follow `SET` contiguously,
97
+ and the redaction pattern allowed only whitespace between the name and the
98
+ `=`, so the closing quote broke each of them. This was not one of the two
99
+ documented limits — the name was perfectly recognisable.
100
+ - **`crs_decisions` and `notes` are redacted on the paths that actually run.**
101
+ Redaction happened at construction, and no engine passes those fields to the
102
+ constructor — every one assigns afterwards. So two of the four fields
103
+ SECURITY.md lists as covered were never redacted in practice, and
104
+ `parameters_redacted` stayed false. Redaction now also runs in `write_for`,
105
+ the single point where a manifest becomes a file, and covers `verification`
106
+ and `repairs` as well. The test that claimed to cover this passed both fields
107
+ as constructor arguments — a shape no caller uses — so it was green and
108
+ proved nothing; it now uses the real one.
109
+ - **A local GDAL indirection file could reach the network from inside a
110
+ workspace.** A `.vrt` is a plain local path, so the path guard, the SQL scan
111
+ and DuckDB's `allowed_directories` all saw a local file while GDAL fetched
112
+ whatever its `<SrcDataSource>` named — measured on 0.2.1 as HEAD and GET
113
+ leaving the process with `MAPSMITH_ALLOW_REMOTE` unset **and**
114
+ `MAPSMITH_WORKSPACE` set, through the GeoPandas/pyogrio path. That
115
+ contradicted the one promise SECURITY.md states as testable, so the fix is at
116
+ GDAL's level: with remote reads off, the indirection and network drivers are
117
+ deregistered before the geospatial stack initialises. The opt-in restores
118
+ them, including when a parent process installed the policy — containers pass
119
+ their whole environment down, and a switch that cannot lift an inherited
120
+ policy is a switch that does nothing. Found by an adversarial audit of the
121
+ commit that introduced the opt-in, i.e. of the fix itself.
122
+ - **Credentials no longer reach provenance manifests or the job ledger.**
123
+ `run_sql` records the query, and manifests are made to be shared, so an agent
124
+ emitting `CREATE SECRET (... SECRET 'AKIA…')` in the same session used to write
125
+ that key into a file destined for a bug report. **SQL that configures a
126
+ credential is now refused before it runs** — `CREATE SECRET` in any spelling,
127
+ `SET`/`PRAGMA` of a credential-bearing setting, `ATTACH` carrying a password or
128
+ URI userinfo — with a message pointing at where credentials belong: the
129
+ environment of the process that starts the server, out of reach of a tool call.
130
+ Nothing documented used that path, and in MapSmith's sandbox only one secret
131
+ type was even constructible.
132
+
133
+ Redaction stays as the second layer for credentials that reach a manifest
134
+ without being SQL (a signed URL as an input path, a connection string as an
135
+ argument), now covering `crs_decisions`, `notes`, input paths and the ledger's
136
+ `error` column — an engine error quotes the statement that failed. Masked
137
+ values are quoted, so a redacted statement still parses when pasted back into
138
+ a client.
139
+
140
+ Refusal came first because redaction alone did not hold: an adversarial audit
141
+ of the shipped version escaped it with `MAP{'Authorization': 'Bearer …'}`, an
142
+ `E'…'` literal, dollar quoting and a comment between name and value — and the
143
+ `E'…'` case masked the *wrong* argument while keeping the secret, producing a
144
+ manifest both misleading and leaky. All four are regression tests now.
145
+ Remaining limits, documented rather than implied: detection is name-based, so
146
+ a bare positional secret is not caught, and neither is a URI that
147
+ percent-encodes the colon of its own userinfo.
148
+ - **GeoParquet 2.0 files are read instead of refused.** 2.0 (`v2.0.0-rc.1`)
149
+ moves geometry into Parquet's own `GEOMETRY`/`GEOGRAPHY` logical types and
150
+ makes the `geo` metadata key optional, and DuckDB already writes such files.
151
+ MapSmith read them inconsistently: `run_sql` worked, `describe_dataset` failed
152
+ with a raw GeoPandas `Missing geo metadata` error, and the CRS probe reported
153
+ `unknown` **even for a file that states its CRS** — which made the CRS
154
+ precondition refuse valid work for a wrong reason. The CRS now comes from the
155
+ logical type when there is no `geo` key, in all the forms met in practice: the
156
+ spec default (`OGC:CRS84`), an authority string, `projjson:<key>`, and the
157
+ whole PROJJSON document inline, which is what DuckDB writes.
158
+
159
+ One form is deliberately not resolved: `srid:<n>`. The spec defines it as a
160
+ numeric identifier and names no authority (its own example is `srid:0`), so
161
+ reading it as `EPSG:<n>` would be MapSmith inventing a coordinate system and
162
+ recording it as fact. Such a file is treated as having no CRS, so the CRS
163
+ precondition refuses it like any other input without one. The message does
164
+ not yet quote the `srid:` declaration that caused it, which would tell the
165
+ agent what to fix — tracked separately rather than claimed here.
166
+
167
+ **Writing now states its flavour instead of inheriting one.** `run_sql`
168
+ materialises with `geoparquet_version 'BOTH'`, so one output file carries
169
+ Parquet's native geometry types (CRS included as PROJJSON) *and* the 1.x `geo`
170
+ metadata — a 2.0-native reader and GeoPandas 1.x both open it, which is
171
+ asserted by a test that reads it back both ways. This was not a free choice:
172
+ DuckDB 1.4 wrote the native types by default and 1.5 changed the default back
173
+ to 1.x, so the installed engine version was silently deciding the canonical
174
+ output format of a provenance product. The GeoPandas writer path stays 1.x,
175
+ because GeoPandas 1.1 caps `schema_version` there.
176
+ - **Dependency floors raised for a correctness reason, not a housekeeping one.**
177
+ `pyarrow>=21`: measured on a file DuckDB itself produced, pyarrow 18 and 19
178
+ **raise** on Parquet's geospatial logical types ("Thrift LogicalType that is
179
+ not recognized"), 20 opens the file but reports the type as `Undefined`, and 21
180
+ reports `Geometry` with its CRS. Below 21, MapSmith could not read back its own
181
+ `run_sql` output — the CRS probe returned `unknown`, which made the CRS
182
+ precondition refuse a file MapSmith had just written. `duckdb>=1.5` is the
183
+ floor for the `geoparquet_version` option above.
184
+ - **Persistent DuckDB secrets can no longer be created from a tool call.** They
185
+ are written to `~/.duckdb/stored_secrets` — outside any workspace and beyond
186
+ the session. A workspace already refused that write; the connection now sets
187
+ `allow_persistent_secrets = false` in both modes, before locking the
188
+ configuration.
189
+ - **The `mcp` floor is 1.28.1.** The SDK range MapSmith allowed admitted three
190
+ High-severity advisories (CVE-2026-59950, CVE-2026-52869, CVE-2026-52870).
191
+ None is exploitable here — MapSmith uses stdio and stateless Streamable HTTP
192
+ with no authentication, never the websocket transport, never task handlers —
193
+ but a fresh install could resolve to a version carrying them, and a scanner
194
+ cannot know the difference.
195
+ - **`duckdb` is now capped below 2.0.** The floor was raised to 1.5 for a
196
+ correctness reason (above); the cap is for a different one. DuckDB 2.0 is on
197
+ the autumn-2026 calendar with breaking changes, and an unbounded requirement
198
+ means the first install after that release can fail with nothing on our side
199
+ having changed. A pin you have to lift deliberately beats an install that
200
+ breaks on someone else's schedule.
201
+
202
+ ## [0.2.1] — 2026-08-20
203
+
204
+ Three fixes you would rather not find yourself. All came from reviewing 0.2.0
205
+ *after* it shipped, and all were reproduced through the real MCP tools instead
206
+ of read off a diff.
207
+
208
+ ### Fixed
209
+
210
+ - **An empty spatial join no longer crashes before writing its manifest.**
211
+ DuckDB writes no GeoParquet `geo` metadata for a zero-row result, so reading
212
+ the output back raised — on the default engine path, for exactly the case the
213
+ verification checks exist to explain, while the tool description promised a
214
+ warning. A zero-row result is now written as a valid empty GeoParquet with the
215
+ analysis CRS and the joined schema, and the join goes through the same audited
216
+ writer as everything else, so the manifest exists even when a check fails.
217
+ - **A GeoParquet declaring `crs: null` is no longer read as CRS84.** MapSmith
218
+ invented a coordinate system and recorded it in the manifest as fact, with
219
+ `verified: true` — the worst class of bug a provenance tool can have. The
220
+ GeoParquet spec distinguishes an *absent* `crs` field, which does mean CRS84,
221
+ from an explicit null, which means unknown; so does MapSmith now, and an
222
+ unknown CRS is refused by the preconditions like any other missing CRS.
223
+ - **The DuckDB sandbox locks its configuration in every mode.** The lock used to
224
+ apply only under `MAPSMITH_WORKSPACE`, so a multi-statement call could switch
225
+ extension autoloading back on, and an explicit `LOAD httpfs` was never blocked
226
+ at all. Locking is now unconditional and DuckDB's HTTP and S3 filesystems are
227
+ disabled, while local reads keep working.
228
+ - **The security documentation no longer claims that unconfined mode blocks
229
+ network egress.** It does not, and it never did: GDAL carries its own HTTP
230
+ client, so `ST_Read('/vsicurl/https://…')` in raw SQL reads whatever the host
231
+ can reach — internal services and metadata endpoints included — and the URL it
232
+ names can carry data out. Remote reads are deliberately available while the
233
+ server is unconfined, because cloud-native data is a feature; the claim that
234
+ the network was closed anyway was the bug. README and SECURITY.md now state
235
+ the price of that choice, and two tests pin both halves of it: without a
236
+ workspace the read succeeds, and with one it is refused *before any request
237
+ leaves*, asserted by counting requests at a loopback server instead of
238
+ matching an error message. If you do not trust your `run_sql` input, set a
239
+ workspace.
240
+ - **The multi-layer guard fails closed.** A container whose layer list could not
241
+ be read looked like a single-layer file, and mechanical geometry repair would
242
+ then have destroyed the other layers while recording success.
243
+ - **`execute_plan` reports `repairs` per step.** Geometry MapSmith rewrote was
244
+ visible in a single-operation result and invisible at plan level.
245
+ - **Manifests record `EPSG:32632`, not 2.5 KB of PROJJSON**, when a GeoParquet
246
+ input carries its CRS as an embedded projection object.
247
+ - **The example `docker-compose.yml` binds MinIO to loopback.** It published
248
+ ports 9000/9001 on every interface with the documented development
249
+ credentials.
250
+
251
+ ### Changed
252
+
253
+ - **The provenance badge has three states instead of two.** "All critical checks
254
+ passed" is vacuously true when no critical check ran — a `run_sql` manifest,
255
+ for one — so an output whose only check had *failed* rendered with the same
256
+ green tick as a verified buffer. `provenance_summary` now reports `verified`,
257
+ `failed` or `unchecked`, with the reason, and the map panel renders all three.
258
+ The `verified` boolean stays in the payload, computed correctly, so a client
259
+ reading it gets a fix rather than a breaking change.
260
+ - **The README says when *not* to use MapSmith**, and the
261
+ [benchmark results](docs/benchmarks.md) are linked from it — they were public
262
+ for a day with nothing pointing at them.
263
+
264
+ ## [0.2.0] — 2026-08-20
265
+
266
+ The first release you can point an agent at and trust the answer: results are
267
+ verified on the way in and on the way out, plans are checked before anything
268
+ runs, and the server can be confined to a single directory.
269
+
270
+ ### Added
271
+
272
+ - **Interactive map inside the chat.** `preview_map` renders your layers on a
273
+ pan/zoom map panel in any client implementing the
274
+ [MCP Apps](https://modelcontextprotocol.io/extensions/apps/overview)
275
+ extension (field-tested on Claude Desktop), with an OpenStreetMap backdrop
276
+ and a provenance card per layer showing operation, engine and one of three
277
+ states: `verified ✓`, `verification failed`, or `not verifiable` when no
278
+ critical check ran. Fully self-contained; on clients without MCP Apps the
279
+ same call returns structured data.
280
+ - **Typed plans.** `validate_plan` statically checks a multi-step analysis —
281
+ operations exist and are installed, arguments complete and well-typed,
282
+ `$step` references resolve backwards, input files exist, outputs don't
283
+ collide, and the CRS of every intermediate is simulated from the real
284
+ inputs — and returns machine-actionable error codes. `execute_plan` then
285
+ runs the validated plan with per-step provenance plus a plan-level manifest
286
+ fingerprinting the exact plan that produced the result.
287
+ - **Terrain and hydrology** on the Whitebox Workflows engine (`[whitebox]`
288
+ extra): `hillshade`, `flow_accumulation` (D8, with depression filling) and
289
+ `watershed` (many pour points at once).
290
+ - **Zonal statistics** with exact fractional pixel coverage via exactextract
291
+ (`[raster]` extra).
292
+ - **A searchable operation catalog.** `list_operations` ranks capabilities by
293
+ relevance (BM25) so an agent can discover what exists — including what is
294
+ planned but not yet available — instead of guessing from a wall of tools.
295
+ - **Workspace confinement.** Set `MAPSMITH_WORKSPACE` and every path argument
296
+ must resolve inside it, `run_sql`'s DuckDB connection is sandboxed to that
297
+ directory with extension loading refused and memory/temp-disk capped, and
298
+ UNC hosts and NTFS alternate data streams are refused in every mode.
299
+ - **Verification on the way in.** Operations check their inputs for the
300
+ failures that produce plausible junk: a missing CRS is refused outright, and
301
+ empty inputs or extents that cannot possibly intersect come back as named
302
+ warnings with hints — in the tool result, not only in the manifest.
303
+ - **Bounded deterministic repair.** Mechanically broken output geometry is
304
+ repaired (`make_valid`, at most two rounds, written atomically) and every
305
+ attempt is recorded in the manifest: a repaired output never looks like one
306
+ that was right the first time.
307
+ - **A notebook gallery** (`examples/`) and a
308
+ [benchmarks page](docs/benchmarks.md) with the harness that produced it.
309
+
310
+ ### Fixed
311
+
312
+ - **Wrong terrain results from ordinary compressed rasters.** Whitebox
313
+ Workflows 2.x does not undo the TIFF predictor when reading, so any DEM
314
+ saved with `PREDICTOR=2` or `3` — the standard encoding for elevation data —
315
+ produced hillshades and flow accumulations that looked like terrain and were
316
+ not. MapSmith now detects the predictor and converts the input first,
317
+ recording it in the manifest.
318
+ ([upstream report](https://github.com/jblindsay/whitebox_next_gen/issues/32))
319
+ - **GeoParquet outputs from the vector engines** were written through a GDAL
320
+ path that produced unreadable files.
321
+ - `run_sql` materialisations and the DuckDB/SedonaDB join fast paths now run
322
+ the same deterministic verification as every other writer, and record their
323
+ CRS decisions.
324
+
325
+ ### Changed
326
+
327
+ - `spatial_join` with `engine="auto"` falls back to GeoPandas when the inputs'
328
+ CRS differ or are unknown, instead of joining mismatched coordinates.
329
+ - The planning-failure figure quoted in the docs is stated as the upper bound
330
+ it is ("up to ~47%"), since the underlying study counts errors multi-label.
331
+
332
+ ## [0.1.0] — 2026-08-18
333
+
334
+ First public release: the engine dispatcher (SedonaDB / DuckDB / GeoPandas),
335
+ `run_sql`, the job ledger, stateless Streamable HTTP transport, and provenance
336
+ manifests with deterministic verification on every writer.
@@ -30,7 +30,7 @@ ruff check .
30
30
 
31
31
  ## Contributor License Agreement
32
32
 
33
- To keep dual-licensing possible (AGPL + commercial), we ask contributors to sign a lightweight CLA on their first PR (automated via bot). Your code always remains available under AGPL-3.0.
33
+ To keep dual-licensing possible (AGPL + commercial), we ask contributors to sign a lightweight CLA on their first PR. There is no bot yet: if you open the first outside PR, we will agree the wording with you in the thread rather than block the merge on tooling that does not exist. Your code always remains available under AGPL-3.0.
34
34
 
35
35
  ## Code style
36
36
 
@@ -0,0 +1,32 @@
1
+ FROM python:3.12-slim
2
+
3
+ # MCP Registry ownership proof: must equal the server.json "name" exactly.
4
+ LABEL io.modelcontextprotocol.server.name="io.github.mapsmith-ai/mapsmith"
5
+
6
+ # Geospatial wheels (pyogrio/shapely/pyproj) bundle their native libs; no system GDAL needed.
7
+ WORKDIR /app
8
+ COPY pyproject.toml README.md LICENSE ./
9
+ COPY src ./src
10
+ # raster + whitebox extras ship manylinux x86_64 wheels; the image is amd64-only.
11
+ RUN pip install --no-cache-dir ".[raster,whitebox]"
12
+
13
+ # Unprivileged by default (#19). uid 1000 on purpose rather than a high system
14
+ # uid: the supported way to run this is a bind mount of your own data directory,
15
+ # which on Linux is usually owned by the first human user — 1000. If yours is
16
+ # not, pass `--user $(id -u):$(id -g)`.
17
+ RUN useradd --create-home --uid 1000 --shell /usr/sbin/nologin mapsmith \
18
+ && mkdir -p /data \
19
+ && chown mapsmith:mapsmith /data
20
+
21
+ # Workspace for datasets: mount your data here. Confined BY DEFAULT — the
22
+ # supported path used to start unconfined unless the operator remembered `-e`,
23
+ # which is the wrong way round for a default.
24
+ VOLUME ["/data"]
25
+ WORKDIR /data
26
+ ENV MAPSMITH_WORKSPACE=/data \
27
+ HOME=/home/mapsmith \
28
+ MPLCONFIGDIR=/home/mapsmith/.config/matplotlib
29
+
30
+ USER mapsmith
31
+
32
+ ENTRYPOINT ["mapsmith"]
@@ -1,7 +1,7 @@
1
1
  Metadata-Version: 2.5
2
2
  Name: mapsmith
3
- Version: 0.2.1
4
- Summary: Professional-grade geoprocessing for AI agents via MCP, with verifiable provenance
3
+ Version: 0.2.2
4
+ Summary: Professional-grade GIS geoprocessing for AI agents via MCP, with verifiable provenance
5
5
  Project-URL: Homepage, https://github.com/mapsmith-ai/MapSmith
6
6
  Project-URL: Repository, https://github.com/mapsmith-ai/MapSmith
7
7
  Author-email: MapSmith <mapsmith@proton.me>
@@ -13,10 +13,10 @@ Classifier: Intended Audience :: Science/Research
13
13
  Classifier: Programming Language :: Python :: 3
14
14
  Classifier: Topic :: Scientific/Engineering :: GIS
15
15
  Requires-Python: >=3.10
16
- Requires-Dist: duckdb>=1.4
16
+ Requires-Dist: duckdb<2,>=1.5
17
17
  Requires-Dist: geopandas>=1.1.2
18
- Requires-Dist: mcp<2,>=1.26
19
- Requires-Dist: pyarrow>=17
18
+ Requires-Dist: mcp<2,>=1.28.1
19
+ Requires-Dist: pyarrow>=21
20
20
  Requires-Dist: pyogrio>=0.9
21
21
  Requires-Dist: pyproj>=3.6
22
22
  Requires-Dist: shapely>=2.0
@@ -42,7 +42,7 @@ Description-Content-Type: text/markdown
42
42
  [![MCP](https://img.shields.io/badge/Model_Context_Protocol-server-654FF0)](https://modelcontextprotocol.io)
43
43
  [![License: AGPL-3.0](https://img.shields.io/badge/license-AGPL--3.0-blue)](LICENSE)
44
44
 
45
- **Professional-grade geoprocessing for AI agents — with provenance you can verify.**
45
+ **Professional-grade GIS geoprocessing for AI agents — with provenance you can verify.**
46
46
 
47
47
  MapSmith is an open-source [MCP](https://modelcontextprotocol.io) server that gives an AI
48
48
  agent real GIS analysis — buffers, overlays, reprojections, zonal statistics, terrain and
@@ -111,6 +111,11 @@ with working wheels — is the only supported installation path**: geospatial na
111
111
  dependencies across three OSes are a support black hole, and issues about broken local
112
112
  environments will be redirected here.
113
113
 
114
+ Two things about the image, because they change what happens on your machine: it sets
115
+ `MAPSMITH_WORKSPACE=/data` itself (the `-e` above is explicit, not required) and runs as
116
+ uid 1000, so pass `--user $(id -u):$(id -g)` if the directory you mount belongs to another
117
+ user; and it is built for amd64 only, so on Apple Silicon it runs under emulation.
118
+
114
119
  ## What you get back
115
120
 
116
121
  Every dataset comes with the file below, written next to it as
@@ -119,7 +124,7 @@ for it:
119
124
 
120
125
  ```json
121
126
  {
122
- "mapsmith_version": "0.2.1",
127
+ "mapsmith_version": "0.2.2",
123
128
  "operation": "buffer_layer",
124
129
  "parameters": {"distance_meters": 300.0},
125
130
  "inputs": [{"path": "rivers.gpkg", "sha256": "9f2c…", "crs": "EPSG:4326"}],
@@ -137,7 +142,7 @@ had to repair. `get_provenance` returns it for any output.
137
142
 
138
143
  - **Real geoprocessing, not map CRUD.** Built on the proven open geospatial stack: GDAL,
139
144
  GeoPandas, Shapely, DuckDB Spatial, Whitebox Workflows and exactextract ship today
140
- (more to come: PDAL, QGIS Processing via sidecar).
145
+ (more to come: QGIS Processing via sidecar).
141
146
  - **Provenance by design.** Every layer MapSmith produces ships with a machine-readable
142
147
  lineage manifest — source datasets with checksums, tools executed, exact parameters, CRS
143
148
  decisions, software versions, timestamps. Everything needed to re-run the analysis
@@ -173,6 +178,28 @@ had to repair. `get_provenance` returns it for any output.
173
178
  | `list_operations` | BM25-ranked catalog search; `detail=true` returns parameters and worked examples |
174
179
  | `server_info` | Version, license, available engines |
175
180
 
181
+ ### Formats
182
+
183
+ | Format | Read | Write |
184
+ |---|---|---|
185
+ | GeoParquet 1.0 / 1.1 — WKB plus `geo` metadata | yes | yes, every path |
186
+ | **GeoParquet 2.0** — Parquet-native `GEOMETRY`/`GEOGRAPHY` logical types | yes, including files that carry no `geo` key at all | yes on the SQL path: `run_sql` writes **both** layers into one file |
187
+ | GeoPackage, Shapefile, FlatGeobuf, GeoJSON, … | anything pyogrio/GDAL opens | via GDAL |
188
+ | GeoTIFF / COG | yes | outputs of the `[raster]` and `[whitebox]` engines |
189
+
190
+ GeoParquet [2.0](https://github.com/opengeospatial/geoparquet/releases) moves geometry
191
+ into Parquet's own logical types and makes the `geo` key optional, so "a Parquet file with
192
+ geometry in it" no longer implies that key. MapSmith reads the CRS from the logical type
193
+ when it is the only place it exists — the spec default, an authority string,
194
+ `projjson:<key>`, or the whole PROJJSON document inline, which is what DuckDB writes.
195
+ `run_sql` emits both layers (`geoparquet_version 'BOTH'`), so one output file satisfies a
196
+ 2.0-native reader and a GeoPandas 1.x one; the GeoPandas writer path stays 1.x because
197
+ GeoPandas 1.1 caps `schema_version` there.
198
+
199
+ One declaration is deliberately refused rather than guessed: `srid:<n>`. The spec defines
200
+ it as a numeric identifier and names no authority — its own example is `srid:0` — so
201
+ reading it as `EPSG:<n>` would be inventing a coordinate system and recording it as fact.
202
+
176
203
  ## Verification, in and out
177
204
 
178
205
  Every tool that writes a dataset also writes `<output>.provenance.json` beside it and
@@ -253,9 +280,9 @@ that produced the result.
253
280
  UNC hosts and NTFS alternate data streams are refused in every path *argument* of every
254
281
  tool call, before anything touches the filesystem (on Windows even an existence check on a
255
282
  UNC path talks to an attacker-chosen host). Remote and virtual forms — GDAL `/vsi*`,
256
- `https://` COGs — stay available while the server is unconfined, because cloud-native data
257
- is a feature, and are refused once a workspace is set. Validated plans are stricter by
258
- design and reject every non-local form.
283
+ `https://` COGs — are refused by default since 0.2.2 and need `MAPSMITH_ALLOW_REMOTE=1`;
284
+ a workspace refuses them whatever that setting says (details below). Validated plans are
285
+ stricter by design and reject every non-local form, opt-in or not.
259
286
 
260
287
  Set `MAPSMITH_WORKSPACE=/data` to confine the server to one directory:
261
288
 
@@ -275,13 +302,15 @@ community extensions are off (`shellfs` turns a filename into a shell command),
275
302
  extensions are refused, DuckDB's HTTP and S3 filesystems are disabled, and the
276
303
  configuration is locked, so untrusted SQL cannot turn file access into code execution.
277
304
 
278
- **The network is not closed in that mode, by the same decision that keeps cloud-native
279
- data working**: GDAL carries its own HTTP client, so `ST_Read('/vsicurl/https://…')` in raw
280
- SQL reads whatever the host can reach — internal services and link-local metadata
281
- endpoints included — and the URL is chosen by whoever wrote the SQL, so it can also carry a
282
- string out. Setting `MAPSMITH_WORKSPACE` closes all of it, GDAL included, and the test
283
- suite asserts both halves (`tests/test_duckdb_sandbox.py`). If your `run_sql` input is not
284
- trusted and your host sits somewhere interesting, set a workspace. The full threat model —
305
+ **The network is closed too, unless you open it.** Remote and virtual forms — GDAL
306
+ `/vsi*`, `https://` COGs — are refused by default in path arguments *and* inside `run_sql`
307
+ text, because the path is written by the model rather than by you: a third-party dataset
308
+ carrying "the updated layer lives at https://evil.tld/x.gpkg" was otherwise enough to have
309
+ GDAL parse attacker-chosen bytes in-process. Set **`MAPSMITH_ALLOW_REMOTE=1`** to allow them
310
+ — cloud-native data is a real use case and the capability is gated, not removed. A workspace
311
+ refuses them regardless, since containment and "fetch whatever URL the model names" cannot
312
+ both be true. The test suite asserts every branch by counting requests at a loopback server
313
+ (`tests/test_duckdb_sandbox.py`). The full threat model —
285
314
  and what is explicitly *not* covered — is in [SECURITY.md](SECURITY.md).
286
315
 
287
316
  Fine print, because it changes how you deploy this: the path jail assumes a single trusted
@@ -318,8 +347,17 @@ in every arm, i.e. wrong parameters and missing outputs at *execution* time,
318
347
  which is why MapSmith enforces its plans at the execution boundary and verifies
319
348
  inputs and outputs at runtime rather than advising an agent that improvises.
320
349
 
350
+ Three further arms then measured the configuration MapSmith actually ships —
351
+ the plan *enforced*, no improvisation between validation and execution — over
352
+ 375 runs, and the result cuts both ways: enforcing reproduces its own score
353
+ 3–18× more tightly than an improvising solver, and it does **not** beat it on
354
+ accuracy (parity on tool selection, measurably worse on ordering). One of those
355
+ arms also refuted a conclusion this page had published two arms earlier; the
356
+ correction is kept in place rather than edited away.
357
+
321
358
  The harness is in [`benchmarks/gabench-ab/`](benchmarks/gabench-ab/), including
322
- the `split_analysis.py` that took our own win apart.
359
+ the `split_analysis.py` that took our own win apart and the
360
+ `rep_analysis.py` that bars every delta against a measured noise floor.
323
361
 
324
362
  ## Notebook gallery
325
363
 
@@ -367,13 +405,19 @@ detects it, converts the input first, and discloses the workaround in the manife
367
405
  - **Your data lives in a database.** MapSmith reads and writes files (GeoParquet,
368
406
  GeoPackage, anything GDAL opens). There is no PostGIS engine and no database catalog —
369
407
  the `[postgres]` extra is for the optional job ledger, not for data.
408
+ - **Your data lives in object storage.** Since 0.2.2 remote and virtual paths are refused
409
+ unless you set `MAPSMITH_ALLOW_REMOTE=1`, and refused whatever that setting says under a
410
+ workspace — which is what the container runs with by default. DuckDB's own HTTP and S3
411
+ filesystems stay off in every mode, so `read_parquet('s3://…')` does not work even with
412
+ the opt-in: fetch the data down first, or run unconfined with remote reads on.
370
413
  - **You want the full breadth of a desktop GIS.** 16 tools plus a catalog that tells the
371
414
  agent what does *not* exist yet. The ~900 QGIS Processing algorithms are on the roadmap,
372
415
  not in the box.
373
416
  - **You expect plan validation to make a weak model strong.** Our own A/B says advisory
374
417
  validation upstream of an improvising solver does approximately nothing at aggregate
375
- level; MapSmith's answer is enforcement at the execution boundary, and that hypothesis
376
- is not measured yet.
418
+ level — and the enforced configuration MapSmith ships, measured afterwards, did not beat
419
+ it on accuracy either. What enforcement buys is reproducibility
420
+ ([the numbers](docs/benchmarks.md)).
377
421
  - **You want us to debug your local geospatial toolchain.** Docker, or `uvx` where the
378
422
  wheels work, are the only supported paths; a hand-built native GDAL stack is not, on
379
423
  purpose.
@@ -385,12 +429,19 @@ detects it, converts the input first, and discloses the workaround in the manife
385
429
  - [x] Typed analysis plans: static validation against the operation registry + simulated CRS flow before execution
386
430
  - [x] Runtime verification: input preconditions, warnings with hints in the tool result, bounded deterministic repair recorded in the manifest
387
431
  - [x] MCP Apps in-chat map panel with provenance cards (self-contained, works under the default host sandbox)
388
- - [ ] Agent-loop repair: hand verification failures back to the agent for a bounded number of retries
432
+ - [x] GeoParquet 2.0: read Parquet-native geometry types (including files with no `geo` key), write both layers from the SQL path — the GeoPandas writer path follows when GeoPandas lifts its `schema_version` cap and 2.0.0 stops being a release candidate
433
+
434
+ Next, in the order we intend to do it. The linked items carry a written spec — a roadmap line without one is a wish, so the rest get theirs before work starts on them:
435
+
436
+ - [ ] [**Silent-corruption suite**](https://github.com/mapsmith-ai/MapSmith/issues/25): an eval for the failure every existing benchmark misses — a result that is wrong and reported as successful. Closed-form truth, no LLM in the evaluator, and it runs against MapSmith with verification *disabled* so it can fail us in public
437
+ - [ ] [Agent-loop repair](https://github.com/mapsmith-ai/MapSmith/issues/26): hand verification failures back to the agent as structured, actionable errors, with a bounded retry budget recorded in the manifest. Our [own measurements](docs/benchmarks.md) say the runtime error message is the information channel that works
438
+ - [ ] [Tool contracts that carry their own rules](https://github.com/mapsmith-ai/MapSmith/issues/27): argument constraints enforced *and* stated, and errors that name the rule rather than only the violation. The one intervention in our benchmark work that moved a metric past its noise floor
439
+ - [ ] [Satellite embeddings as a first-class input](https://github.com/mapsmith-ai/MapSmith/issues/24): per-zone embedding vectors (multiband zonal statistics) and similarity rasters against a reference location, over the open [AlphaEarth annual dataset](https://developers.google.com/earth-engine/guides/aef_on_gcs_readme) (CC-BY 4.0 COGs). Deterministic arithmetic on a raster — no model inference in MapSmith — with the tile, year and reference vector recorded in the manifest
440
+ - [ ] Authenticated remote mode (OAuth on the existing Streamable HTTP transport) and [long-job progress via MCP Tasks](https://github.com/mapsmith-ai/MapSmith/issues/8). This is the item that closes the one limitation [SECURITY.md](SECURITY.md) declares outright: the HTTP transport has no authentication today
389
441
  - [ ] More terrain & hydrology: slope/aspect, stream network extraction
390
- - [ ] QGIS Processing sidecar (subprocess-isolated): ~900 algorithms
391
- - [ ] Sandboxed code-execution tool for the long tail
392
- - [ ] Map panel: MapLibre vector rendering and shareable viewer URLs (raster OSM tiles already ship)
393
- - [ ] Authenticated remote mode (OAuth on the existing Streamable HTTP transport) and long-job progress via MCP Tasks
442
+ - [ ] Map panel: MapLibre vector rendering, and an export of the panel as a self-contained HTML file you host yourself (raster OSM tiles already ship). No hosted viewer — MapSmith runs on your machine and we would rather not own your maps
443
+ - [ ] [Sandboxed code-execution tool](https://github.com/mapsmith-ai/MapSmith/issues/7) for the long tail
444
+ - [ ] QGIS Processing sidecar (subprocess-isolated): ~900 algorithms. By far the largest item on this list — parameter mapping and error handling for an external process, not an afternoon
394
445
 
395
446
  ## License and project
396
447
 
@@ -401,6 +452,11 @@ You can self-host MapSmith freely, forever. If you modify it and offer it as a s
401
452
  AGPL asks you to share your changes — or [talk to us](mailto:mapsmith@proton.me) about a
402
453
  commercial license.
403
454
 
455
+ Nothing here has been funded so far. [`funding.json`](funding.json) states, in the
456
+ [FLOSS/fund](https://fundingjson.org/) format, the two pieces of work that money would go
457
+ to: a public suite of geospatial traps with hand-computable answers, and the provenance
458
+ manifest as a specification other tools can implement.
459
+
404
460
  Release notes are in [CHANGELOG.md](CHANGELOG.md), how to contribute in
405
461
  [CONTRIBUTING.md](CONTRIBUTING.md), how to report a vulnerability in
406
462
  [SECURITY.md](SECURITY.md). "MapSmith" is a trademark of the MapSmith project — see