argleton 0.3.0__tar.gz → 0.5.0__tar.gz

{argleton-0.3.0 → argleton-0.5.0}/.github/workflows/ci.yml

@@ -16,6 +16,12 @@ jobs:
         python: ["3.11", "3.13"]
     steps:
       - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683  # v4.2.2
+        with:
+          # Tags, because one showcase test checks the trap count the README
+          # attributes to a release against the tag that release was cut from.
+          # A shallow checkout has no tags, and the test would skip in exactly
+          # the place it is meant to run.
+          fetch-depth: 0
       - uses: actions/setup-python@42375524e23c412d93fb67b49958b491fce71c38  # v5.4.0
         with:
           python-version: ${{ matrix.python }}
@@ -76,6 +82,22 @@ jobs:
           # point — a trap that stops catching its own naive adapter, or one
           # that starts catching a case it should not, has changed what it
           # measures, and that must be a decision rather than a surprise.
-          # 0.9545 = traps 002 through 022 caught, 001 passed, out of 22.
-          sys.exit(0 if (r["silent_error_rate"], r["completion_rate"]) == (0.9545, 1.0) else 1)
+          # 0.9355 = 29 of 31 traps caught. TWO still pass, unchanged: 001,
+          # where rasterio undoes the predictor on the naive adapter's behalf,
+          # and 026, where `src.res` reports the cell size faithfully whichever
+          # way the rows run — so a plain numpy gradient is more faithful to the
+          # geotransform than a specialised terrain engine is. Moved from
+          # 0.9333 on 2026-09-02 when trap 031 added family 29 — a shapefile
+          # whose inner ring is wound like its outer one, which the naive
+          # composition reads as two shells and adds — so both numerator and
+          # denominator went up by one. Before that 0.931 on 2026-08-31 when
+          # trap 030 added family 28, the same way. Before
+          # that 0.96 when traps 026 to 029 closed the original family list, and
+          # 0.9583, 0.9565 and 0.9545 as traps 025, 024 and 023 landed.
+          # Each time the denominator changed and the numerator changed with
+          # it. This pin failing on such a commit is the
+          # mechanism working — the number moved because a trap was added, which
+          # is a decision, and it has to be written down here rather than
+          # discovered in a published table.
+          sys.exit(0 if (r["silent_error_rate"], r["completion_rate"]) == (0.9355, 1.0) else 1)
           PY

argleton-0.5.0/.zenodo.json

@@ -0,0 +1,19 @@
+{
+  "upload_type": "software",
+  "title": "Argleton: a correctness suite for geospatial systems",
+  "version": "0.5.0",
+  "creators": [
+    {"name": "The Argleton project"}
+  ],
+  "license": "apache-2.0",
+  "access_right": "open",
+  "keywords": [
+    "geospatial",
+    "GIS",
+    "benchmark",
+    "correctness",
+    "silent errors",
+    "evaluation"
+  ],
+  "description": "<p>Argleton measures how often a geospatial system answers confidently and wrongly.</p><p>Each probe has a right answer known <em>by construction</em> &mdash; derived on paper, never measured from a reference implementation &mdash; and a typical wrong answer that looks entirely plausible. Every trap ships with a defect-free twin, so a system cannot score well by refusing to answer: the suite reports a silent-error rate alongside a completion rate on the twins, and a refusal counts as neither a pass nor a silent error.</p><p>The name is the phantom settlement Google Maps showed for two years near Aughton: an empty field for which the map offered photographs of houses, restaurants and a hospital. Well-formed data, valid against its schema, rendered with confidence, entirely false, and crashing nothing. That is the class of error this suite measures.</p><p>Published from an organisation separate from the system it was extracted from, and that relationship is stated in the repository rather than hidden &mdash; an evaluation living inside the thing it evaluates can be dismissed in one line.</p>"
+}

argleton-0.5.0/CHANGELOG.md

@@ -0,0 +1,111 @@
+# Changelog
+
+All notable changes to Argleton are documented here, in the format of
+[Keep a Changelog](https://keepachangelog.com/en/1.1.0/). This project follows
+[semantic versioning](https://semver.org/).
+
+This file starts at 0.5.0. Releases before it are described by their tags and
+by the dated sections of [`results/README.md`](results/README.md), which is
+where this project's findings have always been written — the omission was that
+a reader had nowhere to see, in one place, what changed between two versions of
+the suite itself.
+
+## [Unreleased]
+
+Nothing yet.
+
+## [0.5.0] - 2026-09-21
+
+Thirty-one commits since 0.4.0. The headline is that a third-party system was
+measured against the engine it wraps, for the first time, and the result is
+that the wrapper is not the thing to blame.
+
+### Added
+
+- **Three QGIS rows, and they are the same QGIS.** The processing engine driven
+  headless through `qgis_process`, and the two MCP servers that run inside a
+  live QGIS and forward to it — `nkarasiak/qgis-mcp` 0.14.0 over its plugin
+  socket, QGIS Agent MCP 0.5.0 over its local bridge. All three answer **every
+  one of the thirty-one traps**, nothing unsupported, and all three come out at
+  the same silent-error rate of **0.3871**.
+
+  That equality is the finding, and it is only readable because the engine has
+  a row of its own: the wrappers inherit it, neither adding a correct answer
+  nor losing one. Without the engine's row, three equal numbers read as three
+  equally defective servers, and that reading would be wrong. The engine's row
+  was built first for exactly this reason.
+
+  The chains are shared by all three adapters in one file, so a difference
+  between the rows can only come from the system rather than from a chain that
+  drifted between copies.
+
+- **`gis-mcp` 0.15.0**, the first system here whose defects are not ours to
+  fix, published on 2026-09-10 after its maintainer was told first.
+
+- **Family 29, `ring-role-by-winding`.** A shapefile carries no nesting, so
+  which ring is a hole is decided by the direction it is wound and by nothing
+  else. An inner ring wound like its parent reads as a second shell and its
+  area is added: 31000 m² where the truth is 29000, +6.9% in the owner's
+  favour, with the right bounding box, the right CRS and no warning.
+
+### Changed
+
+- **`argleton.__version__` is derived instead of written.** It answered
+  `0.1.0.dev0` while 0.4.0 was on PyPI, across four releases, because the
+  number lived in two places and only one was on a checklist. Nothing inside
+  the package read it, so nothing caught it. It now comes from the installed
+  distribution, or from `pyproject.toml` when running from a checkout, and a
+  test compares the two.
+
+- **The naive baseline pin moves to 0.9355**, which is the pin doing its job
+  rather than a number being adjusted: the denominator grew when the suite
+  reached thirty-one traps.
+
+- **A fixture build that dies with no output is retried on its signature, not
+  on a count.** One retry had been calibrated against 370 builds, where a
+  doubled failure is about one in a thousand. At nine adapters and 558 builds
+  the same doubled failure is about one in twenty. The constant was not wrong
+  when it was written; it went wrong when the table grew, and nothing was
+  watching it. The retry now keys on what the death looks like — both output
+  streams empty, because the process never ran — which is stricter than what it
+  replaces: a builder that fails while saying something now fails on the first
+  attempt.
+
+### Fixed
+
+- **The site sorted both QGIS MCP servers below the naive baseline**, the row
+  that exists to be the worst line on the page, because neither matched a
+  prefix in the ordering list and an unmatched row sorts last. The comment
+  above that list had named their correct place two months earlier — beside
+  `gis-mcp`, with the engine they call directly underneath. A rule stated in a
+  comment and not executed by code is not a rule; a test covers it now.
+
+- **The site said "every system here completes every clean probe"** directly
+  under a column showing two systems that do not.
+
+- **Three surfaces said every number comes from calling code in the same
+  process rather than over a transport.** Two rows are a TCP socket and a local
+  bridge into a live QGIS, which is what their own labels say.
+
+- **The results page said we had "found nothing of its own" about one wrapper**,
+  seven lines after describing a property of that wrapper. The true sentence is
+  "nothing that makes an answer wrong", and the notification rule is restated
+  around it.
+
+- **"In opposite directions" described two rows with the same probe, the same
+  verdict and the same completion rate.** They differ in how they fail.
+
+- **`FAMILIES.md` published four false sentences about its own count** while
+  two guards on counts were green, because they watched three other sentence
+  shapes. The numbers are derived now, and an unexplained spelled-out number in
+  that file is a failure rather than a gap.
+
+- **The results index counts its own rows** instead of only checking the ones
+  that happen to be listed, and the tool count is asked of the server rather
+  than written down.
+
+- **`publish_run.py`** — in the private workspace, but its effects are here —
+  refuses to publish an adapter whose failures all carry one message: that is a
+  system we could not reach, not a system that got things wrong. A run was
+  minutes from publishing `completion 0.2258` for a third-party server whose
+  62 probes had all failed to connect.

{argleton-0.3.0 → argleton-0.5.0}/CITATION.cff

@@ -5,10 +5,15 @@ type: software
 authors:
   - name: "The Argleton project"
 repository-code: "https://github.com/argleton/argleton"
+doi: "10.5281/zenodo.22206349"
+identifiers:
+  - type: doi
+    value: "10.5281/zenodo.22206349"
+    description: "Concept DOI — always resolves to the latest release"
 url: "https://argleton.org"
 license: Apache-2.0
-version: "0.3.0"
-date-released: "2026-08-29"
+version: "0.5.0"
+date-released: "2026-09-21"
 abstract: >-
   Probes whose right answer is known by construction — derived on paper, never
   measured from a reference implementation — and whose typical wrong answer

{argleton-0.3.0 → argleton-0.5.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: argleton
-Version: 0.3.0
+Version: 0.5.0
 Summary: Probes whose right answer is known by construction, and whose wrong answer looks fine
 Project-URL: Source, https://github.com/argleton/argleton
 Author: Argleton contributors
@@ -42,6 +42,13 @@ known by construction, and every trap has a *wrong* answer that looks fine.
 **[argleton.org](https://argleton.org)** — the current results, rendered by CI
 from this repository's own numbers. Nothing on that page is typed in by hand.
 
+[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.22206349.svg)](https://doi.org/10.5281/zenodo.22206349)
+
+Cite it as `10.5281/zenodo.22206349`, which always resolves to the current release. To cite the
+exact version you measured against, take the version DOI from the
+[Zenodo record](https://doi.org/10.5281/zenodo.22206349) — which is the same reason every run
+here pins its `spec_commit`.
+
 That second half is the whole point. Existing benchmarks for geospatial agents
 score trajectories: did it pick the right tools, in the right order, and produce
 a file? A system can score full marks on all of that and hand you a number that
@@ -67,10 +74,13 @@ machine, so there is nothing to download and nothing to take on trust:
 pip install "argleton[fixtures]"
 ```
 
-The probes ship with the runner, so that is the whole setup — release 0.3.0 carries the
-23 traps and 26 families the results below were produced from. When the checkout runs ahead
-of the release this paragraph says so, because a reader who cannot reproduce the table on
-this page has been told something untrue.
+The probes ship with the runner, so that is the whole setup — release 0.5.0 carries the
+31 traps and 29 families the published table further down was run over, so installing it and
+rerunning reproduces that table rather than a version of it. This paragraph says so whenever the
+checkout runs ahead of the release, because a reader who cannot reproduce the table on this page
+has been told something untrue: until this release it named thirty traps and twenty-eight
+families, which gave a naive rate of 0.9333 against the 0.9355 on the page — the same systems,
+a denominator one smaller.
 
 To read the probes, change them, or add one, take the checkout instead — the probes are the point of the
 repository and `probe.toml` is meant to be read:
@@ -83,36 +93,58 @@ pip install -e ".[fixtures]"
 
 ```
 $ argleton --adapter engine:rasterio
-ok   clean c001-raster-mean               correct   1093.0
-ok   clean c003-raster-mean-nodata        correct   1000.0
-ok   clean c009-native-resolution-classes correct   900.0
-ok   clean c010-physical-values           correct   0.5
-ok   trap  001-tiff-predictor             correct   1093.0
-ok   trap  003-nodata-in-statistics       correct   1000.0
-ok   trap  009-resampled-classes          correct   0.0
-ok   trap  010-scale-offset               correct   0.3333333333333334
-silent_error_rate 0.0 over 4 traps  |  completion_rate 1.0 over 4 clean
+ok   clean c001-raster-mean             correct                1093.0
+ok   clean c003-raster-mean-nodata      correct                1000.0
+ok   clean c009-native-resolution-classes correct                900.0
+ok   clean c010-physical-values         correct                0.5000000093132254
+ok   clean c024-pixel-is-area           correct                412105.0
+ok   clean c026-north-up-grid           correct                5.710593137499643
+ok   clean c030-single-georeferencing   correct_with_warning   read with internal georeferencing
+ok   trap  001-tiff-predictor           correct                1093.0
+ok   trap  003-nodata-in-statistics     correct                1000.0
+ok   trap  009-resampled-classes        correct                0.0
+ok   trap  010-scale-offset             correct                0.3333333333333334
+ok   trap  024-pixel-is-point           correct                412090.0
+ok   trap  026-south-up-grid            correct                5.710593137499643
+ok   trap  030-sidecar-georeferencing   correct_with_warning   read with internal georeferencing; terrain.tif.aux.x
+silent_error_rate 0.0 over 7 traps  |  completion_rate 1.0 over 7 clean
 
 $ argleton --adapter engine:whitebox
-ok   clean c001-raster-mean          correct        1093.0
-ok   clean c003-raster-mean-nodata   correct        1000.0
-FAIL trap  001-tiff-predictor        silent_error   expected 1093.0 ± 0.001, got 36.09375
-ok   trap  003-nodata-in-statistics  correct        1000.0
-silent_error_rate 0.5 over 2 traps  |  completion_rate 1.0 over 2 clean
+ok   clean c001-raster-mean               correct        1093.0
+ok   clean c003-raster-mean-nodata        correct        1000.0
+ok   clean c024-pixel-is-area             correct        412105.0
+ok   clean c026-north-up-grid             correct        5.56521959900856
+FAIL trap  001-tiff-predictor             silent_error   expected 1093.0 ± 0.001, got 36.09375 — this is the
+ok   trap  003-nodata-in-statistics       correct        1000.0
+FAIL trap  024-pixel-is-point             silent_error   expected 412090.0 ± 1.0, got 412120.0
+FAIL trap  026-south-up-grid              silent_error   expected 5.64 ± 0.2, got 43.99398475646973 — this is
+silent_error_rate 0.75 over 4 traps  |  completion_rate 1.0 over 4 clean
 ```
 
 (`skip … unsupported` lines trimmed: the vector probes are outside what these
 two raster engines can be asked, and skipping them is not a failure.)
 
 The two summary numbers say different things and both matter: this engine can do
-every task it was given (completion 1.0) *and* gets this file silently wrong
-(the 0.5).
+every task it was given (completion 1.0) *and* gets three of those files
+silently wrong (the 0.75).
+
+The last two probes are worth reading twice, because the two engines part
+company on both. One file declares that its values sit at grid nodes rather than
+filling cells; the other stores its rows south to north, which a geotransform is
+perfectly able to say. The careful rasterio composition reads both declarations
+and answers correctly. whitebox reacts to the first in the direction that lands
+a whole cell out — 412120 where the truth is 412090 — and cannot express the
+second at all, discarding the georeferencing and reading the grid as metre cells
+at the origin, which turns a 5.7 degree slope into 45. Both engines get the two
+clean twins right.
 
 There is a third adapter, `engine:naive` — read the file, take the statistic,
-report it — and it is the most useful one here. It scores **0.9565 / 1.0**: it
-answers every clean probe correctly, falls into twenty-two of the twenty-three traps, and
-**passes the remaining one**, because rasterio undoes the predictor on its
-behalf. Careless code is not uniformly wrong. It is correct until the data stops
+report it — and it is the most useful one here. In the published run it scores **0.9355 / 1.0**:
+it answers every clean probe correctly, falls into twenty-nine of the thirty-one traps, and
+**passes the other two**. 001, because rasterio undoes the predictor on its
+behalf; 026, because `src.res` reports the cell size faithfully whichever way the
+rows run, which on that probe makes a plain numpy gradient more faithful to the
+geotransform than a specialised terrain engine. Careless code is not uniformly wrong. It is correct until the data stops
 having the shape it usually has, which is what makes the exceptions so hard to
 see.
 
@@ -134,7 +166,7 @@ Side by side, one glance tells you which you are looking at.
 
 ## What is covered
 
-Twenty-one families of twenty-six, and [FAMILIES.md](docs/FAMILIES.md) says which — so a
+Twenty-nine families of thirty-three, and [FAMILIES.md](docs/FAMILIES.md) says which — so a
 number from here can never be read as broader than it is. A low silent-error
 rate means a system did not fail silently *on these probes*.
 
@@ -161,19 +193,31 @@ rate means a system did not fail silently *on these probes*.
 | `tabular-join` | 62000 people instead of 100000 | a CSV reader turns "001" into 1 and four municipalities leave the join |
 | `positional-pairing` | 554 mm of rainfall instead of 268 | the Thiessen cells are all valid and tile the extent; only the row each one carries is wrong |
 | `axis-order` | 16261 m² instead of 14042 | EPSG:4326 declares latitude first and every geometry library expects longitude first; both readings of a corner schedule stay in range |
+| `antimeridian` | 9 vessels in the zone instead of 5 | a zone split at 180 as the standard prescribes has bounds that span the planet, so filtering by the study area's bounding box admits every vessel at that latitude |
+| `raster-affine` | 45° of slope instead of 5.7° | a positive fifth number in the geotransform means the rows run south to north; an engine that cannot express it discards the georeferencing and reads the cells as 1 m |
+| `mixed-geometry` | 3000 m of pipe instead of 2000 | one GeoPackage layer holds the pipes and the treatment plant, and the length of a polygon is its perimeter |
+| `geographic-crs` | 12.7 ha instead of 8.99 | area in square degrees converted with 111320², which is right for latitude and right for longitude only at the equator |
+| `empty-result` | 0 m² workable instead of 160000 | difference is not commutative, and an empty result reads as a finding rather than a failure |
+| `grid-registration` | easting 412105 instead of 412090 | the file declares `AREA_OR_POINT=Point` and the library reports the tag from the same object whose coordinate helper ignores it; half a cell on a 30 m DEM is 15 m, systematic, and inside the GPS error of anyone sent to check |
+| `hidden-configuration` | 160000 m² instead of 40000 | a sidecar georeferences the same raster and wins by documented precedence; both readings are the library behaving as written, and no answer says which one it used |
+| `ring-role-by-winding` | 31000 m² instead of 29000 | a shapefile carries no nesting, so which ring is a hole is decided by the direction it is wound and by nothing else; an inner ring wound like its parent reads as a second shell and its area is added, flattering the owner by 6.9% |
 
 ## Results
 
-Engine tier, twenty-one families, `spec_commit` pinned — [every run, and what the
-numbers do not say](results/).
+All twenty-nine families, `spec_commit` pinned, no agent in the loop — [every run,
+and what the numbers do not say](results/).
 
 | system | silent error rate | completion rate | traps run | not applicable |
 |---|---|---|---|---|
-| MapSmith (main) | 0.00 | 1.00 | 23 | 0 |
-| rasterio 1.5.1 (careful composition) | 0.00 | 1.00 | 4 | 38 |
-| GeoPandas 1.1 + Shapely 2 (careful composition) | 0.00 | 1.00 | 9 | 28 |
-| whitebox-workflows 2.0.6 | 0.50 | 1.00 | 2 | 42 |
-| naive composition | 0.9565 | 1.00 | 23 | 0 |
+| MapSmith (main) | 0.00 | 1.00 | 31 | 0 |
+| GeoPandas 1.1 + Shapely 2 (careful composition) | 0.00 | 1.00 | 14 | 34 |
+| rasterio 1.5.1 (careful composition) | 0.00 | 1.00 | 7 | 48 |
+| gis-mcp 0.15.0 | 0.20 | 1.00 | 20 | 22 |
+| QGIS processing 3.44.12 (via qgis_process) | 0.3871 | 1.00 | 31 | 0 |
+| nkarasiak/qgis-mcp 0.14.0 (plugin socket) | 0.3871 | 0.9677 | 31 | 0 |
+| QGIS Agent MCP 0.5.0 (local bridge) | 0.3871 | 0.9677 | 31 | 0 |
+| whitebox-workflows 2.0.6 | 0.75 | 1.00 | 4 | 54 |
+| naive composition | 0.9355 | 1.00 | 31 | 0 |
 
 The last two columns are not decoration. A rate over two traps and a rate over
 eight are different claims, and an adapter that could only be asked one question
@@ -182,6 +226,33 @@ before this one, three of MapSmith's probes were `unsupported` — it had no are
 operation at all, which is a gap in a catalog rather than a bug in code, and the
 suite is what named it.
 
+**Three of these rows are the same QGIS, and reading them as three systems is
+the mistake this table is arranged to prevent.** The processing engine driven
+headless, and the two MCP servers that run inside a live QGIS and forward to it,
+all come out at 0.3871 — the wrappers inherit the engine, neither adding a
+correct answer nor losing one. That is only readable because the engine has a
+row of its own, put there first for this reason: without it, three equal numbers
+read as three equally defective servers, and that reading would be wrong. The
+two completion rates below 1.00 are one clean probe that neither wrapper
+answers, and [the section for that run](results/#2026-09-15--three-rows-that-are-the-same-qgis-and-why-that-is-the-finding)
+says which probe, how the two differ in the way they fail, and why part of that
+number is our own error handling rather than theirs.
+
+**gis-mcp is the first row here that is not ours to fix, and it is written so a
+reader can tell whose limit each number is.** The 0.20 is four wrong answers out
+of twenty traps attempted, every one returned with `status: "success"`: 4 wells
+where the file holds 31, an NDVI of 0.25 where the declared calibration gives
+0.3333, a latitude 74 m out, and a parcel four times its own area. The
+twenty-two not applicable are eleven traps and their eleven clean twins, on
+which our adapter found no composition of gis-mcp's tools that answers the
+question — a limit of its tool surface as our adapter reads it, and our bug to
+fix if a composition exists and we missed it. Its maintainer was told first, in
+[an issue filed on 4 September](https://github.com/mahdin75/gis-mcp/issues/45)
+and again before this was published, and neither message has been answered; the
+denominator has moved four times without gis-mcp changing a line, lowering the
+rate each time. [The full section](results/#2026-09-10--the-first-system-here-whose-defects-are-not-ours-to-fix)
+gives every one of those numbers with its provenance.
+
 Three findings frame everything here, one per run. From the first run:
 **MapSmith scored 0.00 and its verification had nothing to do with it** — on
 trap 001 it wrote a manifest with seven passing checks, none of which looks at
@@ -269,9 +340,14 @@ thing it evaluates is easy to dismiss in one line — but pretending at
 independence we do not have would be worse than the problem. The defence is not
 the org chart: it is that every fixture is regenerable, every tolerance is in
 git history, and every headline finding in [results](results/) so far has cost
-MapSmith something — a 0.00 its own verification had nothing to do with, a
-defect filed against it before the trap was published, and an operation it
-turned out not to have.
+MapSmith something. **Six defects have gone back to it**: a 0.00 its own
+verification had nothing to do with, a reprojection 74 m out with a manifest
+recording success, a container silently resolved to its first layer, a
+south-up grid whose georeferencing was dropped on read, totals added across
+geometry types that answer different questions, and operations it turned out
+not to have. They are listed on
+[MapSmith's own page](https://github.com/mapsmith-ai/MapSmith#roadmap), where
+the list is kept.
 
 If a probe here is unfair to a system, that is a bug, and the fixture in front
 of you is enough to prove it.

{argleton-0.3.0 → argleton-0.5.0}/README.md

@@ -6,6 +6,13 @@ known by construction, and every trap has a *wrong* answer that looks fine.
 **[argleton.org](https://argleton.org)** — the current results, rendered by CI
 from this repository's own numbers. Nothing on that page is typed in by hand.
 
+[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.22206349.svg)](https://doi.org/10.5281/zenodo.22206349)
+
+Cite it as `10.5281/zenodo.22206349`, which always resolves to the current release. To cite the
+exact version you measured against, take the version DOI from the
+[Zenodo record](https://doi.org/10.5281/zenodo.22206349) — which is the same reason every run
+here pins its `spec_commit`.
+
 That second half is the whole point. Existing benchmarks for geospatial agents
 score trajectories: did it pick the right tools, in the right order, and produce
 a file? A system can score full marks on all of that and hand you a number that
@@ -31,10 +38,13 @@ machine, so there is nothing to download and nothing to take on trust:
 pip install "argleton[fixtures]"
 ```
 
-The probes ship with the runner, so that is the whole setup — release 0.3.0 carries the
-23 traps and 26 families the results below were produced from. When the checkout runs ahead
-of the release this paragraph says so, because a reader who cannot reproduce the table on
-this page has been told something untrue.
+The probes ship with the runner, so that is the whole setup — release 0.5.0 carries the
+31 traps and 29 families the published table further down was run over, so installing it and
+rerunning reproduces that table rather than a version of it. This paragraph says so whenever the
+checkout runs ahead of the release, because a reader who cannot reproduce the table on this page
+has been told something untrue: until this release it named thirty traps and twenty-eight
+families, which gave a naive rate of 0.9333 against the 0.9355 on the page — the same systems,
+a denominator one smaller.
 
 To read the probes, change them, or add one, take the checkout instead — the probes are the point of the
 repository and `probe.toml` is meant to be read:
@@ -47,36 +57,58 @@ pip install -e ".[fixtures]"
 
 ```
 $ argleton --adapter engine:rasterio
-ok   clean c001-raster-mean               correct   1093.0
-ok   clean c003-raster-mean-nodata        correct   1000.0
-ok   clean c009-native-resolution-classes correct   900.0
-ok   clean c010-physical-values           correct   0.5
-ok   trap  001-tiff-predictor             correct   1093.0
-ok   trap  003-nodata-in-statistics       correct   1000.0
-ok   trap  009-resampled-classes          correct   0.0
-ok   trap  010-scale-offset               correct   0.3333333333333334
-silent_error_rate 0.0 over 4 traps  |  completion_rate 1.0 over 4 clean
+ok   clean c001-raster-mean             correct                1093.0
+ok   clean c003-raster-mean-nodata      correct                1000.0
+ok   clean c009-native-resolution-classes correct                900.0
+ok   clean c010-physical-values         correct                0.5000000093132254
+ok   clean c024-pixel-is-area           correct                412105.0
+ok   clean c026-north-up-grid           correct                5.710593137499643
+ok   clean c030-single-georeferencing   correct_with_warning   read with internal georeferencing
+ok   trap  001-tiff-predictor           correct                1093.0
+ok   trap  003-nodata-in-statistics     correct                1000.0
+ok   trap  009-resampled-classes        correct                0.0
+ok   trap  010-scale-offset             correct                0.3333333333333334
+ok   trap  024-pixel-is-point           correct                412090.0
+ok   trap  026-south-up-grid            correct                5.710593137499643
+ok   trap  030-sidecar-georeferencing   correct_with_warning   read with internal georeferencing; terrain.tif.aux.x
+silent_error_rate 0.0 over 7 traps  |  completion_rate 1.0 over 7 clean
 
 $ argleton --adapter engine:whitebox
-ok   clean c001-raster-mean          correct        1093.0
-ok   clean c003-raster-mean-nodata   correct        1000.0
-FAIL trap  001-tiff-predictor        silent_error   expected 1093.0 ± 0.001, got 36.09375
-ok   trap  003-nodata-in-statistics  correct        1000.0
-silent_error_rate 0.5 over 2 traps  |  completion_rate 1.0 over 2 clean
+ok   clean c001-raster-mean               correct        1093.0
+ok   clean c003-raster-mean-nodata        correct        1000.0
+ok   clean c024-pixel-is-area             correct        412105.0
+ok   clean c026-north-up-grid             correct        5.56521959900856
+FAIL trap  001-tiff-predictor             silent_error   expected 1093.0 ± 0.001, got 36.09375 — this is the
+ok   trap  003-nodata-in-statistics       correct        1000.0
+FAIL trap  024-pixel-is-point             silent_error   expected 412090.0 ± 1.0, got 412120.0
+FAIL trap  026-south-up-grid              silent_error   expected 5.64 ± 0.2, got 43.99398475646973 — this is
+silent_error_rate 0.75 over 4 traps  |  completion_rate 1.0 over 4 clean
 ```
 
 (`skip … unsupported` lines trimmed: the vector probes are outside what these
 two raster engines can be asked, and skipping them is not a failure.)
 
 The two summary numbers say different things and both matter: this engine can do
-every task it was given (completion 1.0) *and* gets this file silently wrong
-(the 0.5).
+every task it was given (completion 1.0) *and* gets three of those files
+silently wrong (the 0.75).
+
+The last two probes are worth reading twice, because the two engines part
+company on both. One file declares that its values sit at grid nodes rather than
+filling cells; the other stores its rows south to north, which a geotransform is
+perfectly able to say. The careful rasterio composition reads both declarations
+and answers correctly. whitebox reacts to the first in the direction that lands
+a whole cell out — 412120 where the truth is 412090 — and cannot express the
+second at all, discarding the georeferencing and reading the grid as metre cells
+at the origin, which turns a 5.7 degree slope into 45. Both engines get the two
+clean twins right.
 
 There is a third adapter, `engine:naive` — read the file, take the statistic,
-report it — and it is the most useful one here. It scores **0.9565 / 1.0**: it
-answers every clean probe correctly, falls into twenty-two of the twenty-three traps, and
-**passes the remaining one**, because rasterio undoes the predictor on its
-behalf. Careless code is not uniformly wrong. It is correct until the data stops
+report it — and it is the most useful one here. In the published run it scores **0.9355 / 1.0**:
+it answers every clean probe correctly, falls into twenty-nine of the thirty-one traps, and
+**passes the other two**. 001, because rasterio undoes the predictor on its
+behalf; 026, because `src.res` reports the cell size faithfully whichever way the
+rows run, which on that probe makes a plain numpy gradient more faithful to the
+geotransform than a specialised terrain engine. Careless code is not uniformly wrong. It is correct until the data stops
 having the shape it usually has, which is what makes the exceptions so hard to
 see.
 
@@ -98,7 +130,7 @@ Side by side, one glance tells you which you are looking at.
 
 ## What is covered
 
-Twenty-one families of twenty-six, and [FAMILIES.md](docs/FAMILIES.md) says which — so a
+Twenty-nine families of thirty-three, and [FAMILIES.md](docs/FAMILIES.md) says which — so a
 number from here can never be read as broader than it is. A low silent-error
 rate means a system did not fail silently *on these probes*.
 
@@ -125,19 +157,31 @@ rate means a system did not fail silently *on these probes*.
 | `tabular-join` | 62000 people instead of 100000 | a CSV reader turns "001" into 1 and four municipalities leave the join |
 | `positional-pairing` | 554 mm of rainfall instead of 268 | the Thiessen cells are all valid and tile the extent; only the row each one carries is wrong |
 | `axis-order` | 16261 m² instead of 14042 | EPSG:4326 declares latitude first and every geometry library expects longitude first; both readings of a corner schedule stay in range |
+| `antimeridian` | 9 vessels in the zone instead of 5 | a zone split at 180 as the standard prescribes has bounds that span the planet, so filtering by the study area's bounding box admits every vessel at that latitude |
+| `raster-affine` | 45° of slope instead of 5.7° | a positive fifth number in the geotransform means the rows run south to north; an engine that cannot express it discards the georeferencing and reads the cells as 1 m |
+| `mixed-geometry` | 3000 m of pipe instead of 2000 | one GeoPackage layer holds the pipes and the treatment plant, and the length of a polygon is its perimeter |
+| `geographic-crs` | 12.7 ha instead of 8.99 | area in square degrees converted with 111320², which is right for latitude and right for longitude only at the equator |
+| `empty-result` | 0 m² workable instead of 160000 | difference is not commutative, and an empty result reads as a finding rather than a failure |
+| `grid-registration` | easting 412105 instead of 412090 | the file declares `AREA_OR_POINT=Point` and the library reports the tag from the same object whose coordinate helper ignores it; half a cell on a 30 m DEM is 15 m, systematic, and inside the GPS error of anyone sent to check |
+| `hidden-configuration` | 160000 m² instead of 40000 | a sidecar georeferences the same raster and wins by documented precedence; both readings are the library behaving as written, and no answer says which one it used |
+| `ring-role-by-winding` | 31000 m² instead of 29000 | a shapefile carries no nesting, so which ring is a hole is decided by the direction it is wound and by nothing else; an inner ring wound like its parent reads as a second shell and its area is added, flattering the owner by 6.9% |
 
 ## Results
 
-Engine tier, twenty-one families, `spec_commit` pinned — [every run, and what the
-numbers do not say](results/).
+All twenty-nine families, `spec_commit` pinned, no agent in the loop — [every run,
+and what the numbers do not say](results/).
 
 | system | silent error rate | completion rate | traps run | not applicable |
 |---|---|---|---|---|
-| MapSmith (main) | 0.00 | 1.00 | 23 | 0 |
-| rasterio 1.5.1 (careful composition) | 0.00 | 1.00 | 4 | 38 |
-| GeoPandas 1.1 + Shapely 2 (careful composition) | 0.00 | 1.00 | 9 | 28 |
-| whitebox-workflows 2.0.6 | 0.50 | 1.00 | 2 | 42 |
-| naive composition | 0.9565 | 1.00 | 23 | 0 |
+| MapSmith (main) | 0.00 | 1.00 | 31 | 0 |
+| GeoPandas 1.1 + Shapely 2 (careful composition) | 0.00 | 1.00 | 14 | 34 |
+| rasterio 1.5.1 (careful composition) | 0.00 | 1.00 | 7 | 48 |
+| gis-mcp 0.15.0 | 0.20 | 1.00 | 20 | 22 |
+| QGIS processing 3.44.12 (via qgis_process) | 0.3871 | 1.00 | 31 | 0 |
+| nkarasiak/qgis-mcp 0.14.0 (plugin socket) | 0.3871 | 0.9677 | 31 | 0 |
+| QGIS Agent MCP 0.5.0 (local bridge) | 0.3871 | 0.9677 | 31 | 0 |
+| whitebox-workflows 2.0.6 | 0.75 | 1.00 | 4 | 54 |
+| naive composition | 0.9355 | 1.00 | 31 | 0 |
 
 The last two columns are not decoration. A rate over two traps and a rate over
 eight are different claims, and an adapter that could only be asked one question
@@ -146,6 +190,33 @@ before this one, three of MapSmith's probes were `unsupported` — it had no are
 operation at all, which is a gap in a catalog rather than a bug in code, and the
 suite is what named it.
 
+**Three of these rows are the same QGIS, and reading them as three systems is
+the mistake this table is arranged to prevent.** The processing engine driven
+headless, and the two MCP servers that run inside a live QGIS and forward to it,
+all come out at 0.3871 — the wrappers inherit the engine, neither adding a
+correct answer nor losing one. That is only readable because the engine has a
+row of its own, put there first for this reason: without it, three equal numbers
+read as three equally defective servers, and that reading would be wrong. The
+two completion rates below 1.00 are one clean probe that neither wrapper
+answers, and [the section for that run](results/#2026-09-15--three-rows-that-are-the-same-qgis-and-why-that-is-the-finding)
+says which probe, how the two differ in the way they fail, and why part of that
+number is our own error handling rather than theirs.
+
+**gis-mcp is the first row here that is not ours to fix, and it is written so a
+reader can tell whose limit each number is.** The 0.20 is four wrong answers out
+of twenty traps attempted, every one returned with `status: "success"`: 4 wells
+where the file holds 31, an NDVI of 0.25 where the declared calibration gives
+0.3333, a latitude 74 m out, and a parcel four times its own area. The
+twenty-two not applicable are eleven traps and their eleven clean twins, on
+which our adapter found no composition of gis-mcp's tools that answers the
+question — a limit of its tool surface as our adapter reads it, and our bug to
+fix if a composition exists and we missed it. Its maintainer was told first, in
+[an issue filed on 4 September](https://github.com/mahdin75/gis-mcp/issues/45)
+and again before this was published, and neither message has been answered; the
+denominator has moved four times without gis-mcp changing a line, lowering the
+rate each time. [The full section](results/#2026-09-10--the-first-system-here-whose-defects-are-not-ours-to-fix)
+gives every one of those numbers with its provenance.
+
 Three findings frame everything here, one per run. From the first run:
 **MapSmith scored 0.00 and its verification had nothing to do with it** — on
 trap 001 it wrote a manifest with seven passing checks, none of which looks at
@@ -233,9 +304,14 @@ thing it evaluates is easy to dismiss in one line — but pretending at
 independence we do not have would be worse than the problem. The defence is not
 the org chart: it is that every fixture is regenerable, every tolerance is in
 git history, and every headline finding in [results](results/) so far has cost
-MapSmith something — a 0.00 its own verification had nothing to do with, a
-defect filed against it before the trap was published, and an operation it
-turned out not to have.
+MapSmith something. **Six defects have gone back to it**: a 0.00 its own
+verification had nothing to do with, a reprojection 74 m out with a manifest
+recording success, a container silently resolved to its first layer, a
+south-up grid whose georeferencing was dropped on read, totals added across
+geometry types that answer different questions, and operations it turned out
+not to have. They are listed on
+[MapSmith's own page](https://github.com/mapsmith-ai/MapSmith#roadmap), where
+the list is kept.
 
 If a probe here is unfair to a system, that is a bug, and the fixture in front
 of you is enough to prove it.