@mailwoman/corpus 10.3.0 → 10.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +17 -17
- package/data/PROVENANCE.md +287 -273
- package/data/address-source-register.json +545 -23
- package/data/license-decisions.json +120 -0
- package/data/licenses/PROVENANCE.md +5 -5
- package/data/licenses/cdla-permissive-2.0.md +6 -6
- package/data/licenses/geonames-publication.md +5 -5
- package/data/licenses/licence-ouverte-2.0.md +3 -3
- package/data/licenses/national-address-database.md +12 -12
- package/data/licenses/nlod-2.0.md +3 -3
- package/data/licenses/sirene-publication.md +13 -6
- package/data/licenses/whosonfirst-licenses.md +4 -4
- package/data/source-resolutions.json +46 -0
- package/data/training-manifests/v0.7.2-address-systems.effective.json +1169 -0
- package/data/training-manifests/v0.7.2-address-systems.json +550 -0
- package/lib/adapters/cordis/adapter.ts +9 -9
- package/lib/adapters/gleif/adapter.ts +7 -8
- package/lib/adapters/po-box/adapter.ts +1 -1
- package/lib/adapters/source-id.ts +1 -1
- package/lib/adapters.ts +4 -2
- package/lib/au/adapters/gnaf/adapter.ts +1 -1
- package/lib/be/adapters/brussels/adapter.ts +3 -3
- package/lib/be/adapters/vlaanderen/adapter.ts +1 -1
- package/lib/be/adapters/wallonie/adapter.ts +1 -1
- package/lib/build/align.ts +1 -1
- package/lib/build.ts +1 -1
- package/lib/cz/adapters/cuzk/adapter.ts +2 -2
- package/lib/cz/tools/fetch/cuzk.ts +6 -6
- package/lib/de/recipes/locale.ts +1 -1
- package/lib/dk/adapters/inspire/adapter.ts +1 -1
- package/lib/ee/tools/fetch/ads.ts +2 -2
- package/lib/es/adapters/bizkaia/adapter.ts +1 -1
- package/lib/es/adapters/navarra/adapter.ts +2 -2
- package/lib/es/cadastre.ts +2 -2
- package/lib/es/tools/fetch/bizkaia.ts +8 -8
- package/lib/es/tools/fetch/catastro.ts +6 -6
- package/lib/es/tools/fetch/navarra.ts +4 -4
- package/lib/fingerprints.ts +3 -3
- package/lib/fr/adapters/finess/adapter.ts +6 -5
- package/lib/fr/adapters/finess/voie-type.ts +1 -1
- package/lib/fr/tools/fetch/annuaire-education.ts +3 -3
- package/lib/gb/adapters/os-open/adapter.ts +453 -0
- package/lib/gb/constituent-country.ts +52 -0
- package/lib/gb/os-open/nearest-link.ts +169 -0
- package/lib/gb/os-open/open-names.ts +180 -0
- package/lib/gb/os-open/open-roads.ts +145 -0
- package/lib/gb/tools/fetch/os-open.ts +251 -0
- package/lib/inspire/address.ts +3 -3
- package/lib/international/recipes/locale.ts +41 -7
- package/lib/it/adapters/anac/adapter.ts +4 -3
- package/lib/jp/adapters/wof-admin/adapter.ts +2 -2
- package/lib/kr/adapters/juso/label-rows.ts +1 -1
- package/lib/kr/adapters/localdata/key-index.ts +2 -1
- package/lib/lu/adapters/bd-adresses/adapter.ts +1 -1
- package/lib/nl/adapters/kadaster/adapter.ts +2 -2
- package/lib/nl/tools/fetch/kadaster.ts +2 -2
- package/lib/parquet/shuffle.ts +2 -2
- package/lib/parquet/span-sql.ts +8 -6
- package/lib/parquet/streams.ts +1 -1
- package/lib/parquet/writers.ts +2 -2
- package/lib/pl/tools/fetch/emuia.ts +2 -2
- package/lib/recipes/scaffold.ts +2 -2
- package/lib/recipes/sources.ts +3 -3
- package/lib/recipes/sub/venue/render.ts +1 -1
- package/lib/recipes/sub/venue/sources.ts +4 -3
- package/lib/recipes/sub/venue.ts +4 -3
- package/lib/recipes/venue.ts +11 -11
- package/lib/registers.ts +7 -0
- package/lib/ro/adapters/retea-scolara/adapter.ts +2 -2
- package/lib/runner.ts +3 -3
- package/lib/sk/tools/fetch/inspire.ts +3 -3
- package/lib/source-register/effective-manifest.ts +14 -8
- package/lib/source-register/training-manifest.ts +30 -2
- package/lib/source-register/types.ts +2 -2
- package/lib/source-register.ts +1 -0
- package/lib/subvenue/harvest.ts +6 -5
- package/lib/subvenue/promotions.ts +3 -3
- package/lib/subvenue/surfaces.ts +1 -1
- package/lib/subvenue/table.ts +6 -6
- package/lib/surfaces/locale.ts +1 -1
- package/lib/synthesizers/anchor-absorption.ts +1 -1
- package/lib/synthesizers/boundary-stress.ts +3 -2
- package/lib/synthesizers/no-street.ts +1 -1
- package/lib/synthesizers/po-box.ts +1 -1
- package/lib/synthesizers/tail.ts +2 -2
- package/lib/test-kit/corpus-recipe.ts +1 -1
- package/lib/tools/align-canonical.ts +3 -2
- package/lib/tools/audit.ts +1 -1
- package/lib/tools/corpus-stats.ts +1 -1
- package/lib/tools/fetch/download/network.ts +1 -1
- package/lib/tools/fetch/geonames/dump.ts +1 -1
- package/lib/tools/fetch/gleif.ts +2 -2
- package/lib/tools/fetch/header.ts +1 -1
- package/lib/tools/fetch/inspire-addresses.ts +1 -1
- package/lib/tools/fetch/vlaanderen-ad.ts +1 -1
- package/lib/tools/fetch/wfs-harvest.ts +6 -6
- package/lib/tools/fetch/zip-archive.ts +2 -2
- package/lib/tools/fetch.ts +2 -0
- package/lib/tools/golden/expand.ts +1 -1
- package/lib/tools/golden/promote.ts +3 -3
- package/lib/tools/golden/relabel-street/directory.ts +2 -1
- package/lib/tools/lint/recipe-output/vocab.ts +2 -2
- package/lib/tools/lint/recipe-output.ts +1 -1
- package/lib/tools/migrate/source-names.ts +1 -1
- package/lib/tools/overlay/manifest.ts +62 -10
- package/lib/tools/overlay/split-slice.ts +2 -10
- package/lib/tools/postcode-triples.ts +2 -2
- package/lib/tools/sub/venue/head-nouns.ts +4 -3
- package/lib/types.ts +3 -3
- package/lib/us/adapters/state/tx-notaries/adapter.ts +1 -1
- package/lib/us/adapters/usgov/irs-bmf/adapter.ts +2 -2
- package/lib/us/adapters/usgov/nad/adapter.ts +1 -1
- package/lib/us/adapters/usgov/nppes/adapter.ts +2 -2
- package/lib/us/synthesizers/intersection.ts +4 -4
- package/lib/us/synthesizers/street.ts +3 -3
- package/lib/us/tools/fetch/nad.ts +2 -2
- package/lib/utils/split.ts +59 -6
- package/lib/utils/wof-json.ts +1 -1
- package/out/adapters/cordis/adapter.d.ts +7 -7
- package/out/adapters/cordis/adapter.js +9 -9
- package/out/adapters/cordis/adapter.js.map +1 -1
- package/out/adapters/gleif/adapter.d.ts +7 -8
- package/out/adapters/gleif/adapter.d.ts.map +1 -1
- package/out/adapters/gleif/adapter.js +7 -8
- package/out/adapters/gleif/adapter.js.map +1 -1
- package/out/adapters/po-box/adapter.d.ts +1 -1
- package/out/adapters/source-id.d.ts +1 -1
- package/out/adapters/source-id.js +1 -1
- package/out/adapters.d.ts +2 -2
- package/out/adapters.d.ts.map +1 -1
- package/out/adapters.js +4 -2
- package/out/adapters.js.map +1 -1
- package/out/au/adapters/gnaf/adapter.js +1 -1
- package/out/be/adapters/brussels/adapter.d.ts +2 -2
- package/out/be/adapters/brussels/adapter.js +3 -3
- package/out/be/adapters/vlaanderen/adapter.d.ts +1 -1
- package/out/be/adapters/wallonie/adapter.d.ts +1 -1
- package/out/be/adapters/wallonie/adapter.js +1 -1
- package/out/build/align.d.ts +1 -1
- package/out/build.js +1 -1
- package/out/build.js.map +1 -1
- package/out/cz/adapters/cuzk/adapter.d.ts +2 -2
- package/out/cz/adapters/cuzk/adapter.js +2 -2
- package/out/cz/tools/fetch/cuzk.d.ts +2 -2
- package/out/cz/tools/fetch/cuzk.js +4 -4
- package/out/cz/tools/fetch/cuzk.js.map +1 -1
- package/out/de/recipes/locale.d.ts +1 -1
- package/out/de/recipes/locale.js +1 -1
- package/out/dk/adapters/inspire/adapter.js +1 -1
- package/out/ee/tools/fetch/ads.d.ts +2 -2
- package/out/ee/tools/fetch/ads.js +1 -1
- package/out/es/adapters/bizkaia/adapter.js +1 -1
- package/out/es/adapters/navarra/adapter.d.ts +2 -2
- package/out/es/adapters/navarra/adapter.js +2 -2
- package/out/es/cadastre.d.ts +2 -2
- package/out/es/cadastre.js +2 -2
- package/out/es/tools/fetch/bizkaia.d.ts +5 -5
- package/out/es/tools/fetch/bizkaia.js +6 -6
- package/out/es/tools/fetch/bizkaia.js.map +1 -1
- package/out/es/tools/fetch/catastro.d.ts +4 -4
- package/out/es/tools/fetch/catastro.js +2 -2
- package/out/es/tools/fetch/navarra.d.ts +2 -2
- package/out/es/tools/fetch/navarra.js +2 -2
- package/out/fingerprints.d.ts +3 -3
- package/out/fingerprints.js +3 -3
- package/out/fr/adapters/finess/adapter.d.ts +6 -5
- package/out/fr/adapters/finess/adapter.d.ts.map +1 -1
- package/out/fr/adapters/finess/adapter.js +6 -5
- package/out/fr/adapters/finess/adapter.js.map +1 -1
- package/out/fr/adapters/finess/voie-type.d.ts +1 -1
- package/out/fr/adapters/finess/voie-type.js +1 -1
- package/out/fr/tools/fetch/annuaire-education.d.ts +3 -3
- package/out/fr/tools/fetch/annuaire-education.js +3 -3
- package/out/gb/adapters/os-open/adapter.d.ts +165 -0
- package/out/gb/adapters/os-open/adapter.d.ts.map +1 -0
- package/out/gb/adapters/os-open/adapter.js +298 -0
- package/out/gb/adapters/os-open/adapter.js.map +1 -0
- package/out/gb/constituent-country.d.ts +36 -0
- package/out/gb/constituent-country.d.ts.map +1 -0
- package/out/gb/constituent-country.js +30 -0
- package/out/gb/constituent-country.js.map +1 -0
- package/out/gb/os-open/nearest-link.d.ts +73 -0
- package/out/gb/os-open/nearest-link.d.ts.map +1 -0
- package/out/gb/os-open/nearest-link.js +133 -0
- package/out/gb/os-open/nearest-link.js.map +1 -0
- package/out/gb/os-open/open-names.d.ts +92 -0
- package/out/gb/os-open/open-names.d.ts.map +1 -0
- package/out/gb/os-open/open-names.js +107 -0
- package/out/gb/os-open/open-names.js.map +1 -0
- package/out/gb/os-open/open-roads.d.ts +80 -0
- package/out/gb/os-open/open-roads.d.ts.map +1 -0
- package/out/gb/os-open/open-roads.js +79 -0
- package/out/gb/os-open/open-roads.js.map +1 -0
- package/out/gb/tools/fetch/os-open.d.ts +91 -0
- package/out/gb/tools/fetch/os-open.d.ts.map +1 -0
- package/out/gb/tools/fetch/os-open.js +166 -0
- package/out/gb/tools/fetch/os-open.js.map +1 -0
- package/out/inspire/address.d.ts +3 -3
- package/out/inspire/address.js +3 -3
- package/out/international/recipes/locale.d.ts +6 -0
- package/out/international/recipes/locale.d.ts.map +1 -1
- package/out/international/recipes/locale.js +29 -7
- package/out/international/recipes/locale.js.map +1 -1
- package/out/it/adapters/anac/adapter.d.ts +1 -1
- package/out/it/adapters/anac/adapter.d.ts.map +1 -1
- package/out/it/adapters/anac/adapter.js +4 -3
- package/out/it/adapters/anac/adapter.js.map +1 -1
- package/out/jp/adapters/wof-admin/adapter.d.ts +2 -2
- package/out/jp/adapters/wof-admin/adapter.js +2 -2
- package/out/kr/adapters/juso/label-rows.d.ts +1 -1
- package/out/kr/adapters/localdata/key-index.d.ts +2 -1
- package/out/kr/adapters/localdata/key-index.d.ts.map +1 -1
- package/out/kr/adapters/localdata/key-index.js +2 -1
- package/out/kr/adapters/localdata/key-index.js.map +1 -1
- package/out/lu/adapters/bd-adresses/adapter.d.ts +1 -1
- package/out/lu/adapters/bd-adresses/adapter.js +1 -1
- package/out/nl/adapters/kadaster/adapter.js +2 -2
- package/out/nl/tools/fetch/kadaster.d.ts +2 -2
- package/out/nl/tools/fetch/kadaster.js +2 -2
- package/out/parquet/shuffle.d.ts +2 -2
- package/out/parquet/shuffle.js +2 -2
- package/out/parquet/span-sql.d.ts +8 -6
- package/out/parquet/span-sql.d.ts.map +1 -1
- package/out/parquet/span-sql.js +8 -6
- package/out/parquet/span-sql.js.map +1 -1
- package/out/parquet/streams.d.ts +1 -1
- package/out/parquet/writers.d.ts +2 -2
- package/out/parquet/writers.js +1 -1
- package/out/pl/tools/fetch/emuia.d.ts +2 -2
- package/out/recipes/scaffold.d.ts +2 -2
- package/out/recipes/sources.d.ts +2 -2
- package/out/recipes/sources.js +3 -3
- package/out/recipes/sources.js.map +1 -1
- package/out/recipes/sub/venue/render.d.ts +1 -1
- package/out/recipes/sub/venue/render.js +1 -1
- package/out/recipes/sub/venue/sources.d.ts +4 -3
- package/out/recipes/sub/venue/sources.d.ts.map +1 -1
- package/out/recipes/sub/venue/sources.js +4 -3
- package/out/recipes/sub/venue/sources.js.map +1 -1
- package/out/recipes/sub/venue.d.ts +3 -2
- package/out/recipes/sub/venue.d.ts.map +1 -1
- package/out/recipes/sub/venue.js +2 -2
- package/out/recipes/sub/venue.js.map +1 -1
- package/out/recipes/venue.d.ts +7 -6
- package/out/recipes/venue.d.ts.map +1 -1
- package/out/recipes/venue.js +11 -10
- package/out/recipes/venue.js.map +1 -1
- package/out/registers.d.ts +7 -0
- package/out/registers.d.ts.map +1 -1
- package/out/registers.js +7 -0
- package/out/registers.js.map +1 -1
- package/out/ro/adapters/retea-scolara/adapter.d.ts +2 -2
- package/out/ro/adapters/retea-scolara/adapter.js +2 -2
- package/out/runner.d.ts +3 -3
- package/out/runner.js +2 -2
- package/out/sk/tools/fetch/inspire.d.ts +3 -3
- package/out/sk/tools/fetch/inspire.js +1 -1
- package/out/source-register/effective-manifest.d.ts +11 -5
- package/out/source-register/effective-manifest.d.ts.map +1 -1
- package/out/source-register/effective-manifest.js +9 -4
- package/out/source-register/effective-manifest.js.map +1 -1
- package/out/source-register/training-manifest.d.ts +27 -0
- package/out/source-register/training-manifest.d.ts.map +1 -1
- package/out/source-register/training-manifest.js +13 -1
- package/out/source-register/training-manifest.js.map +1 -1
- package/out/source-register/types.d.ts +2 -2
- package/out/source-register.d.ts +1 -1
- package/out/source-register.d.ts.map +1 -1
- package/out/source-register.js +1 -1
- package/out/source-register.js.map +1 -1
- package/out/subvenue/harvest.d.ts +4 -3
- package/out/subvenue/harvest.d.ts.map +1 -1
- package/out/subvenue/harvest.js +6 -5
- package/out/subvenue/harvest.js.map +1 -1
- package/out/subvenue/promotions.d.ts +2 -2
- package/out/subvenue/promotions.js +3 -3
- package/out/subvenue/promotions.js.map +1 -1
- package/out/subvenue/surfaces.d.ts +1 -1
- package/out/subvenue/table.d.ts +6 -6
- package/out/subvenue/table.js +1 -1
- package/out/surfaces/locale.d.ts +1 -1
- package/out/surfaces/locale.js +1 -1
- package/out/synthesizers/boundary-stress.d.ts.map +1 -1
- package/out/synthesizers/boundary-stress.js +2 -1
- package/out/synthesizers/boundary-stress.js.map +1 -1
- package/out/synthesizers/tail.d.ts +2 -2
- package/out/synthesizers/tail.js +2 -2
- package/out/test-kit/corpus-recipe.d.ts +1 -1
- package/out/test-kit/corpus-recipe.js +1 -1
- package/out/tools/align-canonical.d.ts.map +1 -1
- package/out/tools/align-canonical.js +3 -2
- package/out/tools/align-canonical.js.map +1 -1
- package/out/tools/audit.js +1 -1
- package/out/tools/corpus-stats.d.ts +1 -1
- package/out/tools/fetch/download/network.d.ts +1 -1
- package/out/tools/fetch/download/network.js +1 -1
- package/out/tools/fetch/geonames/dump.d.ts +1 -1
- package/out/tools/fetch/geonames/dump.js +1 -1
- package/out/tools/fetch/gleif.d.ts +2 -2
- package/out/tools/fetch/gleif.js +1 -1
- package/out/tools/fetch/header.d.ts +1 -1
- package/out/tools/fetch/header.js +1 -1
- package/out/tools/fetch/inspire-addresses.d.ts +1 -1
- package/out/tools/fetch/vlaanderen-ad.d.ts +1 -1
- package/out/tools/fetch/wfs-harvest.d.ts +3 -3
- package/out/tools/fetch/wfs-harvest.js +4 -4
- package/out/tools/fetch/wfs-harvest.js.map +1 -1
- package/out/tools/fetch/zip-archive.d.ts +2 -2
- package/out/tools/fetch/zip-archive.js +2 -2
- package/out/tools/fetch.d.ts +2 -0
- package/out/tools/fetch.d.ts.map +1 -1
- package/out/tools/fetch.js +2 -0
- package/out/tools/fetch.js.map +1 -1
- package/out/tools/golden/expand.js +1 -1
- package/out/tools/golden/expand.js.map +1 -1
- package/out/tools/golden/promote.d.ts +1 -1
- package/out/tools/golden/promote.js +3 -3
- package/out/tools/golden/promote.js.map +1 -1
- package/out/tools/golden/relabel-street/directory.d.ts +2 -1
- package/out/tools/golden/relabel-street/directory.d.ts.map +1 -1
- package/out/tools/golden/relabel-street/directory.js.map +1 -1
- package/out/tools/lint/recipe-output/vocab.js +2 -2
- package/out/tools/lint/recipe-output.d.ts +1 -1
- package/out/tools/migrate/source-names.d.ts +1 -1
- package/out/tools/overlay/manifest.d.ts +15 -2
- package/out/tools/overlay/manifest.d.ts.map +1 -1
- package/out/tools/overlay/manifest.js +41 -10
- package/out/tools/overlay/manifest.js.map +1 -1
- package/out/tools/overlay/split-slice.d.ts.map +1 -1
- package/out/tools/overlay/split-slice.js +2 -9
- package/out/tools/overlay/split-slice.js.map +1 -1
- package/out/tools/postcode-triples.d.ts +1 -1
- package/out/tools/postcode-triples.js +2 -2
- package/out/tools/postcode-triples.js.map +1 -1
- package/out/tools/sub/venue/head-nouns.d.ts +1 -1
- package/out/tools/sub/venue/head-nouns.d.ts.map +1 -1
- package/out/tools/sub/venue/head-nouns.js +4 -3
- package/out/tools/sub/venue/head-nouns.js.map +1 -1
- package/out/types.d.ts +3 -3
- package/out/types.js +1 -1
- package/out/us/adapters/state/tx-notaries/adapter.d.ts +1 -1
- package/out/us/adapters/state/tx-notaries/adapter.js +1 -1
- package/out/us/adapters/usgov/irs-bmf/adapter.d.ts +1 -1
- package/out/us/adapters/usgov/irs-bmf/adapter.js +2 -2
- package/out/us/adapters/usgov/nad/adapter.d.ts +1 -1
- package/out/us/adapters/usgov/nad/adapter.js +1 -1
- package/out/us/adapters/usgov/nppes/adapter.d.ts +1 -1
- package/out/us/adapters/usgov/nppes/adapter.js +1 -1
- package/out/us/synthesizers/intersection.js +4 -4
- package/out/us/synthesizers/intersection.js.map +1 -1
- package/out/us/synthesizers/street.js +3 -3
- package/out/us/synthesizers/street.js.map +1 -1
- package/out/us/tools/fetch/nad.d.ts +2 -2
- package/out/us/tools/fetch/nad.js +2 -2
- package/out/utils/split.d.ts +21 -0
- package/out/utils/split.d.ts.map +1 -1
- package/out/utils/split.js +49 -5
- package/out/utils/split.js.map +1 -1
- package/out/utils/wof-json.d.ts +1 -1
- package/out/utils/wof-json.js +1 -1
- package/package.json +15 -14
package/data/PROVENANCE.md
CHANGED
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
## `address-source-register.json` — the address-source register (#2323)
|
|
4
4
|
|
|
5
5
|
Which jurisdictions exist, what research has resolved for each, and what is known about every
|
|
6
|
-
source's terms.
|
|
6
|
+
source's terms. The file is generated, and a hand edit to it is refused.
|
|
7
7
|
|
|
8
8
|
### Regenerate
|
|
9
9
|
|
|
@@ -22,16 +22,16 @@ yarn format
|
|
|
22
22
|
That is the command that produced the committed copy. The two CSVs are research working documents and
|
|
23
23
|
live in the `mailwoman-internal` repository under `research/`, so regenerating needs a checkout of it.
|
|
24
24
|
That repository is checked out at `internal/`, which `.git/info/exclude` keeps out of this one, and
|
|
25
|
-
the paths above read from there. They were under this repository's gitignored `.notes/` until 2026-09-20
|
|
26
|
-
|
|
27
|
-
is the committed record here
|
|
25
|
+
the paths above read from there. They were under this repository's gitignored `.notes/` until 2026-09-20. No
|
|
26
|
+
version control tracked them there. A reviewer could not determine which copy produced the register. The
|
|
27
|
+
register is the committed record here. It stores every field a reader needs to check a row: the
|
|
28
28
|
publisher, the retrieved URL, and the research pass.
|
|
29
29
|
|
|
30
30
|
Two further inputs are committed beside this file, and both exist because the write below replaces the
|
|
31
31
|
register whole: a value recorded in the register itself is erased by the next run with no error and a
|
|
32
32
|
normal-looking count.
|
|
33
33
|
|
|
34
|
-
`--decisions` defaults to `license-decisions.json` and holds every
|
|
34
|
+
`--decisions` defaults to `license-decisions.json` and holds every license decision a reviewer made by
|
|
35
35
|
reading a publisher's terms. The build derives each decision as `unchecked` from the research pass's
|
|
36
36
|
access labels and applies the recorded ones over that, so a decision survives a rebuild (#2351).
|
|
37
37
|
|
|
@@ -41,8 +41,8 @@ first three as the placeholders `varies`, `country-specific` and the empty strin
|
|
|
41
41
|
non-rail rows, and has no column for a personal-data review, so a review records its findings here
|
|
42
42
|
rather than by rewriting the record of what the research pass found on 2026-09-18 (#2323).
|
|
43
43
|
|
|
44
|
-
The `oxfmt` pass is required
|
|
45
|
-
reproduce. The generator is deterministic, so the artifact is reproducible from the same inputs.
|
|
44
|
+
The `oxfmt` pass is required. Committed JSON is oxfmt-clean, and `JSON.stringify` cannot
|
|
45
|
+
reproduce that formatting. The generator is deterministic, so the artifact is reproducible from the same inputs.
|
|
46
46
|
|
|
47
47
|
### What no check can establish about this file (#2352)
|
|
48
48
|
|
|
@@ -52,10 +52,10 @@ of that one, so the register's correctness rests on whoever last ran the command
|
|
|
52
52
|
`auditAddressSourceRegister` does not change that: it checks the file's structure, and a publisher's
|
|
53
53
|
name rewritten to something that names no institution is structurally sound.
|
|
54
54
|
|
|
55
|
-
The inputs
|
|
55
|
+
The inputs have been versioned since 2026-09-20. A reviewer can diff the CSV that
|
|
56
56
|
produced the register against the one a later build reads, and a research pass that moves a
|
|
57
|
-
publisher's name leaves a commit
|
|
58
|
-
inputs still
|
|
57
|
+
publisher's name leaves a commit that records the move. A check that the committed register matches
|
|
58
|
+
its inputs still requires a run of the command with both repositories checked out.
|
|
59
59
|
|
|
60
60
|
A committed digest would not close it either. A sweep that edits the artifact and the digest together
|
|
61
61
|
passes, and one that edits only the artifact fails without saying which side is right.
|
|
@@ -71,74 +71,88 @@ concluding the publisher renamed itself.
|
|
|
71
71
|
| file | rows | what is read |
|
|
72
72
|
| ---------------------------------------------- | ----: | ---------------------------------------------------------------------- |
|
|
73
73
|
| `global-address-jurisdiction-inventory-v3.csv` | 250 | all 250 rows — every ISO 3166-1 alpha-2 code plus the operational `XK` |
|
|
74
|
-
| `global-functional-authority-corpora-v2.csv` | 2,426 | the 426 rows naming a national source
|
|
74
|
+
| `global-functional-authority-corpora-v2.csv` | 2,426 | the 426 rows naming a national source. 2,000 are dropped |
|
|
75
75
|
|
|
76
|
-
The 2,000 dropped rows
|
|
77
|
-
pass repeated once per jurisdiction
|
|
78
|
-
|
|
76
|
+
The 2,000 dropped rows have `origin = global_research_rail`. They are eight global discovery lookups the
|
|
77
|
+
pass repeated once per jurisdiction. With the `iso2` and `jurisdiction` columns ignored, they collapse
|
|
78
|
+
to eight distinct row bodies. They name a lookup to perform rather than a source, so they are the
|
|
79
79
|
README's procedure and not rows here.
|
|
80
80
|
|
|
81
|
-
### Why three columns the CSV
|
|
81
|
+
### Why three columns of the CSV reach no register field
|
|
82
82
|
|
|
83
83
|
`global-functional-authority-corpora-v2.csv` has an `address_role`, a `coverage` and an `upstream`
|
|
84
84
|
column, and the register declares all three unresolved. That is not the build dropping usable data.
|
|
85
85
|
Over the 426 rows it keeps, `address_role` reads `varies` on all 426 and `coverage` reads
|
|
86
86
|
`country-specific` on all 426. `upstream` is empty on 407 of them: the nineteen rows the 2026-10-01
|
|
87
87
|
rights review added name their own upstream, as `adresse.data.gouv.fr`'s department directory and
|
|
88
|
-
Geonorge's area codelist. The first two are the research
|
|
89
|
-
pass
|
|
88
|
+
Geonorge's area codelist. The first two values are the research
|
|
89
|
+
pass's record that it did not determine the field per source.
|
|
90
90
|
|
|
91
|
-
The build reads the columns and refuses to
|
|
91
|
+
The build reads the columns and refuses to copy a placeholder, because
|
|
92
92
|
`ingestEligibilityProblems` reports "no address role is resolved" and "no coverage has been
|
|
93
93
|
measured" as two of its five conditions. The others are the unreviewed license decision, the absent
|
|
94
|
-
personal-data review, and a status that records the source as uninspected.
|
|
95
|
-
into `addressRole` would stop both from firing
|
|
96
|
-
placeholder set is
|
|
94
|
+
personal-data review, and a status that records the source as uninspected. A `varies` written
|
|
95
|
+
into `addressRole` would stop both from firing and would resolve no field. A value outside the declared
|
|
96
|
+
placeholder set is copied through, and an `address_role` that is neither a placeholder nor an
|
|
97
97
|
`AddressRole` fails the build rather than being dropped.
|
|
98
98
|
|
|
99
99
|
### How an edit to the committed file is detected
|
|
100
100
|
|
|
101
|
-
The register
|
|
101
|
+
The register stores `contentDigest`, a sha256 the build writes over every other field in it.
|
|
102
102
|
`readAddressSourceRegister` recomputes it on every read and refuses a file that no longer matches,
|
|
103
103
|
naming the regenerate command above.
|
|
104
104
|
|
|
105
105
|
It exists because no check can regenerate this artifact and compare. The two research CSVs are
|
|
106
|
-
working documents under `.notes/`, which is not committed, so a CI job has
|
|
106
|
+
working documents under `.notes/`, which is not committed, so a CI job has no input to rebuild from.
|
|
107
107
|
Without a recorded digest, a hand edit to a generated file is invisible: a prose sweep rewrote
|
|
108
108
|
`Contracts Finder / Find a Tender` to `Interfaces Finder / Find a Tender` and two other publisher
|
|
109
109
|
names, and the structural audit passed, because it checks shape rather than whether a name is the
|
|
110
110
|
publisher's (#2352).
|
|
111
111
|
|
|
112
|
-
The digest covers the register as parsed
|
|
113
|
-
build writes it. It detects an edit rather than attributing one
|
|
114
|
-
produces a new digest
|
|
112
|
+
The digest covers the register as parsed rather than as bytes, because `oxfmt` reformats the file
|
|
113
|
+
after the build writes it. It detects an edit rather than attributing one. A build rerun after a change
|
|
114
|
+
produces a new digest through the intended path. A hand edit to the committed copy produces none.
|
|
115
115
|
|
|
116
|
-
Of the four artifacts in this directory, the register is the one
|
|
116
|
+
Of the four artifacts in this directory, the register is the one with a digest. The sub-venue
|
|
117
117
|
lexicon, the reviewed VE postcode tuples and the license decisions do not. The license decisions are
|
|
118
|
-
hand-written by design
|
|
118
|
+
hand-written by design. That file is an input to this build rather than an output of one.
|
|
119
119
|
|
|
120
|
-
###
|
|
120
|
+
### How a `sourceID` is numbered, and why a CSV row is appended
|
|
121
121
|
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
122
|
+
A `sourceID` is `<iso2>-<sector>-<ordinal>`, and the ordinal is the row's position among the research
|
|
123
|
+
CSV's rows of the same `iso2` and sector. Row order therefore decides the suffix: inserting a row before
|
|
124
|
+
an existing one of the same `iso2` and sector renumbers the later row and reattaches its entry in
|
|
125
|
+
`source-resolutions.json` to the wrong source, with no error. Append within an `iso2` and sector. Never
|
|
126
|
+
insert.
|
|
127
|
+
|
|
128
|
+
`ASSERTS_BY_ROLE`, `SOURCE_STATUS_BY_NAME`, `GEOMETRY_BY_NAME` and `RESEARCH_PASS_BY_ORIGIN` in
|
|
129
|
+
`packages/corpus/lib/tools/source-register/build.ts` are closed tables, and a CSV spelling outside one
|
|
130
|
+
fails the build. The sector column is the one mapped spelling: the research CSV writes
|
|
131
|
+
`property_building`, the register declares `property-building`, and the build replaces the underscore.
|
|
132
|
+
`ingestEligibilityProblems(source, register)` takes the whole register rather than the license record.
|
|
133
|
+
|
|
134
|
+
### What the register stores
|
|
135
|
+
|
|
136
|
+
The register holds two tables rather than one nested list. The jurisdiction table is a complete
|
|
137
|
+
250-row enumeration. The source table is zero or more rows per jurisdiction, 426 over 240 of them. If
|
|
138
|
+
the second were nested inside the first, a jurisdiction no researcher examined would be structurally identical to one whose
|
|
125
139
|
list happens to be empty, and no reader could tell which they were looking at.
|
|
126
140
|
|
|
127
141
|
| research state | jurisdictions | meaning |
|
|
128
142
|
| -------------- | ------------: | ------------------------------------------------------------- |
|
|
129
143
|
| `seeded` | 240 | at least one national source resolved |
|
|
130
144
|
| `exception` | 7 | somebody looked and recorded that no ordinary registry exists |
|
|
131
|
-
| `unexamined` | 3 |
|
|
145
|
+
| `unexamined` | 3 | no source resolved and no source ruled out |
|
|
132
146
|
|
|
133
|
-
`exception` and `unexamined` are separate values on purpose
|
|
134
|
-
an unexamined jurisdiction is not
|
|
135
|
-
as "
|
|
147
|
+
`exception` and `unexamined` are separate values on purpose. A researched absence is a finding and
|
|
148
|
+
an unexamined jurisdiction is not. If the two were merged, a consumer would read "we could not look"
|
|
149
|
+
as "no registry exists there". Every non-`seeded` row has a `stateReason`, and the audit refuses
|
|
136
150
|
one that does not.
|
|
137
151
|
|
|
138
152
|
| source status | rows | meaning |
|
|
139
153
|
| ----------------------- | ---: | ----------------------------------------------------------------------- |
|
|
140
|
-
| `verified-authority` | 221 | register confirmed
|
|
141
|
-
| `retained-original` | 142 |
|
|
154
|
+
| `verified-authority` | 221 | register confirmed. Address fields, bulk access and terms not inspected |
|
|
155
|
+
| `retained-original` | 142 | copied from the earlier memo without recheck. No publisher or URL |
|
|
142
156
|
| `verified-corpus` | 29 | a bulk corpus confirmed reachable |
|
|
143
157
|
| `verified-corpus-stale` | 1 | reachable, and the copy examined has stopped being updated |
|
|
144
158
|
|
|
@@ -148,12 +162,12 @@ repository already has locale tiers 1 through 5 in `scope.config.json` and the
|
|
|
148
162
|
|
|
149
163
|
### Three things the research pass did not resolve, on any row
|
|
150
164
|
|
|
151
|
-
|
|
165
|
+
This section lists them so a reader does not have to discover them by querying:
|
|
152
166
|
|
|
153
167
|
- **`addressRole` — 0 of 426.** Every row reads `varies`. The specification's section 2 asks for a
|
|
154
168
|
role per source and the pass recorded none, so the research CSV cannot say whether any of these
|
|
155
169
|
sources holds premises, registered offices or mailing addresses.
|
|
156
|
-
- **`upstreamLineage` — 19 of 426.** The 2026-09-18 pass recorded none, so two databases
|
|
170
|
+
- **`upstreamLineage` — 19 of 426.** The 2026-09-18 pass recorded none, so two databases built from one
|
|
157
171
|
upstream submission could not be collapsed into the single observation they are. The 2026-10-01
|
|
158
172
|
rights review fills the column as it goes, and the nineteen rows it added name the directory or
|
|
159
173
|
codelist the source was found through.
|
|
@@ -161,7 +175,7 @@ Say this plainly rather than making a reader discover it by querying:
|
|
|
161
175
|
measurement. A national portal is not evidence of national coverage.
|
|
162
176
|
|
|
163
177
|
A later review resolves these per source in `source-resolutions.json`, and the register's `unresolved`
|
|
164
|
-
array
|
|
178
|
+
array lists any field no row has yet. The audit checks that claim in both directions: a field
|
|
165
179
|
listed there must be absent from every row, and a field not listed must be present on at least one.
|
|
166
180
|
The array is empty today, because a review has resolved each of the four on at least one source.
|
|
167
181
|
|
|
@@ -175,50 +189,50 @@ adapter emits a source's rows, `readSourceEligibility` keys each declaring sourc
|
|
|
175
189
|
`country`. The pair is the key because the register scopes a source to one publisher in one
|
|
176
190
|
jurisdiction while one adapter can serve several: `ban` emits eleven jurisdictions from one schema,
|
|
177
191
|
each under its own license decision. Eleven sources declare an adapter today, all of them BAN's, so a
|
|
178
|
-
release-eligible build admits those eleven and refuses every other row with the jurisdiction
|
|
179
|
-
A source that declares no adapter stays out of that map
|
|
180
|
-
since no adapter emits
|
|
192
|
+
release-eligible build admits those eleven and refuses every other row with an error that identifies the jurisdiction.
|
|
193
|
+
A source that declares no adapter stays out of that map. That exclusion is correct rather than
|
|
194
|
+
restrictive, since no adapter emits the source and no row can arrive under it.
|
|
181
195
|
|
|
182
196
|
### Licenses: 54 read, 372 unchecked
|
|
183
197
|
|
|
184
198
|
All 426 rows point at a decision record of their own. 47 are `elected`, 7 are `refused`, and the
|
|
185
199
|
other 372 read `unchecked`, which is a finding rather than a placeholder. The 253 web-researched rows
|
|
186
|
-
read `CHECK NATIONAL / DATASET TERMS` verbatim. The other 173
|
|
187
|
-
memo
|
|
188
|
-
six compound labels over the remaining 24, as `Free/varied` on 7 and `Free/gated` on 6
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
`unchecked` is a first-class value distinct from a
|
|
193
|
-
|
|
200
|
+
read `CHECK NATIONAL / DATASET TERMS` verbatim. The other 173 have an access label from the original
|
|
201
|
+
memo. The labels are `Free` on 130 rows, `Free-reg` on 15, `Gated` on 2, `Licensed` on 1 and `Varied`
|
|
202
|
+
on 1, with six compound labels over the remaining 24, such as `Free/varied` on 7 and `Free/gated` on 6.
|
|
203
|
+
Such a label states what the download costs and grants no rights. A build that treated one as
|
|
204
|
+
permissive would admit a source on a sentence about price.
|
|
205
|
+
|
|
206
|
+
`unchecked` is a first-class value distinct from a license reviewed and found permissive. The
|
|
207
|
+
`elected` value records the reviewed case: the terms, the retrieved copy, the version, and why that grant rather than
|
|
194
208
|
another. `electedLicenseLabel()` is what the mechanical prefix filter in `utils/license.ts` reads,
|
|
195
209
|
and it answers `undefined` for anything not elected.
|
|
196
210
|
|
|
197
211
|
## `licenses/` — the retrieved texts a decision would be read from
|
|
198
212
|
|
|
199
213
|
Seven grants as retrieved, each with the URL and the retrieval date at its head, plus the failures a
|
|
200
|
-
fetch returned.
|
|
201
|
-
retrieval
|
|
214
|
+
fetch returned. Each file is written by hand from a fetch rather than by a build, and is never
|
|
215
|
+
edited after retrieval. A correction is a new retrieval.
|
|
202
216
|
|
|
203
|
-
The directory
|
|
204
|
-
retrieved, and the two readings a retrieval moved.
|
|
205
|
-
the directory is accounted for rather than overlooked.
|
|
217
|
+
The directory has its own `PROVENANCE.md`. That file lists the directory's contents, the texts that
|
|
218
|
+
could not be retrieved, and the two readings a retrieval moved. This section points to it so a reader
|
|
219
|
+
of this file knows the directory is accounted for rather than overlooked.
|
|
206
220
|
|
|
207
|
-
`ElectedLicense.retrievedCopy` in the source register
|
|
208
|
-
recorded in `license-decisions.json` points at the text it was made from.
|
|
221
|
+
`ElectedLicense.retrievedCopy` in the source register holds the path of a file there. Through that
|
|
222
|
+
path, a decision recorded in `license-decisions.json` points at the text it was made from.
|
|
209
223
|
|
|
210
|
-
## `license-decisions.json` —
|
|
224
|
+
## `license-decisions.json` — license decisions a reviewer made by reading terms (#2351)
|
|
211
225
|
|
|
212
226
|
Hand-written, and the only file in this directory that is. `address-source-register.json` is
|
|
213
227
|
generated and rewritten whole on every build, so a decision recorded there is erased by the next run
|
|
214
228
|
with no error and a normal-looking count. This file is an input to that build instead.
|
|
215
229
|
|
|
216
230
|
Each key is a `licenseID` the register already declares. Each value is an `ElectedLicense` or a
|
|
217
|
-
`RefusedLicense` minus its `licenseID
|
|
218
|
-
two decisions. `buildSourceRegister` applies these over the `unchecked` defaults it derives from the
|
|
219
|
-
research pass's access labels, and refuses a decision naming a
|
|
231
|
+
`RefusedLicense` minus its `licenseID`. The build fills the id from the key, so one license cannot
|
|
232
|
+
hold two decisions. `buildSourceRegister` applies these over the `unchecked` defaults it derives from the
|
|
233
|
+
research pass's access labels, and refuses a decision naming a license no source points at.
|
|
220
234
|
|
|
221
|
-
An election records four things
|
|
235
|
+
An election records the four things the corpus acceptance rules ask for: the elected terms, the
|
|
222
236
|
copy they were read from, that copy's version where the publisher gives one, and why this grant and
|
|
223
237
|
not another. `auditAddressSourceRegister` refuses an incomplete record and the build refuses a
|
|
224
238
|
register that fails its audit, so an incomplete entry never reaches the committed artifact.
|
|
@@ -230,10 +244,10 @@ SIRENE in each of nine overseas territories. Kadaster dedicates the Dutch addres
|
|
|
230
244
|
domain under CC0 1.0, Klimadatastyrelsen licenses the Danish one under CC BY 4.0, and ČÚZK states
|
|
231
245
|
`žádné podmínky neplatí` on all 6,259 rights elements of the Czech feed. Maa- ja Ruumiamet dedicates
|
|
232
246
|
the Estonian theme to the public domain under CC0 1.0, stated in the ISO 19139 dataset record its own
|
|
233
|
-
WFS capabilities links rather than in the capabilities document
|
|
234
|
-
read `puudub` and describe the service. The Dirección General del
|
|
247
|
+
WFS capabilities links rather than in the capabilities document. That document's `Fees` and
|
|
248
|
+
`AccessConstraints` read `puudub` and describe the service. The Dirección General del
|
|
235
249
|
Catastro grants its own INSPIRE
|
|
236
|
-
access-and-use
|
|
250
|
+
access-and-use license over the cadastral addresses of 52 Spanish territorial offices, and the three
|
|
237
251
|
foral councils that publish through the same national feed grant their own terms over the rest of
|
|
238
252
|
Spain's cadastral addresses: Navarra under `CC-BY-4.0`, Gipuzkoa under `CC-BY-SA-4.0`, and Bizkaia
|
|
239
253
|
under a statement naming no instrument. Each is retained under
|
|
@@ -246,43 +260,43 @@ upphovsrätt. Avtal för användning krävs, avgifter för användning tas ut`.
|
|
|
246
260
|
`unchecked`.
|
|
247
261
|
|
|
248
262
|
**A refusal is recorded as a decision rather than left as an absence, and the research pass's access
|
|
249
|
-
label can contradict it.** Sweden's row
|
|
250
|
-
|
|
263
|
+
label can contradict it.** Sweden's row has the label `Free`. That label describes access to the feed.
|
|
264
|
+
The feed itself requires an agreement and charges fees for use. The label is what prompted the
|
|
251
265
|
reading, and the reading is what the register records.
|
|
252
266
|
|
|
253
267
|
A decision may name a body declared once under `sharedReadings` rather than restating it. One publisher
|
|
254
|
-
can hold a
|
|
255
|
-
jurisdiction, so one reading of INSEE's terms produces nine decisions. Each
|
|
256
|
-
own, so no election spans two jurisdictions
|
|
268
|
+
can hold a license over several jurisdictions, and a decision is scoped to one publisher in one
|
|
269
|
+
jurisdiction, so one reading of INSEE's terms produces nine decisions. Each license id still has its
|
|
270
|
+
own decision, so no election spans two jurisdictions. The shared body removes the risk of nine hand-maintained
|
|
257
271
|
copies drifting. The build refuses a `sameAs` naming a reading the file does not declare.
|
|
258
272
|
|
|
259
|
-
|
|
260
|
-
address role per column, measured coverage, and a personal-data review that does not read
|
|
261
|
-
|
|
262
|
-
have exactly one blocking condition left, and fifteen of the sixteen
|
|
263
|
-
reading
|
|
264
|
-
a question about one. The sixteenth is Uruguay, blocked on its address role instead.
|
|
273
|
+
An election of terms is necessary and not sufficient. `ingestEligibilityProblems` also requires a
|
|
274
|
+
resolved address role per column, measured coverage, and a personal-data review that does not read
|
|
275
|
+
`present`. A license decision touches none of these. They live in `source-resolutions.json`. **Sixteen
|
|
276
|
+
sources have exactly one blocking condition left, and for fifteen of the sixteen that condition is the
|
|
277
|
+
personal-data reading.** The personal-data question is therefore one decision that blocks fifteen
|
|
278
|
+
sources rather than a question about one. The sixteenth is Uruguay, blocked on its address role instead.
|
|
265
279
|
|
|
266
280
|
### Why there are 426 of them rather than 12
|
|
267
281
|
|
|
268
|
-
A decision is scoped to one publisher in one jurisdiction, so the register
|
|
269
|
-
|
|
282
|
+
A decision is scoped to one publisher in one jurisdiction, so the register holds one per source.
|
|
283
|
+
A decision keyed only by the research pass's access label was as wide as the label:
|
|
270
284
|
`CHECK NATIONAL / DATASET TERMS` covers 253 of the 426 sources across 228 publishers, and `Free`
|
|
271
|
-
covers 130 across 22.
|
|
285
|
+
covers 130 across 22. One election recorded against either would have granted every source under it on a
|
|
272
286
|
single reading of one publisher's terms.
|
|
273
287
|
|
|
274
|
-
Spain was the worked case of the same defect inside one row. `es-mixed-regulatory-1`
|
|
275
|
-
`Catastro, INE` and
|
|
276
|
-
institutions. The same national feed
|
|
277
|
-
same pass gave three rows of their own. The 2026-09-30 rights review split the row per publisher
|
|
278
|
-
row's scope onto its own publisher.
|
|
279
|
-
that
|
|
288
|
+
Spain was the worked case of the same defect inside one row. `es-mixed-regulatory-1` had the
|
|
289
|
+
name `Catastro, INE` and no publisher, so its decision scoped to the source id and covered two
|
|
290
|
+
institutions. The same national feed publishes three foral cadastres under their own terms, and the
|
|
291
|
+
same pass gave them three rows of their own. The 2026-09-30 rights review split the row per publisher.
|
|
292
|
+
Each row's scope then moved onto its own publisher. The scoping works only when a row identifies one
|
|
293
|
+
publisher, and a row that lists two defeats it.
|
|
280
294
|
|
|
281
295
|
The jurisdiction is part of the scope because a publisher name is not unique across states. The pass
|
|
282
296
|
wrote `Ministry of Justice` for Belarus, Lebanon and Timor-Leste, `Ministry of Commerce and Industry`
|
|
283
297
|
for five countries, and `Commercial-registration authority` as a description rather than a name.
|
|
284
298
|
|
|
285
|
-
The cost falls on a publisher that
|
|
299
|
+
The cost falls on a publisher that serves several jurisdictions. INSEE covers mainland
|
|
286
300
|
France and nine overseas territories, so it holds ten decisions, and a reviewer who reads its terms
|
|
287
301
|
once records that conclusion ten times. That is the direction to err in: an over-wide election is the
|
|
288
302
|
failure review cannot undo.
|
|
@@ -292,53 +306,53 @@ failure review cannot undo.
|
|
|
292
306
|
Hand-written, and an input to the register build for the same reason as `license-decisions.json`: the
|
|
293
307
|
build rewrites the register whole.
|
|
294
308
|
|
|
295
|
-
Each key is a `sourceID` the register declares, and each value
|
|
309
|
+
Each key is a `sourceID` the register declares, and each value holds any of `addressRoles`,
|
|
296
310
|
`coverage`, `upstreamLineage` and `personalDataReview`. `buildSourceRegister` applies these over the
|
|
297
311
|
rows it derives from the research CSV and refuses a resolution naming a source the register does not
|
|
298
|
-
|
|
312
|
+
declare, because a resolution that resolves no row is a typo or a source that has been removed.
|
|
299
313
|
|
|
300
314
|
The fields live here rather than in the CSV because the CSV is the record of a research pass. All 390
|
|
301
315
|
of its non-rail rows write `varies` for `address_role`, `country-specific` for `coverage` and an empty
|
|
302
|
-
`upstream`, and no pass has a personal-data column at all.
|
|
316
|
+
`upstream`, and no pass has a personal-data column at all. An edit to those placeholders would change
|
|
303
317
|
what a pass is recorded as having found. A row's other columns are a different matter: a later pass
|
|
304
318
|
rewrites one when reading the publisher settles something the earlier pass recorded wrongly, and the
|
|
305
319
|
row's `origin` column then names that later pass so the change is legible.
|
|
306
320
|
|
|
307
321
|
`addressRoles` is keyed by the address column's path in the published record rather than holding one
|
|
308
|
-
role for the source, because a publication can
|
|
322
|
+
role for the source, because a publication can hold two roles on one record. AusTender's OCDS release
|
|
309
323
|
gives a `counterparty` address for the supplier and a `facility` address for the procuring entity on
|
|
310
324
|
every contracting process. Taiwan's GCIS register gives a registered company address and a tax-office
|
|
311
325
|
business address in separate columns. `ingestEligibilityProblems` requires at least one column to have
|
|
312
|
-
a role, and the audit refuses an empty map
|
|
326
|
+
a role, and the audit refuses an empty map. An empty map would read as resolved while it states no role.
|
|
313
327
|
|
|
314
|
-
**Fifty-four sources of 426
|
|
315
|
-
Five of those fifty-four
|
|
316
|
-
reachable data
|
|
328
|
+
**Fifty-four sources of 426 have at least one resolved field, and forty-five have all four.**
|
|
329
|
+
Five of those fifty-four have only `upstreamLineage`. Those five are located publishers with no
|
|
330
|
+
reachable data: Aruba, Curaçao, Sint Maarten, the Caribbean Netherlands and Hong Kong. AusTender
|
|
317
331
|
and five OCDS procurement releases are recorded from whole-year measurements of the published data.
|
|
318
332
|
The nine SIRENE territories are recorded from INSEE's own `dessin de fichier`. Seven of the fifteen
|
|
319
333
|
address publications are recorded from an element census over a whole GML file, the largest being the
|
|
320
334
|
Netherlands' 29,677,448,685 bytes and 10,066,060 features and the newest Wallonia's 5,163,917,131
|
|
321
335
|
bytes and 1,772,312 features. Denmark's is recorded from the column schema of its GeoPackage instead,
|
|
322
336
|
because it publishes the theme as a database, and Finland's and France's from the headers of their
|
|
323
|
-
CSVs
|
|
337
|
+
CSVs. Every row of those CSVs has exactly the fields its header declares, 24 for Finland and 23 for
|
|
324
338
|
France. The remaining four are recorded from every
|
|
325
339
|
element of every feature type their WFS publishes, because none of them publishes a file: five types
|
|
326
340
|
and 101 properties for Estonia, five and 44 for Slovakia, four and 32 for Flanders, and one type and
|
|
327
341
|
19 elements for Poland. Each is retained under `internal/strategy/rights-receipts/`.
|
|
328
342
|
|
|
329
343
|
**A whole-file census answers what a sample cannot.** Wallonia's 56 element names were counted over
|
|
330
|
-
all 5,163,917,131 bytes
|
|
344
|
+
all 5,163,917,131 bytes. That count shows that `ad:locatorName` appears 0 times in the publication
|
|
331
345
|
rather than 0 times in the rows that were read. The count is self-checking: `ad:Address`,
|
|
332
346
|
`ad:AddressLocator`, `ad:GeographicPosition`, `ad:locator` and `ad:position` each appear 1,772,312
|
|
333
347
|
times, as a GML file holds one of each per address, and those 1,772,312 addresses plus the file's
|
|
334
348
|
55,766 thoroughfare names, 1,963 address-area names, 598 postal descriptors and 264 administrative-unit
|
|
335
349
|
names equal its 1,830,903 `gml:featureMember` elements exactly. A first pass reported those five
|
|
336
|
-
counts as 1,772,315, 1,772,316 and 1,772,314
|
|
337
|
-
|
|
350
|
+
counts as 1,772,315, 1,772,316 and 1,772,314. Counts slightly above the true value are the symptom
|
|
351
|
+
of a chunked reader that re-scans its own overlap.
|
|
338
352
|
|
|
339
353
|
**Seventeen sources are ingest-eligible and fifteen have one blocking condition each.** Fourteen of the
|
|
340
354
|
fifteen read `personalDataReview: present`. The fifteenth is Uruguay, refused on its address role: no
|
|
341
|
-
party object in 127,885
|
|
355
|
+
party object in 127,885 has an address field of any kind, so that publication is reachable in bulk
|
|
342
356
|
and is not an address source.
|
|
343
357
|
|
|
344
358
|
Sixteen of the seventeen eligible sources are national or regional address publications of European
|
|
@@ -351,32 +365,32 @@ Walloon Region's Service public de Wallonie under CC BY 4.0, Finland's Suomen ym
|
|
|
351
365
|
CC BY 4.0, France's IGN under Licence Ouverte 2.0, Luxembourg's Administration du cadastre et de
|
|
352
366
|
la topographie under CC0 1.0, and Norway's Kartverket under CC BY 4.0.
|
|
353
367
|
|
|
354
|
-
**An elected
|
|
368
|
+
**An elected license does not make a source eligible, and Switzerland is the worked case.** swisstopo's
|
|
355
369
|
own terms grant use, distribution, enrichment, processing and commercial use against a mandatory
|
|
356
|
-
source reference, so its
|
|
357
|
-
`ingestEligibilityProblems` refuses on that
|
|
358
|
-
|
|
370
|
+
source reference, so its license is elected. Its `personalDataReview` reads `present`, and
|
|
371
|
+
`ingestEligibilityProblems` refuses on that condition by itself, so the 3,303,210-row directory stays
|
|
372
|
+
out. The license and the eligibility are separate answers and the register keeps them separate.
|
|
359
373
|
|
|
360
374
|
**The largest of them is France's Base Adresse Nationale at 26,117,350 rows**, measured by parsing
|
|
361
375
|
the whole file, against the `plus de 25 millions d'adresses` its publisher states. Poland's 8,625,921
|
|
362
376
|
follows, then the Flemish Region's 4,563,062, Finland's 3,862,373, the Walloon Region's 1,772,312 and
|
|
363
377
|
Slovakia's 1,704,196. The INSPIRE Addresses model has no element for a party, so a party reaches a
|
|
364
|
-
|
|
378
|
+
harmonized publication only through a free-text name element. INSPIRE Annex I makes Addresses a
|
|
365
379
|
mandatory theme for every EU member state, so the remaining states are the place to look for more.
|
|
366
380
|
|
|
367
381
|
**Belgium takes three rows and Finland's is not an INSPIRE service at all.** Belgium's regions publish
|
|
368
382
|
separately, so Flanders, Wallonia and Brussels each need their own row, of which the first two are
|
|
369
383
|
recorded. Finland's eligible publication is a pair of bulk files on the publisher's own host rather
|
|
370
|
-
than a WFS or an ATOM feed
|
|
384
|
+
than a WFS or an ATOM feed. For that reason `addressRoles` keys it by two CSV columns, `address_fin` and
|
|
371
385
|
`address_swe`, each holding a complete address in its own language.
|
|
372
386
|
|
|
373
387
|
**A service may answer the theme's name with the national schema.** Poland's download service
|
|
374
388
|
publishes one feature type, `ms:AD.Address`, whose element names are the Polish register's own
|
|
375
|
-
columns: `miejscowosc`, `ulica`, `numer`, `kod` and `adres
|
|
389
|
+
columns: `miejscowosc`, `ulica`, `numer`, `kod` and `adres`. The last holds the whole address as
|
|
376
390
|
one string. No `ad:ThoroughfareName`, `ad:PostalDescriptor` or `ad:AdminUnitName` type exists there to
|
|
377
391
|
join to, so that build needs no component resolution and the personal-data review has to ask what the
|
|
378
|
-
columns are rather than whether `AD:LocatorName` is populated.
|
|
379
|
-
of the
|
|
392
|
+
columns are rather than whether `AD:LocatorName` is populated. A reader that took the theme's name as a promise
|
|
393
|
+
of the harmonized model would have found no matching elements.
|
|
380
394
|
|
|
381
395
|
**A regional row covers a region.** The Flemish theme's 4,563,062 addresses are the Flemish Region,
|
|
382
396
|
and Brussels and Wallonia publish their own services, so a Belgian build needs three rows where a
|
|
@@ -385,12 +399,12 @@ cadastres.
|
|
|
385
399
|
|
|
386
400
|
**Switzerland is why that element is opened on every source rather than assumed empty.** Its
|
|
387
401
|
`BDG_NAME` is populated on 20,989 of 3,303,210 rows, 0.64%, and every populated value was read: of
|
|
388
|
-
1,158 matching a two-
|
|
402
|
+
1,158 matching a two-capitalized-word shape, 684 contain a structure word and 474 do not, and **at least
|
|
389
403
|
twelve of those 474 are a surname and a given name on a row whose category reads `residential`** —
|
|
390
404
|
`Chaignat Aimée` in Jura, `Rutz Nelly` and `Frigerio Cesare Vittorio` in Ticino, `NB EFH Lauber Jörg`
|
|
391
|
-
in Valais among them. That is a natural person at their home
|
|
392
|
-
|
|
393
|
-
farm or building name
|
|
405
|
+
in Valais among them. That is a natural person at their home. On 2026-09-30 this repository
|
|
406
|
+
decided it would not include such a row. Every other address publication's name element has read as a place,
|
|
407
|
+
farm or building name. This one does not, and a reader that had assumed the element empty would have
|
|
394
408
|
admitted it.
|
|
395
409
|
|
|
396
410
|
`AD:LocatorName` is that free-text element in the INSPIRE model, typed `GeographicalName`, and each
|
|
@@ -401,49 +415,50 @@ than assumed empty. The Dutch, Czech, Danish, Slovak and Flemish themes leave it
|
|
|
401
415
|
measured at 0 of 800 addresses over 40 windows across its 4,563,062 and Slovakia at 0 of 800 over 40
|
|
402
416
|
windows across its 1,704,196. Estonia fills it on 1,200 of 1,200 addresses sampled over 60 windows
|
|
403
417
|
spread across its 729,973, every one typed `siteName` or `buildingName` in the INSPIRE codelist, with
|
|
404
|
-
0 of the 1,200
|
|
405
|
-
|
|
418
|
+
0 of the 1,200 in the given-name-and-family-name shape. The values are Estonian farm names. A rural
|
|
419
|
+
Estonian address designates its addressable unit by a farm name.
|
|
406
420
|
|
|
407
421
|
**A reader that finds the element on one service can miss it on another.** Slovakia nests the locator
|
|
408
422
|
and omits the `name` key entirely, so its absence is structural as well as sampled. Estonia flattens
|
|
409
423
|
the path to `locator_addresslocator_name_locatorname_name_spelling_text`. One traversal run against
|
|
410
|
-
both would report one of them wrongly
|
|
411
|
-
read, and
|
|
424
|
+
both would report one of them wrongly. For that reason each service's measurement states the shape
|
|
425
|
+
it read, and Slovakia's traversal was checked against Estonia before its 0 was recorded.
|
|
412
426
|
|
|
413
|
-
Those publishers state seven different grants,
|
|
414
|
-
|
|
427
|
+
Those publishers state seven different grants, so a mandatory EU theme does not come with uniform
|
|
428
|
+
terms. Kadaster and Maa- ja Ruumiamet dedicate to the public
|
|
415
429
|
domain, Denmark, Navarra, Wallonia and Finland's SYKE license under CC BY 4.0, Gipuzkoa under
|
|
416
430
|
CC BY-SA 4.0, ČÚZK, the Slovak Ministerstvo vnútra and Poland's GUGiK state the INSPIRE controlled
|
|
417
431
|
value for an absence of conditions, each in its own language, IGN grants Licence Ouverte 2.0 over the
|
|
418
432
|
Base Adresse Nationale, Catastro and Bizkaia grant their own
|
|
419
|
-
access-and-use terms, and Flanders grants its own model
|
|
420
|
-
attribution the sole condition
|
|
421
|
-
property in works created from the data. The Directive mandates the data rather than its
|
|
433
|
+
access-and-use terms, and Flanders grants its own model license for no-charge re-use. Article 4 of that
|
|
434
|
+
license makes attribution the sole condition. Article 5 leaves the licensee holding the intellectual
|
|
435
|
+
property in works created from the data. The Directive mandates the data rather than its license.
|
|
422
436
|
|
|
423
|
-
**A publisher may state a
|
|
437
|
+
**A publisher may state a license that covers part of what it publishes.** SYKE's record reads
|
|
424
438
|
`Avoimen tiedon osalta Creative Commons Nimeä 4.0 Kansainvälinen … Muiden tietotuotteiden osalta
|
|
425
439
|
käyttöehdot on määritelty tietoluvassa`: CC BY 4.0 for the open-data part, and a data permit for other
|
|
426
|
-
products.
|
|
427
|
-
terms page
|
|
428
|
-
|
|
440
|
+
products. A reading of the first clause without the second would grant what the second withholds.
|
|
441
|
+
The publisher's own terms page therefore decides which files are which, and the two files this
|
|
442
|
+
register records have the name `open_address`.
|
|
443
|
+
Finland's other address publication states the permit requirement in its own text: the population register's
|
|
429
444
|
building and dwelling data reads `Tietojen käyttöön saaminen edellyttää Digi- ja väestötietoviraston
|
|
430
445
|
myöntämää käyttöoikeuslupaa`.
|
|
431
446
|
|
|
432
|
-
A member state's WFS states its own access terms in fields that are not the data's
|
|
447
|
+
A member state's WFS states its own access terms in fields that are not the data's license. `Fees`
|
|
433
448
|
and `AccessConstraints` in a capabilities document describe what calling the service costs, and the
|
|
434
449
|
grant over the data sits in the ISO 19139 dataset record the document links through
|
|
435
450
|
`inspire_dls:SpatialDataSetIdentifier`. Estonia's capabilities read `puudub` in both fields while its
|
|
436
451
|
dataset record grants CC0 1.0, and Slovakia's read `NONE` while its register record states that no
|
|
437
|
-
conditions apply and
|
|
438
|
-
Flanders states the price of the service in `Fees` and sends the reader to
|
|
439
|
-
the data's terms
|
|
440
|
-
`No constraints` while its dataset record states
|
|
441
|
-
record is linked per feature type rather than through the
|
|
442
|
-
`inspire_dls:SpatialDataSetIdentifier` states the unreplaced placeholder `mycode`.
|
|
452
|
+
conditions apply and identifies CC0. A reader that takes the service fields as the license
|
|
453
|
+
understates both grants. Flanders states the price of the service in `Fees` and sends the reader to
|
|
454
|
+
each dataset's metadata for the data's terms. That metadata identifies its model license. Poland's
|
|
455
|
+
read `No fees` and `No constraints` while its dataset record states
|
|
456
|
+
`Brak warunków dostępu i użytkowania`. That record is linked per feature type rather than through the
|
|
457
|
+
extended capabilities. The capabilities' `inspire_dls:SpatialDataSetIdentifier` states the unreplaced placeholder `mycode`.
|
|
443
458
|
|
|
444
459
|
**A service's own count can be wrong in more than one direction at once.** Poland states three
|
|
445
|
-
numbers for one type and none is its count: a bare `resultType=hits` answers 1000,
|
|
446
|
-
MapServer
|
|
460
|
+
numbers for one type and none is its count: a bare `resultType=hits` answers 1000, the
|
|
461
|
+
MapServer default feature cap. The same request at `startIndex=1` answers 1, and a feature page
|
|
447
462
|
answers `unknown`. `countWFSFeaturesByPaging` in `@mailwoman/core/api` measures such a type by asking
|
|
448
463
|
which indices hold a feature, doubling then bisecting, and a page straddling the last index confirms
|
|
449
464
|
the boundary. Poland's type holds 8,625,921 features, measured in 51 requests. That function refuses
|
|
@@ -451,42 +466,42 @@ a service which answers two different `startIndex` values with the same leading
|
|
|
451
466
|
paging an ignored parameter would measure the page cap instead.
|
|
452
467
|
|
|
453
468
|
**`Fees` and `AccessConstraints` can disagree in the same document.** Austria's BEV Addresses service
|
|
454
|
-
states `Fees` `no conditions apply` and `AccessConstraints` `restricted`, and its path
|
|
455
|
-
customer token.
|
|
456
|
-
|
|
469
|
+
states `Fees` `no conditions apply` and `AccessConstraints` `restricted`, and its path contains a
|
|
470
|
+
customer token. A reader of the fee field without the access field records that service as free when
|
|
471
|
+
its own access field states a restriction, so both are read before a service is called reachable.
|
|
457
472
|
|
|
458
473
|
**An HTTP 422 is a validation message naming the fix, where a 404 is an absence.** The export URL in
|
|
459
474
|
Slovakia's own capabilities document answers 422 with
|
|
460
|
-
`{"loc":["path","collection_record_id"],"msg":"value is not a valid uuid"}
|
|
461
|
-
the path where the capabilities put it in the query
|
|
462
|
-
`export_format` values,
|
|
475
|
+
`{"loc":["path","collection_record_id"],"msg":"value is not a valid uuid"}`. That message places the
|
|
476
|
+
id in the path where the capabilities put it in the query. A second 422 enumerates the permitted
|
|
477
|
+
`export_format` values, and they exclude the `gmd` the capabilities asks for. Those two bodies
|
|
463
478
|
turned a record recorded as unreachable into
|
|
464
479
|
`https://rpi.gov.sk/api/collection_record/86dea70e-a55b-4241-bd63-4024b3f76b72`. Its own
|
|
465
|
-
`downloadable_urls` holds the service's capabilities URL
|
|
466
|
-
|
|
480
|
+
`downloadable_urls` holds the service's capabilities URL. That URL ties the record to the service,
|
|
481
|
+
rather than a title match.
|
|
467
482
|
|
|
468
|
-
**A dataset record may
|
|
483
|
+
**A dataset record may state more than one instrument, in separate constraint blocks that answer
|
|
469
484
|
different questions.** The Flemish Adressen record states `Geen beperkingen op de publieke toegang` in
|
|
470
|
-
its access block, and in its re-use block names four things: the model
|
|
485
|
+
its access block, and in its re-use block names four things: the model license for no-charge re-use, two
|
|
471
486
|
instruments governing public-task use by GDI-Vlaanderen participants and by bodies that are not
|
|
472
|
-
participants, and the attribution string the licensor requires, `Bron: Digitaal Vlaanderen`.
|
|
473
|
-
those blocks loses which instrument governs which re-user.
|
|
487
|
+
participants, and the attribution string the licensor requires, `Bron: Digitaal Vlaanderen`. A reader
|
|
488
|
+
that flattens those blocks loses which instrument governs which re-user.
|
|
474
489
|
|
|
475
490
|
**An identifier that will not serve its own text may serve it under a file extension.** The Flemish
|
|
476
|
-
|
|
491
|
+
license identifier answers `text/html`, `text/turtle`, `application/ld+json` and `application/rdf+xml`
|
|
477
492
|
with one 541,080-byte application rendering to its title. The same path with `.ttl` appended returns
|
|
478
|
-
the RDF
|
|
479
|
-
|
|
493
|
+
the RDF. The RDF states `cc:requires cc:Attribution`, and the record's `rdfs:seeAlso` gives the page
|
|
494
|
+
that publishes the nine articles.
|
|
480
495
|
|
|
481
496
|
Denmark also shows that the theme is not always GML: it serves a GeoPackage, so its `addressRoles`
|
|
482
497
|
is keyed by a table name rather than an element name.
|
|
483
498
|
|
|
484
499
|
The eighth is Italy's ANAC procurement release, and it is eligible for a different reason.
|
|
485
500
|
`internal/strategy/personal-data-policy-2026-09-30.md` records the operator's decision that a corpus
|
|
486
|
-
|
|
501
|
+
contains the addresses of organizations and no address of an identifiable natural person. Italy's
|
|
487
502
|
release names natural persons among its suppliers and records an address for none of them: 0 of
|
|
488
|
-
11,075 supplier parties
|
|
489
|
-
or `payer`, both Italian public entities. Its `addressRoles`
|
|
503
|
+
11,075 supplier parties have an `address` object, and every address in the release sits on `buyer`
|
|
504
|
+
or `payer`, both Italian public entities. Its `addressRoles` lists only those two fields. That
|
|
490
505
|
reading is `assessed` rather than `absent`, because the publication does name people.
|
|
491
506
|
|
|
492
507
|
The other `verified-corpus` sources are business registers and procurement portals where the address
|
|
@@ -494,19 +509,19 @@ belongs to a party, and some of those parties are natural persons trading on the
|
|
|
494
509
|
policy document records why a name-shape filter, an organization identifier and a diffusion-status
|
|
495
510
|
filter each fail to separate the two kinds of party.
|
|
496
511
|
|
|
497
|
-
Every measured source places the role differently
|
|
498
|
-
than one role per source.
|
|
512
|
+
Every measured source places the role differently. For that reason the field is a map keyed by a path
|
|
513
|
+
rather than one role per source.
|
|
499
514
|
|
|
500
|
-
- AusTender
|
|
501
|
-
- Companies House
|
|
502
|
-
- Catastro's INSPIRE AD theme
|
|
515
|
+
- AusTender has two roles in two column families on one record.
|
|
516
|
+
- Companies House has one role in one family.
|
|
517
|
+
- Catastro's INSPIRE AD theme has one role on the record itself rather than in a column. The
|
|
503
518
|
feature is the address, so the map is keyed by the feature's element name, `AD:Address`.
|
|
504
|
-
- SIRENE
|
|
505
|
-
unit's seat, so the same columns are a `registered-office` address on one row and a `facility` address
|
|
519
|
+
- SIRENE records the role in a row-level flag. `etablissementSiege` states whether the establishment is
|
|
520
|
+
its legal unit's seat, so the same columns are a `registered-office` address on one row and a `facility` address
|
|
506
521
|
on the next.
|
|
507
522
|
- An OCDS release keys addresses by party role, and the roles differ per publisher. Germany uses fifteen
|
|
508
|
-
where AusTender uses two. Italy's `supplier` role
|
|
509
|
-
`buyer`'s and a `payer`'s
|
|
523
|
+
where AusTender uses two. Italy's `supplier` role has no address at all, so its addresses are a
|
|
524
|
+
`buyer`'s and a `payer`'s. That is AusTender's shape inverted. Chile records a region and neither a
|
|
510
525
|
postcode nor a locality on any of 544,754 party objects.
|
|
511
526
|
|
|
512
527
|
An OCDS role takes a role here only where OCDS's own codelist defines it. Germany's eight extension roles
|
|
@@ -515,15 +530,15 @@ take none rather than a guessed one.
|
|
|
515
530
|
## `training-manifests/<version>.json` — what reached one base corpus (#2375)
|
|
516
531
|
|
|
517
532
|
The frozen training manifest `buildCorpus` writes beside the corpus it produced, copied here so a release
|
|
518
|
-
path can read it. Each file
|
|
519
|
-
the register decision in force at build time
|
|
533
|
+
path can read it. Each file lists the sources that contributed, their row counts, their license label, and
|
|
534
|
+
the register decision in force at build time. It also stores a `contentDigest` over its own contents. The
|
|
520
535
|
corpus `MANIFEST.json` records the same digest as `training_manifest_digest`, so a committed copy that
|
|
521
|
-
drifted from the build it claims is detectable
|
|
536
|
+
drifted from the build it claims is detectable. `auditTrainingManifest` recomputes the digest and compares
|
|
522
537
|
the source rows against `totalRows`.
|
|
523
538
|
|
|
524
539
|
`mwops release rights-audit` looks a manifest up by the `training.corpus_version` a package's
|
|
525
|
-
`model-card.json` names. Observed 2026-09-28: of the twelve `neural-weights-*` cards, ten
|
|
526
|
-
`training.corpus_version` at all and the two that do
|
|
540
|
+
`model-card.json` names. Observed 2026-09-28: of the twelve `neural-weights-*` cards, ten have no
|
|
541
|
+
`training.corpus_version` at all and the two that do hold prose rather than a version, so no card yet
|
|
527
542
|
resolves to a file here. The record exists for the corpus the next base model trains on.
|
|
528
543
|
|
|
529
544
|
`v0.7.0-de-holdout.json` is byte-identical to
|
|
@@ -546,11 +561,11 @@ establishes is which path each adapter was pointed at. A corpus `MANIFEST.json`
|
|
|
546
561
|
`assembleOverlayManifest` writes it next to the `MANIFEST.json` it produced, and the copy here is the one a
|
|
547
562
|
later assembly on the same base reads. The `v0.6.0-register-surface` plan (66 files in three lists) was
|
|
548
563
|
produced by three scripts in a session scratch directory with the old data root written into them, and was
|
|
549
|
-
|
|
564
|
+
passed into the next assembly as a hand-written list.
|
|
550
565
|
|
|
551
|
-
No plan exists for `v0.7.0-de-holdout`. Its overlays were assembled before `assembleOverlayManifest`
|
|
566
|
+
No plan exists for `v0.7.0-de-holdout`. Its corpus overlays were assembled before `assembleOverlayManifest`
|
|
552
567
|
wrote one, and its `MANIFEST.json` records the `split`, `source` and `path` of each of the 71 labeled
|
|
553
|
-
|
|
568
|
+
overlay files while recording the `route` of none. A plan reconstructed from the manifest would therefore assert
|
|
554
569
|
a route it cannot read, and that reconstruction is the mechanism `OverlayPlan`'s docstring names: a
|
|
555
570
|
hand-placed split reached `v0.6.0-register-surface` through the caller route and put 770 of DE's
|
|
556
571
|
validation `source_id`s in train, and the filename was the only record that showed it (#2359). The next
|
|
@@ -570,15 +585,15 @@ weight, the sampler, `augment_exclude_sources` and every overlay merged after th
|
|
|
570
585
|
written. `effective-manifest.run.ts` derives it from the frozen manifest and an `audit_epoch_mixture`
|
|
571
586
|
output, and `mwops release rights-audit` reads `<corpus_version>.effective.json` beside the frozen file.
|
|
572
587
|
|
|
573
|
-
Each row
|
|
588
|
+
Each row records the corpus row count, the config weight, the rows drawn, the rows emitted after
|
|
574
589
|
augmentation, and the reason an excluded source was excluded. `emittedButUnrecorded` holds the sources the
|
|
575
590
|
epoch emitted that the frozen manifest does not name, with their row counts.
|
|
576
591
|
|
|
577
592
|
`v0.7.0-de-holdout.effective.json` reads the corpus through
|
|
578
593
|
`corpus-python/src/mailwoman_train/configs/v7.0.0-de-holdout-60k.yaml` at seed 43 over 1,000,000 draws. Of
|
|
579
|
-
the eleven sources the frozen manifest names, nine drew rows and two
|
|
594
|
+
the eleven sources the frozen manifest names, nine drew rows and two have weight 0.0. Thirty-nine further
|
|
580
595
|
sources were emitted under no frozen-manifest entry, 886,620 of the 1,000,000 rows, because the overlays
|
|
581
|
-
|
|
596
|
+
that contain them were merged after that manifest was written.
|
|
582
597
|
|
|
583
598
|
## The `surface` column on a corpus built up to `v0.7.0-de-holdout`
|
|
584
599
|
|
|
@@ -602,9 +617,9 @@ From the next build, an adapter that assembles a line declares `rendered`, a rec
|
|
|
602
617
|
and `attested` is reserved for a source that publishes the written line itself. No adapter in the tree
|
|
603
618
|
emits `attested` today, so a corpus reporting zero attested rows is a correct census rather than a defect.
|
|
604
619
|
|
|
605
|
-
|
|
606
|
-
|
|
607
|
-
and `wof-admin` and
|
|
620
|
+
A measurement on `v0.7.0-de-holdout`, taken when this defect was found, shows that GB holds 813,781
|
|
621
|
+
rows with a street, all of which read `composed`, from `rendered-gb`. Its 13,000,821 rows reading `attested` come from
|
|
622
|
+
`wof-postalcode` and `wof-admin` and have no street.
|
|
608
623
|
|
|
609
624
|
## `reviewed-ve-postcode-tuples.json` — reviewed Venezuelan postcode placement (#1821)
|
|
610
625
|
|
|
@@ -622,15 +637,15 @@ postcode-to-place join unchanged.
|
|
|
622
637
|
## `sub-venue-lexicon.json` — the sub-venue designator lexicon (#35, waves 1–2)
|
|
623
638
|
|
|
624
639
|
The vocabulary a corpus recipe (and eventually the span proposer) reads to recognize `Terminal 5`,
|
|
625
|
-
`North Terminal`, `Concourse B`, `第1ターミナル` as venue-interior structure.
|
|
626
|
-
|
|
627
|
-
|
|
628
|
-
decode weight would take a bias scale near 11 nats where the stronger designator+identifier evidence
|
|
640
|
+
`North Terminal`, `Concourse B`, `第1ターミナル` as venue-interior structure. The file is
|
|
641
|
+
generated and is never edited by hand. `docs/records/engineering/sub-venue-corpus-task.mdx` explains
|
|
642
|
+
why it exists. In short, the `unit` tag was never taught the modifier+designator shape, so closing the
|
|
643
|
+
class by decode weight would take a bias scale near 11 nats where the stronger designator+identifier evidence
|
|
629
644
|
needed 6.
|
|
630
645
|
|
|
631
646
|
The one hand-authored input is `corpus/src/tools/sub-venue-promotions.ts`, the curation ledger. The
|
|
632
647
|
builder cannot decide that Spanish `terminal` is safe and British `hall` is not, because that is a
|
|
633
|
-
|
|
648
|
+
judgment about a language's confounds. The builder applies those decisions.
|
|
634
649
|
|
|
635
650
|
### Regenerate
|
|
636
651
|
|
|
@@ -651,12 +666,12 @@ npx oxfmt corpus/data/sub-venue-lexicon.json
|
|
|
651
666
|
```
|
|
652
667
|
|
|
653
668
|
`$E` is `$MAILWOMAN_DATA_ROOT/sub-venue/extracts`. `--extracts` takes `REGION=path` pairs because the
|
|
654
|
-
region is the axis every curation decision is taken on, and no extract filename
|
|
655
|
-
`ile-de-france` is FR, `great-britain` is GB. A bare path lands region `""`, whose surfaces can never
|
|
669
|
+
region is the axis every curation decision is taken on, and no extract filename states it reliably.
|
|
670
|
+
`ile-de-france` is FR, and `great-britain` is GB. A bare path lands region `""`, whose surfaces can never
|
|
656
671
|
be promoted.
|
|
657
672
|
|
|
658
|
-
The oxfmt pass is required
|
|
659
|
-
reproduce. The generator is byte-deterministic and oxfmt is too, so the artifact is reproducible from
|
|
673
|
+
The oxfmt pass is required. Committed JSON is oxfmt-clean, and raw `JSON.stringify` cannot
|
|
674
|
+
reproduce that formatting. The generator is byte-deterministic and oxfmt is too, so the artifact is reproducible from
|
|
660
675
|
the same fetch outputs. `sub-venue-lexicon.test.ts` pins determinism directly.
|
|
661
676
|
|
|
662
677
|
The extract JSONLs and the `.osm.pbf` files are build inputs under
|
|
@@ -675,32 +690,32 @@ The extract JSONLs and the `.osm.pbf` files are build inputs under
|
|
|
675
690
|
| Overture Places | `poi.db` spatial layer, vintage `2026-05-20.0` | CDLA-Permissive-2.0 | 9,219 |
|
|
676
691
|
|
|
677
692
|
**The ODbL question, unchanged from wave 1 and still open for the recipe output.** The committed artifact
|
|
678
|
-
contains no OSM geometry and no OSM row. What survives the OSM
|
|
679
|
-
|
|
680
|
-
`examples` are gate
|
|
681
|
-
a substantial extraction from a database, so this table is not treated as a
|
|
682
|
-
reading matches `osm/README.md`'s
|
|
683
|
-
rather than
|
|
684
|
-
|
|
693
|
+
contains no OSM geometry and no OSM row. What survives the OSM read is surface counts, such as the count of 1,215
|
|
694
|
+
Japanese feature names that contain the token `ターミナル`. The `identifierShapes` distribution also
|
|
695
|
+
survives. Its `examples` are `aeroway=gate` `ref` strings (`B32`, `1A`, `16-18`). Facts and short
|
|
696
|
+
factual strings are not a substantial extraction from a database, so this table is not treated as a
|
|
697
|
+
Derived Database. That reading matches `osm/README.md`'s position that the ODbL obligation attaches to
|
|
698
|
+
the built recipe output rather than to code. **Whether a corpus recipe output built from OSM rows is a
|
|
699
|
+
derived work is still an open question.** That question blocks step 4 rather than this table.
|
|
685
700
|
|
|
686
|
-
### What
|
|
701
|
+
### What each source is for, and what each cannot do
|
|
687
702
|
|
|
688
|
-
Overture and OSM fail differently,
|
|
703
|
+
Overture and OSM fail differently, so the build reads both.
|
|
689
704
|
|
|
690
705
|
- **Overture (`poi.db`)** is curated venue-interior naming. `concourse` appears 35 times in its
|
|
691
706
|
`airport_terminal` category against 4 in the whole Great Britain OSM extract, and 3 of those 4 are a
|
|
692
707
|
street called CONCOURSE WAY. **But poi.db is four countries** — US 11,521,612 / CA 794,418 /
|
|
693
|
-
FR 721,352 / MX 644,316, and
|
|
694
|
-
and es-MX and
|
|
708
|
+
FR 721,352 / MX 644,316, and no other country (measured 2026-08-05). It can attest en-US, en-CA, fr-FR
|
|
709
|
+
and es-MX and no other locale, so a zero count in it is evidence of absence in four countries rather than in
|
|
695
710
|
the world.
|
|
696
|
-
- **OSM** reaches any region with a Geofabrik extract, and
|
|
697
|
-
|
|
698
|
-
|
|
699
|
-
|
|
700
|
-
|
|
701
|
-
The Overture category set is measured rather than guessed. A
|
|
702
|
-
category,
|
|
703
|
-
predicts. `gas_station` leads the entire table with 12,996 hits, every one of them `station` inside
|
|
711
|
+
- **OSM** reaches any region with a Geofabrik extract, and it has the `name:<lang>` family. Every
|
|
712
|
+
non-Latin surface in this table comes from that family. OSM has no curated notion of "interior":
|
|
713
|
+
3,204 of Great Britain's 3,273 `hall` hits sit on a `public_transport=platform`, because a British
|
|
714
|
+
bus stop takes its name from the village hall it stands outside.
|
|
715
|
+
|
|
716
|
+
The Overture category set is measured rather than guessed. A scan of all 13,681,698 rows counted, per
|
|
717
|
+
category, the rows with a name that contain a designator token. The ranking does not follow what a
|
|
718
|
+
category name predicts. `gas_station` leads the entire table with 12,996 hits, every one of them `station` inside
|
|
704
719
|
"Holiday Station" or "Chevron Station Seward", and `shoe_store` contributes 708 hits of `wing` because
|
|
705
720
|
Red Wing sells boots. Four categories survived reading the distribution — `airport_terminal`,
|
|
706
721
|
`campus_building`, `pier`, `airport_lounge` — and `overture-subvenue.ts` lists the rejects with the
|
|
@@ -708,20 +723,20 @@ number that rejected them.
|
|
|
708
723
|
|
|
709
724
|
### The wave-1 defect this build fixes
|
|
710
725
|
|
|
711
|
-
Wave 1 attributed every harvested phrase to `row.designatorID`, the rule that matched the
|
|
712
|
-
|
|
713
|
-
**108 of its 133 OSM-derived surfaces
|
|
726
|
+
Wave 1 attributed every harvested phrase to `row.designatorID`, the rule that matched the feature.
|
|
727
|
+
A bus stop tagged `public_transport=platform` has a name such as "Village Hall" or "West Kensington".
|
|
728
|
+
**108 of its 133 OSM-derived surfaces therefore identified a different record than the one they pointed at**: the
|
|
714
729
|
shipped artifact claimed `west → platform`, `hall → platform`, `biggin → platform`, `salon →
|
|
715
730
|
platform`. Attribution now runs through a phrase → record index, and the row's own designator is kept
|
|
716
|
-
as `context
|
|
731
|
+
as `context`. A confound board needs that axis, since a `hall` on a platform is a bus stop
|
|
717
732
|
and a `hall` on a terminal is a hall.
|
|
718
733
|
|
|
719
734
|
### Head nouns: what the curation pass needed before it could start
|
|
720
735
|
|
|
721
|
-
Wikidata gives the
|
|
722
|
-
Spanish label is `terminal aeroportuaria
|
|
723
|
-
shipped 1,014 uncurated surfaces and could promote none of them
|
|
724
|
-
|
|
736
|
+
Wikidata gives the encyclopedic name of a concept rather than the designator as addressed. Q849706's
|
|
737
|
+
Spanish label is `terminal aeroportuaria`. The form on an envelope is `Terminal`. For that reason wave 1
|
|
738
|
+
shipped 1,014 uncurated surfaces and could promote none of them, because the table held no promotable
|
|
739
|
+
surface. Two derivations run before the harvest:
|
|
725
740
|
|
|
726
741
|
- **Latin script — the cognate test.** A token whose ASCII fold shares five leading characters with
|
|
727
742
|
the designator's own id. `terminal aeroportuaria` → `terminal`, `letištní terminál` → `terminál`,
|
|
@@ -729,99 +744,98 @@ table. Two derivations run before the harvest:
|
|
|
729
744
|
the record and, because Dutch `universiteit` is one, derived `universitario`, `universitaire` and
|
|
730
745
|
twenty more as head nouns of `campus`. Those are the modifier half, and they would have taught the
|
|
731
746
|
harvest to read "Ciudad Universitaria" as sub-venue structure.
|
|
732
|
-
- **Han / Kana / Hangul — the shared-substring test.** Every substring of length ≥ 2
|
|
747
|
+
- **Han / Kana / Hangul — the shared-substring test.** Every substring of length ≥ 2 that occurs in two or
|
|
733
748
|
more surfaces of the same record and language, maximal-only, capped at six. `ターミナル` is in none
|
|
734
|
-
of the five Japanese labels on its own
|
|
735
|
-
addresses
|
|
736
|
-
|
|
749
|
+
of the five Japanese labels on its own, because every label is a compound. It is the form Japanese
|
|
750
|
+
addresses use. No other step in the pipeline can produce it, and the Japan harvest attests it
|
|
751
|
+
1,215 times.
|
|
737
752
|
|
|
738
753
|
Run over every non-Latin phrase instead, the second derivation produced 90 fragments of Cyrillic,
|
|
739
754
|
Greek, Arabic, Thai, Burmese and Tamil words — `сгра`, `κτίρ`, `ิ่งก่อสร้า` — because those languages
|
|
740
755
|
have one surface per concept and the only shared substrings are pieces of one word. None could ever
|
|
741
|
-
be counted
|
|
756
|
+
be counted, because no source in reach attests a Thai or Burmese surface. The derivation is scoped to the four
|
|
742
757
|
scripts where it works.
|
|
743
758
|
|
|
744
759
|
The harvest also gained a script-aware match. `第1ターミナル` has no word boundaries, so the
|
|
745
|
-
token-boundary rule that
|
|
746
|
-
Han and Kana the match is a substring test, and the Germanic-compound objection does not transfer.
|
|
760
|
+
token-boundary rule that keeps Latin script from matching `Briggate` finds no match at all in Japanese.
|
|
761
|
+
For Han and Kana the match is a substring test, and the Germanic-compound objection does not transfer.
|
|
747
762
|
|
|
748
763
|
### The curation ledger
|
|
749
764
|
|
|
750
|
-
`curated: false` is still the default and
|
|
751
|
-
|
|
752
|
-
|
|
765
|
+
`curated: false` is still the default, and no surface is promoted automatically. A surface becomes
|
|
766
|
+
curated only when it matches a decision in `sub-venue-promotions.ts`. Each decision identifies a
|
|
767
|
+
designator, a phrase and a locale. The locale is part of the decision because the same token is a designator in one language and a disaster in another.
|
|
753
768
|
|
|
754
769
|
Every decision below is backed by a census in that locale's own data. `real` counts occurrences in
|
|
755
|
-
|
|
756
|
-
number is not a board, and "3,273 hits"
|
|
757
|
-
|
|
758
|
-
| designator | phrase | locale | real | confound | what the confound is
|
|
759
|
-
| ---------- | ------------ | ------ | ----: | -------: |
|
|
760
|
-
| gate
|
|
761
|
-
| gate
|
|
762
|
-
| hall | `hall` | en-GB | 0 | 3,273 | 3,204 bus stops
|
|
763
|
-
| hall | `hall` | en-US | 2,095 | 27,081 | City Hall, Kingdom Hall, event halls, dormitory halls
|
|
764
|
-
| hall | `hall` | fr-FR | 35 | 5 | three English `Town Hall` strings on `name:en` tags
|
|
765
|
-
| hall | `halle` | de-DE | 32 | 168 | the CITY Halle (Saale) / Halle (Westf), plus village halls
|
|
766
|
-
| pier | `pier` | en-GB | 120 | 44 | Pier Road / Street / Avenue / Terrace — street names
|
|
767
|
-
| pier | `pier` | en-US | 278 | 2,330 | Pier 1 Imports and franchises, seafood restaurants
|
|
768
|
-
| terminal | `terminal` | ca-ES | 15 | 0 |
|
|
769
|
-
| terminal | `terminal` | es-ES | 190 | 0 |
|
|
770
|
-
| terminal | `terminal` | fr-FR | 169 | 0 |
|
|
771
|
-
| terminal | `ターミナル` | ja-JP | 1,213 | 2 | two `ターミナル前` bus stops, arguably real
|
|
772
|
-
| wing | `wing` | en-GB | 23 | 6 | the Buckinghamshire village of Wing — Wing Close/Road
|
|
773
|
-
| wing | `wing` | en-US | 4 | 3,354 | Red Wing boots (676), chicken-wing restaurants (759)
|
|
774
|
-
| wing | `wing` | fr-FR | 0 | 26 | Wing Chun and Wing Tsun martial-arts clubs
|
|
775
|
-
|
|
776
|
-
|
|
777
|
-
0-of-3,273 in Great Britain and 35-of-40 in France
|
|
778
|
-
4-of-3,358 in the United States
|
|
779
|
-
|
|
780
|
-
**The test that separates a promote from a reject is whether
|
|
770
|
+
venue-interior naming. `confound` counts the rest, and the note states what the rest is. A bare
|
|
771
|
+
number is not a board, and "3,273 hits" did not tell a reader that 3,204 of them were bus stops.
|
|
772
|
+
|
|
773
|
+
| designator | phrase | locale | real | confound | what the confound is |
|
|
774
|
+
| ---------- | ------------ | ------ | ----: | -------: | -------------------------------------------------------------------- |
|
|
775
|
+
| `gate` | `flugsteig` | de-DE | 19 | 0 | none. An aviation-only term with no collision in German |
|
|
776
|
+
| `gate` | `porte` | fr-FR | 19 | 927 | the Portes of Paris and their Métro stations |
|
|
777
|
+
| hall | `hall` | en-GB | 0 | 3,273 | 3,204 bus stops that share a village hall's name, and Hall Lane/Road |
|
|
778
|
+
| hall | `hall` | en-US | 2,095 | 27,081 | City Hall, Kingdom Hall, event halls, dormitory halls |
|
|
779
|
+
| hall | `hall` | fr-FR | 35 | 5 | three English `Town Hall` strings on `name:en` tags |
|
|
780
|
+
| hall | `halle` | de-DE | 32 | 168 | the CITY Halle (Saale) / Halle (Westf), plus village halls |
|
|
781
|
+
| pier | `pier` | en-GB | 120 | 44 | Pier Road / Street / Avenue / Terrace — street names |
|
|
782
|
+
| pier | `pier` | en-US | 278 | 2,330 | Pier 1 Imports and franchises, seafood restaurants |
|
|
783
|
+
| terminal | `terminal` | ca-ES | 15 | 0 | none |
|
|
784
|
+
| terminal | `terminal` | es-ES | 190 | 0 | none |
|
|
785
|
+
| terminal | `terminal` | fr-FR | 169 | 0 | none |
|
|
786
|
+
| terminal | `ターミナル` | ja-JP | 1,213 | 2 | two `ターミナル前` bus stops, arguably real |
|
|
787
|
+
| wing | `wing` | en-GB | 23 | 6 | the Buckinghamshire village of Wing — Wing Close/Road |
|
|
788
|
+
| wing | `wing` | en-US | 4 | 3,354 | Red Wing boots (676), chicken-wing restaurants (759) |
|
|
789
|
+
| wing | `wing` | fr-FR | 0 | 26 | Wing Chun and Wing Tsun martial-arts clubs |
|
|
790
|
+
|
|
791
|
+
The ledger holds nine promotions and six rejections. Three pairs show why the decision is per-locale.
|
|
792
|
+
`hall` is 0-of-3,273 in Great Britain and 35-of-40 in France. `wing` is 23-of-29 in Great Britain and
|
|
793
|
+
4-of-3,358 in the United States. `pier` is promotable in Great Britain and not in the United States.
|
|
794
|
+
|
|
795
|
+
**The test that separates a promote from a reject is whether shape can isolate the confound**, and it
|
|
781
796
|
is checked by enumeration rather than asserted. `halle` keeps a 168-hit confound and is promoted
|
|
782
797
|
because dumping all 32 `<halle> <identifier>` hits returns numbered factory, trade-fair and airport
|
|
783
798
|
halls — VW Halle 42, Audi GVZ Halle G, Messe West Halle 8 — and not one instance of the city of
|
|
784
799
|
240,000 people with the same name. `porte` has a smaller confound ratio in that bucket and is
|
|
785
800
|
rejected, because 17 of its 36 shape hits are Porte Saint-Martin, Porte Saint-Denis and Porte
|
|
786
|
-
Notre-Dame.
|
|
801
|
+
Notre-Dame. The same test rejected `pier` for en-US, because Pier 1 Imports has the designator+identifier shape.
|
|
787
802
|
|
|
788
803
|
**A caveat on `wing`, the designator the corpus task's board rests on.** 24 of its 29 Great Britain
|
|
789
|
-
hits sit on a `public_transport=platform`, because British bus stops
|
|
804
|
+
hits sit on a `public_transport=platform`, because British bus stops take their names from the hospital wing
|
|
790
805
|
they serve. The extractor maps no building wings at all: `aile` in France is **0 hits**, `ala` in
|
|
791
806
|
Spain **0**, `flügel` in Germany **0**. The en-GB evidence is a property of British stop naming rather
|
|
792
807
|
than of a source that has wings in it, and the localized wing surfaces the corpus task asked for are
|
|
793
808
|
out of reach until the extractor gains an `indoor=*` rule or the lexicon gains a hand-seeded surface
|
|
794
809
|
list.
|
|
795
810
|
|
|
796
|
-
**A rejection of a
|
|
797
|
-
English vocabulary with no locale
|
|
811
|
+
**A rejection of a shipped designator is advisory.** `neural/venue-structure.ts` holds a flat
|
|
812
|
+
English vocabulary with no per-locale filter, and `wing`, `terminal` and `concourse` are in it. This table
|
|
798
813
|
cannot un-ship them: the `wing` / en-US rejection tells a recipe author which locale to leave out of a
|
|
799
|
-
generated line, and it does
|
|
800
|
-
vocabulary
|
|
801
|
-
want that does not exist yet.
|
|
814
|
+
generated line, and it does not stop the span proposer from firing on "Red Wing". A per-locale filter on the shipped
|
|
815
|
+
vocabulary belongs to step 4. It is the largest missing piece the recipe will need.
|
|
802
816
|
|
|
803
|
-
The
|
|
817
|
+
The ledger does apply to every surface the lexicon adds. `pier` is promoted for en-GB and rejected for
|
|
804
818
|
en-US, and because it is not in the shipped list, the rejection has teeth: the region-free English
|
|
805
819
|
`pier` surface stays `curated: false` and only the 164 Great Britain attestations are marked usable.
|
|
806
820
|
|
|
807
821
|
### Identifier shapes are per-region now
|
|
808
822
|
|
|
809
823
|
`Gate A12` is a rendering rather than a string anyone wrote down: all 658 Great Britain `aeroway=gate`
|
|
810
|
-
features but 13 are unnamed and
|
|
811
|
-
than a phrase list
|
|
812
|
-
|
|
813
|
-
|
|
814
|
-
| region |
|
|
815
|
-
| ------ |
|
|
816
|
-
| GB |
|
|
817
|
-
| JP |
|
|
818
|
-
| FR |
|
|
819
|
-
| DE |
|
|
820
|
-
| ES |
|
|
821
|
-
|
|
822
|
-
Britain and Japan
|
|
823
|
-
is the outlier
|
|
824
|
-
|
|
824
|
+
features but 13 are unnamed and have only a `ref`. The table therefore stores a distribution rather
|
|
825
|
+
than a phrase list. The distribution differs by country far more than the shared vocabulary suggests,
|
|
826
|
+
so a recipe that generates `Gate <ref>` for a French address has to sample France's:
|
|
827
|
+
|
|
828
|
+
| region | `ref`s | most common shape | second | third |
|
|
829
|
+
| ------ | -----: | ---------------------- | ---------------: | --------------- |
|
|
830
|
+
| GB | 655 | digit 463 (71%) | letter-digit 19% | digit-letter 8% |
|
|
831
|
+
| JP | 450 | digit 402 (89%) | digit-letter 5% | other 3% |
|
|
832
|
+
| FR | 628 | letter-digit 385 (61%) | digit 29% | letter 5% |
|
|
833
|
+
| DE | 642 | letter-digit 387 (60%) | digit 18% | range 11% |
|
|
834
|
+
| ES | 493 | letter-digit 189 (38%) | range 35% | digit 15% |
|
|
835
|
+
|
|
836
|
+
Britain and Japan use bare numbers for `aeroway=gate` refs. France and Germany use a letter and then
|
|
837
|
+
a number (`A37`, `B05`). Spain is the outlier: a third of its refs are a range (`B18-B20`, `D42-D43`).
|
|
838
|
+
No other country uses ranges at that rate. A generator that samples Great Britain's 71%-bare-digit shape into a Spanish line
|
|
825
839
|
produces a plausible string that is wrong about Spain.
|
|
826
840
|
|
|
827
841
|
### The seed duplicates `neural/venue-structure.ts`
|
|
@@ -829,5 +843,5 @@ produces a plausible string that is wrong about Spain.
|
|
|
829
843
|
`@mailwoman/corpus` does not depend on `@mailwoman/neural`, so the shipped designator and modifier
|
|
830
844
|
lists are re-declared in `sub-venue-lexicon.ts`. That is a drift surface, and
|
|
831
845
|
`sub-venue-lexicon.test.ts` pins both lists literally so a change in either place fails a test rather
|
|
832
|
-
than passing silently. The
|
|
846
|
+
than passing silently. The planned change is still the reverse direction: have `neural/venue-structure.ts`
|
|
833
847
|
read a committed lexicon extract and delete both copies.
|