@mailwoman/resolver-wof-sqlite 9.1.0 → 9.3.0

package/README.md

@@ -15,11 +15,11 @@ Requires Node 22+ for built-in `node:sqlite`.
 ## Quick start
 
 ```ts
-import { WofSqlitePlaceLookup } from "@mailwoman/resolver-wof-sqlite"
+import { WOFSQLitePlaceLookup } from "@mailwoman/resolver-wof-sqlite"
 
-const lookup = new WofSqlitePlaceLookup({
+using lookup = new WOFSQLitePlaceLookup({
 	databasePath: "/path/to/whosonfirst-data-admin-us-latest.db",
-	buildFts: true, // build the FTS5 index on first open (one-time cost)
+	buildFTS: true, // build the FTS5 index on first open (one-time cost)
 })
 
 const candidates = await lookup.findPlace({
@@ -31,42 +31,59 @@ const candidates = await lookup.findPlace({
 for (const c of candidates) {
 	console.log(c.id, c.name, c.country, c.lat, c.lon, "score:", c.score)
 }
+```
+
+## A database that cannot answer says so on construction
+
+Databases are `ATTACH`ed by a schema name **derived from the filename**, and queries route to them by matching that name against the requested placetype — `postalcode_us` serves `postalcode`. Two ways that used to fail without logging, and both now throw when you build the lookup:
+
+- The name does not route. `postcode-ca-overture.db` derives `postcode_ca_overture`, and the router tests `startsWith("postalcode_")` — **"postcode" is not "postalcode"**. It held 843,739 Canadian codes and answered every query with zero hits, which is indistinguishable from "this country has no places".
+- The database carries `spr` but no `place_search`. It routes, then dies mid-`SELECT`.
 
-lookup.close()
+The predicate is the **table, not the filename**: a database carrying `spr` is claiming to be a place database, and every lookup path here reaches the FTS index. A relation-table database like `postcode-locality-<cc>.db` carries no `spr`, never makes that claim, and is exempt — which is what keeps the documented default database list working.
+
+```
+WOFSQLitePlaceLookup: …/postcode-ca-overture.db carries "spr" but no "place_search" table, so it
+cannot serve a lookup. Build it with the FTS index, or leave it out — it is usable as a BUILD input
+either way. Its schema name "postcode_ca_overture" also matches no routed placetype (postalcode,
+locality, region, county, country, venue), so it would never have been queried even with the table
+— check the filename's spelling.
 ```
 
-## Multi-shard (admin + postcode in one connection)
+The second sentence appears only when the name routes nowhere. It is the half that turns zero hits into a diagnosis.
+
+## Multiple databases (admin + postcode in one connection)
 
-Pass an array of paths to open multiple WOF shards on a single connection — each is opened as a
+Pass an array of paths to open multiple WOF databases on a single connection — each is opened as a
 separate SQLite schema via `ATTACH DATABASE`. Schema names auto-derive from filenames
 (`whosonfirst-data-admin-us-latest.db` → `admin_us`, `whosonfirst-data-postalcode-us-latest.db` →
 `postalcode_us`). Queries route by `placetype` — a `postalcode` query goes to the
-`postalcode_us` shard automatically, everything else hits main.
+`postalcode_us` database automatically, everything else hits main.
 
 ```ts
-const lookup = new WofSqlitePlaceLookup({
+const lookup = new WOFSQLitePlaceLookup({
 	databasePath: ["/data/wof/whosonfirst-data-admin-us-latest.db", "/data/wof/whosonfirst-data-postalcode-us-latest.db"],
 })
 
-await lookup.findPlace({ text: "Springfield", placetype: "locality" }) // → admin shard
-await lookup.findPlace({ text: "62701", placetype: "postalcode" }) // → postcode shard
+await lookup.findPlace({ text: "Springfield", placetype: "locality" }) // → admin extract
+await lookup.findPlace({ text: "62701", placetype: "postalcode" }) // → postcode extract
 ```
 
 Override schema names or routing explicitly when needed:
 
 ```ts
-new WofSqlitePlaceLookup({
+new WOFSQLitePlaceLookup({
 	databasePath: ["/data/wof/admin.db", { path: "/data/oddly-named.db", schemaName: "pc", placetypes: ["postalcode"] }],
 })
 ```
 
-Cross-shard `UNION` queries are not supported in one `findPlace` call — BM25 scores aren't
+Cross-database `UNION` queries are not supported in one `findPlace` call — BM25 scores aren't
 comparable across separately-indexed corpora. Issue two `findPlace` calls and merge in your
 caller if you need that.
 
 ## Getting the WOF SQLite distribution
 
-The Geocode Earth team mirrors WOF SQLite distributions at <https://data.geocode.earth/wof/dist/sqlite/>. The two relevant shards for v1:
+The Geocode Earth team mirrors WOF SQLite distributions at <https://data.geocode.earth/wof/dist/sqlite/>. The two relevant distributions for v1:
 
 | Distribution                                   | Size (bz2) | Use                                                               |
 | ---------------------------------------------- | ---------- | ----------------------------------------------------------------- |
@@ -83,7 +100,7 @@ bunzip2 whosonfirst-data-admin-us-latest.db.bz2
 
 Upstream WOF SQLite distributions ship a `places` table but **not** an FTS5 index. The resolver needs FTS5 to do fast prefix + token-bag matching. Two options:
 
-1. **`buildFts: true` on construction** — builds the index lazily on first open. Cost is one-time but expensive (~minutes on the full US admin shard). Use for prototyping.
+1. **`buildFTS: true` on construction** — builds the index lazily on first open. Cost is one-time but expensive (~minutes on the full US admin database). Use for prototyping.
 2. **Pre-build the index with `mailwoman gazetteer build fts`** — ship the DB with the index included so first-open is fast. Recommended for production.
 
 ### `mailwoman gazetteer build fts`
@@ -143,7 +160,7 @@ What survives in the slim DB:
 
 What gets dropped: the `geojson` table, which is build-time only — `lookup.ts` never reads it at query time, and it accounts for ~95% of the on-disk size. The `place_population` aux table consumes `wof:population` from geojson before we drop it.
 
-`WofSqlitePlaceLookup` opens the slim DB without any code change. Out-of-set queries (a locality not in the top-K) correctly return zero hits.
+`WOFSQLitePlaceLookup` opens the slim DB without any code change. Out-of-set queries (a locality not in the top-K) correctly return zero hits.
 
 You can also build the index programmatically via the package's `./fts` subpath:
 
@@ -151,12 +168,11 @@ You can also build the index programmatically via the package's `./fts` subpath:
 import { DatabaseSync } from "node:sqlite"
 import { buildPlaceSearchFts } from "@mailwoman/resolver-wof-sqlite/fts"
 
-const db = new DatabaseSync("/path/to/wof.db")
+using db = new DatabaseSync("/path/to/wof.db")
 const { created, indexedRows, durationMs } = buildPlaceSearchFts(db, {
 	drop: false,
 	onProgress: (phase, detail) => console.log(phase, detail),
 })
-db.close()
 ```
 
 ## Ranking
@@ -209,7 +225,7 @@ R*Tree because it computes haversine distance against the centroid columns direc
 All weights are configurable via the second ctor argument:
 
 ```ts
-new WofSqlitePlaceLookup({ databasePath }, { countryMatchBoost: 0.5 })
+new WOFSQLitePlaceLookup({ databasePath }, { countryMatchBoost: 0.5 })
 ```
 
 Defaults are in `lookup.ts::DEFAULT_WEIGHTS`.
@@ -242,11 +258,11 @@ This package itself is AGPL-3.0; the WOF data it indexes is CC-BY 4.0. The two l
 
 `resolver-wof-sqlite/integration.test.ts` exercises the resolver against a real WOF SQLite distribution. The suite is **skipped** when no DB is present — set `MAILWOMAN_WOF_DB` to override the lookup path, otherwise it defaults to `/mnt/playpen/mailwoman-data/wof/whosonfirst-data-admin-us-latest.db` (the canonical lab location). CI runs against the fixture-only suites; operators with real WOF data locally get an extra layer of validation.
 
-Coverage includes: placetype filtering, country filtering, the empty-result case, FTS5 special-character sanitization, Japanese alt-name resolution, parent-constrained lookup, and a performance budget (`findPlace` < 250 ms against the 142 k-row US admin shard).
+Coverage includes: placetype filtering, country filtering, the empty-result case, FTS5 special-character sanitization, Japanese alt-name resolution, parent-constrained lookup, and a performance budget (`findPlace` < 250 ms against the 142 k-row US admin database).
 
 ## Concurrency model
 
-This package opens a single `node:sqlite` connection per `WofSqlitePlaceLookup` instance. SQLite is single-writer / many-reader; the Kysely wrapper around the connection serializes all queries through a mutex. For high-concurrency HTTP servers, instantiate one resolver per request handler or per pool slot — sharing a single instance across concurrent requests is fine (queries queue) but won't parallelize across cores.
+This package opens a single `node:sqlite` connection per `WOFSQLitePlaceLookup` instance. SQLite is single-writer / multiple-reader; the Kysely wrapper around the connection serializes all queries through a mutex. For high-concurrency HTTP servers, instantiate one resolver per request handler or per pool slot — sharing a single instance across concurrent requests is safe (queries queue) but won't parallelize across cores.
 
 ## License
 

package/lib/address/index.ts

@@ -0,0 +1,9 @@
+/**
+ * @copyright Sister Software
+ * @license AGPL-3.0
+ * @author Teffen Ellis, et al.
+ */
+
+export * from "#address/point"
+export * from "#address/point-interpolation"
+export * from "#address/point-schema"

package/{address-point-interpolation.ts → lib/address/point-interpolation.ts}

@@ -6,13 +6,13 @@
  *   Address-point interpolation — "Method 2" of the resolution ladder (#483, Phase 1 of
  *   `docs/articles/plan/2026-06-11-resolution-ladder.md`): when the exact address-point tier (#476)
  *   misses a house number, bracket the number with REAL neighbor points on the same street from the
- *   same #476 shard and interpolate linearly in house-number space between them. Real occupancy
- *   replaces TIGER's uniform-spacing assumption — the dominant error term of the TIGER pilot's gate
+ *   same #476 extract and interpolate linearly in house-number space between them. Real occupancy
+ *   replaces TIGER's uniform-spacing assumption — the dominant error term of the TIGER pilot's check
  *   miss; TIGER range interpolation (`StreetInterpolator`) demotes to the fallback for streets too
  *   sparse to bracket.
  *
  *   Matching key is `street_key` — THE shared normalizer plus the route fold
- *   (`canonicalizeRouteKey`), identical at build time (`scripts/build-address-point-shard.ts`) and
+ *   (`canonicalizeRouteKey`), identical at build time (`scripts/build-address-point-extract.ts`) and
  *   query time, by construction. Scope is postcode-first like the segment tier; a query without a
  *   postcode goes straight to the fallback (which carries its own statewide-ambiguity abstention).
  *
@@ -20,7 +20,7 @@
  *
  *   - Neighbor candidates NEVER include the queried number itself (any unit/duplicate row of it) — in
  *       production the exact tier would already have answered an on-file number, and in the eval
- *       this is what makes grading against the same shard non-circular by construction.
+ *       this is what makes grading against the same extract non-circular by construction.
  *   - Both-sided bracket (`bracket: "both"`): linear interpolation between the nearest known number
  *       below and above; `uncertaintyM` = half the distance between them.
  *   - Single-sided (`bracket: "single"`): linear extrapolation along the two nearest known numbers on
@@ -33,14 +33,14 @@
  *   Standalone like the segment tier — core wiring rides the Phase 2 ordered `spatialTiers` list.
  */
 
-import { DatabaseSync } from "node:sqlite"
-
 import type { InterpolationLookup } from "@mailwoman/resolver"
+import { haversineKm } from "@mailwoman/spatial"
+import { DatabaseClient } from "@mailwoman/sqlite/client"
 
-import { haversineKm } from "./geo.ts"
-import type { InterpolatedHit, InterpolationQuery, StreetInterpolator } from "./interpolation.ts"
-import { hasTable } from "./sqlite-utils.ts"
-import { canonicalizeRouteKey, normalizeStreetForKey } from "./street-normalize.ts"
+import type { AddressPointDatabase } from "#address/point-schema"
+import type { InterpolatedHit, InterpolationQuery, StreetInterpolator } from "#interpolation"
+import { hasTable, prepareAll, type PreparedAll } from "#sqlite-utils"
+import { canonicalizeRouteKey, type RouteKey, streetKeyVariants } from "#street/normalize"
 
 /**
  * Extrapolation cap for a single-sided bracket: at most one pair-span beyond the nearest known point (`t = 2`). Past
@@ -67,31 +67,36 @@ interface NumberAnchor {
 	release: string
 }
 
-export class AddressPointInterpolator implements InterpolationLookup {
-	readonly #db: DatabaseSync
-	readonly #ownsDB: boolean
+export class AddressPointInterpolator<
+	DB extends AddressPointDatabase = AddressPointDatabase,
+> implements InterpolationLookup {
+	readonly #db: DatabaseClient<DB>
+	/**
+	 * Resources this instance opened. A connection handed in by a caller is NOT in here, so disposal cannot reach it —
+	 * ownership is membership rather than a flag a later branch has to check.
+	 */
+	readonly #resources = new DisposableStack()
 	readonly #fallback: StreetInterpolator | undefined
-	readonly #byPostcode: ReturnType<DatabaseSync["prepare"]> | undefined
+	readonly #byPostcode: PreparedAll<[postcode: string, street: RouteKey, number: number], PointRow> | undefined
 
-	constructor(opts: { dbPath?: string; database?: DatabaseSync; fallback?: StreetInterpolator }) {
+	constructor(opts: { dbPath?: string; database?: DatabaseClient<DB>; fallback?: StreetInterpolator }) {
 		if (opts.database) {
 			this.#db = opts.database
-			this.#ownsDB = false
 		} else if (opts.dbPath) {
-			this.#db = new DatabaseSync(opts.dbPath, { readOnly: true })
-			this.#ownsDB = true
+			this.#db = this.#resources.use(new DatabaseClient<DB>(opts.dbPath, { readOnly: true }))
 		} else {
 			throw new Error("AddressPointInterpolator: one of dbPath or database is required")
 		}
 
 		this.#fallback = opts.fallback
 
-		// Degrade gracefully on an empty/tableless shard (#568): with no `address_point` table this tier
+		// Degrade gracefully on an empty/tableless extract (#568): with no `address_point` table this tier
 		// is skipped, deferring to the segment fallback rather than crashing at construction.
 		if (hasTable(this.#db, "address_point")) {
 			// Strictly-numeric neighbor numbers on the route-folded street key within the ZIP. The
 			// queried number itself is excluded HERE (see module doc: non-circular by construction).
-			this.#byPostcode = this.#db.prepare(
+			this.#byPostcode = prepareAll(
+				this.#db,
 				`SELECT CAST(number AS INTEGER) AS n, lat, lon, source, release
 				 FROM address_point
 				 WHERE postcode = ? AND street_key = ?
@@ -102,25 +107,32 @@ export class AddressPointInterpolator implements InterpolationLookup {
 	}
 
 	find(query: InterpolationQuery): InterpolatedHit | null {
-		const streetKey = canonicalizeRouteKey(normalizeStreetForKey(query.street))
 		const numberRaw = query.number.trim()
 
-		if (!streetKey || !/^\d+$/.test(numberRaw)) return null
+		if (!/^\d+$/.test(numberRaw)) return null
 		const n = Number(numberRaw)
 
-		// No own table (empty shard) or no postcode → defer to the segment fallback rather than query.
+		// No own table (empty extract) or no postcode → defer to the segment fallback rather than query.
 		if (!this.#byPostcode || !query.postcode) return this.#fallback?.find(query) ?? null
 
-		const rows = this.#byPostcode.all(query.postcode.trim(), streetKey, n) as unknown as PointRow[]
+		// Key-variant ladder (see `streetKeyVariants`) — same probe order as the exact-point reader.
+		let rows: PointRow[] = []
+
+		for (const variant of streetKeyVariants(query.street)) {
+			const streetKey = canonicalizeRouteKey(variant)
+
+			rows = this.#byPostcode(query.postcode.trim(), streetKey, n)
+
+			if (rows.length) break
+		}
+
 		const hit = rows.length >= 2 ? interpolateFromNeighbors(rows, n) : null
 
 		return hit ?? this.#fallback?.find(query) ?? null
 	}
 
-	close(): void {
-		if (this.#ownsDB) {
-			this.#db.close()
-		}
+	[Symbol.dispose](): void {
+		this.#resources[Symbol.dispose]()
 	}
 }
 

package/{address-point-schema.ts → lib/address/point-schema.ts}

@@ -3,8 +3,8 @@
  * @license AGPL-3.0
  * @author Teffen Ellis, et al.
  *
- *   Typed schema for the SITUS / rooftop ADDRESS-POINT shards (`address-points-<cc>-<slug>.db`, built
- *   by `scripts/build-address-point-shard.ts` — the #476/#567 national rooftop tier behind the
+ *   Typed schema for the SITUS / rooftop ADDRESS-POINT extracts (`address-points-<cc>-<slug>.db`, built
+ *   by `scripts/build-address-point-extract.ts` — the #476/#567 national rooftop tier behind the
  *   demo's "type any US address, get the building"). Single source of truth for the columns shared
  *   by the BUILDER and the READER ({@link AddressPointSqliteLookup}), so a column rename in one is a
  *   compile error in the other.
@@ -18,6 +18,8 @@
 
 import type { Kysely } from "kysely"
 
+import type { NameKey, RouteKey, StreetKey } from "#street/normalize"
+
 /**
  * One rooftop address point. `(street_norm, number)` within a `postcode` (preferred) or `locality_norm` scope is the
  * lookup; `street_key` is the #483 route-fold key for interpolation. Coordinates are non-null (the builder drops
@@ -27,11 +29,12 @@ export interface AddressPointTable {
 	/**
 	 * Shared {@link normalizeStreetForKey} of the street — the build/query-consistent probe key.
 	 */
-	street_norm: string
+	street_norm: StreetKey
 	/**
-	 * `canonicalizeRouteKey(street_norm)` — the route-fold key (#483 Method 2).
+	 * `canonicalizeRouteKey(street_norm)` — the route-fold key (#483 Method 2). Its own brand, so it cannot be
+	 * interchanged with the plain `street_norm` above.
 	 */
-	street_key: string
+	street_key: RouteKey
 	/**
 	 * House number, normalized lower-case (kept TEXT — "123-A", "12 1/2" must survive).
 	 */
@@ -41,7 +44,7 @@ export interface AddressPointTable {
 	/**
 	 * Shared {@link normalizeLocalityForKey} of the locality — the fallback scope.
 	 */
-	locality_norm: string | null
+	locality_norm: NameKey | null
 	/**
 	 * The street as it appeared in the source (kept for display / debugging).
 	 */
@@ -56,6 +59,16 @@ export interface AddressPointTable {
 	 * The pinned data release the point was ingested from.
 	 */
 	release: string
+	/**
+	 * The source register's stable administrative key for the point's commune or municipality — BAN's `code_insee`. A
+	 * display name (`locality_norm`) is not a key; the coverage basis is computed per THIS.
+	 */
+	admin_code: string | null
+	/**
+	 * The register's own certification flag for the point (BAN `certification_commune`: 1 certified by the commune, 0
+	 * not), or null for a source that states none. A basis is never inferred from a share of these.
+	 */
+	certified: number | null
 }
 
 /**
@@ -65,6 +78,15 @@ export interface AddressPointDatabase {
 	address_point: AddressPointTable
 }
 
+/**
+ * The slice of a Kysely handle the `address_point` DDL touches — the parameter type its builders take.
+ *
+ * Kysely is invariant in its schema parameter (the incompatibility is in `transaction()`), so a extract that EXTENDS
+ * `AddressPointTable` — OSM adds `h3_cell` — cannot pass its own handle to a `Kysely<AddressPointDatabase>` parameter.
+ * Naming only `schema` lets it, and the DDL below needs nothing else.
+ */
+export type AddressPointSchemaHandle = Pick<Kysely<AddressPointDatabase>, "schema">
+
 /**
  * The `address_point` columns in INSERT order. The builder's positional prepared statement derives its placeholder list
  * from this, so the positional order can't drift from the DDL / the reader.
@@ -81,12 +103,14 @@ export const ADDRESS_POINT_COLUMNS = [
 	"lon",
 	"source",
 	"release",
+	"admin_code",
+	"certified",
 ] as const
 
 /**
  * Create the `address_point` table — called before the streaming bulk load.
  */
-export async function createAddressPointTable(db: Kysely<AddressPointDatabase>): Promise<void> {
+export async function createAddressPointTable(db: AddressPointSchemaHandle): Promise<void> {
 	await db.schema
 		.createTable("address_point")
 		.addColumn("street_norm", "text", (c) => c.notNull())
@@ -101,13 +125,15 @@ export async function createAddressPointTable(db: Kysely<AddressPointDatabase>):
 		.addColumn("lon", "real", (c) => c.notNull())
 		.addColumn("source", "text", (c) => c.notNull())
 		.addColumn("release", "text", (c) => c.notNull())
+		.addColumn("admin_code", "text")
+		.addColumn("certified", "integer")
 		.execute()
 }
 
 /**
  * Create the three probe indexes the reader relies on (postcode-scope, locality-scope, route-key).
  */
-export async function createAddressPointIndexes(db: Kysely<AddressPointDatabase>): Promise<void> {
+export async function createAddressPointIndexes(db: AddressPointSchemaHandle): Promise<void> {
 	await db.schema
 		.createIndex("idx_ap_postcode")
 		.on("address_point")