curtilage 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: 536f2e3c92ff702ee5a38e10ef6f494b11f72d5a34d30bca4c0cf205e3527727
4
+ data.tar.gz: 303d2c493d3aca7f31f2902699ba57678dc08e65d0370b590a683459294e0b95
5
+ SHA512:
6
+ metadata.gz: 1684ce45a76140ecb7fc11f8dbd5f58061db7c715d52c554dd47c96fe25de2875b6328bbf3228782c21ed775f911a99d779488f89e7fdd71cc8e8a1061e955ad
7
+ data.tar.gz: 53c66c6d5eff5e9e661b91eea8e0906e708809ebfc37f1efe59da3cc2f0b2613094694518b641b3bc7768f2f7d79fae36f0a1ecd529391cad1f1d76c0b11c8ee
data/CHANGELOG.md ADDED
@@ -0,0 +1,39 @@
1
+ # Changelog
2
+
3
+ ## [Unreleased]
4
+
5
+ ## [0.1.0] - 2026-11-21
6
+
7
+ First release. What it does:
8
+
9
+ ### Premises
10
+ - Grows the building's rectangle 30 m toward its lot, trims at public roads and railways, and claims the site's parking; falls back to the building rectangle plus a margin when that fails validation. Every result has a step-by-step explanation.
11
+ - Roads: stops half a typical road width short of the centreline (by road class), but always at least 3 m out from the building (`building_clearance_m`). Cuts follow each road's fitted direction beside the site, nearest road first, snapped square to the building within 5 degrees. Roads and railways in tunnels or on bridges are ignored.
12
+ - Railways at ground level bound the premises like roads.
13
+ - Neighbours: stops 3 m short of a neighbouring building, square to the building (halfway when the gap is narrower). Homes (typed in OSM, or untyped small buildings in a row) get the halfway split when the site is behind them (their back yard, judged by which way they face their street).
14
+ - Lot: mapped parking within 20 m of the building is the site's, up to the kerb and at most 60 m out. A lot shared with one or two other buildings is claimed whole (split halfway only with homes); a shopping-centre lot (three or more buildings) is split by nearest building. Parking garages, carports and canopies next to the building count as lot, and driveways linking the lot to a public road are taken in up to the kerb.
15
+ - When the pin isn't on a building, the nearest real building within 30 m is used, not a tiny canopy next to it.
16
+ - Presets: `:whole_lot` (default), `:tight` (building plus a small margin) and `:centre` (a store on a shared lot gets the whole shopping centre, flagged `shopping_centre`).
17
+
18
+ ### Store zone
19
+ - `result.store_ring` is the building's OSM outline (any shape, at most 20 corners, never self-crossing), always inside the premises: the premises take in the whole building and the outline is clipped 5 cm inside them. `store_zone: false` turns it off.
20
+
21
+ ### Flags and validation
22
+ - Flags, checked against reviewed sites: `near_houses`, `large_building` (over 5,000 m2), `small_building` (under 50 m2), `building_not_under_pin`, `store_may_be_unit` (over 1,500 m2), `shopping_centre`, `fallback`, `ml_footprint`, `no_data`.
23
+ - `Validation.check` errors on missing, unclosed, self-intersecting or swapped rings, a pin outside, vertices too far away, too many vertices, or an area over 100,000 m2; warns on small, large or off-centre polygons.
24
+ - `Audit.check` scores an existing polygon (crosses a road, covers a neighbour, differs from what would be generated).
25
+
26
+ ### Data
27
+ - On demand: `generate_many(pins)` finds each pin's region in Geofabrik's index (worldwide), builds only those regions into a temporary folder or `cache_dir:` (reused for a month, locked so concurrent jobs build once) and cleans up.
28
+ - `source: :overture`: reads features in place from Overture Maps' latest release, using its file index and sharing reads between nearby pins; a silent network stall is retried once.
29
+ - `curtilage build REGION`: any US state or Geofabrik region path; downloads only the layers it uses with HTTP range requests (CRC-checked, full-file fallback), reads every part of split layers and multi-part regions (California), keeps building types, and builds with DuckDB capped at 400 MB and 1 thread. Writes `<region>.json` metadata beside the Parquet file.
30
+ - `Catalogue`, batch lookups (`Lookup.features_many`), Parquet read from `s3://`, `gs://` or `https://`.
31
+ - Optional machine-learned building footprints (Microsoft, ODbL): `--ml-footprints microsoft`, then `buildings: :osm_filled` or `:ml`. OSM only is the default.
32
+ - Map data is treated as untrusted: size and vertex caps, invalid geometry dropped and counted.
33
+
34
+ ### Interface
35
+ - `Curtilage.premises(pin)` for one pin with no setup; `Curtilage.generate_many`, `Curtilage.generate`, `Result#to_geojson` (with attribution; `include_store: true` for both shapes).
36
+ - `Curtilage.configure` (or `CURTILAGE_*` environment variables) for the DuckDB path, cache folder and data source; DuckDB's extensions install themselves.
37
+ - Commands: `curtilage generate` (works with only `--pin`), `curtilage build`, `curtilage doctor`.
38
+ - `docs/reference.md` documents every option, setting, flag and command.
39
+ - MIT licence.
data/LICENSE.txt ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Emil M
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,134 @@
1
+ # curtilage
2
+
3
+ *Curtilage: the land immediately around a building, such as its yard, drive and lot.*
4
+
5
+ Generates a site's **premises polygon** (the building plus its customer-facing lot) and a **store zone** (the building's own outline) from free map data, so nobody has to draw them by hand.
6
+
7
+ **Status:** 0.1.0, the first release. Checked on 20 real store sites across five US states against hand-drawn premises and by eye on aerial imagery. The API may still change before 1.0.
8
+
9
+ ## Quick start
10
+
11
+ 1. Install the gem (`gem "curtilage"` in your Gemfile, once published) and the [DuckDB CLI](https://duckdb.org/docs/installation/) 1.4. Check with `curtilage doctor`; DuckDB's extensions install themselves.
12
+ 2. Ask for premises:
13
+
14
+ ```ruby
15
+ require "curtilage"
16
+
17
+ result = Curtilage.premises([-80.0, 35.0]) # [lng, lat]
18
+ result.ok? # true when it passed validation
19
+ result.ring # the premises polygon, [[lng, lat], ...]
20
+ result.store_ring # the building's outline, inside the premises
21
+ result.flags # anything a person should check, e.g. [:near_houses]
22
+ result.to_geojson # ready to draw or save, with the attribution to show
23
+ ```
24
+
25
+ Or from the shell: `curtilage generate --pin -80.0,35.0`.
26
+
27
+ That's all the setup: map data is fetched for the pin's region as needed. Everything else (batches, Overture instead of Geofabrik, caching, styles, every option and flag) is in the [reference](docs/reference.md).
28
+
29
+ ## How it works
30
+
31
+ 1. **Data.** Buildings, parking lots, roads and railways from OpenStreetMap, from one of:
32
+ - **Geofabrik, built on demand** (default): the gem finds each pin's region in Geofabrik's region index, downloads only the layers it needs from that region's shapefiles, and builds a compact Parquet file. 1-5 minutes per US state the first time; nothing to host.
33
+ - **Overture Maps, read in place**: the same OpenStreetMap data from Overture's public release, read straight from their storage with no download up front. About 11 s per area; nearby pins share the read.
34
+ - **Your own prebuilt files** (`curtilage build`), local or in a bucket, for instant lookups.
35
+ 2. **Generate.** Start from the building at the pin and grow its rectangle 30 m toward its lot, then:
36
+ - stop at public roads, half a typical road width short of the centreline (by road class) but at least 3 m out from the building, following each road's direction; and at railways;
37
+ - take in the site's mapped parking (a lot shared with a few buildings is claimed whole; a shopping-centre lot is split by nearest building), parking garages, canopies, and driveways to the street;
38
+ - stop 3 m short of neighbouring buildings, or halfway when the site is behind a house (its back yard);
39
+ - fall back to the building's rectangle plus a margin if that fails validation.
40
+ Every result explains each step and carries review flags.
41
+ 3. **Validate.** Reject or flag polygons that are unsafe to use: pin outside, unclosed, self-intersecting, swapped coordinates, too far from the pin, an implausible size. The store zone always sits inside the premises.
42
+
43
+ It's rule-based: no machine learning in the method. Coordinates are `[lng, lat]` throughout, as in GeoJSON.
44
+
45
+ ## Usage
46
+
47
+ ```ruby
48
+ pins = [[-80.0, 35.0], [-80.1, 35.2]]
49
+ results = Curtilage.generate_many(pins) # Geofabrik, built on demand, cleaned up after
50
+ results = Curtilage.generate_many(pins, source: :overture) # Overture Maps, nothing downloaded up front
51
+ results = Curtilage.generate_many(pins, preset: :centre) # whole shopping centre where on a shared lot
52
+
53
+ Curtilage.configure do |c| # once, instead of options on every call
54
+ c.cache_dir = "/var/cache/curtilage" # reuse on-demand builds for a month
55
+ end
56
+ ```
57
+
58
+ - Styles: `preset: :whole_lot` (default), `:tight` (building plus a small margin) or `:centre`. `store_zone: false` skips the store zone. Every rule's distance is an option ([reference](docs/reference.md#generation-options)).
59
+ - Built ahead of time, for instant lookups: `curtilage build tennessee`, then `generate_many(pins, catalogue: Curtilage::Catalogue.from_dir("."))`. Files can also live in a bucket (`s3://`, `gs://`, `https://`).
60
+ - Check an existing polygon: `Curtilage::Audit.check(pin, ring, features)` returns `:ok`, `:review` (with reasons) or `:invalid`.
61
+
62
+ ### Flags
63
+
64
+ | Flag | Meaning |
65
+ |---|---|
66
+ | `near_houses` | The lot borders a home: check where the yard or property line is |
67
+ | `large_building` | Building over 5,000 m2: may be shared by several tenants |
68
+ | `small_building` | Building under 50 m2: may be a kiosk or canopy rather than the store |
69
+ | `building_not_under_pin` | No building under the pin; the nearest one within 30 m was used |
70
+ | `store_may_be_unit` | Store zone is a building over 1,500 m2: the store may be one unit of it |
71
+ | `shopping_centre` | `scope: :centre` mapped the whole centre |
72
+ | `fallback` | The main method failed validation; the building rectangle was used |
73
+ | `ml_footprint` | Built on a machine-learned footprint, not an OSM building |
74
+ | `no_data` | No map data covers the pin |
75
+
76
+ ### Speed
77
+
78
+ - Generate: about 100 ms per site (median; a few hundred ms in dense blocks).
79
+ - Prebuilt files: about 85 ms per lookup through the DuckDB CLI; one call serves many pins.
80
+ - Built on demand: 1-5 minutes per US state the first time (Texas, the largest tested, about 4.5 minutes and under 650 MB of memory), then as above.
81
+ - Overture: about 11 s per area; pins within a few km of each other share it.
82
+
83
+ ## Building footprints: OSM, or OSM plus machine-learned
84
+
85
+ By default all buildings come from OpenStreetMap. Where OSM is missing buildings, you can opt in to [Microsoft's Global ML Building Footprints](https://github.com/microsoft/GlobalMLBuildingFootprints) (ODbL, like OSM): build with `--ml-footprints microsoft` (or `ml_footprints: "microsoft"`), then generate with `buildings: :osm_filled` or `:ml`. With `source: :overture`, Overture's non-OSM buildings play the same role. Results built on one carry the `ml_footprint` flag.
86
+
87
+ ## Network access
88
+
89
+ The gem only talks to public map-data hosts, and only when it needs data: Geofabrik (region index and extracts), Overture Maps (file index and data), Microsoft's footprint files (opt-in), and your own bucket if you point it at one. Reading by area tells the host roughly which areas you're interested in. Nothing else is sent anywhere.
90
+
91
+ ## Layout
92
+
93
+ | Path | What |
94
+ |---|---|
95
+ | `lib/curtilage.rb` | Entry points: `generate`, `generate_many`, presets |
96
+ | `lib/curtilage/generator.rb` | Methods, the fallback chain, road and neighbour trims, flags |
97
+ | `lib/curtilage/generator_lot.rb` | Which parking, garages, canopies and driveways are the site's |
98
+ | `lib/curtilage/generator_store.rb` | The store zone |
99
+ | `lib/curtilage/features.rb` | Parses nearby map features (untrusted) and projects them to metres |
100
+ | `lib/curtilage/geometry.rb` | Plain-Ruby planar geometry |
101
+ | `lib/curtilage/validation.rb` | Checks a polygon against its pin |
102
+ | `lib/curtilage/audit.rb` | Checks an existing polygon against the map and the generated one |
103
+ | `lib/curtilage/auto_data.rb` | Builds the regions pins need on demand, into a temporary or cache folder |
104
+ | `lib/curtilage/regions.rb` | Geofabrik's region index: which region a pin is in |
105
+ | `lib/curtilage/build.rb` | Builds a region's features Parquet file from Geofabrik shapefiles |
106
+ | `lib/curtilage/zip_range.rb` | Downloads only the needed files from a remote zip |
107
+ | `lib/curtilage/catalogue.rb` | Finds the built region file(s) covering a pin |
108
+ | `lib/curtilage/lookup.rb` | Reads the features around pins from Parquet files |
109
+ | `lib/curtilage/overture.rb` | Reads features in place from Overture Maps |
110
+ | `lib/curtilage/footprints.rb` | Optional machine-learned footprints (Microsoft) |
111
+ | `lib/curtilage/download.rb` | HTTPS downloads from allowed hosts, with size caps |
112
+ | `lib/curtilage/config.rb` | Shared settings (`Curtilage.configure`, environment variables) |
113
+ | `exe/curtilage` | `curtilage generate`, `build` and `doctor` |
114
+ | `docs/reference.md` | Every option, setting, flag and command |
115
+ | `sql/build_from_shapefiles.sql` | DuckDB query: shapefiles to a features Parquet file |
116
+ | `test/` | Tests, using made-up coordinates only |
117
+
118
+ ## Development
119
+
120
+ ```sh
121
+ bin/setup
122
+ bundle exec rake # tests and RuboCop
123
+ ```
124
+
125
+ The gem itself is plain Ruby with no runtime gem dependencies. Getting data needs the [DuckDB](https://duckdb.org/) CLI (1.4 LTS); its `spatial` and `httpfs` extensions install themselves on first use. Point `CURTILAGE_DUCKDB` and `CURTILAGE_EXTENSIONS` at them if they aren't in the default places (`curtilage doctor` checks). `unzip` is only needed if a server can't serve byte ranges.
126
+
127
+ ## Data and licensing
128
+
129
+ - Map data © [OpenStreetMap contributors](https://www.openstreetmap.org/copyright), available under the [ODbL](https://opendatacommons.org/licenses/odbl/). Show this attribution wherever generated premises are displayed; results carry it (`to_geojson`).
130
+ - With `source: :overture`: also © Overture Maps Foundation. With ML footprints: also Microsoft Global ML Building Footprints (ODbL).
131
+ - Downloaded files and built Parquet files are OSM-derived data. They're never committed to this repository (see `.gitignore`).
132
+ - This repository must never contain real site locations, client data or credentials. Tests and examples use made-up coordinates.
133
+ - The code is MIT-licensed (see `LICENSE.txt`). The map data keeps its own licence (ODbL).
134
+ - Security issues: see `SECURITY.md` (report privately through GitHub).
data/SECURITY.md ADDED
@@ -0,0 +1,15 @@
1
+ # Security
2
+
3
+ ## Reporting a vulnerability
4
+
5
+ Please report security issues privately through GitHub: on this repository, open the **Security** tab and choose **Report a vulnerability**. Don't open a public issue for them.
6
+
7
+ Include what you found, how to reproduce it, and the version you used. You should get a reply within a week.
8
+
9
+ ## Supported versions
10
+
11
+ Fixes go into the latest release only.
12
+
13
+ ## Scope
14
+
15
+ curtilage downloads public map data (Geofabrik, Overture Maps, and opt-in Microsoft footprint files), runs the DuckDB CLI on it, and treats all of that data as untrusted: downloads are limited to allowed HTTPS hosts and capped in size, and map features with oversized or invalid geometry are dropped. Reports about any of these paths are welcome.
data/docs/reference.md ADDED
@@ -0,0 +1,140 @@
1
+ # Reference
2
+
3
+ Everything curtilage offers, in one place. For an overview, see the [README](../README.md).
4
+
5
+ ## Ruby API
6
+
7
+ ### `Curtilage.premises(pin, **options) → Result`
8
+
9
+ The simplest call: premises for one `[lng, lat]` pin, fetching map data as configured. Takes the same options as `generate_many`.
10
+
11
+ ### `Curtilage.generate_many(pins, **options) → [Result]`
12
+
13
+ Premises for many pins, in pin order. Pins that share map data share the work.
14
+
15
+ | Option | Default | What |
16
+ |---|---|---|
17
+ | `source:` | `Curtilage.config.source` (`:geofabrik`) | `:geofabrik`: build the pins' regions on demand. `:overture`: read Overture Maps in place. |
18
+ | `cache_dir:` | `Curtilage.config.cache_dir` (none) | Keep on-demand builds here and reuse them for a month. Without it, builds go to a temporary folder deleted afterwards. |
19
+ | `catalogue:` | none | A `Curtilage::Catalogue` of regions built ahead of time (overrides `source:`). |
20
+ | `parquet:` | none | One built file, or a list, local or `s3://`, `gs://`, `https://` (overrides `source:`). |
21
+ | `lookup:` | `{}` | `duckdb:`, `extension_directory:` (default: the configuration), `setup_sql:` (e.g. `CREATE SECRET` for a private bucket), `radius_deg:` (how far around each pin to read, default 0.004). |
22
+ | `data:` | `{}` | For on-demand builds: `max_age_days:` (31), `ml_footprints:` (`"microsoft"`). For Overture: `release:` (default: the latest). |
23
+ | `preset:` | none | `:whole_lot`, `:tight` or `:centre` (below). |
24
+ | `method:` | none | `:grow_trim`, `:building` or `:box` to force one method, no fallback. |
25
+ | any generation option | | See the table below. |
26
+
27
+ ### `Curtilage.generate(pin, features, **options) → Result`
28
+
29
+ Premises from features you already have (from `Curtilage::Lookup.features` or `Curtilage::Overture.features_many`), with the same `preset:`, `method:` and generation options.
30
+
31
+ ### `Result`
32
+
33
+ | Field | What |
34
+ |---|---|
35
+ | `ok?` | The premises passed validation. Never save one that isn't. |
36
+ | `ring` | The premises: `[[lng, lat], ...]`, closed, convex, at most `max_vertices` corners. `nil` if nothing could be made. |
37
+ | `store_ring` | The store zone: the building's own outline, always inside `ring`. `nil` with `store_zone: false`. |
38
+ | `flags` | Reasons a person should look first (below). |
39
+ | `explanation` | Each step, in plain words. |
40
+ | `validation` | `errors`, `warnings` and `metrics` (area, rectangularity, pin inside, distances). |
41
+ | `method_name` | `:grow_trim`, or `:building` after a fallback. |
42
+ | `building` | The id of the building used. |
43
+ | `to_geojson(include_store: false)` | A GeoJSON Feature (premises) with everything above in its properties, including the attribution to show; with `include_store: true`, a FeatureCollection of premises and store zone (`properties.role`). |
44
+
45
+ ### Presets
46
+
47
+ | Preset | What |
48
+ |---|---|
49
+ | `:whole_lot` | The default: the building and its whole customer-facing lot. |
50
+ | `:tight` | The building plus an 8 m margin, still trimmed at roads. |
51
+ | `:centre` | Like `:whole_lot`, but a store on a shared shopping-centre lot gets the whole centre (flag `shopping_centre`). |
52
+
53
+ Any option passed with a preset overrides it.
54
+
55
+ ### Generation options
56
+
57
+ All distances in metres, areas in square metres. These are `Curtilage::Generator::DEFAULTS`.
58
+
59
+ | Option | Default | What |
60
+ |---|---|---|
61
+ | `store_zone` | `true` | Also return the building's outline as the store zone. |
62
+ | `buildings` | `:osm` | `:osm`, `:osm_filled` (machine-learned footprints only where OSM has no building) or `:ml`. |
63
+ | `scope` | `:site` | `:site`, or `:centre` for the whole shopping centre when on a shared lot. |
64
+ | `grow_m` | `30` | How far the building's rectangle grows toward its lot before trimming. |
65
+ | `road_offset_m` | `nil` | How far short of road centrelines to stop; `nil` uses half a typical road width by class. |
66
+ | `building_clearance_m` | `3` | Road trims stop at least this far from the building. |
67
+ | `neighbour_gap_m` | `3` | Strip left to a neighbouring building; `nil` splits halfway. |
68
+ | `neighbour_min_area_m2` | `40` | Smaller buildings (sheds) don't bound the lot. |
69
+ | `neighbour_strip_min_area_m2` | `0` | Neighbours smaller than this are split halfway instead of getting the strip. |
70
+ | `house_max_m2` | `250` | An untyped building smaller than this, among other small ones, counts as a home. |
71
+ | `house_street_m` | `25` | ... when two more small buildings are this close. |
72
+ | `parking_reach_m` | `20` | Mapped parking this close to the building is its lot. |
73
+ | `parking_max_extent_m` | `60` | Parking is claimed at most this far from the building. |
74
+ | `shared_lot_buildings` | `3` | A lot touching this many other buildings is a shopping-centre lot. |
75
+ | `building_max_distance_m` | `30` | When the pin isn't on a building, use the nearest one only this close. |
76
+ | `pin_clearance_m` | `5` | Room kept around the pin. |
77
+ | `small_building_m2` | `50` | Flag `small_building` below this; such structures count as lot next to a real building. |
78
+ | `large_building_m2` | `5000` | Flag `large_building` above this. |
79
+ | `large_building_note_m2` | `1500` | Above this, explained (and the store zone flagged `store_may_be_unit`). |
80
+ | `building_margin_m` | `20` | The fallback's margin around the building rectangle. |
81
+ | `box_side_m` | `60` | Side of the baseline `:box` method's square. |
82
+ | `max_vertices` | `10` | The premises are simplified to at most this many corners (never cutting the building). |
83
+ | `max_store_vertices` | `20` | The store outline is simplified to at most this many corners. |
84
+
85
+ ### Flags
86
+
87
+ | Flag | Meaning |
88
+ |---|---|
89
+ | `near_houses` | The lot borders a home: check where the yard or property line is. |
90
+ | `large_building` | Building over 5,000 m2: may be shared by several tenants. |
91
+ | `small_building` | Building under 50 m2: may be a kiosk or canopy rather than the store. |
92
+ | `building_not_under_pin` | No building under the pin; the nearest one within 30 m was used. |
93
+ | `store_may_be_unit` | The store zone is a building over 1,500 m2: the store may be one unit of it. |
94
+ | `shopping_centre` | `scope: :centre` mapped the whole centre. |
95
+ | `fallback` | The main method failed validation; the building rectangle was used. |
96
+ | `ml_footprint` | Built on a machine-learned footprint, not an OSM building. |
97
+ | `no_data` | No map data covers the pin. |
98
+
99
+ ### Other entry points
100
+
101
+ - `Curtilage::Audit.check(pin, ring, features, thresholds: {}, **options)`: checks an existing polygon. Returns a report with `verdict` (`:ok`, `:review`, `:invalid`), `reasons` (`crosses_road`, `covers_neighbour`, `differs_from_generated`, validation warnings), `overlap`, `road_m`, `neighbour_m2` and the `generated` result.
102
+ - `Curtilage::Validation.check(pin, ring, **thresholds)`: the checks every polygon goes through.
103
+ - `Curtilage::Build.run(region, out_dir:, work_dir:, **options)`: build a region's files ahead of time (`date:`, `ml_footprints:`, `memory_limit:` `"400MB"`, `threads:` `1`, `keep_downloads:`).
104
+ - `Curtilage::Catalogue.from_dir(dir, base_url: nil)`: the regions built into a folder; `base_url:` when the Parquet files live in a bucket.
105
+ - `Curtilage::Lookup.features(parquet:, pin:)` / `features_many(parquet:, pins:)`: the map features around pins, from built files.
106
+ - `Curtilage::Overture.features_many(pins)`: the map features around pins, from Overture Maps.
107
+
108
+ ## Configuration
109
+
110
+ Set once instead of passing on every call. Defaults come from the environment.
111
+
112
+ | Setting | Environment variable | Default |
113
+ |---|---|---|
114
+ | `duckdb` | `CURTILAGE_DUCKDB` | `duckdb` on the PATH |
115
+ | `extension_directory` | `CURTILAGE_EXTENSIONS` | DuckDB's own |
116
+ | `cache_dir` | `CURTILAGE_CACHE_DIR` | none (temporary folders) |
117
+ | `source` | `CURTILAGE_SOURCE` | `geofabrik` |
118
+
119
+ ```ruby
120
+ Curtilage.configure do |c|
121
+ c.cache_dir = "/var/cache/curtilage"
122
+ c.source = :overture
123
+ end
124
+ ```
125
+
126
+ DuckDB's `spatial` and `httpfs` extensions install themselves on first use.
127
+
128
+ ## Command line
129
+
130
+ ```
131
+ curtilage generate --pin LNG,LAT [--pin LNG,LAT ...] [options]
132
+ curtilage build REGION [options]
133
+ curtilage doctor
134
+ ```
135
+
136
+ - `generate` prints GeoJSON (premises and store zone; several pins print a list). Options: `--source geofabrik|overture`, `--cache-dir DIR`, `--catalogue DIR`, `--parquet FILE`, `--preset NAME`, `--buildings FLAVOUR`, `--[no-]store`, `--duckdb PATH`, `--extension-directory DIR`. Exits 1 if any result isn't usable.
137
+ - `build REGION` writes `REGION.parquet` and `REGION.json`. REGION is a US state (`tennessee`, `new-york`) or a Geofabrik path (`europe/germany/bayern`). Options: `--out DIR` (default `.`), `--work DIR`, `--date YYMMDD`, `--keep-downloads`, `--ml-footprints microsoft`, `--duckdb`, `--extension-directory`.
138
+ - `doctor` checks that DuckDB runs and its extensions load, and says what to install if not.
139
+
140
+ Run `curtilage COMMAND --help` for the same list.
data/exe/curtilage ADDED
@@ -0,0 +1,134 @@
1
+ #!/usr/bin/env ruby
2
+ # frozen_string_literal: true
3
+
4
+ require "optparse"
5
+ require "json"
6
+ require "open3"
7
+ require "tmpdir"
8
+ require "curtilage"
9
+
10
+ USAGE = <<~TEXT
11
+ Usage:
12
+ curtilage generate --pin LNG,LAT [--pin LNG,LAT ...] [options]
13
+ curtilage build REGION [options]
14
+ curtilage doctor
15
+
16
+ generate Print the premises (and store zone) for one or more pins as GeoJSON.
17
+ Map data is fetched as needed: nothing to set up or host.
18
+ build Build a region's map data ahead of time (a US state such as
19
+ "tennessee", or a Geofabrik path such as "europe/germany/bayern").
20
+ doctor Check that DuckDB and its extensions are ready.
21
+
22
+ Run "curtilage COMMAND --help" for a command's options.
23
+ TEXT
24
+
25
+ def duckdb_flags(parser, opts)
26
+ parser.on("--duckdb PATH", "DuckDB CLI (default: $CURTILAGE_DUCKDB or duckdb on PATH)") { |v| opts[:duckdb] = v }
27
+ parser.on("--extension-directory DIR", "DuckDB extension folder (default: $CURTILAGE_EXTENSIONS)") do |v|
28
+ opts[:extension_directory] = File.expand_path(v)
29
+ end
30
+ end
31
+
32
+ def generate_options(args)
33
+ opts = { pins: [] }
34
+ OptionParser.new do |o|
35
+ o.banner = "Usage: curtilage generate --pin LNG,LAT [--pin LNG,LAT ...] [options]"
36
+ o.on("--pin LNG,LAT", "A location; repeat for several") { |v| opts[:pins] << v.split(",").map { |n| Float(n) } }
37
+ data_flags(o, opts)
38
+ style_flags(o, opts)
39
+ duckdb_flags(o, opts)
40
+ end.parse!(args)
41
+ opts
42
+ end
43
+
44
+ def data_flags(o, opts)
45
+ o.on("--source NAME", %w[geofabrik overture], "geofabrik (default: built on demand) or overture") do |v|
46
+ opts[:source] = v.to_sym
47
+ end
48
+ o.on("--cache-dir DIR", "Keep on-demand builds here and reuse them for a month") { |v| opts[:cache_dir] = v }
49
+ o.on("--catalogue DIR", "Use regions built ahead of time (curtilage build)") { |v| opts[:catalogue] = v }
50
+ o.on("--parquet FILE", "Use one built region file") { |v| opts[:parquet] = v }
51
+ end
52
+
53
+ def style_flags(o, opts)
54
+ o.on("--preset NAME", Curtilage::PRESETS.keys.map(&:to_s), "whole_lot (default), tight or centre") do |v|
55
+ opts[:preset] = v.to_sym
56
+ end
57
+ o.on("--buildings FLAVOUR", Curtilage::Generator::BUILDING_SOURCES.map(&:to_s),
58
+ "osm (default), osm_filled or ml") { |v| opts[:buildings] = v.to_sym }
59
+ o.on("--[no-]store", "Include the store zone (default: yes)") { |v| opts[:store_zone] = v }
60
+ end
61
+
62
+ def generate(args)
63
+ opts = generate_options(args)
64
+ abort "curtilage generate: give at least one --pin LNG,LAT" if opts[:pins].empty?
65
+ abort "curtilage generate: --pin takes LNG,LAT" unless opts[:pins].all? { |p| p.size == 2 }
66
+
67
+ catalogue = opts[:catalogue] && Curtilage::Catalogue.from_dir(opts[:catalogue])
68
+ results = Curtilage.generate_many(opts[:pins], catalogue: catalogue, **opts.slice(:parquet, :source, :cache_dir),
69
+ lookup: opts.slice(:duckdb, :extension_directory),
70
+ **opts.slice(:preset, :buildings, :store_zone))
71
+ json = results.map { |r| r.to_geojson(include_store: opts.fetch(:store_zone, true)) }
72
+ puts JSON.pretty_generate(json.size == 1 ? json.first : json)
73
+ exit(results.all?(&:ok?) ? 0 : 1)
74
+ end
75
+
76
+ def build(args)
77
+ opts = { out: ".", work: File.join(Dir.tmpdir, "curtilage-build") }
78
+ OptionParser.new do |o|
79
+ o.banner = "Usage: curtilage build REGION [options]"
80
+ o.on("--out DIR", "Where to write REGION.parquet and REGION.json (default: .)") { |v| opts[:out] = v }
81
+ o.on("--work DIR", "Download folder (default: a temporary one)") { |v| opts[:work] = v }
82
+ o.on("--date YYMMDD", /\A\d{6}\z/, "A specific Geofabrik extract date (default: newest)") { |v| opts[:date] = v }
83
+ o.on("--keep-downloads", "Keep downloaded files in the work folder") { opts[:keep] = true }
84
+ o.on("--ml-footprints SOURCE", Curtilage::Footprints::SOURCES, "Add machine-learned footprints (microsoft)") do |v|
85
+ opts[:ml] = v
86
+ end
87
+ duckdb_flags(o, opts)
88
+ end.parse!(args)
89
+ region = args.shift or abort "curtilage build: give a REGION, e.g. tennessee"
90
+
91
+ puts Curtilage::Build.run(region, out_dir: opts[:out], work_dir: opts[:work], date: opts[:date],
92
+ ml_footprints: opts[:ml], keep_downloads: opts[:keep],
93
+ **opts.slice(:duckdb, :extension_directory))
94
+ end
95
+
96
+ def doctor(args)
97
+ opts = {}
98
+ OptionParser.new { |o| duckdb_flags(o, opts) }.parse!(args)
99
+ settings = Curtilage.duckdb_options(opts)
100
+ check_duckdb(settings)
101
+ check_extensions(settings)
102
+ puts "Ready."
103
+ end
104
+
105
+ def check_duckdb(settings)
106
+ version, status = Open3.capture2(settings[:duckdb], "--version")
107
+ raise Errno::ENOENT unless status.success?
108
+
109
+ puts "DuckDB: #{version.strip}"
110
+ rescue Errno::ENOENT
111
+ abort "DuckDB CLI not found (#{settings[:duckdb]}). Install DuckDB 1.4 (https://duckdb.org/docs/installation/), " \
112
+ "or set CURTILAGE_DUCKDB to its path."
113
+ end
114
+
115
+ def check_extensions(settings)
116
+ dir = settings[:extension_directory]
117
+ sql = [("SET extension_directory = '#{dir.gsub("'", "''")}';" if dir),
118
+ "INSTALL spatial; LOAD spatial; INSTALL httpfs; LOAD httpfs;"].compact.join("\n")
119
+ _, err, status = Open3.capture3(settings[:duckdb], stdin_data: sql)
120
+ unless status.success?
121
+ abort "DuckDB extensions (spatial, httpfs) could not be installed or loaded: #{err.strip[0,
122
+ 300]}"
123
+ end
124
+
125
+ puts "Extensions: spatial and httpfs ready"
126
+ end
127
+
128
+ case ARGV.shift
129
+ when "generate" then generate(ARGV)
130
+ when "build" then build(ARGV)
131
+ when "doctor" then doctor(ARGV)
132
+ when "-h", "--help", "help", nil then puts USAGE
133
+ else abort USAGE
134
+ end
@@ -0,0 +1,88 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Curtilage
4
+ # Checks an existing premises polygon (hand-drawn, or saved earlier)
5
+ # against the map and against what Curtilage would generate, to find the
6
+ # ones worth a second look.
7
+ module Audit
8
+ Report = Struct.new(:verdict, :reasons, :validation, :generated, :overlap, :road_m, :neighbour_m2,
9
+ keyword_init: true)
10
+
11
+ THRESHOLDS = {
12
+ road_m: 5, # public road or railway inside the polygon, metres
13
+ neighbour_m2: 25, # other buildings covered, square metres
14
+ min_overlap: 0.3, # overlap (IoU) with the generated polygon below this
15
+ }.freeze
16
+
17
+ module_function
18
+
19
+ # verdict: :invalid (fails validation), :review (see reasons) or :ok.
20
+ # options go to Curtilage.generate.
21
+ def check(pin, ring, features, thresholds: {}, **)
22
+ limits = THRESHOLDS.merge(thresholds)
23
+ features = Features.new(pin, features) unless features.is_a?(Features)
24
+ validation = Validation.check(pin, ring)
25
+ generated = Curtilage.generate(pin, features, **)
26
+ return Report.new(verdict: :invalid, reasons: validation.errors, validation: validation, generated: generated) \
27
+ unless validation.ok?
28
+
29
+ xy = Geometry.counter_clockwise(Geometry.open_ring(Geometry.project(ring, pin)))
30
+ report = measure(xy, features, generated, pin)
31
+ report.validation = validation
32
+ report.reasons = reasons(report, limits) + validation.warnings
33
+ report.verdict = report.reasons.empty? ? :ok : :review
34
+ report
35
+ end
36
+
37
+ def measure(xy, features, generated, pin)
38
+ Report.new(generated: generated, overlap: overlap(xy, generated, pin),
39
+ road_m: road_length(xy, features).round,
40
+ neighbour_m2: neighbour_area(xy, features, generated.building).round)
41
+ end
42
+
43
+ def reasons(report, limits)
44
+ reasons = []
45
+ reasons << "crosses_road" if report.road_m > limits[:road_m]
46
+ reasons << "covers_neighbour" if report.neighbour_m2 > limits[:neighbour_m2]
47
+ reasons << "differs_from_generated" if report.overlap && report.overlap < limits[:min_overlap]
48
+ reasons
49
+ end
50
+
51
+ # Intersection over union with the generated polygon (convex, so it can
52
+ # clip any shape), or nil when nothing was generated.
53
+ def overlap(xy, generated, pin)
54
+ return nil unless generated.ok?
55
+
56
+ gen = Geometry.counter_clockwise(Geometry.open_ring(Geometry.project(generated.ring, pin)))
57
+ inter = Geometry.clip_convex(xy, gen)
58
+ common = inter.size >= 3 ? Geometry.area(inter) : 0
59
+ (common / (Geometry.area(xy) + Geometry.area(gen) - common)).round(2)
60
+ end
61
+
62
+ # Metres of public road and railway more than 1 m inside the polygon.
63
+ def road_length(xy, features)
64
+ lines = features.roads.select { |r| Generator::PUBLIC_HIGHWAYS.include?(r.highway) } + features.railways
65
+ lines.sum { |line| line.xy.each_cons(2).sum { |a, b| inside_length(xy, a, b) } }
66
+ end
67
+
68
+ def inside_length(xy, a, b)
69
+ length = Math.hypot(b[0] - a[0], b[1] - a[1])
70
+ n = [length.ceil, 1].max
71
+ inside = n.times.count do |i|
72
+ p = Geometry.lerp(a, b, (i + 0.5) / n)
73
+ Geometry.contains?(xy, p) && Geometry.distance_to_edges(xy, p) > 1
74
+ end
75
+ inside * length / n
76
+ end
77
+
78
+ # Area of other buildings (their convex hulls) inside the polygon.
79
+ def neighbour_area(xy, features, building_id)
80
+ features.buildings.sum do |b|
81
+ next 0 if b.id == building_id || Geometry.contains?(b.xy, [0.0, 0.0])
82
+
83
+ part = Geometry.clip_convex(xy, b.hull)
84
+ part.size >= 3 ? Geometry.area(part) : 0
85
+ end
86
+ end
87
+ end
88
+ end