sailkick-boat 0.18.2 → 0.18.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +184 -52
- package/lib/proxy/index.js +100 -16
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1,35 +1,98 @@
|
|
|
1
1
|
# sailkick-boat
|
|
2
2
|
|
|
3
|
-
> ## ⚠️ Project status:
|
|
3
|
+
> ## ⚠️ Project status: alpha
|
|
4
4
|
>
|
|
5
|
-
>
|
|
6
|
-
>
|
|
7
|
-
> and
|
|
8
|
-
>
|
|
5
|
+
> An offline webapp and mobile app for boat metrics, with weather, climatology and
|
|
6
|
+
> basic routing. Add a sailkick account and you also get real-time cloud sync of those
|
|
7
|
+
> metrics, polars, and an optional public page — so others can follow the boat, or you
|
|
8
|
+
> can check on it while you are away.
|
|
9
9
|
>
|
|
10
|
-
> **Registration is invite-only
|
|
11
|
-
>
|
|
12
|
-
>
|
|
10
|
+
> **Registration is free, and invite-only.** I'm looking for courageous early testers:
|
|
11
|
+
> if you would like to try the plugin, email **[info@sailkick.io](mailto:info@sailkick.io)**.
|
|
12
|
+
> Self-hosters can point it at their own sailkick server and InfluxDB v2 instead — see
|
|
13
|
+
> [Config](#config).
|
|
13
14
|
>
|
|
14
|
-
>
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
15
|
+
> Expect breaking changes while the version is 0.x.
|
|
16
|
+
|
|
17
|
+
One Signal K plugin doing four jobs, each independently toggleable — so the boat stays
|
|
18
|
+
"just SignalK + plugins". They are deliberately separate modules: a fault in the cache
|
|
19
|
+
must never wedge the data-critical sync path.
|
|
20
|
+
|
|
21
|
+
**1. Data out — telemetry and AIS to the cloud.** Gapless store-and-forward of this
|
|
22
|
+
vessel's data into your sailkick account, so the cloud holds your history and (later)
|
|
23
|
+
can analyse it. Buffered on disk, so an offline passage or a restart loses nothing.
|
|
24
|
+
Locally-received AIS targets go up too, letting you see the boat's surroundings from
|
|
25
|
+
shore.
|
|
26
|
+
|
|
27
|
+
**2. Data in — the app and its maps, cached for offline.** An offline-first mirror of
|
|
28
|
+
the sailkick host: fetch once online, serve from disk forever. Charts, terrain, the app
|
|
29
|
+
itself. Plus a worldwide base map seeded on start and an on-demand download of the area
|
|
30
|
+
around the boat, so a usable chart exists before you lose connectivity — not only where
|
|
31
|
+
you happened to browse.
|
|
32
|
+
|
|
33
|
+
**3. Live data, served by the boat itself.** Caching alone would leave you offline with
|
|
34
|
+
a dead app: no position, no instruments, no trends, no AIS, and a login wall. So the
|
|
35
|
+
boat *answers* the app's live contracts from its own SignalK — `/ws/telemetry`,
|
|
36
|
+
`/api/history/{series,track}`, `/api/ais` — and serves `/api/config` with the cloud
|
|
37
|
+
login disabled. Same JSON the cloud returns, so the browser cannot tell the difference.
|
|
38
|
+
|
|
39
|
+
**4. Backfill — an existing InfluxDB archive into the cloud.** If the boat recorded
|
|
40
|
+
into its own database before it ever synced (a `signalk-to-influxdb-v2` bucket, or an
|
|
41
|
+
imported logbook), a one-time resumable copy lifts that history into the cloud where the
|
|
42
|
+
app can reach it.
|
|
43
|
+
|
|
44
|
+
|
|
45
|
+
# 1 · Data out — telemetry and AIS to the cloud
|
|
46
|
+
|
|
47
|
+
## Telemetry sync — gapless by design
|
|
48
|
+
Every value on `vessels.self` is batched, written to a durable on-disk spool as one
|
|
49
|
+
atomic file, and only then uploaded. A file is deleted **only** after InfluxDB
|
|
50
|
+
acknowledges it with a `204`, so an offline stretch or a Signal K restart simply leaves
|
|
51
|
+
files to send later — nothing is lost in the gap.
|
|
52
|
+
|
|
53
|
+
- Network errors, `429` and `5xx` are retried with backoff (1 s → 60 s); the data stays
|
|
54
|
+
on disk.
|
|
55
|
+
- A `4xx` is quarantined to `spool/dead/` rather than retried forever, because a
|
|
56
|
+
malformed or unauthorised batch would otherwise wedge the queue behind it.
|
|
57
|
+
- The buffer is bounded (500 MB). On overflow the **oldest** files are dropped and
|
|
58
|
+
logged — a long-offline boat fills its own disk otherwise.
|
|
59
|
+
- Timestamps are nanosecond-precise, so replaying after a reconnect overwrites rather
|
|
60
|
+
than duplicating. Re-sending is always safe.
|
|
61
|
+
|
|
62
|
+
The destination is fixed at `https://sync.sailkick.io` and cannot be changed from the
|
|
63
|
+
config page — a wrong endpoint here is invisible, since telemetry piling up in the spool
|
|
64
|
+
looks exactly like a normal offline backlog. See
|
|
65
|
+
[Troubleshooting](#troubleshooting-is-telemetry-actually-leaving-the-boat).
|
|
66
|
+
|
|
67
|
+
## Uploading AIS targets
|
|
68
|
+
The cloud app already draws other vessels, but its AIS source polls a SignalK server over
|
|
69
|
+
the LAN and keeps everything in memory — which cannot work once a boat is on a mobile
|
|
70
|
+
link. Enable **Upload AIS targets** and the boat pushes what its own receiver hears, so
|
|
71
|
+
the web app can show other boats, their heading and their trail from stored data.
|
|
72
|
+
|
|
73
|
+
Only **locally received** AIS is forwarded. A boat running an internet feed such as
|
|
74
|
+
`signalk-aisstream` would otherwise spend uplink bandwidth sending data the cloud can
|
|
75
|
+
fetch directly from the same API — known feeds are skipped automatically. The plugin logs
|
|
76
|
+
the AIS sources it sees, so you can name your own receiver in **Only this AIS source** if
|
|
77
|
+
you want to be explicit.
|
|
78
|
+
|
|
79
|
+
There is no radius or rate limit: a real AIS receiver is bounded by VHF line-of-sight,
|
|
80
|
+
which is the honest limiter, and offshore — where this data is most valuable, because
|
|
81
|
+
commercial feeds are blind there — it tends to zero. Vessel identity (name, dimensions,
|
|
82
|
+
ship type) repeats every few minutes and never changes, so it is re-sent at most hourly;
|
|
83
|
+
positions are never throttled.
|
|
84
|
+
|
|
85
|
+
Telemetry always wins the link. AIS buffers in its **own** spool with its own cap and
|
|
86
|
+
stands down completely whenever the telemetry spool has a backlog, so a busy anchorage
|
|
87
|
+
can never delay or evict your own boat's data.
|
|
88
|
+
|
|
89
|
+
> ⚠️ **Requires a cloud that filters history on `self`.** Every AIS row is tagged
|
|
90
|
+
> `self=false`, and the cloud's Trends and track queries must filter `self == "true"`.
|
|
91
|
+
> Without that, other ships' speed and heading appear in *your* charts. Leave this off
|
|
92
|
+
> until the server side is in place.
|
|
93
|
+
|
|
94
|
+
|
|
95
|
+
# 2 · Data in — the app and its maps, cached offline
|
|
33
96
|
|
|
34
97
|
## Proxy: how it works
|
|
35
98
|
```
|
|
@@ -62,6 +125,14 @@ comes from the cloud **announcing bakes**, not from a clock:
|
|
|
62
125
|
|
|
63
126
|
`X-Sailkick-Cache` reports `HIT` / `MISS` / `UPDATED` / `STALE` / `LIVE` per response.
|
|
64
127
|
|
|
128
|
+
**The app itself is never pinned.** `/`, any `index.html`, the web manifest and `/health`
|
|
129
|
+
are fetched fresh whenever online. They are the only files whose URL does not change
|
|
130
|
+
between deploys — everything they pull in is content-hashed (`main-Cm1RhM4y.js`), so a
|
|
131
|
+
fresh shell drags in a new build as ordinary cache misses, and the old hashed files just
|
|
132
|
+
sit there harmlessly. Pin the shell instead and the boat stays on whatever build it
|
|
133
|
+
cached first, forever. Offline the last-seen shell is served as `STALE`, so the app still
|
|
134
|
+
opens with no uplink.
|
|
135
|
+
|
|
65
136
|
**Static vs dynamic — two strategies.** Tiles and app assets are **cache-first**
|
|
66
137
|
(pinned, offline forever). Dynamic `/api/*` data (AIS, weather, lightning) is
|
|
67
138
|
**network-first**: fetched live every time online (so it never goes stale), with the
|
|
@@ -113,6 +184,31 @@ passage area on demand.
|
|
|
113
184
|
boat's current position from local SignalK, builds a box, and warms the chart layers
|
|
114
185
|
in the background (progress in the status line). Idempotent; re-saving tops up. An
|
|
115
186
|
oversized radius+detail is refused (reduce one). Config `prefetchRadiusNm` / `prefetchDetailZoom`.
|
|
187
|
+
|
|
188
|
+
**Each layer is clamped to the zoom it actually publishes**, read from the upstream's
|
|
189
|
+
`/api/assets` (with a built-in fallback so it still works offline). Coastline tops out
|
|
190
|
+
at z13 where osm-standard reaches z19, so asking for "Harbor (z15)" fetches osm, seamap
|
|
191
|
+
and bathy to 15 and coastline only to 13 — instead of spending ~23% of the budget on
|
|
192
|
+
coastline tiles that can only 404. Those refusals also counted toward the cap, so a
|
|
193
|
+
request could be turned away for tiles that were never there.
|
|
194
|
+
|
|
195
|
+
**What fits under the 150k cap** (four layers, mid-latitude, from z6):
|
|
196
|
+
|
|
197
|
+
| radius | z12 | z13 | z14 | z15 |
|
|
198
|
+
|---|---|---|---|---|
|
|
199
|
+
| 25 nm | 1.1k | 4.1k | 12.5k | 46k |
|
|
200
|
+
| 50 nm | 4.1k | 15.3k | 48k | 179k ✗ |
|
|
201
|
+
| 100 nm | 15.5k | 60k | 192k ✗ | 715k ✗ |
|
|
202
|
+
|
|
203
|
+
So **50 nm at z14** or **25 nm at z15** are the practical maxima. A refused request
|
|
204
|
+
pre-warms *nothing* — check the status line rather than assuming coverage exists.
|
|
205
|
+
|
|
206
|
+
**Resolution is only limited when pre-warming.** On-demand caching has no zoom ceiling:
|
|
207
|
+
whatever the browser requests while online is stored and served offline afterwards, up to
|
|
208
|
+
the upstream's own maximum (z19 for osm-standard, z18 seamap). Browse a harbour approach
|
|
209
|
+
once with a connection and it is yours. Pre-warming is capped at z15 by the settings
|
|
210
|
+
dropdown — 3.5 m/px, ample for coastal work but coarser than the z17–18 you might want
|
|
211
|
+
alongside a berth.
|
|
116
212
|
- **Region prefetch (API)** — for scripted/arbitrary boxes, warm the detailed chart layers for an area:
|
|
117
213
|
```
|
|
118
214
|
POST /plugins/sailkick-boat/prefetch/region
|
|
@@ -128,6 +224,9 @@ passage area on demand.
|
|
|
128
224
|
Note: the app's **Coastline** and depth layers are default-off toggles — enable them in
|
|
129
225
|
the app to see the seeded base.
|
|
130
226
|
|
|
227
|
+
|
|
228
|
+
# 3 · Live data, served by the boat
|
|
229
|
+
|
|
131
230
|
## No login on the boat (single-tenant)
|
|
132
231
|
The cloud app gates behind a boat-account login (a `Secure` session cookie), which
|
|
133
232
|
can't work over the boat's plain-HTTP offline mirror — the browser drops a `Secure`
|
|
@@ -171,32 +270,8 @@ the chart.
|
|
|
171
270
|
This works **with no uplink at all**, which is when other vessels on your chart matter
|
|
172
271
|
most. Turn it off by hand-editing `proxy.serveAis: false`.
|
|
173
272
|
|
|
174
|
-
## Uploading AIS targets
|
|
175
|
-
The cloud app already draws other vessels, but its AIS source polls a SignalK server over
|
|
176
|
-
the LAN and keeps everything in memory — which cannot work once a boat is on a mobile
|
|
177
|
-
link. Enable **Upload AIS targets** and the boat pushes what its own receiver hears, so
|
|
178
|
-
the web app can show other boats, their heading and their trail from stored data.
|
|
179
|
-
|
|
180
|
-
Only **locally received** AIS is forwarded. A boat running an internet feed such as
|
|
181
|
-
`signalk-aisstream` would otherwise spend uplink bandwidth sending data the cloud can
|
|
182
|
-
fetch directly from the same API — known feeds are skipped automatically. The plugin logs
|
|
183
|
-
the AIS sources it sees, so you can name your own receiver in **Only this AIS source** if
|
|
184
|
-
you want to be explicit.
|
|
185
|
-
|
|
186
|
-
There is no radius or rate limit: a real AIS receiver is bounded by VHF line-of-sight,
|
|
187
|
-
which is the honest limiter, and offshore — where this data is most valuable, because
|
|
188
|
-
commercial feeds are blind there — it tends to zero. Vessel identity (name, dimensions,
|
|
189
|
-
ship type) repeats every few minutes and never changes, so it is re-sent at most hourly;
|
|
190
|
-
positions are never throttled.
|
|
191
273
|
|
|
192
|
-
|
|
193
|
-
stands down completely whenever the telemetry spool has a backlog, so a busy anchorage
|
|
194
|
-
can never delay or evict your own boat's data.
|
|
195
|
-
|
|
196
|
-
> ⚠️ **Requires a cloud that filters history on `self`.** Every AIS row is tagged
|
|
197
|
-
> `self=false`, and the cloud's Trends and track queries must filter `self == "true"`.
|
|
198
|
-
> Without that, other ships' speed and heading appear in *your* charts. Leave this off
|
|
199
|
-
> until the server side is in place.
|
|
274
|
+
# 4 · Backfill — an existing archive into the cloud
|
|
200
275
|
|
|
201
276
|
## Copying older history to the cloud (one-time)
|
|
202
277
|
If the boat recorded into its own InfluxDB before it started syncing — a
|
|
@@ -215,6 +290,43 @@ uploads is verified by counting the destination, and a write-only token cannot r
|
|
|
215
290
|
a partial write would be marked done and lost. The token is only needed while the
|
|
216
291
|
backfill runs: **revoke it afterwards**, live sync is unaffected.
|
|
217
292
|
|
|
293
|
+
> The two tokens a boat normally has are **both insufficient**: the scoped read token
|
|
294
|
+
> cannot write (`403 insufficient permissions for write`) and the scoped write token
|
|
295
|
+
> cannot verify. Mint one carrying *both* permissions on `<slug>_raw`.
|
|
296
|
+
|
|
297
|
+
### What to expect
|
|
298
|
+
This is a background job measured in **days or weeks**, not minutes. Measured on a real
|
|
299
|
+
boat — a Raspberry Pi over Starlink, migrating a `signalk-to-influxdb-v2` archive of
|
|
300
|
+
~57 GB going back 20 months:
|
|
301
|
+
|
|
302
|
+
| | |
|
|
303
|
+
|---|---|
|
|
304
|
+
| sustained throughput | ~10,000 points/s |
|
|
305
|
+
| archive consumed | ~24× realtime |
|
|
306
|
+
| a dense hour (2.7M points) | subdivides into ~30 chunks |
|
|
307
|
+
|
|
308
|
+
It gets faster as it goes: the walk is newest-first and older data is usually sparser
|
|
309
|
+
(that boat's recent hours held ~2.7M points, its oldest ~0.8M). Leave it running — it
|
|
310
|
+
survives restarts, and it yields to live telemetry so it cannot delay your own boat's
|
|
311
|
+
data.
|
|
312
|
+
|
|
313
|
+
**Watching progress.** The status line shows the current window and running total. For
|
|
314
|
+
detail, the manifest lists every completed hour:
|
|
315
|
+
|
|
316
|
+
```bash
|
|
317
|
+
cat <dataDir>/backfill.json # {"done":{...},"points":532062426,"complete":false}
|
|
318
|
+
```
|
|
319
|
+
|
|
320
|
+
From the cloud side, the oldest point in your bucket marches backwards as it works — that
|
|
321
|
+
is the single clearest signal that it is delivering:
|
|
322
|
+
|
|
323
|
+
```flux
|
|
324
|
+
from(bucket:"<slug>_raw")|>range(start:0)|>keep(columns:["_time"])|>group()|>min(column:"_time")
|
|
325
|
+
```
|
|
326
|
+
|
|
327
|
+
Once the bucket is large that query gets expensive; counting a single one-hour window
|
|
328
|
+
near the frontier is cheaper and tells you the same thing.
|
|
329
|
+
|
|
218
330
|
Safe to re-run. Points are keyed by (measurement, tagset, nanosecond timestamp), so an
|
|
219
331
|
identical point overwrites rather than duplicating — an interrupted migration is simply
|
|
220
332
|
run again.
|
|
@@ -287,6 +399,9 @@ works **offline** with the boat's own data. Only when no telemetry source is
|
|
|
287
399
|
available at all do these paths **fall through to the cloud mirror**, so an
|
|
288
400
|
online boat is never worse off than before.
|
|
289
401
|
|
|
402
|
+
|
|
403
|
+
# Running it
|
|
404
|
+
|
|
290
405
|
## Setup: register on the web, then paste the token
|
|
291
406
|
1. Register your boat at **[www.sailkick.io](https://www.sailkick.io)** (an invite code
|
|
292
407
|
is needed — see above). The signup screen shows your boat's ingest credentials.
|
|
@@ -389,6 +504,23 @@ curl -i -XPOST "https://sync.sailkick.io/api/v2/write?org=sailkick&bucket=<slug>
|
|
|
389
504
|
```
|
|
390
505
|
`204` = good, `401` = token not valid for that bucket.
|
|
391
506
|
|
|
507
|
+
## Known gaps
|
|
508
|
+
Things this deliberately does not do yet, so they don't come as a surprise:
|
|
509
|
+
|
|
510
|
+
- **Wind and current fields are not prefetched.** Velocity tiles are pinned once fetched
|
|
511
|
+
(they are keyed by forecast run, so they never go stale), but nothing warms them ahead
|
|
512
|
+
of time — offline, the wind field covers only where you have already panned. Old
|
|
513
|
+
forecast runs are also never pruned, so their tiles accumulate.
|
|
514
|
+
- **The backfill cannot fill gaps in live coverage.** It only copies data *older* than
|
|
515
|
+
the point where cloud sync began, so if live sync ever dropped data — an outage longer
|
|
516
|
+
than the spool's capacity — that hole stays, even when the local archive still has it.
|
|
517
|
+
- **Backfilled history is not browsable in the app.** `/api/history/*` accepts only a
|
|
518
|
+
relative window clamped to 24 h, so once 2024 is in the cloud there is still no way to
|
|
519
|
+
display it. That needs `from`/`to` support server-side.
|
|
520
|
+
- **Per-path sync rate is approximate.** The subscription sets `period` without a
|
|
521
|
+
`policy`, so Signal K's default governs and a few chatty paths exceed the configured
|
|
522
|
+
interval.
|
|
523
|
+
|
|
392
524
|
## Dev / tests
|
|
393
525
|
```bash
|
|
394
526
|
npm install && npm test # proxy: mirror/cache/offline + Express route; sync: subscribe+buffer
|
package/lib/proxy/index.js
CHANGED
|
@@ -30,7 +30,39 @@ const { countBboxTiles, bboxTiles, boxAround } = require('./tiles')
|
|
|
30
30
|
// must stay live, otherwise the boat pins itself to a stale forecast run forever.
|
|
31
31
|
const IMMUTABLE_API_PREFIXES = ['/api/velocity/tiles/']
|
|
32
32
|
const isImmutableApi = (p) => IMMUTABLE_API_PREFIXES.some((x) => p.startsWith(x))
|
|
33
|
-
|
|
33
|
+
|
|
34
|
+
// HTML entry documents are network-first too, for the opposite reason to tiles: they are
|
|
35
|
+
// the ONE file whose URL never changes across deploys. Everything they pull in is
|
|
36
|
+
// content-hashed (main-Cm1RhM4y.js, main-BboaeyYc.css), so a fresh index.html
|
|
37
|
+
// automatically drags in the new build as ordinary cache misses — but a pinned one keeps
|
|
38
|
+
// the boat on whatever version it happened to cache first, forever.
|
|
39
|
+
//
|
|
40
|
+
// The cache manifest was supposed to handle this by announcing a new `app` id. In
|
|
41
|
+
// practice that id is the app's package version, which does not change on every deploy
|
|
42
|
+
// (three deploys in one day all reported "0.2.0"), so nothing was ever invalidated.
|
|
43
|
+
// Not caching the entry document removes the dependency on that signal entirely.
|
|
44
|
+
//
|
|
45
|
+
// Costs one ~8 KB request per app load while online. Offline it falls back to the cached
|
|
46
|
+
// copy as STALE like any network-first path, so the app still opens with no uplink.
|
|
47
|
+
// /health reports the running build and uptime — pinning it freezes the version the UI
|
|
48
|
+
// displays, which is its own small version of this bug.
|
|
49
|
+
const LIVE_PATHS = new Set(['/health'])
|
|
50
|
+
const isEntryDocument = (p) => {
|
|
51
|
+
const path = p.split('?')[0]
|
|
52
|
+
return path === '/' || path.endsWith('/') || path.endsWith('.html') ||
|
|
53
|
+
path.endsWith('.webmanifest') || LIVE_PATHS.has(path)
|
|
54
|
+
}
|
|
55
|
+
const isNetworkFirst = (p) => isEntryDocument(p) || (p.startsWith('/api/') && !isImmutableApi(p))
|
|
56
|
+
|
|
57
|
+
// A prefetch must not spend its tile budget on tiles that cannot exist. Layers top out
|
|
58
|
+
// at very different zooms — coastline at 13 where osm-standard goes to 19 — so applying
|
|
59
|
+
// one "detail level" to all of them wasted 23% of a real 50nm/z15 run on ~55k coastline
|
|
60
|
+
// requests that could only 404. Those also counted toward the cap, so a request could be
|
|
61
|
+
// refused for tiles that were never there.
|
|
62
|
+
//
|
|
63
|
+
// The upstream publishes the real limits at /api/assets, so ask rather than hardcode;
|
|
64
|
+
// these are only the fallback for when it cannot be reached (offline, or an older host).
|
|
65
|
+
const FALLBACK_MAX_ZOOM = { 'osm-standard': 19, seamap: 18, bathy: 16, coastline: 13, 'natural-earth': 7 }
|
|
34
66
|
|
|
35
67
|
function createProxy (app, options) {
|
|
36
68
|
const log = (m) => (app.debug ? app.debug('[proxy] ' + m) : console.log('[sailkick-boat:proxy]', m))
|
|
@@ -129,6 +161,56 @@ function createProxy (app, options) {
|
|
|
129
161
|
return `proxy: mirror ${cfg.upstream}${cfg.port ? ' :' + cfg.port : ''}; live -> ${cfg.localSignalk}${m}${s}${areaStatus()}`
|
|
130
162
|
}
|
|
131
163
|
|
|
164
|
+
// Per-layer max zoom from the upstream asset manifest, cached. Never throws: on any
|
|
165
|
+
// failure the fallback table is used, so a prefetch still runs offline.
|
|
166
|
+
let zoomCache = null
|
|
167
|
+
// Synchronous view, so planning a prefetch never waits on the network. The fallback
|
|
168
|
+
// matches what the upstream publishes today; refreshLayerZooms() replaces it in the
|
|
169
|
+
// background at start, and the region API awaits it for the freshest numbers.
|
|
170
|
+
const zooms = () => zoomCache || FALLBACK_MAX_ZOOM
|
|
171
|
+
async function layerMaxZooms () {
|
|
172
|
+
if (zoomCache) return zoomCache
|
|
173
|
+
const out = { ...FALLBACK_MAX_ZOOM }
|
|
174
|
+
try {
|
|
175
|
+
const r = await fetch(cfg.upstream + '/api/assets', { signal: AbortSignal.timeout(cfg.timeoutMs) })
|
|
176
|
+
if (r.ok) {
|
|
177
|
+
const j = await r.json()
|
|
178
|
+
const raster = j && j.tiles && j.tiles.manifest && j.tiles.manifest.maxZoom
|
|
179
|
+
if (raster && typeof raster === 'object') Object.assign(out, raster)
|
|
180
|
+
// coastline is a vector layer and reports its own bounds separately
|
|
181
|
+
if (j && j.coastline && Number.isFinite(j.coastline.maxZoom)) out.coastline = j.coastline.maxZoom
|
|
182
|
+
log(`layer max zooms: ${Object.entries(out).map(([k, v]) => k + '=' + v).join(' ')}`)
|
|
183
|
+
}
|
|
184
|
+
} catch { log('could not read /api/assets — using fallback layer zoom limits') }
|
|
185
|
+
zoomCache = out
|
|
186
|
+
return out
|
|
187
|
+
}
|
|
188
|
+
|
|
189
|
+
// COUNT ONLY — arithmetic, no allocation. The cap exists to stop absurd requests, so
|
|
190
|
+
// it has to be checked before anything is enumerated: a global z0-15 box is ~1.4
|
|
191
|
+
// billion tiles per layer, which must never reach an array.
|
|
192
|
+
function planTiles (bbox, minZoom, maxZoom, layers, zooms) {
|
|
193
|
+
const perLayer = {}
|
|
194
|
+
let total = 0
|
|
195
|
+
for (const l of layers) {
|
|
196
|
+
const top = Math.min(maxZoom, zooms[l] == null ? maxZoom : zooms[l])
|
|
197
|
+
const count = top < minZoom ? 0 : countBboxTiles(bbox, minZoom, top)
|
|
198
|
+
perLayer[l] = { top, count }
|
|
199
|
+
total += count
|
|
200
|
+
}
|
|
201
|
+
return { total, perLayer }
|
|
202
|
+
}
|
|
203
|
+
|
|
204
|
+
// Enumerate only once the plan is known to be within the cap. A generator, so even a
|
|
205
|
+
// large accepted plan is streamed rather than held twice.
|
|
206
|
+
function * enumerateTiles (bbox, minZoom, perLayer) {
|
|
207
|
+
const ext = (l) => (l === 'coastline' ? 'pbf' : 'png')
|
|
208
|
+
for (const [l, { top }] of Object.entries(perLayer)) {
|
|
209
|
+
if (top < minZoom) continue
|
|
210
|
+
for (const { z, x, y } of bboxTiles(bbox, minZoom, top)) yield `/tiles/${l}/${z}/${x}/${y}.${ext(l)}`
|
|
211
|
+
}
|
|
212
|
+
}
|
|
213
|
+
|
|
132
214
|
const isLocal = (p) => cfg.localPaths.some((lp) => p === lp || p.startsWith(lp + '/') || p.startsWith(lp + '?'))
|
|
133
215
|
|
|
134
216
|
async function serveMirror (req, res) {
|
|
@@ -351,18 +433,15 @@ function createProxy (app, options) {
|
|
|
351
433
|
const maxZoom = Math.max(minZoom, clampInt(body.maxZoom, 15, 0, 20))
|
|
352
434
|
const layers = (Array.isArray(body.layers) && body.layers.length) ? body.layers : ['osm-standard', 'bathy', 'seamap', 'coastline']
|
|
353
435
|
const CAP = 50000
|
|
354
|
-
|
|
355
|
-
|
|
356
|
-
|
|
436
|
+
if (!zoomCache) layerMaxZooms().catch(() => {}) // refresh for next time; never block a request
|
|
437
|
+
const { total, perLayer } = planTiles(bbox, minZoom, maxZoom, layers, zooms())
|
|
438
|
+
if (total > CAP && !body.force) {
|
|
439
|
+
res.json({ ok: true, capped: true, estimate: total, cap: CAP, perLayer, message: `~${total} tiles exceeds ${CAP}; narrow bbox/zoom or pass force:true` })
|
|
357
440
|
return
|
|
358
441
|
}
|
|
359
|
-
const
|
|
360
|
-
const paths = []
|
|
361
|
-
for (const layer of layers) {
|
|
362
|
-
for (const { z, x, y } of bboxTiles(bbox, minZoom, maxZoom)) paths.push(`/tiles/${layer}/${z}/${x}/${y}.${ext(layer)}`)
|
|
363
|
-
}
|
|
442
|
+
const paths = [...enumerateTiles(bbox, minZoom, perLayer)]
|
|
364
443
|
const r = await warmMany(paths, { concurrency: body.concurrency || 6 })
|
|
365
|
-
res.json({ ok: true, requested: paths.length, ...r })
|
|
444
|
+
res.json({ ok: true, requested: paths.length, perLayer, ...r })
|
|
366
445
|
})
|
|
367
446
|
}
|
|
368
447
|
|
|
@@ -392,14 +471,19 @@ function createProxy (app, options) {
|
|
|
392
471
|
if (attempt < 30 && !stopping) { const t = setTimeout(() => startAreaPrefetch(attempt + 1), 10000); if (t.unref) t.unref() } else log('area prefetch: no boat position — skipped')
|
|
393
472
|
return null
|
|
394
473
|
}
|
|
474
|
+
// Learn the real per-layer limits for next time; plan now with what we have. Only
|
|
475
|
+
// probed when a prefetch is actually configured — a boat that never prefetches
|
|
476
|
+
// should not make the request at all.
|
|
477
|
+
if (!zoomCache) layerMaxZooms().catch(() => {})
|
|
478
|
+
|
|
395
479
|
const bbox = boxAround(pos.latitude, pos.longitude, radius)
|
|
396
480
|
const CAP = 150000
|
|
397
|
-
const total =
|
|
481
|
+
const { total, perLayer } = planTiles(bbox, minZoom, maxZoom, layers, zooms())
|
|
482
|
+
const clamped = Object.entries(perLayer).filter(([, v]) => v.top < maxZoom)
|
|
483
|
+
if (clamped.length) log(`area prefetch: ${clamped.map(([l, v]) => `${l} capped at z${v.top}`).join(', ')} (upstream limit — those tiles do not exist)`)
|
|
398
484
|
if (total > CAP) { area = { capped: true, total, radius, maxZoom }; log(`area prefetch: ~${total} tiles too large — reduce radius/detail`); return null }
|
|
399
|
-
const
|
|
400
|
-
|
|
401
|
-
for (const l of layers) { for (const { z, x, y } of bboxTiles(bbox, minZoom, maxZoom)) paths.push(`/tiles/${l}/${z}/${x}/${y}.${ext(l)}`) }
|
|
402
|
-
area = { done: 0, total: paths.length, running: true, radius, maxZoom }
|
|
485
|
+
const paths = [...enumerateTiles(bbox, minZoom, perLayer)]
|
|
486
|
+
area = { done: 0, total: paths.length, running: true, radius, maxZoom, perLayer }
|
|
403
487
|
log(`area prefetch: ${radius}nm around ${pos.latitude.toFixed(2)},${pos.longitude.toFixed(2)} to z${maxZoom} — ${paths.length} tiles`)
|
|
404
488
|
area.promise = warmMany(paths, { concurrency: pf.concurrency || 4, onProgress: () => { area.done++ } })
|
|
405
489
|
.then((r) => { area.running = false; area.result = r; log(`area prefetch done: ${r.cached} cached, ${r.empty} empty, ${r.failed} failed`); return r })
|
|
@@ -416,4 +500,4 @@ function createProxy (app, options) {
|
|
|
416
500
|
return { start, stop, status, handleGet, handlePrefetch, handlePrefetchRegion, handleClear, _serveMirror: serveMirror, _serveConfig: serveConfig, _manifest: () => manifest, _seeder: () => seeder, _startAreaPrefetch: startAreaPrefetch, _area: () => area }
|
|
417
501
|
}
|
|
418
502
|
|
|
419
|
-
module.exports = { createProxy, isNetworkFirst, isImmutableApi }
|
|
503
|
+
module.exports = { createProxy, isNetworkFirst, isImmutableApi, isEntryDocument }
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "sailkick-boat",
|
|
3
|
-
"version": "0.18.
|
|
3
|
+
"version": "0.18.4",
|
|
4
4
|
"description": "EARLY ALPHA — cloud telemetry + offline maps for sailkick boats (www.sailkick.io; register on the web, paste the write token). Gapless boat→cloud telemetry sync to InfluxDB, and a local proxy that keeps the sailkick app and its charts/maps working fully offline on board.",
|
|
5
5
|
"main": "index.js",
|
|
6
6
|
"scripts": {
|