sailkick-boat 0.14.5 → 0.17.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +79 -15
- package/index.js +112 -15
- package/lib/ais/index.js +262 -0
- package/lib/backfill/index.js +257 -0
- package/lib/backfill/lineproto.js +114 -0
- package/lib/history/index.js +21 -138
- package/lib/history/ring.js +24 -4
- package/lib/sync/index.js +8 -1
- package/lib/sync/lineprotocol.js +6 -1
- package/lib/telemetry/signalk-map.js +27 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -27,7 +27,7 @@ One Signal K plugin, independently-toggleable modules — so the boat stays
|
|
|
27
27
|
same contracts as the cloud, offline-first):
|
|
28
28
|
- **`/ws/telemetry`** — the app's live telemetry bus, fed from local SignalK.
|
|
29
29
|
- **`/api/history/{series,track}`** — the app's Trends panel + track, served
|
|
30
|
-
from
|
|
30
|
+
from a live ring sampled on the boat — full local history, no database.
|
|
31
31
|
|
|
32
32
|
Kept as separate modules so a proxy fault can't wedge the data-critical sync path.
|
|
33
33
|
|
|
@@ -139,13 +139,79 @@ config passes through untouched. (Hand-edit `proxy.openAccess: false` to keep th
|
|
|
139
139
|
login gate — there is no toggle, since the gate cannot complete over the mirror anyway.)
|
|
140
140
|
|
|
141
141
|
## Local history (offline Trends + track)
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
142
|
+
**One source: a live ring**, sampled from the same BoatState that feeds `/ws/telemetry`
|
|
143
|
+
— no database, works on a Victron GX with nothing else installed. `historyAvailable` is
|
|
144
|
+
forced on so the app shows the Trends panel, and the eight channels match what the cloud
|
|
145
|
+
serves, `stw` included.
|
|
146
|
+
|
|
147
|
+
There is deliberately **no way to point this at a local InfluxDB**. Until v0.15.0 a read
|
|
148
|
+
token did exactly that, and it was a trap: the app only ever asks for a *relative* window
|
|
149
|
+
clamped to 24 h, so aiming it at a bucket of older data matched nothing — Trends went
|
|
150
|
+
blank **and** the working live ring was switched off. If you still have a token in your
|
|
151
|
+
config it is now inert; the plugin logs `history -> live ring` regardless.
|
|
152
|
+
|
|
153
|
+
A local InfluxDB is not a competitor here anyway. The app never requests finer than
|
|
154
|
+
`every=5s` over a 24 h window, and the ring's floor at that window is 2 s — set
|
|
155
|
+
`ringSampleSec: 5` and it matches anything the UI can draw, from live state. What an old
|
|
156
|
+
database *is* good for is its contents ending up **in the cloud** — see below.
|
|
157
|
+
|
|
158
|
+
## Uploading AIS targets
|
|
159
|
+
The cloud app already draws other vessels, but its AIS source polls a SignalK server over
|
|
160
|
+
the LAN and keeps everything in memory — which cannot work once a boat is on a mobile
|
|
161
|
+
link. Enable **Upload AIS targets** and the boat pushes what its own receiver hears, so
|
|
162
|
+
the web app can show other boats, their heading and their trail from stored data.
|
|
163
|
+
|
|
164
|
+
Only **locally received** AIS is forwarded. A boat running an internet feed such as
|
|
165
|
+
`signalk-aisstream` would otherwise spend uplink bandwidth sending data the cloud can
|
|
166
|
+
fetch directly from the same API — known feeds are skipped automatically. The plugin logs
|
|
167
|
+
the AIS sources it sees, so you can name your own receiver in **Only this AIS source** if
|
|
168
|
+
you want to be explicit.
|
|
169
|
+
|
|
170
|
+
There is no radius or rate limit: a real AIS receiver is bounded by VHF line-of-sight,
|
|
171
|
+
which is the honest limiter, and offshore — where this data is most valuable, because
|
|
172
|
+
commercial feeds are blind there — it tends to zero. Vessel identity (name, dimensions,
|
|
173
|
+
ship type) repeats every few minutes and never changes, so it is re-sent at most hourly;
|
|
174
|
+
positions are never throttled.
|
|
175
|
+
|
|
176
|
+
Telemetry always wins the link. AIS buffers in its **own** spool with its own cap and
|
|
177
|
+
stands down completely whenever the telemetry spool has a backlog, so a busy anchorage
|
|
178
|
+
can never delay or evict your own boat's data.
|
|
179
|
+
|
|
180
|
+
> ⚠️ **Requires a cloud that filters history on `self`.** Every AIS row is tagged
|
|
181
|
+
> `self=false`, and the cloud's Trends and track queries must filter `self == "true"`.
|
|
182
|
+
> Without that, other ships' speed and heading appear in *your* charts. Leave this off
|
|
183
|
+
> until the server side is in place.
|
|
184
|
+
|
|
185
|
+
## Copying older history to the cloud (one-time)
|
|
186
|
+
If the boat recorded into its own InfluxDB before it started syncing — a
|
|
187
|
+
`signalk-to-influxdb-v2` bucket, or an imported logbook — the **backfill** copies it up
|
|
188
|
+
so the cloud holds your full history. Live sync can't do this: it only ever sees deltas
|
|
189
|
+
arriving now, and the spool only replays what it captured itself while offline.
|
|
190
|
+
|
|
191
|
+
Fill the **Copy older history to the cloud** section and save. It walks backwards in
|
|
192
|
+
one-hour windows (newest first, so recent history lands first), resumes after a restart
|
|
193
|
+
from a manifest, and stands aside whenever live telemetry has a backlog — the data-
|
|
194
|
+
critical path is never starved by a bulk upload. Progress shows in the status line.
|
|
195
|
+
|
|
196
|
+
It needs a **cloud read+write token**, not the write token from signup. Every hour it
|
|
197
|
+
uploads is verified by counting the destination, and a write-only token cannot read. A
|
|
198
|
+
`204` means InfluxDB accepted the bytes, not that every point landed — without the count
|
|
199
|
+
a partial write would be marked done and lost. The token is only needed while the
|
|
200
|
+
backfill runs: **revoke it afterwards**, live sync is unaffected.
|
|
201
|
+
|
|
202
|
+
Safe to re-run. Points are keyed by (measurement, tagset, nanosecond timestamp), so an
|
|
203
|
+
identical point overwrites rather than duplicating — an interrupted migration is simply
|
|
204
|
+
run again. Everything in the bucket is copied, including AIS contexts; hand-edit
|
|
205
|
+
`backfill.selfOnly: true` to restrict it to your own vessel if cloud series cardinality
|
|
206
|
+
becomes a problem.
|
|
207
|
+
|
|
208
|
+
**True wind comes from your instruments.** If the boat publishes
|
|
209
|
+
`environment.wind.speedTrue` / `directionTrue`, those are stored verbatim — a wind
|
|
210
|
+
system corrects for heel and leeway against water-referenced boat speed, and every other
|
|
211
|
+
display aboard shows its numbers. Only when the boat publishes none is true wind
|
|
212
|
+
derived, and then from **STW**, not SOG: true wind is relative to motion through the
|
|
213
|
+
water. (Before v0.14.6 the derivation used SOG, which in 3 kt of foul tide skewed TWD by
|
|
214
|
+
~12° and TWS by ~1.8 kt.) SOG is the last resort for boats with no paddlewheel.
|
|
149
215
|
- **Persistent (append-log):** the ring is saved as a JSONL append-log at
|
|
150
216
|
`<dataDir>/history/history-ring.jsonl` (on the SSD/USB with the tiles; override with
|
|
151
217
|
`proxy.history.ringDir`), so it **survives restarts**. Each sample appends one line; the file is
|
|
@@ -158,14 +224,14 @@ Two ways, chosen automatically:
|
|
|
158
224
|
|
|
159
225
|
The sailkick app is deployment-agnostic about history: *central Influx in the
|
|
160
226
|
cloud, in-memory ring on a DB-less edge*. The boat is a third case — an edge that
|
|
161
|
-
serves the app's history endpoints from its **own** data
|
|
227
|
+
serves the app's history endpoints from its **own** live data:
|
|
162
228
|
```
|
|
163
229
|
GET /api/history/series?window=3600s&every=30s -> { series: { sog|heading|tws|… : [[tMs,val],…] } }
|
|
164
230
|
GET /api/history/track?window=3600s -> { track: [{ t, lat, lon }, …] }
|
|
165
231
|
```
|
|
166
232
|
Same JSON the cloud returns, so the browser can't tell the difference — but it
|
|
167
|
-
works **offline** with the boat's own data. Only when
|
|
168
|
-
|
|
233
|
+
works **offline** with the boat's own data. Only when no telemetry source is
|
|
234
|
+
available at all do these paths **fall through to the cloud mirror**, so an
|
|
169
235
|
online boat is never worse off than before.
|
|
170
236
|
|
|
171
237
|
## Setup: register on the web, then paste the token
|
|
@@ -205,18 +271,16 @@ in `index.js`.
|
|
|
205
271
|
|
|
206
272
|
- **Sailkick account**: `slug` (boat name), `writeToken`
|
|
207
273
|
- **Telemetry sync → cloud**: `enabled`
|
|
274
|
+
- **Upload AIS targets**: `enabled` (default off), `source`
|
|
208
275
|
- **Offline app & maps**: `enabled`, `proxyPort` (default 8080), `localSignalkUrl`
|
|
209
276
|
(default `http://127.0.0.1:3000`), `dataDir`, `seedEnabled`, `prefetchRadiusNm`,
|
|
210
|
-
`prefetchDetailZoom
|
|
277
|
+
`prefetchDetailZoom`
|
|
211
278
|
|
|
212
279
|
`dataDir` is the one storage location — cached maps, the telemetry spool and the
|
|
213
280
|
history ring log all live under it. **Put it on the SSD/USB disk, not the SD card.**
|
|
214
281
|
Leave it blank and each part falls back to its historical spot under the plugin data
|
|
215
282
|
dir.
|
|
216
283
|
|
|
217
|
-
`historyToken` is optional and only matters if the boat already runs its own InfluxDB.
|
|
218
|
-
Blank — the normal case — uses the built-in DB-less ring.
|
|
219
|
-
|
|
220
284
|
Cache-manifest polling is always on (no toggle): tile freshness comes from the cloud
|
|
221
285
|
announcing bakes, and without it a re-baked dataset would never refresh.
|
|
222
286
|
|
package/index.js
CHANGED
|
@@ -6,6 +6,8 @@ const { createSync } = require('./lib/sync')
|
|
|
6
6
|
const { createProxy } = require('./lib/proxy')
|
|
7
7
|
const { createTelemetry } = require('./lib/telemetry')
|
|
8
8
|
const { createHistory } = require('./lib/history')
|
|
9
|
+
const { createBackfill } = require('./lib/backfill')
|
|
10
|
+
const { createAis } = require('./lib/ais')
|
|
9
11
|
const { resolveAccountConfig } = require('./lib/account')
|
|
10
12
|
|
|
11
13
|
// sailkick-boat: one Signal K plugin, two independently-toggleable modules —
|
|
@@ -43,15 +45,8 @@ const PROXY_TUNING = { requestTimeoutMs: 20000, localPaths: ['/signalk'], teleme
|
|
|
43
45
|
const MANIFEST = { enabled: true, path: '/api/cache-manifest', pollIntervalSec: 300 }
|
|
44
46
|
const SEED_TUNING = { coastlineMaxZoom: 8, seabedMaxZoom: 6, concurrency: 4 }
|
|
45
47
|
const PREFETCH_TUNING = { concurrency: 4 }
|
|
46
|
-
const HISTORY_TUNING = {
|
|
47
|
-
|
|
48
|
-
org: 'signalk',
|
|
49
|
-
bucket: 'signalk',
|
|
50
|
-
requestTimeoutMs: 15000,
|
|
51
|
-
ringPersist: true,
|
|
52
|
-
ringWindowSec: 86400,
|
|
53
|
-
ringSampleSec: 15
|
|
54
|
-
}
|
|
48
|
+
const HISTORY_TUNING = { ringPersist: true, ringWindowSec: 86400, ringSampleSec: 15 }
|
|
49
|
+
const BACKFILL_SRC_DEFAULTS = { influxUrl: 'http://127.0.0.1:8086', org: 'signalk', bucket: 'signalk' }
|
|
55
50
|
|
|
56
51
|
// A *cloud* endpoint on a loopback or private address means telemetry never leaves the
|
|
57
52
|
// LAN. On the wire that is indistinguishable from a normal offline backlog — the spool
|
|
@@ -69,6 +64,19 @@ function isPrivateHostUrl (u) {
|
|
|
69
64
|
} catch { return false }
|
|
70
65
|
}
|
|
71
66
|
|
|
67
|
+
// The Signal K config UI writes every schema default on save, so a freshly pre-filled
|
|
68
|
+
// `backfillOrg: "signalk"` can appear on a boat whose archive has always lived in org
|
|
69
|
+
// "addiction" under the pre-0.15 `proxy.history` block — and the backfill would read an
|
|
70
|
+
// empty bucket. Rule: an explicitly-changed field wins; else a legacy value that differs
|
|
71
|
+
// from the default wins; else the default.
|
|
72
|
+
function pickField (flat, legacy, dflt) {
|
|
73
|
+
const f = String(flat == null ? '' : flat).trim()
|
|
74
|
+
const l = String(legacy == null ? '' : legacy).trim()
|
|
75
|
+
if (f && f !== dflt) return { value: f, from: 'config' }
|
|
76
|
+
if (l && l !== dflt) return { value: l, from: 'legacy' }
|
|
77
|
+
return { value: f || l || dflt, from: 'default' }
|
|
78
|
+
}
|
|
79
|
+
|
|
72
80
|
// Endpoint resolution with one rule: the constant wins unless self-hosting is declared.
|
|
73
81
|
// Returns { url, ignored } — `ignored` is the stale value we refused, so the caller can
|
|
74
82
|
// say so out loud instead of leaving the owner to guess why nothing arrives.
|
|
@@ -84,6 +92,8 @@ module.exports = function (app) {
|
|
|
84
92
|
let proxy = null
|
|
85
93
|
let telemetry = null
|
|
86
94
|
let history = null
|
|
95
|
+
let backfill = null
|
|
96
|
+
let ais = null
|
|
87
97
|
let statusTimer = null
|
|
88
98
|
let accountStatus = null
|
|
89
99
|
let syncWarning = null
|
|
@@ -116,6 +126,28 @@ module.exports = function (app) {
|
|
|
116
126
|
enabled: { type: 'boolean', title: 'Enable telemetry sync', default: true }
|
|
117
127
|
}
|
|
118
128
|
},
|
|
129
|
+
ais: {
|
|
130
|
+
type: 'object',
|
|
131
|
+
title: 'Upload AIS targets',
|
|
132
|
+
description: 'Send the AIS this boat\'s own receiver hears to the cloud, so the web app can show other vessels, their heading and their trail. Only locally received AIS is forwarded — an internet feed such as signalk-aisstream is skipped, since the cloud can fetch that itself without spending your uplink. REQUIRES a cloud that filters history on self; until then leave this off or the owner\'s SOG and heading charts will pick up other ships.',
|
|
133
|
+
properties: {
|
|
134
|
+
enabled: { type: 'boolean', title: 'Upload AIS targets', default: false },
|
|
135
|
+
source: { type: 'string', title: 'Only this AIS source', description: 'Blank forwards every source except known internet feeds. The plugin logs the AIS sources it sees — copy the one for your own receiver here if you want to be explicit.' }
|
|
136
|
+
}
|
|
137
|
+
},
|
|
138
|
+
backfill: {
|
|
139
|
+
type: 'object',
|
|
140
|
+
title: 'Copy older history to the cloud (one-time)',
|
|
141
|
+
description: 'If this boat recorded data into its own InfluxDB before it started syncing — a signalk-to-influxdb-v2 bucket, or an imported logbook — this copies it up so the app can chart it. It runs in the background, resumes after a restart, verifies every hour it uploads, and stands aside whenever live telemetry is behind. Safe to re-run: identical points overwrite rather than duplicate.',
|
|
142
|
+
properties: {
|
|
143
|
+
enabled: { type: 'boolean', title: 'Run the backfill', default: false },
|
|
144
|
+
cloudToken: { type: 'string', title: 'Cloud read+write token', description: 'A token with READ and WRITE on your cloud bucket. Your normal write token cannot read, and reading is how each uploaded hour is verified. Needed only while the backfill runs — revoke it afterwards; live sync is unaffected.' },
|
|
145
|
+
sourceToken: { type: 'string', title: 'Local InfluxDB read token' },
|
|
146
|
+
sourceUrl: { type: 'string', title: '…local InfluxDB URL', default: 'http://127.0.0.1:8086' },
|
|
147
|
+
sourceOrg: { type: 'string', title: '…organization', default: 'signalk' },
|
|
148
|
+
sourceBucket: { type: 'string', title: '…bucket', default: 'signalk' }
|
|
149
|
+
}
|
|
150
|
+
},
|
|
119
151
|
proxy: {
|
|
120
152
|
type: 'object',
|
|
121
153
|
title: 'Offline app & maps',
|
|
@@ -140,8 +172,7 @@ module.exports = function (app) {
|
|
|
140
172
|
enum: [12, 13, 14, 15],
|
|
141
173
|
enumNames: ['Overview (z12)', 'Coastal (z13)', 'Detailed (z14)', 'Harbor (z15)'],
|
|
142
174
|
default: 13
|
|
143
|
-
}
|
|
144
|
-
historyToken: { type: 'string', title: 'Local InfluxDB read token (optional)', description: 'Only if this boat already runs its own InfluxDB (e.g. signalk-to-influxdb-v2) and you want full history from it. Leave blank — the plugin then keeps its own lightweight history, which is what most boats want.' }
|
|
175
|
+
}
|
|
145
176
|
}
|
|
146
177
|
}
|
|
147
178
|
}
|
|
@@ -220,6 +251,8 @@ module.exports = function (app) {
|
|
|
220
251
|
const oldSeed = p.seed || {}
|
|
221
252
|
const oldPrefetch = p.prefetch || {}
|
|
222
253
|
const oldHistory = p.history || {}
|
|
254
|
+
console.log('[sailkick-boat] history -> live ring')
|
|
255
|
+
|
|
223
256
|
if (upstream.ignored) {
|
|
224
257
|
;(app.error || console.error)(`[sailkick-boat] ignoring proxy.sailkickUrl "${upstream.ignored}" left over from an older config — mirroring ${upstream.url}. Set proxy.selfHosted:true to keep your own server.`)
|
|
225
258
|
}
|
|
@@ -251,10 +284,6 @@ module.exports = function (app) {
|
|
|
251
284
|
history: {
|
|
252
285
|
...HISTORY_TUNING,
|
|
253
286
|
enabled: oldHistory.enabled !== false,
|
|
254
|
-
influxUrl: oldHistory.influxUrl || HISTORY_TUNING.influxUrl,
|
|
255
|
-
org: oldHistory.org || HISTORY_TUNING.org,
|
|
256
|
-
bucket: oldHistory.bucket || HISTORY_TUNING.bucket,
|
|
257
|
-
token: p.historyToken || oldHistory.token || '',
|
|
258
287
|
ringPersist: oldHistory.ringPersist !== false,
|
|
259
288
|
ringWindowSec: oldHistory.ringWindowSec || HISTORY_TUNING.ringWindowSec,
|
|
260
289
|
ringSampleSec: oldHistory.ringSampleSec || HISTORY_TUNING.ringSampleSec,
|
|
@@ -295,6 +324,68 @@ module.exports = function (app) {
|
|
|
295
324
|
}
|
|
296
325
|
}
|
|
297
326
|
|
|
327
|
+
// --- AIS upload (isolated; yields to telemetry, its own spool) ---
|
|
328
|
+
const aisOpts = opts.ais || {}
|
|
329
|
+
if (aisOpts.enabled === true) {
|
|
330
|
+
if (!b.writeToken) {
|
|
331
|
+
(app.error || console.error)('[sailkick-boat] AIS upload needs a paired account for its destination bucket — skipped')
|
|
332
|
+
} else {
|
|
333
|
+
try {
|
|
334
|
+
ais = createAis(app, {
|
|
335
|
+
influxUrl: influx.url,
|
|
336
|
+
org: b.org || SYNC_TUNING.org,
|
|
337
|
+
bucket: b.bucket,
|
|
338
|
+
token: b.writeToken,
|
|
339
|
+
source: String(aisOpts.source || '').trim() || null,
|
|
340
|
+
spoolDir: store ? path.join(store, 'ais-spool') : undefined,
|
|
341
|
+
pending: sync ? sync.pending : null
|
|
342
|
+
})
|
|
343
|
+
ais.start()
|
|
344
|
+
console.log(`[sailkick-boat] ais -> ${influx.url} bucket=${b.bucket}${aisOpts.source ? ' source=' + aisOpts.source : ''}`)
|
|
345
|
+
} catch (e) {
|
|
346
|
+
(app.error || console.error)('[sailkick-boat] AIS start failed: ' + e.message)
|
|
347
|
+
ais = null
|
|
348
|
+
}
|
|
349
|
+
}
|
|
350
|
+
}
|
|
351
|
+
|
|
352
|
+
// --- backfill (best-effort, isolated: it must never disturb sync or the proxy) ---
|
|
353
|
+
const bf = opts.backfill || {}
|
|
354
|
+
const oldHist = (opts.proxy || {}).history || {}
|
|
355
|
+
if (bf.enabled === true) {
|
|
356
|
+
// Pre-0.15 installs kept the archive's coordinates in proxy.history.* — reuse them
|
|
357
|
+
// so nobody retypes an org/bucket the plugin already knows.
|
|
358
|
+
const src = {
|
|
359
|
+
url: pickField(bf.sourceUrl, oldHist.influxUrl, BACKFILL_SRC_DEFAULTS.influxUrl).value,
|
|
360
|
+
org: pickField(bf.sourceOrg, oldHist.org, BACKFILL_SRC_DEFAULTS.org).value,
|
|
361
|
+
bucket: pickField(bf.sourceBucket, oldHist.bucket, BACKFILL_SRC_DEFAULTS.bucket).value,
|
|
362
|
+
token: String(bf.sourceToken || oldHist.token || '').trim()
|
|
363
|
+
}
|
|
364
|
+
const dst = { url: influx.url, org: b.org || SYNC_TUNING.org, bucket: b.bucket, token: String(bf.cloudToken || '').trim() }
|
|
365
|
+
if (!b.writeToken) {
|
|
366
|
+
(app.error || console.error)('[sailkick-boat] backfill needs a paired account for its destination bucket — skipped')
|
|
367
|
+
} else if (!dst.token) {
|
|
368
|
+
(app.error || console.error)('[sailkick-boat] backfill needs a cloud READ+WRITE token: every uploaded hour is verified by counting the destination, which the write-only sync token cannot do — skipped')
|
|
369
|
+
} else if (!src.token) {
|
|
370
|
+
(app.error || console.error)('[sailkick-boat] backfill needs a read token for the local InfluxDB — skipped')
|
|
371
|
+
} else {
|
|
372
|
+
try {
|
|
373
|
+
backfill = createBackfill(app, {
|
|
374
|
+
src,
|
|
375
|
+
dst,
|
|
376
|
+
selfOnly: bf.selfOnly === true,
|
|
377
|
+
startBound: bf.startBound,
|
|
378
|
+
stateFile: path.join((app.getDataDirPath && app.getDataDirPath()) || '.', 'backfill.json'),
|
|
379
|
+
pending: sync ? sync.pending : null
|
|
380
|
+
})
|
|
381
|
+
backfill.start()
|
|
382
|
+
} catch (e) {
|
|
383
|
+
(app.error || console.error)('[sailkick-boat] backfill start failed: ' + e.message)
|
|
384
|
+
backfill = null
|
|
385
|
+
}
|
|
386
|
+
}
|
|
387
|
+
}
|
|
388
|
+
|
|
298
389
|
statusTimer = setInterval(updateStatus, 5000)
|
|
299
390
|
updateStatus()
|
|
300
391
|
}
|
|
@@ -308,6 +399,8 @@ module.exports = function (app) {
|
|
|
308
399
|
if (proxy) parts.push(proxy.status())
|
|
309
400
|
if (telemetry) parts.push(telemetry.status())
|
|
310
401
|
if (history) parts.push(history.status())
|
|
402
|
+
if (ais) parts.push(ais.status())
|
|
403
|
+
if (backfill) parts.push(backfill.status())
|
|
311
404
|
try { app.setPluginStatus(parts.join(' | ') || 'idle (both features off)') } catch {}
|
|
312
405
|
}
|
|
313
406
|
|
|
@@ -317,10 +410,14 @@ module.exports = function (app) {
|
|
|
317
410
|
try { if (sync) sync.stop() } catch {}
|
|
318
411
|
try { if (telemetry) telemetry.stop() } catch {}
|
|
319
412
|
try { if (history) history.stop() } catch {}
|
|
413
|
+
try { if (ais) ais.stop() } catch {}
|
|
414
|
+
try { if (backfill) backfill.stop() } catch {}
|
|
320
415
|
try { if (proxy) proxy.stop() } catch {}
|
|
321
416
|
sync = null
|
|
322
417
|
telemetry = null
|
|
323
418
|
history = null
|
|
419
|
+
backfill = null
|
|
420
|
+
ais = null
|
|
324
421
|
proxy = null
|
|
325
422
|
accountStatus = null
|
|
326
423
|
syncWarning = null
|
package/lib/ais/index.js
ADDED
|
@@ -0,0 +1,262 @@
|
|
|
1
|
+
'use strict'
|
|
2
|
+
|
|
3
|
+
// AIS upload — forward the targets this boat's own receiver hears, so the cloud app can
|
|
4
|
+
// draw other vessels, their heading and their trail.
|
|
5
|
+
//
|
|
6
|
+
// The cloud already renders AIS (public/viewer/ais.js → /api/ais), but its source polls a
|
|
7
|
+
// SignalK server over the LAN and keeps everything in memory. That only works while the
|
|
8
|
+
// cloud can reach the boat inbound, which stops being true on a mobile link. So the boat
|
|
9
|
+
// pushes instead.
|
|
10
|
+
//
|
|
11
|
+
// Only LOCALLY RECEIVED AIS is worth uploading. A boat running an internet feed like
|
|
12
|
+
// signalk-aisstream would otherwise spend uplink bandwidth sending data the cloud could
|
|
13
|
+
// fetch directly from the same API over a fat pipe. The value is what your own VHF hears
|
|
14
|
+
// offshore, where commercial feeds are blind — which is also why there is no radius or
|
|
15
|
+
// rate limit here: VHF line-of-sight is the honest limiter, and offshore it tends to zero.
|
|
16
|
+
//
|
|
17
|
+
// Two invariants this module must not break:
|
|
18
|
+
// - Telemetry always wins the link. Separate spool, separate uploader, and it stands
|
|
19
|
+
// down whenever the telemetry spool has a backlog.
|
|
20
|
+
// - Every row is tagged self=false. The cloud's history queries filter on self=="true"
|
|
21
|
+
// to keep other vessels out of the boat's own Trends and track; if that tag were
|
|
22
|
+
// wrong, AIS would land in the owner's SOG and heading charts.
|
|
23
|
+
|
|
24
|
+
const fs = require('fs')
|
|
25
|
+
const path = require('path')
|
|
26
|
+
const { Spool } = require('../sync/spool')
|
|
27
|
+
const { writeLines } = require('../sync/influxWrite')
|
|
28
|
+
const { deltaToLines } = require('../sync/lineprotocol')
|
|
29
|
+
|
|
30
|
+
// What the cloud's /api/ais envelope needs. Subscribing to these rather than '*' keeps
|
|
31
|
+
// delta volume down without imposing a rate limit.
|
|
32
|
+
const POSITION_PATHS = [
|
|
33
|
+
'navigation.position',
|
|
34
|
+
'navigation.speedOverGround',
|
|
35
|
+
'navigation.courseOverGroundTrue',
|
|
36
|
+
'navigation.headingTrue',
|
|
37
|
+
'navigation.headingMagnetic',
|
|
38
|
+
'navigation.magneticVariation',
|
|
39
|
+
'navigation.rateOfTurn'
|
|
40
|
+
]
|
|
41
|
+
// Static/identity data: repeats every ~6 min per vessel and essentially never changes.
|
|
42
|
+
const STATIC_PATHS = ['design.length', 'design.beam', 'design.aisShipType']
|
|
43
|
+
const ALL_PATHS = [...POSITION_PATHS, ...STATIC_PATHS]
|
|
44
|
+
|
|
45
|
+
// Sources that are internet feeds rather than a receiver on this boat. Uploading these
|
|
46
|
+
// is pure round-tripping. Matched case-insensitively as a substring of $source.
|
|
47
|
+
const INTERNET_FEEDS = ['aisstream', 'aishub', 'marinetraffic', 'vesselfinder']
|
|
48
|
+
|
|
49
|
+
const DEFAULTS = {
|
|
50
|
+
staticIntervalMs: 3600000, // re-send a vessel's identity at most hourly
|
|
51
|
+
flushIntervalMs: 5000,
|
|
52
|
+
batchSize: 2000,
|
|
53
|
+
maxBufferBytes: 50 * 1024 * 1024, // its own, smaller cap — see below
|
|
54
|
+
idlePollMs: 5000,
|
|
55
|
+
retryMinMs: 2000,
|
|
56
|
+
retryMaxMs: 120000,
|
|
57
|
+
requestTimeoutMs: 30000
|
|
58
|
+
}
|
|
59
|
+
|
|
60
|
+
const isInternetFeed = (src) => {
|
|
61
|
+
const s = String(src || '').toLowerCase()
|
|
62
|
+
return INTERNET_FEEDS.some((f) => s.includes(f))
|
|
63
|
+
}
|
|
64
|
+
|
|
65
|
+
function createAis (app, options) {
|
|
66
|
+
const log = (m) => (app.debug ? app.debug('[ais] ' + m) : console.log('[sailkick-boat:ais]', m))
|
|
67
|
+
const warn = (m) => (app.error ? app.error('[sailkick-boat:ais] ' + m) : console.error('[sailkick-boat:ais]', m))
|
|
68
|
+
|
|
69
|
+
const cfg = { ...DEFAULTS, ...options }
|
|
70
|
+
let state = null
|
|
71
|
+
|
|
72
|
+
function start () {
|
|
73
|
+
if (!cfg.influxUrl || !cfg.bucket || !cfg.token) {
|
|
74
|
+
warn('not started — the boat is not paired, so there is no destination bucket')
|
|
75
|
+
state = { statusLine: 'ais: not configured', stopped: true }
|
|
76
|
+
return
|
|
77
|
+
}
|
|
78
|
+
const dataDir = (app.getDataDirPath && app.getDataDirPath()) || '.'
|
|
79
|
+
// A SEPARATE spool. Sharing the telemetry one would be dangerous: it drops the
|
|
80
|
+
// OLDEST files on overflow, so an AIS flood in a busy anchorage could evict
|
|
81
|
+
// telemetry that had not been sent yet.
|
|
82
|
+
const spoolDir = cfg.spoolDir || path.join(dataDir, 'ais-spool')
|
|
83
|
+
const spool = new Spool({ dir: spoolDir, maxBytes: cfg.maxBufferBytes, logger: log })
|
|
84
|
+
const selfContext = app.selfContext || ('vessels.' + (app.selfId || 'self'))
|
|
85
|
+
|
|
86
|
+
state = {
|
|
87
|
+
spool,
|
|
88
|
+
selfContext,
|
|
89
|
+
batch: [],
|
|
90
|
+
lastStatic: new Map(), // context -> ms, so identity is re-sent at most hourly
|
|
91
|
+
sourcesSeen: new Map(), // $source -> count, for the discovery log
|
|
92
|
+
targets: new Set(),
|
|
93
|
+
forwarded: 0,
|
|
94
|
+
dropped: 0,
|
|
95
|
+
lastOkAt: null,
|
|
96
|
+
backoff: cfg.retryMinMs,
|
|
97
|
+
pumping: false,
|
|
98
|
+
stopped: false,
|
|
99
|
+
unsubscribes: [],
|
|
100
|
+
statusLine: 'ais: starting'
|
|
101
|
+
}
|
|
102
|
+
|
|
103
|
+
spool.init().then(() => {
|
|
104
|
+
// stop() can land before init resolves (disable during startup, or a fast
|
|
105
|
+
// restart). Without this the timers are installed on an already-stopped module
|
|
106
|
+
// and never cleared — a leaked interval that also keeps the host process alive.
|
|
107
|
+
if (!state || state.stopped) return
|
|
108
|
+
state.flushTimer = setInterval(flush, cfg.flushIntervalMs)
|
|
109
|
+
state.reportTimer = setInterval(reportSources, 300000)
|
|
110
|
+
// Unref every timer: this module must never be the reason the Signal K server
|
|
111
|
+
// cannot exit.
|
|
112
|
+
if (state.flushTimer.unref) state.flushTimer.unref()
|
|
113
|
+
if (state.reportTimer.unref) state.reportTimer.unref()
|
|
114
|
+
subscribe()
|
|
115
|
+
pump()
|
|
116
|
+
log(`started; ${cfg.source ? 'source "' + cfg.source + '" only' : 'all sources except known internet feeds'}; buffer ${spoolDir}`)
|
|
117
|
+
}).catch((e) => warn('init failed: ' + e.message))
|
|
118
|
+
}
|
|
119
|
+
|
|
120
|
+
// Which $source values are actually producing AIS, so the config field can be filled
|
|
121
|
+
// in from the log instead of guessed.
|
|
122
|
+
function reportSources () {
|
|
123
|
+
if (!state || !state.sourcesSeen.size) return
|
|
124
|
+
const list = [...state.sourcesSeen.entries()].sort((a, b) => b[1] - a[1])
|
|
125
|
+
.map(([s, n]) => `${s} (${n}${isInternetFeed(s) ? ', internet feed — not forwarded' : ''})`)
|
|
126
|
+
log('AIS sources seen: ' + list.join('; '))
|
|
127
|
+
}
|
|
128
|
+
|
|
129
|
+
function accept (context, source) {
|
|
130
|
+
if (!context || context === state.selfContext) return false // never our own vessel
|
|
131
|
+
if (cfg.source) return String(source) === String(cfg.source)
|
|
132
|
+
return !isInternetFeed(source)
|
|
133
|
+
}
|
|
134
|
+
|
|
135
|
+
function handleDelta (delta) {
|
|
136
|
+
if (!state || state.stopped || !delta || !Array.isArray(delta.updates)) return
|
|
137
|
+
const context = delta.context
|
|
138
|
+
if (!context || context === state.selfContext) return
|
|
139
|
+
|
|
140
|
+
for (const update of delta.updates) {
|
|
141
|
+
if (!update || !Array.isArray(update.values)) continue
|
|
142
|
+
const source = update.$source || (update.source && (update.source.label || update.source.type)) || 'unknown'
|
|
143
|
+
state.sourcesSeen.set(source, (state.sourcesSeen.get(source) || 0) + 1)
|
|
144
|
+
if (!accept(context, source)) { state.dropped++; continue }
|
|
145
|
+
|
|
146
|
+
// Split identity from position: identity repeats constantly and never changes.
|
|
147
|
+
const now = Date.now()
|
|
148
|
+
const staticDue = (now - (state.lastStatic.get(context) || 0)) >= cfg.staticIntervalMs
|
|
149
|
+
const values = update.values.filter((pv) => {
|
|
150
|
+
if (!pv || !pv.path) return false
|
|
151
|
+
if (STATIC_PATHS.includes(pv.path)) return staticDue
|
|
152
|
+
return POSITION_PATHS.includes(pv.path)
|
|
153
|
+
})
|
|
154
|
+
if (!values.length) continue
|
|
155
|
+
if (staticDue && values.some((pv) => STATIC_PATHS.includes(pv.path))) state.lastStatic.set(context, now)
|
|
156
|
+
|
|
157
|
+
// self:false is the tag the cloud filters on to keep AIS out of the owner's charts.
|
|
158
|
+
const lines = deltaToLines({ context, updates: [{ ...update, values }] }, { self: false })
|
|
159
|
+
if (lines.length) {
|
|
160
|
+
state.batch.push(...lines)
|
|
161
|
+
state.targets.add(context)
|
|
162
|
+
state.forwarded += lines.length
|
|
163
|
+
}
|
|
164
|
+
if (state.batch.length >= cfg.batchSize) flush()
|
|
165
|
+
}
|
|
166
|
+
}
|
|
167
|
+
|
|
168
|
+
function subscribe () {
|
|
169
|
+
const sub = { context: '*', subscribe: ALL_PATHS.map((p) => ({ path: p, period: cfg.periodMs || 10000 })) }
|
|
170
|
+
if (app.subscriptionmanager && app.subscriptionmanager.subscribe) {
|
|
171
|
+
app.subscriptionmanager.subscribe(sub, state.unsubscribes, (err) => warn('subscription error: ' + err), handleDelta)
|
|
172
|
+
} else if (app.signalk && app.signalk.on) {
|
|
173
|
+
const h = (d) => handleDelta(d)
|
|
174
|
+
app.signalk.on('delta', h)
|
|
175
|
+
state.unsubscribes.push(() => app.signalk.removeListener('delta', h))
|
|
176
|
+
} else {
|
|
177
|
+
warn('no subscription mechanism available — nothing will be forwarded')
|
|
178
|
+
}
|
|
179
|
+
}
|
|
180
|
+
|
|
181
|
+
function flush () {
|
|
182
|
+
if (!state || state.stopped || !state.batch.length) return
|
|
183
|
+
const lines = state.batch
|
|
184
|
+
state.batch = []
|
|
185
|
+
state.spool.append(lines).then(() => pump()).catch((e) => {
|
|
186
|
+
warn('spool append failed: ' + e.message)
|
|
187
|
+
state.batch.unshift(...lines)
|
|
188
|
+
})
|
|
189
|
+
}
|
|
190
|
+
|
|
191
|
+
async function pump () {
|
|
192
|
+
if (!state || state.stopped || state.pumping) return
|
|
193
|
+
// Telemetry always wins the link: stand down entirely while its spool is behind.
|
|
194
|
+
if (cfg.pending) {
|
|
195
|
+
let depth = 0
|
|
196
|
+
try { depth = (await cfg.pending()).count || 0 } catch {}
|
|
197
|
+
if (depth) { refreshStatus(`waiting — telemetry backlog (${depth})`); scheduleIdle(); return }
|
|
198
|
+
}
|
|
199
|
+
state.pumping = true
|
|
200
|
+
try {
|
|
201
|
+
for (const file of await state.spool.pending()) {
|
|
202
|
+
if (state.stopped) break
|
|
203
|
+
let body
|
|
204
|
+
try { body = await fs.promises.readFile(file, 'utf8') } catch { continue }
|
|
205
|
+
if (!body.trim()) { await state.spool.remove(file); continue }
|
|
206
|
+
const res = await writeLines({ influxUrl: cfg.influxUrl, org: cfg.org, bucket: cfg.bucket, token: cfg.token, timeoutMs: cfg.requestTimeoutMs }, body)
|
|
207
|
+
if (res.ok) {
|
|
208
|
+
await state.spool.remove(file); state.lastOkAt = new Date(); state.backoff = cfg.retryMinMs
|
|
209
|
+
} else if (res.retryable) {
|
|
210
|
+
state.pumping = false; scheduleRetry(res); return
|
|
211
|
+
} else {
|
|
212
|
+
warn(`batch rejected (HTTP ${res.status}) — quarantined`)
|
|
213
|
+
await state.spool.quarantine(file)
|
|
214
|
+
}
|
|
215
|
+
}
|
|
216
|
+
} catch (e) {
|
|
217
|
+
warn('pump error: ' + e.message); state.pumping = false; scheduleRetry(); return
|
|
218
|
+
}
|
|
219
|
+
state.pumping = false
|
|
220
|
+
scheduleIdle()
|
|
221
|
+
refreshStatus()
|
|
222
|
+
}
|
|
223
|
+
|
|
224
|
+
function scheduleRetry (res) {
|
|
225
|
+
if (!state || state.stopped) return
|
|
226
|
+
clearTimeout(state.pumpTimer)
|
|
227
|
+
const delay = state.backoff
|
|
228
|
+
state.backoff = Math.min(state.backoff * 2, cfg.retryMaxMs)
|
|
229
|
+
state.pumpTimer = setTimeout(pump, delay)
|
|
230
|
+
if (state.pumpTimer.unref) state.pumpTimer.unref()
|
|
231
|
+
refreshStatus(`${res && res.status ? 'HTTP ' + res.status : 'offline'} — retry ${Math.round(delay / 1000)}s`)
|
|
232
|
+
}
|
|
233
|
+
function scheduleIdle () {
|
|
234
|
+
if (!state || state.stopped) return
|
|
235
|
+
clearTimeout(state.pumpTimer)
|
|
236
|
+
state.pumpTimer = setTimeout(pump, cfg.idlePollMs)
|
|
237
|
+
if (state.pumpTimer.unref) state.pumpTimer.unref()
|
|
238
|
+
}
|
|
239
|
+
function refreshStatus (suffix) {
|
|
240
|
+
if (!state) return
|
|
241
|
+
state.statusLine = `ais: ${state.targets.size} target(s), ${state.forwarded} point(s)${state.dropped ? `, ${state.dropped} skipped` : ''}${suffix ? '; ' + suffix : ''}`
|
|
242
|
+
}
|
|
243
|
+
|
|
244
|
+
function stop () {
|
|
245
|
+
if (!state) return
|
|
246
|
+
state.stopped = true
|
|
247
|
+
clearInterval(state.flushTimer)
|
|
248
|
+
clearInterval(state.reportTimer)
|
|
249
|
+
clearTimeout(state.pumpTimer)
|
|
250
|
+
for (const u of (state.unsubscribes || [])) { try { u() } catch {} }
|
|
251
|
+
if (state.batch && state.batch.length && state.spool) {
|
|
252
|
+
try { fs.writeFileSync(path.join(state.spool.dir, `${Date.now()}-final.lp`), state.batch.join('\n') + '\n') } catch {}
|
|
253
|
+
state.batch = []
|
|
254
|
+
}
|
|
255
|
+
}
|
|
256
|
+
|
|
257
|
+
function status () { return state ? state.statusLine : 'ais: off' }
|
|
258
|
+
|
|
259
|
+
return { start, stop, status, _handleDelta: handleDelta, _state: () => state, _flush: flush }
|
|
260
|
+
}
|
|
261
|
+
|
|
262
|
+
module.exports = { createAis, isInternetFeed, POSITION_PATHS, STATIC_PATHS }
|