sailkick-boat 0.14.5 → 0.17.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -27,7 +27,7 @@ One Signal K plugin, independently-toggleable modules — so the boat stays
27
27
  same contracts as the cloud, offline-first):
28
28
  - **`/ws/telemetry`** — the app's live telemetry bus, fed from local SignalK.
29
29
  - **`/api/history/{series,track}`** — the app's Trends panel + track, served
30
- from the boat's local InfluxDB (or a DB-less telemetry ring) — full local history.
30
+ from a live ring sampled on the boat — full local history, no database.
31
31
 
32
32
  Kept as separate modules so a proxy fault can't wedge the data-critical sync path.
33
33
 
@@ -139,13 +139,79 @@ config passes through untouched. (Hand-edit `proxy.openAccess: false` to keep th
139
139
  login gate — there is no toggle, since the gate cannot complete over the mirror anyway.)
140
140
 
141
141
  ## Local history (offline Trends + track)
142
- Two ways, chosen automatically:
143
- - **Read token set** → full history queried from a **local InfluxDB** (e.g. a bucket
144
- written by `signalk-to-influxdb-v2`).
145
- - **No token, telemetry on** → a **DB-less ring** sampled from live telemetry (the
146
- same BoatState feeding `/ws/telemetry`) — for boats with no local InfluxDB, e.g.
147
- **SignalK on a Victron GX / Venus OS**. Same JSON contract, fully offline, no database.
148
- (`historyAvailable` reports true either way, so the app shows the Trends panel.)
142
+ **One source: a live ring**, sampled from the same BoatState that feeds `/ws/telemetry`
143
+ — no database, works on a Victron GX with nothing else installed. `historyAvailable` is
144
+ forced on so the app shows the Trends panel, and the eight channels match what the cloud
145
+ serves, `stw` included.
146
+
147
+ There is deliberately **no way to point this at a local InfluxDB**. Until v0.15.0 a read
148
+ token did exactly that, and it was a trap: the app only ever asks for a *relative* window
149
+ clamped to 24 h, so aiming it at a bucket of older data matched nothing — Trends went
150
+ blank **and** the working live ring was switched off. If you still have a token in your
151
+ config it is now inert; the plugin logs `history -> live ring` regardless.
152
+
153
+ A local InfluxDB is not a competitor here anyway. The app never requests finer than
154
+ `every=5s` over a 24 h window, and the ring's floor at that window is 2 s — set
155
+ `ringSampleSec: 5` and it matches anything the UI can draw, from live state. What an old
156
+ database *is* good for is its contents ending up **in the cloud** — see below.
157
+
158
+ ## Uploading AIS targets
159
+ The cloud app already draws other vessels, but its AIS source polls a SignalK server over
160
+ the LAN and keeps everything in memory — which cannot work once a boat is on a mobile
161
+ link. Enable **Upload AIS targets** and the boat pushes what its own receiver hears, so
162
+ the web app can show other boats, their heading and their trail from stored data.
163
+
164
+ Only **locally received** AIS is forwarded. A boat running an internet feed such as
165
+ `signalk-aisstream` would otherwise spend uplink bandwidth sending data the cloud can
166
+ fetch directly from the same API — known feeds are skipped automatically. The plugin logs
167
+ the AIS sources it sees, so you can name your own receiver in **Only this AIS source** if
168
+ you want to be explicit.
169
+
170
+ There is no radius or rate limit: a real AIS receiver is bounded by VHF line-of-sight,
171
+ which is the honest limiter, and offshore — where this data is most valuable, because
172
+ commercial feeds are blind there — it tends to zero. Vessel identity (name, dimensions,
173
+ ship type) repeats every few minutes and never changes, so it is re-sent at most hourly;
174
+ positions are never throttled.
175
+
176
+ Telemetry always wins the link. AIS buffers in its **own** spool with its own cap and
177
+ stands down completely whenever the telemetry spool has a backlog, so a busy anchorage
178
+ can never delay or evict your own boat's data.
179
+
180
+ > ⚠️ **Requires a cloud that filters history on `self`.** Every AIS row is tagged
181
+ > `self=false`, and the cloud's Trends and track queries must filter `self == "true"`.
182
+ > Without that, other ships' speed and heading appear in *your* charts. Leave this off
183
+ > until the server side is in place.
184
+
185
+ ## Copying older history to the cloud (one-time)
186
+ If the boat recorded into its own InfluxDB before it started syncing — a
187
+ `signalk-to-influxdb-v2` bucket, or an imported logbook — the **backfill** copies it up
188
+ so the cloud holds your full history. Live sync can't do this: it only ever sees deltas
189
+ arriving now, and the spool only replays what it captured itself while offline.
190
+
191
+ Fill the **Copy older history to the cloud** section and save. It walks backwards in
192
+ one-hour windows (newest first, so recent history lands first), resumes after a restart
193
+ from a manifest, and stands aside whenever live telemetry has a backlog — the data-
194
+ critical path is never starved by a bulk upload. Progress shows in the status line.
195
+
196
+ It needs a **cloud read+write token**, not the write token from signup. Every hour it
197
+ uploads is verified by counting the destination, and a write-only token cannot read. A
198
+ `204` means InfluxDB accepted the bytes, not that every point landed — without the count
199
+ a partial write would be marked done and lost. The token is only needed while the
200
+ backfill runs: **revoke it afterwards**, live sync is unaffected.
201
+
202
+ Safe to re-run. Points are keyed by (measurement, tagset, nanosecond timestamp), so an
203
+ identical point overwrites rather than duplicating — an interrupted migration is simply
204
+ run again. Everything in the bucket is copied, including AIS contexts; hand-edit
205
+ `backfill.selfOnly: true` to restrict it to your own vessel if cloud series cardinality
206
+ becomes a problem.
207
+
208
+ **True wind comes from your instruments.** If the boat publishes
209
+ `environment.wind.speedTrue` / `directionTrue`, those are stored verbatim — a wind
210
+ system corrects for heel and leeway against water-referenced boat speed, and every other
211
+ display aboard shows its numbers. Only when the boat publishes none is true wind
212
+ derived, and then from **STW**, not SOG: true wind is relative to motion through the
213
+ water. (Before v0.14.6 the derivation used SOG, which in 3 kt of foul tide skewed TWD by
214
+ ~12° and TWS by ~1.8 kt.) SOG is the last resort for boats with no paddlewheel.
149
215
  - **Persistent (append-log):** the ring is saved as a JSONL append-log at
150
216
  `<dataDir>/history/history-ring.jsonl` (on the SSD/USB with the tiles; override with
151
217
  `proxy.history.ringDir`), so it **survives restarts**. Each sample appends one line; the file is
@@ -158,14 +224,14 @@ Two ways, chosen automatically:
158
224
 
159
225
  The sailkick app is deployment-agnostic about history: *central Influx in the
160
226
  cloud, in-memory ring on a DB-less edge*. The boat is a third case — an edge that
161
- serves the app's history endpoints from its **own** data (local InfluxDB or the ring):
227
+ serves the app's history endpoints from its **own** live data:
162
228
  ```
163
229
  GET /api/history/series?window=3600s&every=30s -> { series: { sog|heading|tws|… : [[tMs,val],…] } }
164
230
  GET /api/history/track?window=3600s -> { track: [{ t, lat, lon }, …] }
165
231
  ```
166
232
  Same JSON the cloud returns, so the browser can't tell the difference — but it
167
- works **offline** with the boat's own data. Only when neither a local InfluxDB nor
168
- telemetry is available do these paths **fall through to the cloud mirror**, so an
233
+ works **offline** with the boat's own data. Only when no telemetry source is
234
+ available at all do these paths **fall through to the cloud mirror**, so an
169
235
  online boat is never worse off than before.
170
236
 
171
237
  ## Setup: register on the web, then paste the token
@@ -205,18 +271,16 @@ in `index.js`.
205
271
 
206
272
  - **Sailkick account**: `slug` (boat name), `writeToken`
207
273
  - **Telemetry sync → cloud**: `enabled`
274
+ - **Upload AIS targets**: `enabled` (default off), `source`
208
275
  - **Offline app & maps**: `enabled`, `proxyPort` (default 8080), `localSignalkUrl`
209
276
  (default `http://127.0.0.1:3000`), `dataDir`, `seedEnabled`, `prefetchRadiusNm`,
210
- `prefetchDetailZoom`, `historyToken`
277
+ `prefetchDetailZoom`
211
278
 
212
279
  `dataDir` is the one storage location — cached maps, the telemetry spool and the
213
280
  history ring log all live under it. **Put it on the SSD/USB disk, not the SD card.**
214
281
  Leave it blank and each part falls back to its historical spot under the plugin data
215
282
  dir.
216
283
 
217
- `historyToken` is optional and only matters if the boat already runs its own InfluxDB.
218
- Blank — the normal case — uses the built-in DB-less ring.
219
-
220
284
  Cache-manifest polling is always on (no toggle): tile freshness comes from the cloud
221
285
  announcing bakes, and without it a re-baked dataset would never refresh.
222
286
 
package/index.js CHANGED
@@ -6,6 +6,8 @@ const { createSync } = require('./lib/sync')
6
6
  const { createProxy } = require('./lib/proxy')
7
7
  const { createTelemetry } = require('./lib/telemetry')
8
8
  const { createHistory } = require('./lib/history')
9
+ const { createBackfill } = require('./lib/backfill')
10
+ const { createAis } = require('./lib/ais')
9
11
  const { resolveAccountConfig } = require('./lib/account')
10
12
 
11
13
  // sailkick-boat: one Signal K plugin, two independently-toggleable modules —
@@ -43,15 +45,8 @@ const PROXY_TUNING = { requestTimeoutMs: 20000, localPaths: ['/signalk'], teleme
43
45
  const MANIFEST = { enabled: true, path: '/api/cache-manifest', pollIntervalSec: 300 }
44
46
  const SEED_TUNING = { coastlineMaxZoom: 8, seabedMaxZoom: 6, concurrency: 4 }
45
47
  const PREFETCH_TUNING = { concurrency: 4 }
46
- const HISTORY_TUNING = {
47
- influxUrl: 'http://127.0.0.1:8086',
48
- org: 'signalk',
49
- bucket: 'signalk',
50
- requestTimeoutMs: 15000,
51
- ringPersist: true,
52
- ringWindowSec: 86400,
53
- ringSampleSec: 15
54
- }
48
+ const HISTORY_TUNING = { ringPersist: true, ringWindowSec: 86400, ringSampleSec: 15 }
49
+ const BACKFILL_SRC_DEFAULTS = { influxUrl: 'http://127.0.0.1:8086', org: 'signalk', bucket: 'signalk' }
55
50
 
56
51
  // A *cloud* endpoint on a loopback or private address means telemetry never leaves the
57
52
  // LAN. On the wire that is indistinguishable from a normal offline backlog — the spool
@@ -69,6 +64,19 @@ function isPrivateHostUrl (u) {
69
64
  } catch { return false }
70
65
  }
71
66
 
67
+ // The Signal K config UI writes every schema default on save, so a freshly pre-filled
68
+ // `backfillOrg: "signalk"` can appear on a boat whose archive has always lived in org
69
+ // "addiction" under the pre-0.15 `proxy.history` block — and the backfill would read an
70
+ // empty bucket. Rule: an explicitly-changed field wins; else a legacy value that differs
71
+ // from the default wins; else the default.
72
+ function pickField (flat, legacy, dflt) {
73
+ const f = String(flat == null ? '' : flat).trim()
74
+ const l = String(legacy == null ? '' : legacy).trim()
75
+ if (f && f !== dflt) return { value: f, from: 'config' }
76
+ if (l && l !== dflt) return { value: l, from: 'legacy' }
77
+ return { value: f || l || dflt, from: 'default' }
78
+ }
79
+
72
80
  // Endpoint resolution with one rule: the constant wins unless self-hosting is declared.
73
81
  // Returns { url, ignored } — `ignored` is the stale value we refused, so the caller can
74
82
  // say so out loud instead of leaving the owner to guess why nothing arrives.
@@ -84,6 +92,8 @@ module.exports = function (app) {
84
92
  let proxy = null
85
93
  let telemetry = null
86
94
  let history = null
95
+ let backfill = null
96
+ let ais = null
87
97
  let statusTimer = null
88
98
  let accountStatus = null
89
99
  let syncWarning = null
@@ -116,6 +126,28 @@ module.exports = function (app) {
116
126
  enabled: { type: 'boolean', title: 'Enable telemetry sync', default: true }
117
127
  }
118
128
  },
129
+ ais: {
130
+ type: 'object',
131
+ title: 'Upload AIS targets',
132
+ description: 'Send the AIS this boat\'s own receiver hears to the cloud, so the web app can show other vessels, their heading and their trail. Only locally received AIS is forwarded — an internet feed such as signalk-aisstream is skipped, since the cloud can fetch that itself without spending your uplink. REQUIRES a cloud that filters history on self; until then leave this off or the owner\'s SOG and heading charts will pick up other ships.',
133
+ properties: {
134
+ enabled: { type: 'boolean', title: 'Upload AIS targets', default: false },
135
+ source: { type: 'string', title: 'Only this AIS source', description: 'Blank forwards every source except known internet feeds. The plugin logs the AIS sources it sees — copy the one for your own receiver here if you want to be explicit.' }
136
+ }
137
+ },
138
+ backfill: {
139
+ type: 'object',
140
+ title: 'Copy older history to the cloud (one-time)',
141
+ description: 'If this boat recorded data into its own InfluxDB before it started syncing — a signalk-to-influxdb-v2 bucket, or an imported logbook — this copies it up so the app can chart it. It runs in the background, resumes after a restart, verifies every hour it uploads, and stands aside whenever live telemetry is behind. Safe to re-run: identical points overwrite rather than duplicate.',
142
+ properties: {
143
+ enabled: { type: 'boolean', title: 'Run the backfill', default: false },
144
+ cloudToken: { type: 'string', title: 'Cloud read+write token', description: 'A token with READ and WRITE on your cloud bucket. Your normal write token cannot read, and reading is how each uploaded hour is verified. Needed only while the backfill runs — revoke it afterwards; live sync is unaffected.' },
145
+ sourceToken: { type: 'string', title: 'Local InfluxDB read token' },
146
+ sourceUrl: { type: 'string', title: '…local InfluxDB URL', default: 'http://127.0.0.1:8086' },
147
+ sourceOrg: { type: 'string', title: '…organization', default: 'signalk' },
148
+ sourceBucket: { type: 'string', title: '…bucket', default: 'signalk' }
149
+ }
150
+ },
119
151
  proxy: {
120
152
  type: 'object',
121
153
  title: 'Offline app & maps',
@@ -140,8 +172,7 @@ module.exports = function (app) {
140
172
  enum: [12, 13, 14, 15],
141
173
  enumNames: ['Overview (z12)', 'Coastal (z13)', 'Detailed (z14)', 'Harbor (z15)'],
142
174
  default: 13
143
- },
144
- historyToken: { type: 'string', title: 'Local InfluxDB read token (optional)', description: 'Only if this boat already runs its own InfluxDB (e.g. signalk-to-influxdb-v2) and you want full history from it. Leave blank — the plugin then keeps its own lightweight history, which is what most boats want.' }
175
+ }
145
176
  }
146
177
  }
147
178
  }
@@ -220,6 +251,8 @@ module.exports = function (app) {
220
251
  const oldSeed = p.seed || {}
221
252
  const oldPrefetch = p.prefetch || {}
222
253
  const oldHistory = p.history || {}
254
+ console.log('[sailkick-boat] history -> live ring')
255
+
223
256
  if (upstream.ignored) {
224
257
  ;(app.error || console.error)(`[sailkick-boat] ignoring proxy.sailkickUrl "${upstream.ignored}" left over from an older config — mirroring ${upstream.url}. Set proxy.selfHosted:true to keep your own server.`)
225
258
  }
@@ -251,10 +284,6 @@ module.exports = function (app) {
251
284
  history: {
252
285
  ...HISTORY_TUNING,
253
286
  enabled: oldHistory.enabled !== false,
254
- influxUrl: oldHistory.influxUrl || HISTORY_TUNING.influxUrl,
255
- org: oldHistory.org || HISTORY_TUNING.org,
256
- bucket: oldHistory.bucket || HISTORY_TUNING.bucket,
257
- token: p.historyToken || oldHistory.token || '',
258
287
  ringPersist: oldHistory.ringPersist !== false,
259
288
  ringWindowSec: oldHistory.ringWindowSec || HISTORY_TUNING.ringWindowSec,
260
289
  ringSampleSec: oldHistory.ringSampleSec || HISTORY_TUNING.ringSampleSec,
@@ -295,6 +324,68 @@ module.exports = function (app) {
295
324
  }
296
325
  }
297
326
 
327
+ // --- AIS upload (isolated; yields to telemetry, its own spool) ---
328
+ const aisOpts = opts.ais || {}
329
+ if (aisOpts.enabled === true) {
330
+ if (!b.writeToken) {
331
+ (app.error || console.error)('[sailkick-boat] AIS upload needs a paired account for its destination bucket — skipped')
332
+ } else {
333
+ try {
334
+ ais = createAis(app, {
335
+ influxUrl: influx.url,
336
+ org: b.org || SYNC_TUNING.org,
337
+ bucket: b.bucket,
338
+ token: b.writeToken,
339
+ source: String(aisOpts.source || '').trim() || null,
340
+ spoolDir: store ? path.join(store, 'ais-spool') : undefined,
341
+ pending: sync ? sync.pending : null
342
+ })
343
+ ais.start()
344
+ console.log(`[sailkick-boat] ais -> ${influx.url} bucket=${b.bucket}${aisOpts.source ? ' source=' + aisOpts.source : ''}`)
345
+ } catch (e) {
346
+ (app.error || console.error)('[sailkick-boat] AIS start failed: ' + e.message)
347
+ ais = null
348
+ }
349
+ }
350
+ }
351
+
352
+ // --- backfill (best-effort, isolated: it must never disturb sync or the proxy) ---
353
+ const bf = opts.backfill || {}
354
+ const oldHist = (opts.proxy || {}).history || {}
355
+ if (bf.enabled === true) {
356
+ // Pre-0.15 installs kept the archive's coordinates in proxy.history.* — reuse them
357
+ // so nobody retypes an org/bucket the plugin already knows.
358
+ const src = {
359
+ url: pickField(bf.sourceUrl, oldHist.influxUrl, BACKFILL_SRC_DEFAULTS.influxUrl).value,
360
+ org: pickField(bf.sourceOrg, oldHist.org, BACKFILL_SRC_DEFAULTS.org).value,
361
+ bucket: pickField(bf.sourceBucket, oldHist.bucket, BACKFILL_SRC_DEFAULTS.bucket).value,
362
+ token: String(bf.sourceToken || oldHist.token || '').trim()
363
+ }
364
+ const dst = { url: influx.url, org: b.org || SYNC_TUNING.org, bucket: b.bucket, token: String(bf.cloudToken || '').trim() }
365
+ if (!b.writeToken) {
366
+ (app.error || console.error)('[sailkick-boat] backfill needs a paired account for its destination bucket — skipped')
367
+ } else if (!dst.token) {
368
+ (app.error || console.error)('[sailkick-boat] backfill needs a cloud READ+WRITE token: every uploaded hour is verified by counting the destination, which the write-only sync token cannot do — skipped')
369
+ } else if (!src.token) {
370
+ (app.error || console.error)('[sailkick-boat] backfill needs a read token for the local InfluxDB — skipped')
371
+ } else {
372
+ try {
373
+ backfill = createBackfill(app, {
374
+ src,
375
+ dst,
376
+ selfOnly: bf.selfOnly === true,
377
+ startBound: bf.startBound,
378
+ stateFile: path.join((app.getDataDirPath && app.getDataDirPath()) || '.', 'backfill.json'),
379
+ pending: sync ? sync.pending : null
380
+ })
381
+ backfill.start()
382
+ } catch (e) {
383
+ (app.error || console.error)('[sailkick-boat] backfill start failed: ' + e.message)
384
+ backfill = null
385
+ }
386
+ }
387
+ }
388
+
298
389
  statusTimer = setInterval(updateStatus, 5000)
299
390
  updateStatus()
300
391
  }
@@ -308,6 +399,8 @@ module.exports = function (app) {
308
399
  if (proxy) parts.push(proxy.status())
309
400
  if (telemetry) parts.push(telemetry.status())
310
401
  if (history) parts.push(history.status())
402
+ if (ais) parts.push(ais.status())
403
+ if (backfill) parts.push(backfill.status())
311
404
  try { app.setPluginStatus(parts.join(' | ') || 'idle (both features off)') } catch {}
312
405
  }
313
406
 
@@ -317,10 +410,14 @@ module.exports = function (app) {
317
410
  try { if (sync) sync.stop() } catch {}
318
411
  try { if (telemetry) telemetry.stop() } catch {}
319
412
  try { if (history) history.stop() } catch {}
413
+ try { if (ais) ais.stop() } catch {}
414
+ try { if (backfill) backfill.stop() } catch {}
320
415
  try { if (proxy) proxy.stop() } catch {}
321
416
  sync = null
322
417
  telemetry = null
323
418
  history = null
419
+ backfill = null
420
+ ais = null
324
421
  proxy = null
325
422
  accountStatus = null
326
423
  syncWarning = null
@@ -0,0 +1,262 @@
1
+ 'use strict'
2
+
3
+ // AIS upload — forward the targets this boat's own receiver hears, so the cloud app can
4
+ // draw other vessels, their heading and their trail.
5
+ //
6
+ // The cloud already renders AIS (public/viewer/ais.js → /api/ais), but its source polls a
7
+ // SignalK server over the LAN and keeps everything in memory. That only works while the
8
+ // cloud can reach the boat inbound, which stops being true on a mobile link. So the boat
9
+ // pushes instead.
10
+ //
11
+ // Only LOCALLY RECEIVED AIS is worth uploading. A boat running an internet feed like
12
+ // signalk-aisstream would otherwise spend uplink bandwidth sending data the cloud could
13
+ // fetch directly from the same API over a fat pipe. The value is what your own VHF hears
14
+ // offshore, where commercial feeds are blind — which is also why there is no radius or
15
+ // rate limit here: VHF line-of-sight is the honest limiter, and offshore it tends to zero.
16
+ //
17
+ // Two invariants this module must not break:
18
+ // - Telemetry always wins the link. Separate spool, separate uploader, and it stands
19
+ // down whenever the telemetry spool has a backlog.
20
+ // - Every row is tagged self=false. The cloud's history queries filter on self=="true"
21
+ // to keep other vessels out of the boat's own Trends and track; if that tag were
22
+ // wrong, AIS would land in the owner's SOG and heading charts.
23
+
24
+ const fs = require('fs')
25
+ const path = require('path')
26
+ const { Spool } = require('../sync/spool')
27
+ const { writeLines } = require('../sync/influxWrite')
28
+ const { deltaToLines } = require('../sync/lineprotocol')
29
+
30
+ // What the cloud's /api/ais envelope needs. Subscribing to these rather than '*' keeps
31
+ // delta volume down without imposing a rate limit.
32
+ const POSITION_PATHS = [
33
+ 'navigation.position',
34
+ 'navigation.speedOverGround',
35
+ 'navigation.courseOverGroundTrue',
36
+ 'navigation.headingTrue',
37
+ 'navigation.headingMagnetic',
38
+ 'navigation.magneticVariation',
39
+ 'navigation.rateOfTurn'
40
+ ]
41
+ // Static/identity data: repeats every ~6 min per vessel and essentially never changes.
42
+ const STATIC_PATHS = ['design.length', 'design.beam', 'design.aisShipType']
43
+ const ALL_PATHS = [...POSITION_PATHS, ...STATIC_PATHS]
44
+
45
+ // Sources that are internet feeds rather than a receiver on this boat. Uploading these
46
+ // is pure round-tripping. Matched case-insensitively as a substring of $source.
47
+ const INTERNET_FEEDS = ['aisstream', 'aishub', 'marinetraffic', 'vesselfinder']
48
+
49
+ const DEFAULTS = {
50
+ staticIntervalMs: 3600000, // re-send a vessel's identity at most hourly
51
+ flushIntervalMs: 5000,
52
+ batchSize: 2000,
53
+ maxBufferBytes: 50 * 1024 * 1024, // its own, smaller cap — see below
54
+ idlePollMs: 5000,
55
+ retryMinMs: 2000,
56
+ retryMaxMs: 120000,
57
+ requestTimeoutMs: 30000
58
+ }
59
+
60
+ const isInternetFeed = (src) => {
61
+ const s = String(src || '').toLowerCase()
62
+ return INTERNET_FEEDS.some((f) => s.includes(f))
63
+ }
64
+
65
+ function createAis (app, options) {
66
+ const log = (m) => (app.debug ? app.debug('[ais] ' + m) : console.log('[sailkick-boat:ais]', m))
67
+ const warn = (m) => (app.error ? app.error('[sailkick-boat:ais] ' + m) : console.error('[sailkick-boat:ais]', m))
68
+
69
+ const cfg = { ...DEFAULTS, ...options }
70
+ let state = null
71
+
72
+ function start () {
73
+ if (!cfg.influxUrl || !cfg.bucket || !cfg.token) {
74
+ warn('not started — the boat is not paired, so there is no destination bucket')
75
+ state = { statusLine: 'ais: not configured', stopped: true }
76
+ return
77
+ }
78
+ const dataDir = (app.getDataDirPath && app.getDataDirPath()) || '.'
79
+ // A SEPARATE spool. Sharing the telemetry one would be dangerous: it drops the
80
+ // OLDEST files on overflow, so an AIS flood in a busy anchorage could evict
81
+ // telemetry that had not been sent yet.
82
+ const spoolDir = cfg.spoolDir || path.join(dataDir, 'ais-spool')
83
+ const spool = new Spool({ dir: spoolDir, maxBytes: cfg.maxBufferBytes, logger: log })
84
+ const selfContext = app.selfContext || ('vessels.' + (app.selfId || 'self'))
85
+
86
+ state = {
87
+ spool,
88
+ selfContext,
89
+ batch: [],
90
+ lastStatic: new Map(), // context -> ms, so identity is re-sent at most hourly
91
+ sourcesSeen: new Map(), // $source -> count, for the discovery log
92
+ targets: new Set(),
93
+ forwarded: 0,
94
+ dropped: 0,
95
+ lastOkAt: null,
96
+ backoff: cfg.retryMinMs,
97
+ pumping: false,
98
+ stopped: false,
99
+ unsubscribes: [],
100
+ statusLine: 'ais: starting'
101
+ }
102
+
103
+ spool.init().then(() => {
104
+ // stop() can land before init resolves (disable during startup, or a fast
105
+ // restart). Without this the timers are installed on an already-stopped module
106
+ // and never cleared — a leaked interval that also keeps the host process alive.
107
+ if (!state || state.stopped) return
108
+ state.flushTimer = setInterval(flush, cfg.flushIntervalMs)
109
+ state.reportTimer = setInterval(reportSources, 300000)
110
+ // Unref every timer: this module must never be the reason the Signal K server
111
+ // cannot exit.
112
+ if (state.flushTimer.unref) state.flushTimer.unref()
113
+ if (state.reportTimer.unref) state.reportTimer.unref()
114
+ subscribe()
115
+ pump()
116
+ log(`started; ${cfg.source ? 'source "' + cfg.source + '" only' : 'all sources except known internet feeds'}; buffer ${spoolDir}`)
117
+ }).catch((e) => warn('init failed: ' + e.message))
118
+ }
119
+
120
+ // Which $source values are actually producing AIS, so the config field can be filled
121
+ // in from the log instead of guessed.
122
+ function reportSources () {
123
+ if (!state || !state.sourcesSeen.size) return
124
+ const list = [...state.sourcesSeen.entries()].sort((a, b) => b[1] - a[1])
125
+ .map(([s, n]) => `${s} (${n}${isInternetFeed(s) ? ', internet feed — not forwarded' : ''})`)
126
+ log('AIS sources seen: ' + list.join('; '))
127
+ }
128
+
129
+ function accept (context, source) {
130
+ if (!context || context === state.selfContext) return false // never our own vessel
131
+ if (cfg.source) return String(source) === String(cfg.source)
132
+ return !isInternetFeed(source)
133
+ }
134
+
135
+ function handleDelta (delta) {
136
+ if (!state || state.stopped || !delta || !Array.isArray(delta.updates)) return
137
+ const context = delta.context
138
+ if (!context || context === state.selfContext) return
139
+
140
+ for (const update of delta.updates) {
141
+ if (!update || !Array.isArray(update.values)) continue
142
+ const source = update.$source || (update.source && (update.source.label || update.source.type)) || 'unknown'
143
+ state.sourcesSeen.set(source, (state.sourcesSeen.get(source) || 0) + 1)
144
+ if (!accept(context, source)) { state.dropped++; continue }
145
+
146
+ // Split identity from position: identity repeats constantly and never changes.
147
+ const now = Date.now()
148
+ const staticDue = (now - (state.lastStatic.get(context) || 0)) >= cfg.staticIntervalMs
149
+ const values = update.values.filter((pv) => {
150
+ if (!pv || !pv.path) return false
151
+ if (STATIC_PATHS.includes(pv.path)) return staticDue
152
+ return POSITION_PATHS.includes(pv.path)
153
+ })
154
+ if (!values.length) continue
155
+ if (staticDue && values.some((pv) => STATIC_PATHS.includes(pv.path))) state.lastStatic.set(context, now)
156
+
157
+ // self:false is the tag the cloud filters on to keep AIS out of the owner's charts.
158
+ const lines = deltaToLines({ context, updates: [{ ...update, values }] }, { self: false })
159
+ if (lines.length) {
160
+ state.batch.push(...lines)
161
+ state.targets.add(context)
162
+ state.forwarded += lines.length
163
+ }
164
+ if (state.batch.length >= cfg.batchSize) flush()
165
+ }
166
+ }
167
+
168
+ function subscribe () {
169
+ const sub = { context: '*', subscribe: ALL_PATHS.map((p) => ({ path: p, period: cfg.periodMs || 10000 })) }
170
+ if (app.subscriptionmanager && app.subscriptionmanager.subscribe) {
171
+ app.subscriptionmanager.subscribe(sub, state.unsubscribes, (err) => warn('subscription error: ' + err), handleDelta)
172
+ } else if (app.signalk && app.signalk.on) {
173
+ const h = (d) => handleDelta(d)
174
+ app.signalk.on('delta', h)
175
+ state.unsubscribes.push(() => app.signalk.removeListener('delta', h))
176
+ } else {
177
+ warn('no subscription mechanism available — nothing will be forwarded')
178
+ }
179
+ }
180
+
181
+ function flush () {
182
+ if (!state || state.stopped || !state.batch.length) return
183
+ const lines = state.batch
184
+ state.batch = []
185
+ state.spool.append(lines).then(() => pump()).catch((e) => {
186
+ warn('spool append failed: ' + e.message)
187
+ state.batch.unshift(...lines)
188
+ })
189
+ }
190
+
191
+ async function pump () {
192
+ if (!state || state.stopped || state.pumping) return
193
+ // Telemetry always wins the link: stand down entirely while its spool is behind.
194
+ if (cfg.pending) {
195
+ let depth = 0
196
+ try { depth = (await cfg.pending()).count || 0 } catch {}
197
+ if (depth) { refreshStatus(`waiting — telemetry backlog (${depth})`); scheduleIdle(); return }
198
+ }
199
+ state.pumping = true
200
+ try {
201
+ for (const file of await state.spool.pending()) {
202
+ if (state.stopped) break
203
+ let body
204
+ try { body = await fs.promises.readFile(file, 'utf8') } catch { continue }
205
+ if (!body.trim()) { await state.spool.remove(file); continue }
206
+ const res = await writeLines({ influxUrl: cfg.influxUrl, org: cfg.org, bucket: cfg.bucket, token: cfg.token, timeoutMs: cfg.requestTimeoutMs }, body)
207
+ if (res.ok) {
208
+ await state.spool.remove(file); state.lastOkAt = new Date(); state.backoff = cfg.retryMinMs
209
+ } else if (res.retryable) {
210
+ state.pumping = false; scheduleRetry(res); return
211
+ } else {
212
+ warn(`batch rejected (HTTP ${res.status}) — quarantined`)
213
+ await state.spool.quarantine(file)
214
+ }
215
+ }
216
+ } catch (e) {
217
+ warn('pump error: ' + e.message); state.pumping = false; scheduleRetry(); return
218
+ }
219
+ state.pumping = false
220
+ scheduleIdle()
221
+ refreshStatus()
222
+ }
223
+
224
+ function scheduleRetry (res) {
225
+ if (!state || state.stopped) return
226
+ clearTimeout(state.pumpTimer)
227
+ const delay = state.backoff
228
+ state.backoff = Math.min(state.backoff * 2, cfg.retryMaxMs)
229
+ state.pumpTimer = setTimeout(pump, delay)
230
+ if (state.pumpTimer.unref) state.pumpTimer.unref()
231
+ refreshStatus(`${res && res.status ? 'HTTP ' + res.status : 'offline'} — retry ${Math.round(delay / 1000)}s`)
232
+ }
233
+ function scheduleIdle () {
234
+ if (!state || state.stopped) return
235
+ clearTimeout(state.pumpTimer)
236
+ state.pumpTimer = setTimeout(pump, cfg.idlePollMs)
237
+ if (state.pumpTimer.unref) state.pumpTimer.unref()
238
+ }
239
+ function refreshStatus (suffix) {
240
+ if (!state) return
241
+ state.statusLine = `ais: ${state.targets.size} target(s), ${state.forwarded} point(s)${state.dropped ? `, ${state.dropped} skipped` : ''}${suffix ? '; ' + suffix : ''}`
242
+ }
243
+
244
+ function stop () {
245
+ if (!state) return
246
+ state.stopped = true
247
+ clearInterval(state.flushTimer)
248
+ clearInterval(state.reportTimer)
249
+ clearTimeout(state.pumpTimer)
250
+ for (const u of (state.unsubscribes || [])) { try { u() } catch {} }
251
+ if (state.batch && state.batch.length && state.spool) {
252
+ try { fs.writeFileSync(path.join(state.spool.dir, `${Date.now()}-final.lp`), state.batch.join('\n') + '\n') } catch {}
253
+ state.batch = []
254
+ }
255
+ }
256
+
257
+ function status () { return state ? state.statusLine : 'ais: off' }
258
+
259
+ return { start, stop, status, _handleDelta: handleDelta, _state: () => state, _flush: flush }
260
+ }
261
+
262
+ module.exports = { createAis, isInternetFeed, POSITION_PATHS, STATIC_PATHS }