sailkick-boat 0.14.4 → 0.14.7

package/README.md

@@ -63,10 +63,19 @@ comes from the cloud **announcing bakes**, not from a clock:
 `X-Sailkick-Cache` reports `HIT` / `MISS` / `UPDATED` / `STALE` / `LIVE` per response.
 
 **Static vs dynamic — two strategies.** Tiles and app assets are **cache-first**
-(pinned, offline forever). Dynamic `/api/*` data (AIS, weather) is **network-first**:
-fetched live every time online (so AIS/weather never go stale), with the last
-response kept only as an **offline fallback** (`STALE`). Local `/api/history` and the
-patched `/api/config` are served specially (above).
+(pinned, offline forever). Dynamic `/api/*` data (AIS, weather, lightning) is
+**network-first**: fetched live every time online (so it never goes stale), with the
+last response kept only as an **offline fallback** (`STALE`). Local `/api/history` and
+the patched `/api/config` are served specially (above).
+
+**Exception — velocity tiles are pinned.** The app serves its wind/current field from
+`/api/velocity/tiles/<layer>/<runId>/<z>/<x>/<y>/<hour>.f32`. The forecast **run id is
+in the path**, so a URL's bytes never change — a new run means new URLs. Those are
+cache-first like map tiles: each tile downloads once instead of on every pan, and the
+wind particles + storm field keep rendering **offline**. The velocity *manifest*
+(`?run=latest`) stays network-first, or the boat would pin itself to a stale run
+forever. Nothing prefetches velocity tiles yet, so offline wind covers only what you
+have panned over, and old runs are not pruned.
 
 **Offline circuit breaker.** Some boat routers return errors (or hang) for outbound
 requests when the uplink is down, instead of failing cleanly. Once a fetch fails, the
@@ -137,6 +146,38 @@ Two ways, chosen automatically:
   same BoatState feeding `/ws/telemetry`) — for boats with no local InfluxDB, e.g.
   **SignalK on a Victron GX / Venus OS**. Same JSON contract, fully offline, no database.
   (`historyAvailable` reports true either way, so the app shows the Trends panel.)
+  Same eight channels as the InfluxDB path, `stw` included.
+
+**Serving an archive.** Set `historyToken` and the three `history*` fields point at any
+InfluxDB holding `signalk-to-influxdb-v2`-schema data — a still-running local database,
+or a bucket you imported an old logbook into. That is the archive's real value: history
+from before this boat ever synced to the cloud. The token is the switch; blank means the
+built-in ring. The plugin logs which source it chose at startup:
+
+```
+[sailkick-boat] history -> archive http://127.0.0.1:8086 org=addiction bucket=bandg
+[sailkick-boat] history -> built-in ring (no archive token set)
+```
+
+Upgrading from before 0.14.7, when these lived in a hidden `proxy.history` block: your
+saved values still win over the newly pre-filled defaults, so an archive in a
+non-default org/bucket is not silently lost when the config UI writes `signalk` over it.
+The plugin says so in the log — copy the values into the visible fields when convenient.
+
+**Do you actually need a local InfluxDB?** For *live* trends, usually not. The app never requests finer
+than `every=5s` over a `window` of 24 h, and the ring's floor at that window is 2 s — so
+set `ringSampleSec: 5` and it matches anything the UI can draw, from live state, with no
+database to run. A local InfluxDB is worth it only for history predating cloud sync,
+more than 30 days offline, or channels outside BoatState (engine, batteries) that the
+Trends panel does not chart anyway.
+
+**True wind comes from your instruments.** If the boat publishes
+`environment.wind.speedTrue` / `directionTrue`, those are stored verbatim — a wind
+system corrects for heel and leeway against water-referenced boat speed, and every other
+display aboard shows its numbers. Only when the boat publishes none is true wind
+derived, and then from **STW**, not SOG: true wind is relative to motion through the
+water. (Before v0.14.6 the derivation used SOG, which in 3 kt of foul tide skewed TWD by
+~12° and TWS by ~1.8 kt.) SOG is the last resort for boats with no paddlewheel.
   - **Persistent (append-log):** the ring is saved as a JSONL append-log at
     `<dataDir>/history/history-ring.jsonl` (on the SSD/USB with the tiles; override with
     `proxy.history.ringDir`), so it **survives restarts**. Each sample appends one line; the file is
@@ -198,7 +239,10 @@ in `index.js`.
 - **Telemetry sync → cloud**: `enabled`
 - **Offline app & maps**: `enabled`, `proxyPort` (default 8080), `localSignalkUrl`
   (default `http://127.0.0.1:3000`), `dataDir`, `seedEnabled`, `prefetchRadiusNm`,
-  `prefetchDetailZoom`, `historyToken`
+  `prefetchDetailZoom`
+- **History archive** (optional): `historyToken`, `historyInfluxUrl`
+  (default `http://127.0.0.1:8086`), `historyOrg` + `historyBucket` (default `signalk`,
+  matching `signalk-to-influxdb-v2`)
 
 `dataDir` is the one storage location — cached maps, the telemetry spool and the
 history ring log all live under it. **Put it on the SSD/USB disk, not the SD card.**

package/index.js

@@ -69,6 +69,20 @@ function isPrivateHostUrl (u) {
   } catch { return false }
 }
 
+// A pre-filled schema default must not silently shadow a legacy value that differs. The
+// Signal K config UI writes every default on save, so `historyOrg: "signalk"` appears on
+// a boat whose archive has always lived in org "addiction" under the old nested
+// proxy.history block — and the archive would go quiet with no explanation. Rule: an
+// explicitly-changed field wins; otherwise a legacy value that differs from the default
+// wins; otherwise the default.
+function pickHistoryField (flat, legacy, dflt) {
+  const f = String(flat == null ? '' : flat).trim()
+  const l = String(legacy == null ? '' : legacy).trim()
+  if (f && f !== dflt) return { value: f, from: 'config' }
+  if (l && l !== dflt) return { value: l, from: 'legacy' }
+  return { value: f || l || dflt, from: 'default' }
+}
+
 // Endpoint resolution with one rule: the constant wins unless self-hosting is declared.
 // Returns { url, ignored } — `ignored` is the stale value we refused, so the caller can
 // say so out loud instead of leaving the owner to guess why nothing arrives.
@@ -141,7 +155,10 @@ module.exports = function (app) {
             enumNames: ['Overview (z12)', 'Coastal (z13)', 'Detailed (z14)', 'Harbor (z15)'],
             default: 13
           },
-          historyToken: { type: 'string', title: 'Local InfluxDB read token (optional)', description: 'Only if this boat already runs its own InfluxDB (e.g. signalk-to-influxdb-v2) and you want full history from it. Leave blank — the plugin then keeps its own lightweight history, which is what most boats want.' }
+          historyToken: { type: 'string', title: 'History archive: read token', description: 'Leave blank for the built-in lightweight history, which is what most boats want and needs no database. Paste a read token to serve Trends and the track from a local InfluxDB instead — worth it for history recorded before this boat synced to the cloud, for more than 30 days offline, or for an imported archive. Setting a token switches the source; the three fields below say where to find it.' },
+          historyInfluxUrl: { type: 'string', title: '…InfluxDB URL', description: 'Where the archive lives. The default is an InfluxDB on this machine.', default: 'http://127.0.0.1:8086' },
+          historyOrg: { type: 'string', title: '…organization', description: 'Default matches signalk-to-influxdb-v2.', default: 'signalk' },
+          historyBucket: { type: 'string', title: '…bucket', description: 'Default matches signalk-to-influxdb-v2. An imported archive often uses its own name.', default: 'signalk' }
         }
       }
     }
@@ -220,6 +237,22 @@ module.exports = function (app) {
       const oldSeed = p.seed || {}
       const oldPrefetch = p.prefetch || {}
       const oldHistory = p.history || {}
+      // History source, resolved before the modules start so it can be reported. A token
+      // is the switch: present = query the archive, absent = the built-in ring.
+      const hUrl = pickHistoryField(p.historyInfluxUrl, oldHistory.influxUrl, HISTORY_TUNING.influxUrl)
+      const hOrg = pickHistoryField(p.historyOrg, oldHistory.org, HISTORY_TUNING.org)
+      const hBucket = pickHistoryField(p.historyBucket, oldHistory.bucket, HISTORY_TUNING.bucket)
+      const hToken = String(p.historyToken || oldHistory.token || '').trim()
+      const fromLegacy = [['URL', hUrl], ['org', hOrg], ['bucket', hBucket]].filter(([, v]) => v.from === 'legacy')
+      if (hToken) {
+        console.log(`[sailkick-boat] history -> archive ${hUrl.value} org=${hOrg.value} bucket=${hBucket.value}`)
+        if (fromLegacy.length) {
+          ;(app.error || console.error)(`[sailkick-boat] history archive ${fromLegacy.map(([n]) => n).join(' + ')} taken from the old proxy.history config, not the new field(s) — copy the value(s) up to keep them after the next save`)
+        }
+      } else {
+        console.log('[sailkick-boat] history -> built-in ring (no archive token set)')
+      }
+
       if (upstream.ignored) {
         ;(app.error || console.error)(`[sailkick-boat] ignoring proxy.sailkickUrl "${upstream.ignored}" left over from an older config — mirroring ${upstream.url}. Set proxy.selfHosted:true to keep your own server.`)
       }
@@ -251,10 +284,10 @@ module.exports = function (app) {
         history: {
           ...HISTORY_TUNING,
           enabled: oldHistory.enabled !== false,
-          influxUrl: oldHistory.influxUrl || HISTORY_TUNING.influxUrl,
-          org: oldHistory.org || HISTORY_TUNING.org,
-          bucket: oldHistory.bucket || HISTORY_TUNING.bucket,
-          token: p.historyToken || oldHistory.token || '',
+          influxUrl: hUrl.value,
+          org: hOrg.value,
+          bucket: hBucket.value,
+          token: hToken,
           ringPersist: oldHistory.ringPersist !== false,
           ringWindowSec: oldHistory.ringWindowSec || HISTORY_TUNING.ringWindowSec,
           ringSampleSec: oldHistory.ringSampleSec || HISTORY_TUNING.ringSampleSec,

package/lib/history/ring.js

@@ -24,12 +24,28 @@ const DEG = Math.PI / 180
 const wrap360 = (d) => ((d % 360) + 360) % 360
 const MAX_SAMPLES = 50000
 
-// True wind (TWS kt, TWD ° the wind blows FROM) from apparent + motion over ground.
+// True wind (TWS kt, TWD ° the wind blows FROM).
+//
+// Prefer what the instruments publish. A wind system computes true wind from its own
+// calibrated apparent wind with heel and leeway correction and a water-referenced boat
+// speed — strictly better than anything derivable here, and it is what every other
+// display on board shows, so deriving a second answer would just disagree with them.
+//
+// Only when the boat publishes no true wind do we derive it, and then from speed
+// THROUGH WATER: true wind is defined relative to the boat's motion through the water,
+// not over the ground. Using SOG (as this did before v0.14.6) skews both TWS and TWD in
+// any tidal stream — several degrees and about a knot in 3 kt of tide, which is exactly
+// the error you would be trimming against. SOG remains the last resort for boats with
+// no paddlewheel, where it is at least right in slack water.
 function trueWind (s) {
+  if (Number.isFinite(s.twsKt) && Number.isFinite(s.twdDeg)) {
+    return { tws: s.twsKt, twd: wrap360(s.twdDeg), measured: true }
+  }
   if (!Number.isFinite(s.awsKt) || !Number.isFinite(s.awaDeg) || !Number.isFinite(s.headingDeg)) return null
+  const boatSpeed = Number.isFinite(s.stwKt) ? s.stwKt : (Number.isFinite(s.sogKt) ? s.sogKt : 0)
   const a = s.awaDeg * DEG
-  const x = s.awsKt * Math.cos(a) - (s.sogKt || 0); const y = s.awsKt * Math.sin(a)
-  return { tws: Math.hypot(x, y), twd: wrap360(s.headingDeg + Math.atan2(y, x) / DEG) }
+  const x = s.awsKt * Math.cos(a) - boatSpeed; const y = s.awsKt * Math.sin(a)
+  return { tws: Math.hypot(x, y), twd: wrap360(s.headingDeg + Math.atan2(y, x) / DEG), measured: false }
 }
 
 class RingHistoryProvider {
@@ -61,6 +77,7 @@ class RingHistoryProvider {
     const row = {
       t: Date.now(),
       sog: num(s.sogKt), cog: num(s.cogDeg), heading: num(s.headingDeg),
+      stw: num(s.stwKt),
       aws: num(s.awsKt), awa: num(s.awaDeg), depth: num(s.depthM),
       tws: tw ? tw.tws : null, twd: tw ? tw.twd : null,
       lat: num(s.lat), lon: num(s.lon)
@@ -80,7 +97,10 @@ class RingHistoryProvider {
 
   getSeries ({ windowSec } = {}) {
     const rows = this._window(windowSec)
-    const chans = ['tws', 'twd', 'aws', 'awa', 'sog', 'heading', 'depth'] // no STW in BoatState
+    // Matches the InfluxDB provider's channel set exactly (lib/history/index.js MAP), so
+    // the Trends panel looks the same whichever provider is serving. Empty channels are
+    // omitted, so a boat with no paddlewheel simply has no `stw` key.
+    const chans = ['tws', 'twd', 'aws', 'awa', 'sog', 'stw', 'heading', 'depth']
     const series = {}
     for (const c of chans) {
       const pts = rows.filter((r) => r[c] != null).map((r) => [r.t, r[c]])

package/lib/proxy/index.js

@@ -18,6 +18,20 @@ const { countBboxTiles, bboxTiles, boxAround } = require('./tiles')
 // get its live data from the boat's local SignalK and its charts/weather cached.
 // Also exposes the auth'd /plugins/sailkick-boat/p/* router handlers.
 
+// Everything under /api/ is dynamic and network-first — EXCEPT paths whose URL is
+// immutable by construction. Velocity tiles embed the forecast run id
+// (/api/velocity/tiles/<layer>/<runId>/<z>/<x>/<y>/<hour>.f32, see the app's
+// public/engine/velocity-client.js), so a given URL's bytes never change: a new
+// forecast run produces new URLs. Treating them as network-first re-downloaded the
+// whole wind field on every pan while online, and left nothing usable offline —
+// pinning them is what makes wind particles and the storm field work with no uplink.
+//
+// The velocity *manifest* is deliberately NOT in here: it resolves `run=latest` and
+// must stay live, otherwise the boat pins itself to a stale forecast run forever.
+const IMMUTABLE_API_PREFIXES = ['/api/velocity/tiles/']
+const isImmutableApi = (p) => IMMUTABLE_API_PREFIXES.some((x) => p.startsWith(x))
+const isNetworkFirst = (p) => p.startsWith('/api/') && !isImmutableApi(p)
+
 function createProxy (app, options) {
   const log = (m) => (app.debug ? app.debug('[proxy] ' + m) : console.log('[sailkick-boat:proxy]', m))
   let cfg = null
@@ -138,8 +152,10 @@ function createProxy (app, options) {
       if (req.method === 'GET' || req.method === 'HEAD') {
         const { buffer, contentType, cacheState } = await getResource({
           storeDir: cfg.storeDir, upstream: cfg.upstream, reqPath: req.url, timeoutMs: cfg.timeoutMs,
-          invalidatedAt: manifest ? manifest.invalidatedAtFor(req.url) : 0,
-          networkFirst: req.url.startsWith('/api/'), // live data (AIS, weather): fresh online, cache-fallback offline
+          // An immutable URL needs no invalidation: a bake announcement can't change
+          // bytes that are keyed by run id, so don't let an app rebuild churn them.
+          invalidatedAt: isImmutableApi(req.url) ? 0 : (manifest ? manifest.invalidatedAtFor(req.url) : 0),
+          networkFirst: isNetworkFirst(req.url), // live data (AIS, weather, lightning): fresh online, cache-fallback offline
           health
         })
         res.setHeader('Content-Type', contentType)
@@ -242,8 +258,8 @@ function createProxy (app, options) {
     try {
       const { buffer, contentType, cacheState } = await getResource({
         storeDir: cfg.storeDir, upstream: cfg.upstream, reqPath, timeoutMs: cfg.timeoutMs,
-        invalidatedAt: manifest ? manifest.invalidatedAtFor(reqPath) : 0,
-        networkFirst: reqPath.startsWith('/api/'), health
+        invalidatedAt: isImmutableApi(reqPath) ? 0 : (manifest ? manifest.invalidatedAtFor(reqPath) : 0),
+        networkFirst: isNetworkFirst(reqPath), health
       })
       res.set('Content-Type', contentType)
       res.set('X-Sailkick-Cache', cacheState)
@@ -390,4 +406,4 @@ function createProxy (app, options) {
   return { start, stop, status, handleGet, handlePrefetch, handlePrefetchRegion, handleClear, _serveMirror: serveMirror, _serveConfig: serveConfig, _manifest: () => manifest, _seeder: () => seeder, _startAreaPrefetch: startAreaPrefetch, _area: () => area }
 }
 
-module.exports = { createProxy }
+module.exports = { createProxy, isNetworkFirst, isImmutableApi }

package/lib/telemetry/signalk-map.js

@@ -27,6 +27,9 @@ function signalkValuesToPatch (values) {
       case 'navigation.speedOverGround':
         if (Number.isFinite(v.value)) patch.sogKt = Math.max(0, v.value * MS_TO_KT)
         break
+      case 'navigation.speedThroughWater':
+        if (Number.isFinite(v.value)) patch.stwKt = Math.max(0, v.value * MS_TO_KT)
+        break
       case 'navigation.courseOverGroundTrue':
         if (Number.isFinite(v.value)) patch.cogDeg = wrap360(v.value * RAD2DEG)
         break
@@ -45,10 +48,34 @@ function signalkValuesToPatch (values) {
       case 'environment.wind.angleApparent':
         if (Number.isFinite(v.value)) patch.awaDeg = wrap180(v.value * RAD2DEG)
         break
+      case 'environment.wind.speedTrue':
+        if (Number.isFinite(v.value)) patch.twsKt = Math.max(0, v.value * MS_TO_KT)
+        break
+      case 'environment.wind.directionTrue': // absolute compass direction (not off-bow)
+        if (Number.isFinite(v.value)) patch.twdDeg = wrap360(v.value * RAD2DEG)
+        break
       case 'environment.depth.belowSurface':
       case 'environment.depth.belowTransducer':
         if (Number.isFinite(v.value)) patch.depthM = v.value
         break
+      case 'steering.rudderAngle':
+        if (Number.isFinite(v.value)) patch.rudderDeg = v.value * RAD2DEG
+        break
+      case 'navigation.rateOfTurn':
+        if (Number.isFinite(v.value)) patch.rotDegMin = v.value * RAD2DEG * 60
+        break
+      case 'navigation.attitude': // object { roll, pitch, yaw } (rad)
+        if (v.value && Number.isFinite(v.value.roll)) patch.heelDeg = v.value.roll * RAD2DEG
+        break
+      case 'steering.autopilot.state':
+        if (typeof v.value === 'string') patch.apState = v.value
+        break
+      case 'steering.autopilot.target.headingTrue':
+        if (Number.isFinite(v.value)) patch.apTargetDeg = wrap360(v.value * RAD2DEG)
+        break
+      case 'steering.autopilot.target.windAngleApparent':
+        if (Number.isFinite(v.value)) patch.apTargetAwa = wrap180(v.value * RAD2DEG)
+        break
       case 'navigation.datetime':
         if (typeof v.value === 'string') patch.gpsTime = v.value
         break

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "sailkick-boat",
-  "version": "0.14.4",
+  "version": "0.14.7",
   "description": "EARLY ALPHA — cloud telemetry + offline maps for sailkick boats (www.sailkick.io; register on the web, paste the write token). Gapless boat→cloud telemetry sync to InfluxDB, and a local proxy that keeps the sailkick app and its charts/maps working fully offline on board.",
   "main": "index.js",
   "scripts": {