signalk-performance-processor 0.0.1 → 0.0.2

package/CHANGELOG.md

@@ -1,5 +1,13 @@
 # Changelog
 
+## 0.0.2 — 2026-10-29
+
+Still a proof of concept: run it in Shadow mode alongside your instruments.
+
+- Heading for TWD and current comes from `navigation.headingTrue` if present, otherwise magnetic heading plus `navigation.magneticVariation` (normally from the GPS). The variation in Settings is now only a fallback, and the Live page says which is in use.
+- Configuration is edited only in the webapp and saved to `config.json` in the plugin's data directory. The admin Plugin Config form no longer shows the tables, so saving it cannot overwrite them. A configuration saved by 0.0.1 is picked up until the webapp first saves.
+- The configuration file records its format version (`"schema": 1`). Future releases upgrade older files as they load; a file from a newer release is refused and never overwritten.
+
 ## 0.0.1 — 2026-10-29
 
 First release, a proof of concept. Run it in Shadow mode alongside your instruments before relying on it.

package/README.md

@@ -2,7 +2,7 @@
 
 # signalk-performance-processor
 
-> **Proof of concept (0.0.1).** Run it in Shadow mode alongside your instruments and compare before relying on it.
+> **Proof of concept (0.0.2).** Run it in Shadow mode alongside your instruments and compare before relying on it.
 
 Signal K plugin and webapp that does the calibration job of an instrument processor such as a B&G H5000: it corrects boat speed and leeway, works out true wind from apparent wind, applies H5000-style TWA and TWS correction tables, and publishes true wind and current.
 
@@ -29,7 +29,7 @@ The corrections run in this order, each on the output of the one before:
 |---|---|
 | Apparent wind | `environment.wind.angleApparent`, `environment.wind.speedApparent` |
 | Speed through water | `navigation.speedThroughWater` |
-| Heading | `navigation.headingTrue`, or `navigation.headingMagnetic` + the variation in Settings |
+| Heading | `navigation.headingTrue`; otherwise `navigation.headingMagnetic` + `navigation.magneticVariation` (normally from the GPS); otherwise magnetic heading + the fallback variation in Settings |
 | Heel | `navigation.attitude` (roll) |
 | SOG and COG | `navigation.speedOverGround`, `navigation.courseOverGroundTrue` |
 
@@ -63,9 +63,9 @@ Open **Webapps → Performance Processor**.
 
 - **Live**: the outputs next to their uncorrected values, what each step is doing right now, and which source and how fresh each input is.
 - **Calibration**: every correction in pipeline order with an on/off switch. Select one to edit its values, then **Save**. **Reset to defaults** puts back this boat's fitted values for that correction.
-- **Settings**: Shadow or Replace, magnetic variation, and which source to read for each input.
+- **Settings**: Shadow or Replace, the fallback magnetic variation, and which source to read for each input.
 
-Saving needs write access when Signal K security is on. The settings are stored as the plugin's normal configuration, so they are also visible (and editable as a fallback) under **Server → Plugin Config**.
+All configuration lives in the webapp. It is saved to `config.json` in the plugin's data directory (under `~/.signalk/plugin-config-data/`), not in the admin **Plugin Config** form, which only points here; saving that form cannot overwrite the tables. Saving needs write access when Signal K security is on.
 
 ## Defaults for this boat
 
@@ -83,7 +83,23 @@ Fitted to the 2025 race logs and checked on Jan–Mar 2026 races not used in fit
 
   The reach row is provisional: it rests on roundings only.
 - **TWS table (%):** off. Downwind −11.1% at 15 kn and −14.4% from 20 kn; light air and upwind left at 0.
-- **Variation:** 12.8° E.
+- **Fallback variation:** 12.8° E, used only when nothing on the boat sends true heading or magnetic variation. The Live page shows which applies.
+
+## Configuration file
+
+`config.json` in the plugin's data directory, written by the webapp:
+
+```json
+{
+  "schema": 1,
+  "mode": "shadow",
+  "variation": 12.8,
+  "sources": { "apparentWind": "", "stw": "", "heading": "", "attitude": "", "gps": "", "variation": "" },
+  "corrections": { "stw": { … }, "leeway": { … }, "twa": { … }, "tws": { … } }
+}
+```
+
+`schema` is the version of the file's shape. A release that changes the shape bumps `CONFIG_SCHEMA` in `lib/processor.js` and adds a migration from the previous version, so older files are upgraded as they load and nothing is reset. A file without `schema` (saved by 0.0.1) is read as schema 1. A file from a newer release is refused rather than guessed at, and is never overwritten by an older plugin.
 
 ## Adding a correction
 

package/index.js

@@ -1,11 +1,17 @@
 'use strict'
 
+const fs = require('fs')
+const path = require('path')
 const { DEFAULTS } = require('./lib/defaults')
-const { process, correctInputs, normalizeConfig, CORRECTIONS } = require('./lib/processor')
+const { process, correctInputs, normalizeConfig, CORRECTIONS, CONFIG_SCHEMA } = require('./lib/processor')
 
 const MS_TO_KN = 1.94384
 const RAD_TO_DEG = 180 / Math.PI
 const STALE_MS = 3000 // an input older than this is not used
+// Variation changes over days, not seconds; a GPS that drops it briefly should not
+// push heading onto the fallback value.
+const VARIATION_MAX_AGE_MS = 10 * 60 * 1000
+const CONFIG_FILE = 'config.json'
 const STATUS_INTERVAL_MS = 1000
 // Bounds for sanity-checking sensor input. Anything outside these is a fault, not a reading.
 const MAX_PLAUSIBLE_STW_KN = 60
@@ -21,7 +27,8 @@ const INPUTS = {
   stw: ['navigation.speedThroughWater'],
   heading: ['navigation.headingMagnetic', 'navigation.headingTrue'],
   attitude: ['navigation.attitude'],
-  gps: ['navigation.speedOverGround', 'navigation.courseOverGroundTrue']
+  gps: ['navigation.speedOverGround', 'navigation.courseOverGroundTrue'],
+  variation: ['navigation.magneticVariation']
 }
 const INPUT_OF = Object.fromEntries(Object.entries(INPUTS).flatMap(([k, ps]) => ps.map(p => [p, k])))
 
@@ -56,7 +63,14 @@ module.exports = function (app) {
   let lastResultAt = 0
   let lastStatus = ''
 
-  plugin.schema = schemaFor(DEFAULTS)
+  // Everything is configured in the webapp and kept in this plugin's data directory,
+  // not in the plugin options: the admin form would otherwise show the tables, and
+  // saving it could overwrite the webapp's values.
+  plugin.schema = {
+    type: 'object',
+    description: 'Calibration tables and settings are edited in Webapps → Performance Processor.',
+    properties: {}
+  }
 
   function outPath (key) {
     return config.mode === 'shadow' ? OUTPUTS[key].path + SHADOW_SUFFIX : OUTPUTS[key].path
@@ -80,9 +94,9 @@ module.exports = function (app) {
   }
   const isStwEcho = ms => recentStw.some(o => Math.abs(o - ms) < 1e-12)
 
-  function fresh (path, now) {
-    const l = latest[path]
-    return l && now - l.at < STALE_MS ? l.value : undefined
+  function fresh (p, now, maxAge = STALE_MS) {
+    const l = latest[p]
+    return l && now - l.at < maxAge ? l.value : undefined
   }
 
   // Refresh on a change of state, and at most once a second within a state so the
@@ -148,6 +162,7 @@ module.exports = function (app) {
       heel: heelDeg(now),
       headingMagnetic: deg(fresh('navigation.headingMagnetic', now)),
       headingTrue: deg(fresh('navigation.headingTrue', now)),
+      variation: deg(fresh('navigation.magneticVariation', now, VARIATION_MAX_AGE_MS)),
       sog: (v => v == null ? undefined : v * MS_TO_KN)(fresh('navigation.speedOverGround', now)),
       cog: deg(fresh('navigation.courseOverGroundTrue', now))
     }
@@ -265,10 +280,43 @@ module.exports = function (app) {
     unsubscribes = []
   }
 
+  // ---- stored configuration ---------------------------------------------------
+
+  function configFile () {
+    return path.join(app.getDataDirPath(), CONFIG_FILE)
+  }
+
+  // The webapp's saved configuration. Before the file exists, take a configuration left
+  // in the plugin options by 0.0.1, which stored it there; otherwise start from defaults.
+  function loadStored (options) {
+    try {
+      return JSON.parse(fs.readFileSync(configFile(), 'utf8'))
+    } catch (e) {
+      if (e.code !== 'ENOENT') app.error(`could not read ${configFile()}: ${e.message}`)
+    }
+    return options && options.corrections ? options : {}
+  }
+
+  // Write to a temporary file and rename, so a crash mid-write never leaves half a file.
+  // Never overwrite a file from a newer release: this version would drop what it
+  // does not understand.
+  function saveStored (cfg) {
+    const file = configFile()
+    let onDisk = null
+    try { onDisk = JSON.parse(fs.readFileSync(file, 'utf8')).schema } catch (e) {}
+    if (Number.isInteger(onDisk) && onDisk > CONFIG_SCHEMA) {
+      throw new Error(`the saved configuration is schema ${onDisk}, from a newer version of this plugin; update the plugin rather than overwrite it`)
+    }
+    fs.mkdirSync(path.dirname(file), { recursive: true })
+    fs.writeFileSync(file + '.tmp', JSON.stringify(cfg, null, 2))
+    fs.renameSync(file + '.tmp', file)
+  }
+
   plugin.start = function (options) {
-    const { config: cfg, errors } = normalizeConfig(options || {})
+    const { config: cfg, errors } = normalizeConfig(loadStored(options))
     if (errors.length) {
-      app.setPluginError(`Configuration: ${errors.join('; ')}`)
+      app.setPluginError(`Configuration: ${errors.join('; ')} — fix it in the webapp`)
+      config = normalizeConfig({}).config // so the webapp can still load and repair it
       return
     }
     startWith(cfg)
@@ -310,9 +358,7 @@ module.exports = function (app) {
   plugin.registerWithRouter = function (router) {
     router.get('/config', (req, res) => {
       res.set('Cache-Control', 'no-store')
-      let stored = {}
-      try { stored = app.readPluginOptions().configuration || {} } catch (e) {}
-      res.json({ config: config || normalizeConfig(stored).config, defaults: DEFAULTS })
+      res.json({ config: config || normalizeConfig(loadStored()).config, defaults: DEFAULTS })
     })
 
     router.put('/config', async (req, res) => {
@@ -320,11 +366,13 @@ module.exports = function (app) {
       try { body = await readBody(req) } catch (e) { return res.status(400).json({ errors: ['body is not valid JSON'] }) }
       const { config: cfg, errors } = normalizeConfig(body)
       if (errors.length) return res.status(400).json({ errors })
-      app.savePluginOptions(cfg, err => {
-        if (err) return res.status(500).json({ errors: [`could not save: ${err.message || err}`] })
-        startWith(cfg)
-        res.json({ config: cfg })
-      })
+      try {
+        saveStored(cfg)
+      } catch (e) {
+        return res.status(500).json({ errors: [`could not save: ${e.message}`] })
+      }
+      startWith(cfg)
+      res.json({ config: cfg })
     })
 
     router.get('/live', (req, res) => {
@@ -336,33 +384,5 @@ module.exports = function (app) {
   return plugin
 }
 
-// JSON schema for the admin "Plugin Config" form, built from the defaults so the form
-// is a working fallback editor and a fresh install starts from this boat's values.
-function schemaFor (defaults) {
-  const titles = {
-    mode: 'Output: shadow publishes …Corrected paths alongside the instruments; replace publishes the standard paths (rank this plugin first in Source Priorities)',
-    variation: 'Magnetic variation (degrees, east positive)',
-    sources: 'Input sources ($source to read; blank = any source except this plugin)',
-    corrections: 'Corrections (easier to edit in the Performance Processor webapp)'
-  }
-  function node (value, title) {
-    if (Array.isArray(value)) return { type: 'array', title, items: { type: 'number' }, default: value }
-    if (value && typeof value === 'object') {
-      return {
-        type: 'object',
-        title,
-        properties: Object.fromEntries(Object.entries(value).map(([k, v]) => [k, node(v, k)]))
-      }
-    }
-    if (typeof value === 'boolean') return { type: 'boolean', title, default: value }
-    if (typeof value === 'number') return { type: 'number', title, default: value }
-    return { type: 'string', title, default: value }
-  }
-  const schema = node(defaults)
-  for (const [k, t] of Object.entries(titles)) schema.properties[k].title = t
-  schema.properties.mode.enum = ['shadow', 'replace']
-  return schema
-}
-
 // Exposed for the test suite only; not part of the Signal K plugin API.
-module.exports._internals = { schemaFor, OUTPUTS, INPUTS, SHADOW_SUFFIX }
+module.exports._internals = { OUTPUTS, INPUTS, SHADOW_SUFFIX, CONFIG_FILE }

package/lib/defaults.js

@@ -8,8 +8,10 @@ const TWS_BINS = [5, 10, 15, 20, 25, 30]
 
 const DEFAULTS = {
   mode: 'shadow', // 'shadow' publishes …Corrected paths; 'replace' publishes the standard paths
-  variation: 12.8, // degrees, east positive; Sydney
-  sources: { apparentWind: '', stw: '', heading: '', attitude: '', gps: '' }, // '' = any but this plugin
+  // Degrees, east positive. Used only when nothing on the boat publishes
+  // navigation.headingTrue or navigation.magneticVariation. Sydney.
+  variation: 12.8,
+  sources: { apparentWind: '', stw: '', heading: '', attitude: '', gps: '', variation: '' }, // '' = any but this plugin
   corrections: {
     // Knots added to STW: linearity by speed, plus heel correction by |heel|.
     // The 185-cell heel x STW table reduced to two rows (every cell within 0.03 kn).

package/lib/processor.js

@@ -145,7 +145,7 @@ function correctInputs (input, config) {
   return { c, steps }
 }
 
-// input: { awa, aws, stw, heel?, headingMagnetic?, headingTrue?, sog?, cog? }
+// input: { awa, aws, stw, heel?, headingMagnetic?, headingTrue?, variation?, sog?, cog? }
 // returns { out: { stw, leeway, twa, tws, heading?, twd?, current? }, steps }
 function process (input, config) {
   const { c, steps } = correctInputs(input, config)
@@ -171,9 +171,19 @@ function process (input, config) {
 
   const out = { stw: c.stw, leeway: c.leeway, twa: c.twa, tws: c.tws }
 
-  const heading = input.headingTrue != null
-    ? input.headingTrue
-    : input.headingMagnetic != null ? input.headingMagnetic + config.variation : null
+  // True heading: from a true-heading source if there is one, otherwise magnetic
+  // heading plus variation, taken from Signal K (the GPS) when available and from the
+  // configured fallback when not.
+  let heading = null
+  if (input.headingTrue != null) {
+    heading = input.headingTrue
+    out.headingFrom = 'true heading'
+  } else if (input.headingMagnetic != null) {
+    const fromSignalK = input.variation != null
+    out.variation = fromSignalK ? input.variation : config.variation
+    out.headingFrom = fromSignalK ? 'magnetic + variation from Signal K' : 'magnetic + fallback variation'
+    heading = input.headingMagnetic + out.variation
+  }
   if (heading != null) {
     out.heading = norm360(heading)
     out.twd = norm360(heading + c.twa)
@@ -233,6 +243,37 @@ function checkNumber (errors, label, v) {
   if (typeof v !== 'number' || !Number.isFinite(v)) errors.push(`${label}: must be a number`)
 }
 
+// ---- file format versions ----------------------------------------------------
+//
+// Every saved config carries `schema`, the version of its shape. When a release changes
+// that shape, bump CONFIG_SCHEMA and add MIGRATIONS[old], a function that turns a
+// config of schema `old` into schema `old + 1`. Older files are then upgraded step by
+// step as they load and saved back in the current shape; nothing is lost or reset.
+//
+// Schema 1 (0.0.1): mode, variation, sources, corrections { stw, leeway, twa, tws }.
+
+const CONFIG_SCHEMA = 1
+const MIGRATIONS = {
+  // 1: cfg => ({ ...cfg, schema: 2, ... }),
+}
+
+// Bring a raw config up to the current schema. A config without `schema` predates the
+// field and has the schema 1 shape. One from a newer release is refused rather than
+// guessed at, since this version cannot know what its fields mean.
+function migrate (raw, migrations = MIGRATIONS, current = CONFIG_SCHEMA) {
+  let cfg = clone(raw)
+  let v = cfg.schema == null ? 1 : cfg.schema
+  if (!Number.isInteger(v) || v < 1) return { error: `schema: "${cfg.schema}" is not a schema version` }
+  if (v > current) return { error: `schema ${v} was saved by a newer version of this plugin (this one reads up to ${current}); update the plugin` }
+  while (v < current) {
+    if (!migrations[v]) return { error: `schema: no upgrade from ${v} to ${v + 1}` }
+    cfg = migrations[v](cfg)
+    v++
+  }
+  cfg.schema = current
+  return { config: cfg }
+}
+
 // Merge a stored or submitted config over the defaults and check it. Numbers typed
 // into a form arrive as strings, so coerce numeric-looking strings first.
 function normalizeConfig (input) {
@@ -242,10 +283,12 @@ function normalizeConfig (input) {
     if (typeof v === 'string' && v.trim() !== '' && Number.isFinite(Number(v))) return Number(v)
     return v
   }
-  const raw = isPlainObject(input) ? clone(input) : {}
+  const migrated = migrate(isPlainObject(input) ? input : {})
+  if (migrated.error) return { config: { ...clone(DEFAULTS), schema: CONFIG_SCHEMA }, errors: [migrated.error] }
+  const raw = migrated.config
   // Source names are text even when they look numeric.
   const keepText = { sources: raw.sources, mode: raw.mode }
-  const config = merge(DEFAULTS, { ...coerce(raw), ...Object.fromEntries(Object.entries(keepText).filter(([, v]) => v !== undefined)) })
+  const config = merge({ schema: CONFIG_SCHEMA, ...DEFAULTS }, { ...coerce(raw), ...Object.fromEntries(Object.entries(keepText).filter(([, v]) => v !== undefined)) })
   const errors = []
   const k = config.corrections
 
@@ -283,5 +326,6 @@ module.exports = {
   correctInputs,
   normalizeConfig,
   CORRECTIONS,
-  _internals: { interp, blendRows, tableLookup, merge, norm180 }
+  CONFIG_SCHEMA,
+  _internals: { interp, blendRows, tableLookup, merge, norm180, migrate }
 }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "signalk-performance-processor",
-  "version": "0.0.1",
+  "version": "0.0.2",
   "description": "H5000-style calibration for Signal K: corrects boat speed and leeway, applies TWA and TWS correction tables, and publishes true wind and current. Proof of concept.",
   "main": "index.js",
   "scripts": {

package/public/app.js

@@ -19,7 +19,8 @@
     stw: 'Speed through water',
     heading: 'Heading',
     attitude: 'Heel (attitude)',
-    gps: 'SOG and COG'
+    gps: 'SOG and COG',
+    variation: 'Magnetic variation'
   }
 
   const state = {
@@ -160,6 +161,13 @@
     return { twa: Math.atan2(y, x) * 180 / Math.PI, tws: Math.hypot(x, y) }
   }
 
+  // Where true heading came from, so a fallback variation is visible at a glance.
+  function headingNote (o) {
+    if (o.headingFrom === 'true heading') return 'from true heading'
+    const v = o.variation == null ? '' : ` ${Math.abs(o.variation).toFixed(1)}° ${o.variation < 0 ? 'W' : 'E'}`
+    return o.headingFrom === 'magnetic + fallback variation' ? `fallback variation${v}` : `variation${v} from GPS`
+  }
+
   function tile (label, value, unit, raw) {
     return `<div class="tile"><div class="label">${esc(label)}</div>` +
       `<div class="value">${esc(value)}<span class="unit">${esc(unit)}</span></div>` +
@@ -182,7 +190,7 @@
       $('tiles').innerHTML = [
         tile('TWA', fmt(Math.abs(o.twa)), `° ${side(o.twa)}`, `uncorrected ${fmt(Math.abs(u.twa))}°`),
         tile('TWS', fmt(o.tws), 'kn', `uncorrected ${fmt(u.tws)} kn`),
-        tile('TWD', o.twd == null ? '--' : fmt(o.twd, 0), '° T', o.twd == null ? 'no heading' : ''),
+        tile('TWD', o.twd == null ? '--' : fmt(o.twd, 0), '° T', o.twd == null ? 'no heading' : headingNote(o)),
         tile('STW', fmt(o.stw, 2), 'kn', `paddlewheel ${fmt(r.input.stw, 2)} kn`),
         tile('Leeway', fmt(Math.abs(o.leeway)), `° ${side(o.leeway)}`, r.input.heel == null ? 'no heel' : `heel ${fmt(r.input.heel, 0)}°`),
         tile('Current', o.current ? fmt(o.current.drift, 2) : '--', o.current ? `kn · ${fmt(o.current.set, 0)}°` : 'kn', o.current ? '' : 'no SOG/COG or heading')

package/public/index.html

@@ -68,8 +68,9 @@
         </fieldset>
       </div>
       <div class="panel">
-        <h2>Magnetic variation</h2>
-        <label class="field">Degrees east, for true heading
+        <h2>Fallback magnetic variation</h2>
+        <p class="muted sub">Used only when nothing on the boat sends true heading or magnetic variation. Normally your GPS supplies variation and this is never used; the Live page shows which applies.</p>
+        <label class="field">Degrees east (west negative)
           <input id="variation" type="number" step="0.1" data-path="variation">
         </label>
       </div>