@meri-imperiumi/signalk-dead-reckoning 0.7.0 → 0.8.0

package/CHANGELOG.md

@@ -5,6 +5,100 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [Unreleased]
+
+## [0.8.0] - 2026-09-09
+
+### Fixed
+- **Celestial sights are no longer silently forced through the noon
+  (meridian-altitude) reducer** (sea trial 2026-08-31, Aitutaki→Niue:
+  both Sun sights reduced to −69.3°/−31.9°). The sight panel's
+  `readForm()` read checkboxes via `el.value` — always the string
+  `"on"` regardless of checked state — so every sight was POSTed with
+  `noon: true`. Checkboxes are now read via `el.checked` (both panels).
+- **The webapp no longer overwrites its own-vessel GPS position with
+  the DR shadow boat's** (sea trial: the map boat flipped ~50 NM
+  between GPS and the ghost, and fix #9 recorded the ghost position as
+  a "GNSS" fix 87.5 km from the boat). Deltas whose context is the
+  shadow vessel now route to neither the AIS target store nor the
+  own-vessel view-model.
+
+### Changed
+- **The uncertainty cone now grows with current-knowledge**, not just
+  distance run (sea trial: 88 km of DR-vs-GPS divergence while the
+  polygon "expected" ~2 NM). Radius combines the distance-run error
+  and a per-tier current residual (manual 0.25 kn, weather/pilot
+  0.3 kn, zero-vector 1.0 kn) root-sum-square; the fallback margin
+  rose from 1° to 4° per NM run (measured open-loop rates were
+  7–12× the old value); the empirical rate is now a median with a
+  2 kn per-row cap (the spec's original intent — a garbage 3058 NM
+  correction previously poisoned an EWMA at "145 kn"); the floor rose
+  to 0.05 NM and the origin's own error radius seeds it (a celestial
+  fix is realistically ~5 nm — the cone never claims GPS confidence
+  below that).
+- **The divergence advisory no longer flaps after every fix or print
+  "0.27 nm exceeds expected 0.27 nm"**: the exceedance deadband is
+  instrument-scale (0.005 NM) instead of float epsilon, and the alert
+  message states the exceedance margin.
+- **"Tack/gybe in progress" no longer latches open for the whole
+  passage** in ss3–4 seaway: rate-of-turn is measured over a 6 s
+  rolling window (a single second of wave yaw can't open it), the
+  re-stabilization tolerances scale with sea state, and a window still
+  open after 5 minutes force-closes without classifying a maneuver.
+  The pre-maneuver AWA for tack/gybe classification is now taken from
+  the base of the ROT window (the previous tick's AWA has already
+  flipped with the bow by the time the window opens).
+
+### Added
+- **Passage replay backtest tool** (`tools/replay-passage.js`, minimal
+  SPEC §10.2 scope): replays a historical passage from the Signal K
+  History API through the real DR engine and reports how the shadow
+  boat diverged from GPS. Fetches a range in chunks at native 10s
+  resolution, forward-fills sparse sensors (unwrapping the signed AWA
+  across ±π through gybes), queries `navigation.attitude.roll`
+  (radians) directly — the attitude object itself is not recorded, only
+  its component paths — so heel reaches the matrix bins, seeds the DR
+  origin at the first GPS fix and never re-anchors it. Four variants run side by side: cold (no
+  training, tier-5 zero current), learning (Training Mode on, mirroring
+  a first live passage), plus the same two with a SCUD satellite-derived
+  current source — daily 0.25° fields from the PacIOOS ERDDAP,
+  nearest-cell in space, time-interpolated over source holes (and held
+  at the span ends), resolved at the shadow boat's own position. Also
+  derives the implied current (ground vector minus water-track vector)
+  as a diagnostic of what DR is missing. Outputs an hourly divergence
+  table, an implied-vs-SCUD current comparison, a GeoJSON track file,
+  full-resolution samples, and a self-contained SVG HTML report. A
+  `--stw-scale` option multiplies STW end-to-end (engine, training,
+  matrix lookup) to test paddlewheel-calibration hypotheses against a
+  passage.
+  Smoketests cover URL building, row parsing/filling, the current-grid
+  interpolation, and three synthetic passages (no-drift, known-current,
+  leeway absorbed by training).
+- **Observation submission is speed-plausibility-gated**: a sight,
+  bearing or vertical-angle observation whose reduction implies the
+  vessel traveled faster than 50 kn since the last fix (e.g. the
+  trial's 3058 NM Antarctica sight ≈ 140 kn) is rejected with an
+  explanatory message in the form, so the user can fix the entry.
+  Displacements within realistic observation quality (~5 NM) always
+  pass.
+- **Noon sights carry meridian and sanity guards**: a sight whose Sun
+  is more than 20° from the meridian (LHA), or whose computed latitude
+  lands more than 12° from the assumed/DR position, is refused rather
+  than reduced into garbage.
+- **Fix confirmation carries a gross-displacement guard**: confirming
+  a fix more than 100 NM (configurable, `fixes.maxDisplacementNm`)
+  from the current DR origin is rejected with 422 unless the request
+  carries `force: true`; `/fix/resolve` previews the candidate's
+  displacement and flags gross candidates.
+- **Raw sight inputs are persisted** (`raw_hs_deg`, index correction,
+  eye height, limb, computed Ho/Hc on lines_of_position; angle/height
+  on circular_position_lines) so reductions can be re-run and
+  backtested — reconstructing the trial's sights from the stored
+  noon results required algebraic archaeology.
+- **`elapsedSinceOriginS` and the origin error radius survive plugin
+  restarts** (persisted alongside `last_known_good_fix`), so "since
+  last fix" and the cone no longer reset mid-excursion on a restart.
+
 ## [0.7.0] - 2026-08-29
 
 ### Added
@@ -137,6 +231,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   anchors: a from-scratch Meeus reduction for the Sun (≤0.7′, the
   anchor's own accuracy class), the AA low-precision series for the
   Moon (≤20′), and paper-almanac star values.
+
 - **The webapp's "Ghost Track" heading above the map is gone.** It
   wasted vertical space the map could use — the map card now opens with
   no chrome above it, so the chart starts higher on the page. The

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@meri-imperiumi/signalk-dead-reckoning",
-  "version": "0.7.0",
+  "version": "0.8.0",
   "description": "Offline-first dead reckoning and sensor fusion engine for Signal K",
   "main": "plugin/index.js",
   "scripts": {

package/plugin/celestial.js

@@ -57,6 +57,22 @@ const MOON_RADIUS_KM = 1737.4;
 /** Earth equatorial radius, km. */
 const EARTH_EQ_RADIUS_KM = 6378.14;
 
+/**
+/**
+ * Maximum distance of the Sun's LHA from the meridian (deg) for a
+ * sight to be accepted as a noon sight (sea trial 2026-08-31 guard).
+ * ~20° ≈ 80 minutes from transit — generous versus the ~30 min a
+ * navigator would actually shoot around LAN, but far below the ~100°
+ * the trial's 07:05-local sights carried.
+ */
+const NOON_MAX_LHA_DEG = 20;
+
+/**
+ * Maximum computed-vs-assumed latitude discrepancy (deg) accepted from
+ * a noon sight before it is refused as implausible (~720 NM).
+ */
+const NOON_MAX_LAT_DELTA_DEG = 12;
+
 /**
  * Reduces an ecliptic longitude/latitude to right ascension (degrees),
  * for a given obliquity (degrees).
@@ -437,6 +453,18 @@ function reduceNoonSight(input) {
   const gp = sunGeographicPosition(epochMs);
   const dec = gp.declination_deg;
 
+  // 1b. Meridian-transit guard (sea trial 2026-08-31): a sight taken
+  // hours from local noon reduces to garbage latitudes (both trial
+  // sights were morning sights that landed at −69°/−32°). LHA is 0
+  // (or 360) exactly on the meridian; refuse sights far from transit.
+  const lhaDeg = normalizeDeg360(gp.gha_deg + assumed.longitude);
+  const fromMeridianDeg = Math.min(lhaDeg, 360 - lhaDeg);
+  if (fromMeridianDeg > NOON_MAX_LHA_DEG) {
+    throw new Error(
+      `Sun is ${fromMeridianDeg.toFixed(0)}° from the meridian (LHA ${lhaDeg.toFixed(1)}°) at the sight time — not a noon sight. Uncheck "Noon sight" for an intercept-method reduction, or verify the sight time`,
+    );
+  }
+
   // 2. Ho from Hs (same corrections as a normal sight; limb sights apply).
   const sd = 0.2666;
   const semiDiameterDeg =
@@ -462,6 +490,17 @@ function reduceNoonSight(input) {
   const sunSouth = dec < assumed.latitude;
   const latitude = sunSouth ? dec + z : dec - z;
 
+  // 3b. Computed-vs-assumed sanity (sea trial 2026-08-31): even a sight
+  // near transit can be mis-entered (wrong limb, IC, time). A computed
+  // latitude a whole ocean away from the assumed/DR position is far
+  // more likely a bad sight than a 700 NM DR error — refuse it.
+  const latDeltaDeg = Math.abs(latitude - assumed.latitude);
+  if (latDeltaDeg > NOON_MAX_LAT_DELTA_DEG) {
+    throw new Error(
+      `Noon sight latitude ${latitude.toFixed(1)}° is ${latDeltaDeg.toFixed(1)}° from the assumed ${assumed.latitude.toFixed(1)}° — check the sight altitude, limb and index correction`,
+    );
+  }
+
   // Azimuth: 180° (due south) when the Sun is south of the observer,
   // 0° (due north) when it's north. Either yields an east-west LOP.
   const azimuth = sunSouth ? 180 : 0;
@@ -484,6 +523,8 @@ function reduceNoonSight(input) {
 }
 
 module.exports = {
+  NOON_MAX_LHA_DEG,
+  NOON_MAX_LAT_DELTA_DEG,
   raFromEcliptic,
   decFromEcliptic,
   sunGeographicPosition,

package/plugin/db.js

@@ -88,6 +88,12 @@ const SCHEMA_DDL = [
     body_or_object TEXT,
     confirmed_by TEXT,
     used_in_fix_id INTEGER,
+    raw_hs_deg REAL,
+    index_correction_deg REAL,
+    eye_height_m REAL,
+    limb TEXT,
+    ho_deg REAL,
+    hc_deg REAL,
     FOREIGN KEY (used_in_fix_id) REFERENCES fixes(fix_id)
   )`,
 
@@ -102,6 +108,8 @@ const SCHEMA_DDL = [
     source_object TEXT,
     confirmed_by TEXT,
     used_in_fix_id INTEGER,
+    raw_angle_deg REAL,
+    object_height_m REAL,
     FOREIGN KEY (used_in_fix_id) REFERENCES fixes(fix_id)
   )`,
 
@@ -193,20 +201,88 @@ function openDatabase(dbPath) {
   for (const stmt of SCHEMA_DDL) {
     db.exec(stmt);
   }
-  // v1 → v2: running-fix provenance column on `fixes`. Fresh databases
-  // get the column from the DDL above; existing ones are altered in
-  // place. The stored version is read *after* the DDL (it creates
-  // dr_state_store on a fresh database, where there is nothing to
-  // migrate) and *before* the new version is recorded below.
-  const storedVersion = getState(db, "schema_version");
-  if (storedVersion != null && Number(storedVersion) < 2) {
-    db.exec("ALTER TABLE fixes ADD COLUMN derived_from_fix_id INTEGER");
-  }
+  migrate(db);
   // Record schema version so future migrations can branch on it.
   setState(db, "schema_version", String(SCHEMA_VERSION));
   return db;
 }
 
+/**
+ * Column additions applied via ALTER TABLE, guarded by PRAGMA table_info
+ * so each is idempotent. Run unconditionally on open — the sea-trial
+ * database carried a schema_version written by a build whose numbering
+ * doesn't match this tree's history, so version-gated migrations would
+ * silently skip. Each step is cheap (one PRAGMA read).
+ *
+ * @type {Array<{table: string, column: string, ddl: string}>}
+ */
+const COLUMN_MIGRATIONS = [
+  // Schema v2 (running fix): provenance for single-observation running
+  // fixes advanced from a previous confirmed fix.
+  {
+    table: "fixes",
+    column: "derived_from_fix_id",
+    ddl: "ALTER TABLE fixes ADD COLUMN derived_from_fix_id INTEGER",
+  },
+  // Schema v2 (sea trial 2026-09-06): persist the raw user-entered sight
+  // inputs so reductions can be re-run/backtested without algebraic
+  // archaeology.
+  {
+    table: "lines_of_position",
+    column: "raw_hs_deg",
+    ddl: "ALTER TABLE lines_of_position ADD COLUMN raw_hs_deg REAL",
+  },
+  {
+    table: "lines_of_position",
+    column: "index_correction_deg",
+    ddl: "ALTER TABLE lines_of_position ADD COLUMN index_correction_deg REAL",
+  },
+  {
+    table: "lines_of_position",
+    column: "eye_height_m",
+    ddl: "ALTER TABLE lines_of_position ADD COLUMN eye_height_m REAL",
+  },
+  {
+    table: "lines_of_position",
+    column: "limb",
+    ddl: "ALTER TABLE lines_of_position ADD COLUMN limb TEXT",
+  },
+  {
+    table: "lines_of_position",
+    column: "ho_deg",
+    ddl: "ALTER TABLE lines_of_position ADD COLUMN ho_deg REAL",
+  },
+  {
+    table: "lines_of_position",
+    column: "hc_deg",
+    ddl: "ALTER TABLE lines_of_position ADD COLUMN hc_deg REAL",
+  },
+  {
+    table: "circular_position_lines",
+    column: "raw_angle_deg",
+    ddl: "ALTER TABLE circular_position_lines ADD COLUMN raw_angle_deg REAL",
+  },
+  {
+    table: "circular_position_lines",
+    column: "object_height_m",
+    ddl: "ALTER TABLE circular_position_lines ADD COLUMN object_height_m REAL",
+  },
+];
+
+/**
+ * Applies pending column migrations.
+ *
+ * @param {import("node:sqlite").DatabaseSync} db
+ * @returns {void}
+ */
+function migrate(db) {
+  for (const step of COLUMN_MIGRATIONS) {
+    const columns = db.prepare(`PRAGMA table_info(${step.table})`).all();
+    const exists = columns.some((c) => c.name === step.column);
+    if (!exists) db.exec(step.ddl);
+  }
+}
+
 /**
  * Reads a scalar value from `dr_state_store`.
  *
@@ -348,8 +424,9 @@ function recordLineOfPosition(db, r) {
   const stmt = db.prepare(
     `INSERT INTO lines_of_position (
        timestamp, lop_type, assumed_lat, assumed_lon, azimuth_true,
-       intercept_nm, body_or_object, confirmed_by
-     ) VALUES (?, ?, ?, ?, ?, ?, ?, ?)`,
+       intercept_nm, body_or_object, confirmed_by,
+       raw_hs_deg, index_correction_deg, eye_height_m, limb, ho_deg, hc_deg
+     ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
   );
   const info = stmt.run(
     r.timestamp,
@@ -360,6 +437,14 @@ function recordLineOfPosition(db, r) {
     r.intercept_nm ?? null,
     r.body_or_object ?? null,
     r.confirmed_by ?? null,
+    // Raw user-entered sight inputs (schema v2): so a reduction can be
+    // re-run/backtested later without reconstructing Hs from the result.
+    r.raw_hs_deg ?? null,
+    r.index_correction_deg ?? null,
+    r.eye_height_m ?? null,
+    r.limb ?? null,
+    r.ho_deg ?? null,
+    r.hc_deg ?? null,
   );
   return Number(info.lastInsertRowid);
 }
@@ -386,8 +471,9 @@ function recordCircularPositionLine(db, r) {
   const stmt = db.prepare(
     `INSERT INTO circular_position_lines (
        timestamp, cpl_type, center_lat, center_lon, radius_nm,
-       radius_uncertainty_nm, source_object, confirmed_by
-     ) VALUES (?, ?, ?, ?, ?, ?, ?, ?)`,
+       radius_uncertainty_nm, source_object, confirmed_by,
+       raw_angle_deg, object_height_m
+     ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
   );
   const info = stmt.run(
     r.timestamp,
@@ -398,6 +484,8 @@ function recordCircularPositionLine(db, r) {
     r.radius_uncertainty_nm ?? null,
     r.source_object ?? null,
     r.confirmed_by ?? null,
+    r.raw_angle_deg ?? null,
+    r.object_height_m ?? null,
   );
   return Number(info.lastInsertRowid);
 }

package/plugin/divergence.js

@@ -44,12 +44,14 @@ const DEFAULT_SUSTAIN_S = 30;
 const DEFAULT_CLEAR_S = 30;
 
 /**
- * Deadband (nm) on the exceedance comparison. At 2 mm this is far below
- * any instrument resolution; it exists so that a post-snap divergence of
- * floating-point epsilon (~1e-9 nm from repeated destinationPoint calls)
- * against an exactly-zero radius does not count as exceeding.
+ * Deadband (nm) on the exceedance comparison — instrument resolution,
+ * not floating-point epsilon. At 2 mm (the previous value) a divergence
+ * a hair over the threshold prints as "0.27 nm exceeds expected 0.27 nm"
+ * (sea trial 2026-08-30…09-05), and a zero-radius fresh fix lets GPS
+ * noise raise the advisory as "0.00 nm exceeds expected 0.00 nm". At
+ * ~9 m real disagreement is required before it counts as exceeding.
  */
-const EPS_NM = 1e-6;
+const EPS_NM = 0.005;
 
 /**
  * Creates a fresh monitor state.

package/plugin/engine.js

@@ -51,6 +51,10 @@ class DeadReckoningEngine {
    * @param {{latitude: number, longitude: number}|null} [opts.origin]
    * @param {number} [opts.logNm] - cumulative water-track log (nm)
    * @param {number} [opts.tripLogNm]
+   * @param {number} [opts.originErrorNm] - error radius of the fix that
+   *   seeded the origin (nm); floors the uncertainty cone until distance
+   *   run outgrows it (a celestial fix is realistically ~5 nm accurate —
+   *   the cone must not collapse to GPS-level confidence below that)
    */
   constructor(opts = {}) {
     /** @type {{latitude: number, longitude: number}|null} */
@@ -66,6 +70,8 @@ class DeadReckoningEngine {
      * uncertainty-polygon growth axis (DR error compounds with distance
      * more honestly than with time). */
     this.logNmSinceOrigin = 0;
+    /** @type {number} error radius of the origin-seeding fix (nm) */
+    this.originErrorNm = opts.originErrorNm ?? 0;
     /** @type {string} active calculation method (SPEC §3.1) */
     this.method = "inertial-paddlewheel";
     /** @type {boolean} whether DR is authoritative for navigation.position */
@@ -75,15 +81,18 @@ class DeadReckoningEngine {
   /**
    * Snaps the DR origin to a confirmed fix, recording the elapsed time so
    * that deviation-rate can be computed later (SPEC §4.5, §9.3). Does not
-   * touch the running logs.
+   * touch the running logs. `originErrorNm` seeds the uncertainty cone's
+   * floor with the fix's own error radius.
    *
    * @param {{latitude: number, longitude: number}} fix
+   * @param {number} [originErrorNm] - the fix's error radius (nm)
    * @returns {void}
    */
-  snapToFix(fix) {
+  snapToFix(fix, originErrorNm = 0) {
     this.origin = { latitude: fix.latitude, longitude: fix.longitude };
     this.elapsedSinceOriginS = 0;
     this.logNmSinceOrigin = 0;
+    this.originErrorNm = Math.max(0, originErrorNm);
   }
 
   /**

package/plugin/fix-pipeline.js

@@ -453,10 +453,10 @@ function confirmFix(db, candidate, engine, helpers, opts = {}) {
   }
 
   if (resets && engine) {
-    engine.snapToFix({
-      latitude: candidate.latitude,
-      longitude: candidate.longitude,
-    });
+    engine.snapToFix(
+      { latitude: candidate.latitude, longitude: candidate.longitude },
+      opts.estimatedErrorRadius ?? defaultOriginErrorNm(candidate.source_type),
+    );
   }
 
   return {
@@ -467,9 +467,81 @@ function confirmFix(db, candidate, engine, helpers, opts = {}) {
   };
 }
 
+/**
+ * Default origin error radius (nm) by fix source when the client doesn't
+ * supply one. GNSS positions are metre-scale; every human observation
+ * (celestial sight, bearing, vertical angle, manual entry) is realistically
+ * ~5 nm in seagoing conditions — the uncertainty cone must not collapse to
+ * GPS-level confidence after a sight fix (sea-trial discussion 2026-09-06:
+ * a celestial sight is rarely more accurate than ~5 nm).
+ *
+ * @param {string} sourceType
+ * @returns {number}
+ */
+function defaultOriginErrorNm(sourceType) {
+  return sourceType === "gps" ? 0.05 : 5;
+}
+
+/**
+ * Maximum implied vessel speed (kn) for an observation to be physically
+ * plausible: how fast the boat would have had to travel from the last
+ * origin-reset fix to be where the reduced observation puts it (sea
+ * trial 2026-08-31: the Antarctica sight implied ~140 kn).
+ */
+const MAX_IMPLIED_SPEED_KN = 50;
+
+/**
+ * Displacement (nm) below which the gate never rejects: a real-world
+ * celestial sight or bearing is rarely better than ~5 nm accurate, so a
+ * displacement inside that band is normal observation quality and none
+ * of the gate's business (it also keeps right-after-a-fix geometry —
+ * elapsed ≈ 0, small offsets — from implying absurd speeds). Beyond it,
+ * the implied-speed test applies.
+ */
+const MIN_GATE_DISPLACEMENT_NM = 5;
+
+/**
+ * Speed-plausibility gate for observation submission. Given the
+ * displacement a reduced observation implies (perpendicular distance from
+ * the DR origin to the LOP/CPL, or the intercept magnitude for a celestial
+ * sight) and the seconds elapsed from the last origin-reset fix *at the
+ * observation time*, returns whether the implied speed is physically
+ * plausible. Displacements below MIN_GATE_DISPLACEMENT_NM always pass.
+ *
+ * Pure logic — unit-testable without Signal K.
+ *
+ * @param {object} input
+ * @param {number} [input.displacementNm] - implied displacement (nm);
+ *   null/undefined skips the gate (no reduction to judge)
+ * @param {number|null} [input.elapsedS] - seconds since the last
+ *   origin-reset fix at the observation time; null/negative (observation
+ *   predates the origin, e.g. a backfill) skips the gate
+ * @returns {{ok: boolean, skipped: boolean, impliedKn: number|null}}
+ */
+function evaluateObservationPlausibility(input) {
+  const disp = input.displacementNm;
+  if (disp == null || !Number.isFinite(disp) || disp < 0) {
+    return { ok: true, skipped: true, impliedKn: null };
+  }
+  if (disp <= MIN_GATE_DISPLACEMENT_NM) {
+    return { ok: true, skipped: false, impliedKn: null };
+  }
+  const elapsedS = input.elapsedS;
+  if (elapsedS == null || !Number.isFinite(elapsedS) || elapsedS < 0) {
+    return { ok: true, skipped: true, impliedKn: null };
+  }
+  // Zero elapsed with a large displacement implies infinite speed.
+  const impliedKn = elapsedS > 0 ? disp / (elapsedS / 3600) : Infinity;
+  return { ok: impliedKn <= MAX_IMPLIED_SPEED_KN, skipped: false, impliedKn };
+}
+
 module.exports = {
   resolveCandidateFix,
   confirmFix,
   loadObservationsById,
   advanceToLatest,
+  defaultOriginErrorNm,
+  evaluateObservationPlausibility,
+  MAX_IMPLIED_SPEED_KN,
+  MIN_GATE_DISPLACEMENT_NM,
 };