@signalk/tracks-plugin 2.0.2 → 3.0.0-beta.0

package/README.md

@@ -1,30 +1,168 @@
-# @signalk/tracks and @signalk/tracks-plugin
-This repo contains two npm modules:
-- `@signalk/tracks` - RxJs based module that accumulates positions into a track, using a configured time resolution and retains a sliding window of points. Contains the code for both a Signal K server plugin that implements the track api and client side "TrackAccumulator" class that manages the track for a single vessel. TrackAccumulator can optionally bootstrap the track data from the server. The result is available as `Observable<LatLng[]>`.
-- `@signalk/tracks-plugin` Convenience module that exposes the plugin part of `@signalk/tracks`.
+# @signalk/tracks-plugin
 
+Signal K server plugin that accumulates vessel positions into tracks and implements the track API.
 
-# Tracks
-Plugin for tracks accumulation and the track API
+Positions are accumulated into a per-vessel sliding window using a configured time resolution. The
+**Where tracks come from after a restart** setting chooses what happens to them:
 
+| Setting   | Behaviour                                                                                                                                                  |
+| --------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `history` | _(default)_ Held in memory, and the own vessel's track is refilled on startup from a history provider such as `signalk-to-influxdb2` or `signalk-questdb`. |
+| `sqlite`  | Recorded to a database file in the plugin's data directory, so every vessel's track survives a restart with no other plugin required.                      |
+| `memory`  | Held in memory only; tracks start empty after a restart.                                                                                                   |
+
+All three work with no other plugin installed — with `history` and no provider present, tracks simply
+build up from live data. `sqlite` also takes a retention setting, in days, for how long to keep
+positions.
+
+This replaces the earlier `bootstrapFromHistory` checkbox. Existing configurations keep working:
+the setting is still read, with `false` mapping to `memory` and anything else to `history`.
+
+The package also exports a client side `TrackAccumulator` class that manages the track for a single
+vessel, exposing the result as `Observable<LatLngTuple[]>`.
+
+## Where a track comes from
+
+With a history provider installed — [signalk-questdb](https://www.npmjs.com/package/signalk-questdb),
+[signalk-to-influxdb2](https://www.npmjs.com/package/signalk-to-influxdb2) — a track is answered from
+both it and the plugin's own store: the provider is the finer record for as long as its retention
+reaches, and the store is what remains of everything older or of any period the provider missed.
+
+Nothing needs configuring for this, and the plugin works exactly as before with no provider
+installed. See [docs/history-and-storage.md](docs/history-and-storage.md) for what that means in
+practice.
+
+## Glitch filtering
+
+A receiver occasionally reports a position far from the vessel — a bad almanac, a multipath
+reflection, a unit resetting. On a live map it flickers past. In a stored track it is permanent:
+one spike stretches the bounding box across an ocean and draws a line over the chart every time
+the track is rendered.
+
+Positions implying a speed above **Discard positions implying a speed above this (knots)** since
+the previous accepted fix are discarded, defaulting to 100 knots. That is well above any real
+vessel, and glitches usually miss it by orders of magnitude rather than by a little. The test is
+speed rather than distance, so a track resuming after a long gap — a passage with the plugin
+stopped, or an AIS target reappearing — is not filtered. Set it to 0 to record everything.
+
+The same check runs over positions loaded from a history provider at startup, since those carry
+the same glitches.
+
+## Pausing while not under way
+
+A boat on a mooring for the winter emits a position every second and travels nowhere. Setting
+**Pause recording while navigation.state is one of** stops those months costing any rows.
+
+Off by default, and it needs `navigation.state` to be set — by
+[signalk-autostate](https://github.com/meri-imperiumi/signalk-autostate) or by hand. A vessel
+that reports no state is always recorded, so nothing changes for an install that has not opted in.
+
+Two limits are deliberate. **`anchored` is offered but rarely wanted**: an anchor alarm watches
+exactly the track a vessel makes while swinging on its rode, so pausing there would break it.
+And **AIS targets are never paused** — their navigational status comes from the transponder and
+is often stale, so a vessel under way still reporting `moored` would otherwise vanish from the
+track.
+
+While paused, the plugin says so in its status on the server dashboard.
+
+## Position sources
+
+`navigation.position` often arrives from several sources at once — an internal GPS, an AIS
+transponder, a plotter echoing its own fix. Signal K decides which one wins through **source
+priority**, and the stream this plugin records from is already filtered by it, so normally only
+the winning source is stored.
+
+When no priority rule matches the path, though, every source comes through. Their fixes are
+metres apart, so the track zigzags between receivers instead of following the boat. If that is
+happening, the plugin says so in its status on the server dashboard, naming the sources it has
+seen. The fix is to set a source priority for `navigation.position` in the server settings.
 
 # Usage:
 
-__Retrieve tracks for individual vessel:__
+**Retrieve track for an individual vessel:**
+
+`/signalk/v1/api/vessels/<vesselId>/track`
+
+_`<vesselId>` may be `self` or a fully qualified context such as `urn:mrn:imo:mmsi:123456789`._
 
-`/signalk/v1/api/tracks/<vesselId>`
+**Retrieve the own vessel's track:**
+
+`/signalk/v1/api/self/track`
+
+---
+
+**Narrow a track to a time window:**
+
+`/signalk/v1/api/self/track?from=2026-08-09T06:00:00Z&to=2026-08-09T12:00:00Z`
+
+`/signalk/v1/api/self/track?duration=6h`
+
+_`from` and `to` are ISO-8601 timestamps; `to` defaults to now. `duration` is a window
+ending now. A window ending at now includes its most recent point; one ending earlier is
+half-open, so consecutive windows tile without returning the shared point twice._
+
+_`timespan` and `timespanOffset` are also accepted for Freeboard-SK compatibility, where
+`timespan=23h&timespanOffset=1` means "23 hours ending an hour ago". They are not part of
+the proposed track API and are expected to be superseded by `from`/`to`._
 
 ---
 
-__Retrieve tracks for all vessels:__
+**Reduce the number of points returned:**
+
+`/signalk/v1/api/self/track?duration=24h&resolution=5m`
+
+_`resolution` is the minimum spacing between returned points. Durations accept a bare
+number of seconds or an `s`/`m`/`h`/`d` suffix. The first and last points are always kept,
+so thinning never shortens the track._
+
+---
+
+**Retrieve the time each position was recorded:**
+
+`/signalk/v1/api/self/track?duration=6h&times`
+
+```json
+{
+  "type": "MultiLineString",
+  "coordinates": [
+    [
+      [24.9, 60.1],
+      [25.0, 60.2]
+    ]
+  ],
+  "times": [["2026-08-14T09:00:00.000Z", "2026-08-14T09:01:00.000Z"]],
+  "context": "vessels.urn:mrn:imo:mmsi:123456789",
+  "isSelf": true
+}
+```
+
+_`times` adds a `times` array positionally aligned with `coordinates`: `times[i][j]` is when
+`coordinates[i][j]` was recorded, as ISO-8601 UTC. It is opt-in because the response grows by
+roughly a third and clients that only draw the geometry have no use for it. Accepts
+`true`/`1`/`yes` and `false`/`0`/`no`; a valueless `?times` reads as true._
+
+_`context` is the fully qualified context the track belongs to, and `isSelf` says whether it is
+the own vessel. Asking for `self` resolves the alias, so the response tells you which vessel
+`self` actually is._
+
+---
+
+**Retrieve tracks for all vessels:**
 
 `/signalk/v1/api/tracks`
 
 _If `maxRadius` is specified only vessels with last track position within this distance are returned._
 
+_Each entry carries `isSelf`, so the own vessel can be told from an AIS target without
+string-matching the context against the server's self identity._
+
+_`?times` works here too, adding a `times` array to every vessel's entry. Note that asking for
+times also segments each track on the gap threshold, so `coordinates` and `times` line up;
+without `times` each vessel keeps its single unsegmented line._
+
 ---
 
-__Retrieve tracks for all vessels within a given radius (in meters) from your vessel position:__
+**Retrieve tracks for all vessels within a given radius (in meters) from your vessel position:**
 
 `/signalk/v1/api/tracks?radius=50000`
 
@@ -32,14 +170,43 @@ _Note: This value overrides the `maxRadius` value specified in plugin configurat
 
 ---
 
-__Retrieve tracks for all vessels within a bounded area:__
+**Retrieve tracks for all vessels within a bounded area:**
 
 `/signalk/v1/api/tracks?bbox=130,-35,139,-33`
 
-_Bounded area is defined as `lon1, lat1, lon2, lat2`_
+_Bounded area is defined as `west, south, east, north` — GeoJSON coordinate order, the
+same as the coordinates this endpoint returns, the Resources API, and the v2 Track API._
 
-_`lon1, lat1` = lower left corner of bounded area_
+_A box crossing the antimeridian is expressed with `west` greater than `east`, for
+example `bbox=175,-10,-175,10`._
 
-_`lon2, lat2` = upper right corner of bounded area_
+> **Changed:** this parameter was `lat1, lon1, lat2, lon2` — latitude first — up to and
+> including 2.0.2. A box in the old order is still four valid numbers, so it will not be
+> rejected; it will simply describe a different area. Freeboard-SK is unaffected, as it
+> filters by `radius` rather than `bbox`.
 
 ---
+
+# Development
+
+```bash
+npm ci
+npm run build      # vite library build -> dist/
+npm test           # vitest
+npm run test:e2e   # against a real signalk-server; see below
+npm run typecheck  # tsc --noEmit
+npm run lint       # eslint
+npm run format     # prettier --write
+```
+
+`npm run test:e2e` packs the plugin, installs it into a throwaway config directory, boots a real
+Signal K server against it and feeds positions as deltas — so it covers plugin loading, route
+mounting and the delta path, none of which the unit suite can. It needs a built server checkout;
+set `SIGNALK_SERVER_DIR` if yours is not at `~/dev/xxx_signalk-server`. A second tier installs a
+real history provider (signalk-questdb) into that server and exercises the startup bootstrap
+through it; it skips itself if no QuestDB is reachable at `QUESTDB_URL`. Neither tier runs in CI.
+
+The package is ESM only and targets Node >= 22.5.0, the release that added `node:sqlite`. ESM alone
+would only need 20.19, the first release in which the Signal K server's `require()`-based plugin
+loader can load an ES module, but the sqlite track source raises the floor. The server itself
+requires Node >= 22, so this rules out nothing that could have run the plugin anyway.

package/dist/bootstrap.test.d.ts

@@ -0,0 +1 @@
+export {};

package/dist/e2e.test-utils.d.ts

@@ -0,0 +1,49 @@
+/** QuestDB endpoint used by the history-provider tier. */
+export declare const QUESTDB_URL: string;
+export interface E2EServer {
+    url: string;
+    configDir: string;
+    /** Send a position delta as a provider would, over the WebSocket API. */
+    feed: (context: string, position: [number, number], timestamp?: number, source?: string) => Promise<void>;
+    /** GET a path under /signalk/v1/api and parse the JSON. */
+    api: (path: string) => Promise<unknown>;
+    /**
+     * GET a path under /signalk/v2/api, keeping the status.
+     *
+     * The status is what distinguishes "no provider registered" (501) from an
+     * answered query, which is the thing a provider registration test is about.
+     */
+    apiV2: (path: string) => Promise<{
+        status: number;
+        body: unknown;
+    }>;
+    /** The server's own vessel context, as it resolved it. */
+    selfContext: string;
+    stop: () => void;
+}
+export interface E2EOptions {
+    /** Plugin configuration written to plugin-config-data/tracks.json. */
+    config?: Record<string, unknown>;
+    /** Seconds to wait for the server to answer before giving up. */
+    timeoutSeconds?: number;
+    /**
+     * Other plugins to install and enable alongside this one, as
+     * `{ 'npm-package-name': pluginConfiguration }`.
+     *
+     * Used to stand up a real history provider so the bootstrap can be exercised
+     * through `getHistoryApi()` — the interface this plugin actually depends on —
+     * rather than against a provider's private storage.
+     */
+    plugins?: Record<string, Record<string, unknown>>;
+    /** Port to boot on, when a test needs its own server. */
+    port?: number;
+}
+export declare function startServer(options?: E2EOptions): Promise<E2EServer>;
+/**
+ * Whether a QuestDB is listening, so the provider tier can skip cleanly.
+ *
+ * A liveness probe only. What the provider stores, and how, is its own
+ * business: this plugin reaches it through `getHistoryApi()` and must keep
+ * working across any change to that.
+ */
+export declare function questdbAvailable(): Promise<boolean>;

package/dist/glitchFilter.d.ts

@@ -0,0 +1,51 @@
+import { Context, LatLngTuple } from './types.js';
+/**
+ * Drops positions that imply an impossible speed since the last accepted one.
+ *
+ * A GPS occasionally emits a fix hundreds of miles away — a bad almanac, a
+ * multipath reflection, a receiver resetting. On a live map it flickers past.
+ * In a stored track it is permanent: one spike stretches the bounding box
+ * across an ocean, inflates the distance travelled, and drags the line across
+ * the chart every time the track is drawn.
+ *
+ * Speed, not distance, is the test. A vessel can legitimately jump a long way
+ * between two fixes if the gap between them is long enough — after a passage
+ * with the plugin stopped, or when a slow-reporting AIS target reappears. What
+ * cannot happen is covering that distance faster than a vessel moves.
+ *
+ * Requested in https://github.com/SignalK/tracks/issues/48
+ */
+/**
+ * Default ceiling in knots.
+ *
+ * Above any displacement vessel and most planing craft, and well above the
+ * ~50kn an AIS Class A target reports at the top of its scale, so a real fix is
+ * never dropped. Glitches are typically wrong by whole degrees of latitude, so
+ * they miss this by orders of magnitude rather than by a little.
+ */
+export declare const DEFAULT_MAX_SPEED_KNOTS = 100;
+export interface GlitchFilterConfig {
+    /** Ceiling in knots. 0 or below disables the filter. */
+    maxSpeedKnots: number;
+}
+export declare class GlitchFilter {
+    private readonly last;
+    private readonly maxSpeedKnots;
+    /** Rejections per context, for the plugin status line. */
+    private readonly rejected;
+    constructor(config: GlitchFilterConfig);
+    /**
+     * Whether to keep this position.
+     *
+     * Accepting updates the reference point, so a sustained move — a genuinely
+     * fast vessel, or a track resuming after a gap — is not fought after the
+     * first fix.
+     */
+    accept(context: Context, position: LatLngTuple, timestamp: number): boolean;
+    /** How many positions have been rejected for a context. */
+    rejectedCount(context: Context): number;
+    /** Total rejections across every context. */
+    totalRejected(): number;
+    /** Forget every reference point; used when the plugin restarts. */
+    clear(): void;
+}

package/dist/glitchFilter.test.d.ts

@@ -0,0 +1 @@
+export {};

package/dist/harness.test-utils.d.ts

@@ -0,0 +1,59 @@
+import { Express } from 'express';
+import { TrackApi } from './trackApi.js';
+import { LatLngTuple } from './types.js';
+/**
+ * Test harness that stands the plugin up behind a real express app, mounted the
+ * way the Signal K server mounts it (`app.use('/signalk/v1/api', …)`), so route
+ * patterns and parameter parsing are exercised rather than mocked.
+ *
+ * Note the server runs express 4; `express` is pinned to ^4 in devDependencies
+ * so the wildcard route syntax under test matches production.
+ */
+export declare const SELF_CONTEXT = "vessels.urn:mrn:imo:mmsi:123456789";
+export declare const OTHER_CONTEXT = "vessels.urn:mrn:imo:mmsi:987654321";
+export interface TestHarness {
+    app: Express;
+    /**
+     * Feed a position delta as the streambundle would. `timestamp` dates the
+     * point, so time-window queries can be tested without faking the clock.
+     *
+     * Note `throttleTime` thins on the leading edge against the wall clock, so a
+     * synchronous burst yields a single point regardless of the timestamps given.
+     * Use `seedTrack` to install a back-dated track instead.
+     */
+    emit: (context: string, position: LatLngTuple, timestamp?: number, source?: string) => void;
+    /** Status messages the plugin has pushed to the server dashboard. */
+    statuses: string[];
+    /**
+     * Install a track with explicit timestamps, bypassing the throttled bus. This
+     * is the entry point the History API bootstrap uses.
+     */
+    seedTrack: (context: string, positions: LatLngTuple[], timestamps: number[]) => void;
+    /** Self position as reported by `getSelfPath`, used for radius filtering. */
+    setSelfPosition: (position: LatLngTuple | undefined) => void;
+    /** The own vessel's navigation.state, as reported by `getSelfPath`. */
+    setSelfState: (state: string | undefined) => void;
+    stop: () => void;
+    errors: unknown[][];
+    /**
+     * The v2 Track API provider the plugin registered, or undefined when the
+     * server offered no `registerTrackApiProvider`.
+     */
+    trackProvider: () => TrackApi | undefined;
+    /** How many times the plugin registered a provider. */
+    registrations: () => number;
+}
+export interface HarnessOptions {
+    /** Plugin config overrides. Bootstrap is disabled by default so tests stay hermetic. */
+    config?: Record<string, unknown>;
+    selfContext?: string;
+    selfPosition?: LatLngTuple;
+    /** Initial navigation.state for the own vessel. */
+    selfState?: string;
+    /**
+     * Stand in for a server without the v2 Track API, to check the plugin still
+     * starts when `registerTrackApiProvider` is absent.
+     */
+    withoutTrackApi?: boolean;
+}
+export declare function createHarness(options?: HarnessOptions): TestHarness;

package/dist/historyReconcile.test.d.ts

@@ -0,0 +1 @@
+export {};

package/dist/index.d.ts

@@ -0,0 +1,135 @@
+import { Temporal } from '@js-temporal/polyfill';
+import { Router } from 'express';
+import { TrackApi } from './trackApi.js';
+import { TrackStore } from './store.js';
+import { Context, Debug, Position } from './types.js';
+export interface ContextPosition {
+    context: Context;
+    value: Position;
+    /** ISO-8601 time the value was recorded, as carried by the Signal K delta. */
+    timestamp?: string;
+    /**
+     * Which source produced the value.
+     *
+     * The bus is priority-filtered, so normally this is the one source the
+     * server selected. Seeing several for one context means no priority rule is
+     * matching the path — which is what `SourceWatch` reports on.
+     */
+    $source?: string;
+}
+interface HistoryValuesQuery {
+    context: string;
+    from: Temporal.Instant;
+    to: Temporal.Instant;
+    pathSpecs: {
+        path: string;
+        aggregate: string;
+    }[];
+    resolution: number;
+}
+interface HistoryApi {
+    getValues(query: HistoryValuesQuery): Promise<HistoryValuesResponse>;
+}
+interface HistoryValuesResponse {
+    context: string;
+    range: {
+        from: string;
+        to: string;
+    };
+    values: unknown[];
+    /** Each element: [timestamp_string, [lon, lat]] */
+    data: unknown[];
+}
+interface App {
+    debug: Debug;
+    error: (...args: unknown[]) => void;
+    streambundle: {
+        getBus: (path: string) => {
+            onValue: (cb: (x: ContextPosition) => void) => () => void;
+        };
+    };
+    getSelfPath: (path: string) => unknown;
+    selfContext: string;
+    /** Shown against the plugin in the server's dashboard. Absent on older servers. */
+    setPluginStatus?: (msg: string) => void;
+    /** Plugin-private directory for persistent data; absent on older servers. */
+    getDataDirPath?: () => string;
+    /** Resolves the named provider, or the configured default when omitted. */
+    getHistoryApi?: (providerId?: string) => Promise<HistoryApi>;
+    /**
+     * Offer this plugin's tracks to the v2 Track API. Absent on servers older
+     * than SignalK/signalk-server#2995.
+     *
+     * The server unregisters the provider itself when the plugin stops, so
+     * `stop()` here must not do it a second time.
+     */
+    registerTrackApiProvider?: (provider: TrackApi) => void;
+    config?: {
+        settings?: {
+            historyApi?: {
+                defaultProvider?: string;
+            };
+        };
+    };
+}
+interface Plugin {
+    start: (c: TracksPluginConfig) => void;
+    stop: () => void;
+    signalKApiRoutes: (r: Router) => Router;
+    id: string;
+    name: string;
+    description: string;
+    schema: Record<string, unknown>;
+    /**
+     * The live accumulator, or undefined before `start()`.
+     *
+     * Exposed so a track can be installed or inspected without going through the
+     * position bus, which throttles against the wall clock. The Signal K server
+     * does not use this.
+     */
+    getTracks: () => TrackStore | undefined;
+}
+/**
+ * Where tracks come from when the server starts.
+ *
+ * - `memory`: start empty, accumulate from the position bus only.
+ * - `history`: refill from a History provider at startup, then accumulate.
+ * - `sqlite`: persist to disk, so a restart resumes from what was recorded.
+ *
+ * One setting rather than a boolean per source: they are alternatives, and as
+ * independent booleans a user could enable two and have the same positions
+ * loaded twice.
+ */
+export type TrackSource = 'memory' | 'history' | 'sqlite';
+interface TracksPluginConfig {
+    resolution?: number;
+    pointsToKeep?: number;
+    maxAge?: number;
+    maxRadius?: number;
+    source?: TrackSource;
+    /** @deprecated superseded by `source`; still honoured for existing configs. */
+    bootstrapFromHistory?: boolean;
+    /** Days of history to keep when `source` is `sqlite`. 0 keeps everything. */
+    retentionDays?: number;
+    /** Minutes without a fix that start a new track segment. 0 disables. */
+    segmentGapMinutes?: number;
+    /** Speed above which a position is treated as a glitch. 0 disables. */
+    maxSpeedKnots?: number;
+    /** navigation.state values that pause recording of the own vessel. */
+    pauseWhenState?: string[];
+}
+/**
+ * Resolve the effective source, honouring the setting this replaced.
+ *
+ * `bootstrapFromHistory` shipped as a boolean defaulting to true, so an
+ * installed plugin has it saved in its config either way. Reading `source`
+ * first and only then falling back keeps those installs working: an explicit
+ * `false` still means "do not touch the History API", and everything else
+ * lands on the old default of `history`.
+ */
+export declare const resolveSource: (config: TracksPluginConfig) => TrackSource;
+export default function ThePlugin(app: App): Plugin;
+export { Tracks, TrackAccumulator } from './tracks.js';
+export type { TracksConfig } from './tracks.js';
+export type { TrackStore } from './store.js';
+export type * from './types.js';