signalk-questdb 0.1.0

package/.prettierignore

@@ -0,0 +1,2 @@
+dist/
+node_modules/

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Dirk Wahrheit
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,140 @@
+# signalk-questdb
+
+QuestDB history provider for Signal K -- a drop-in replacement for signalk-to-influxdb and signalk-to-influxdb2.
+
+Stores all vessel data in QuestDB running as a managed container (via [signalk-container](https://github.com/dirkwa/signalk-container)). Implements both the modern v2 History API and the legacy v1 playback API.
+
+## Features
+
+- **Automatic container management** -- QuestDB runs in Podman/Docker, managed by signalk-container
+- **ILP ingestion** -- writes via InfluxDB Line Protocol over raw TCP (no client library needed)
+- **v2 History API** -- `getValues`, `getPaths`, `getContexts` with all aggregate methods
+- **v1 Legacy API** -- `hasAnyData`, `streamHistory`, `getHistory` for WebSocket playback
+- **Path filtering** -- include/exclude paths with glob patterns
+- **Sampling rates** -- per-path throttling to control write volume
+- **Retention policy** -- automatic partition drop after N days
+- **AIS recording** -- optionally record other vessels
+- **Position tracking** -- separate optimized table for lat/lon
+- **On-disk compression** -- LZ4 (fast) or ZSTD (smaller) via QuestDB WAL segment compression
+- **Parquet export** -- native QuestDB Parquet export with configurable compression
+- **CSV export** -- download historical data via REST endpoint
+- **InfluxDB migration** -- auto-detect InfluxDB 1.x/2.x on localhost or remote URL
+- **One-click updates** -- check for new QuestDB releases and update from the config panel
+- **Config panel** -- status dashboard with row counts, version picker, update check, collapsible compression/migration/export sections
+- **SQL injection protection** -- strict input validation on all query endpoints
+- **Container lifecycle** -- container stops when plugin is disabled, starts on enable
+
+## Config Panel
+
+The plugin embeds a React config panel in the Signal K Admin UI showing:
+
+- **QuestDB Status** -- running/not running indicator, total rows, active paths today
+- **Update check** -- compares running version against latest GitHub release, one-click update
+- **Image Version** -- dropdown with latest, pre-releases, and last 3 stable releases
+- **Connection** -- managed container toggle, host/ports, PostgreSQL port for Grafana
+- **Recording** -- record self, record AIS targets, retention days
+- **Compression** (collapsible) -- LZ4/ZSTD codec selection for on-disk storage
+- **InfluxDB Migration** (collapsible) -- auto-detect with manual URL for remote instances
+- **Data Export** (collapsible) -- date range picker, Parquet/CSV format, download button
+
+## QuestDB Schema
+
+Three tables, all with WAL mode, daily partitioning, and deduplication:
+
+| Table              | Purpose        | Columns                                                          |
+| ------------------ | -------------- | ---------------------------------------------------------------- |
+| `signalk`          | Numeric values | `ts`, `path` (SYMBOL), `context` (SYMBOL), `value` (DOUBLE)      |
+| `signalk_str`      | String values  | `ts`, `path` (SYMBOL), `context` (SYMBOL), `value_str` (VARCHAR) |
+| `signalk_position` | Positions      | `ts`, `context` (SYMBOL), `lat` (DOUBLE), `lon` (DOUBLE)         |
+
+## History API
+
+### v2 (REST -- `/signalk/v2/api/history/`)
+
+Registered via `app.registerHistoryApiProvider()`. Supports all aggregate methods:
+
+| Method    | QuestDB mapping                        |
+| --------- | -------------------------------------- |
+| `average` | `avg(value)`                           |
+| `min`     | `min(value)`                           |
+| `max`     | `max(value)`                           |
+| `first`   | `first(value)`                         |
+| `last`    | `last(value)`                          |
+| `mid`     | `(min + max) / 2`                      |
+| `sma`     | Client-side N-sample moving average    |
+| `ema`     | Client-side exponential moving average |
+
+Query example:
+
+```
+GET /signalk/v2/api/history/values?paths=navigation.speedOverGround&duration=PT1H&resolution=60
+```
+
+### v1 (WebSocket playback)
+
+Registered via `app.registerHistoryProvider()`. Supports playback at configurable speed multipliers using chunked reads from QuestDB.
+
+## REST Endpoints
+
+All mounted at `/plugins/signalk-questdb/api/`:
+
+| Method | Path                                     | Description                                        |
+| ------ | ---------------------------------------- | -------------------------------------------------- |
+| GET    | `/status`                                | QuestDB health, row counts, active paths           |
+| GET    | `/query?sql=...`                         | Read-only SQL proxy (DDL/DML blocked)              |
+| GET    | `/paths`                                 | All recorded paths with row counts and time range  |
+| GET    | `/versions`                              | QuestDB releases from GitHub (for version picker)  |
+| GET    | `/update/check`                          | Compare running version against latest release     |
+| POST   | `/update/apply`                          | Pull latest image, recreate container, reconnect   |
+| GET    | `/migration/detect`                      | Auto-detect InfluxDB (supports `?url=` for remote) |
+| GET    | `/export?from=...&to=...&format=parquet` | Parquet or CSV export                              |
+
+## Configuration
+
+| Setting            | Default     | Description                                                  |
+| ------------------ | ----------- | ------------------------------------------------------------ |
+| QuestDB version    | `latest`    | Docker image tag (dropdown shows stable + pre-releases)      |
+| Managed container  | `true`      | Let signalk-container manage QuestDB, or connect to external |
+| QuestDB host       | `127.0.0.1` | Host (only used when managed=false)                          |
+| HTTP port          | `9000`      | QuestDB REST API port                                        |
+| ILP port           | `9009`      | InfluxDB Line Protocol write port                            |
+| PostgreSQL port    | `8812`      | For Grafana connections                                      |
+| Record own vessel  | `true`      | Record self context                                          |
+| Record AIS targets | `false`     | Record other vessels                                         |
+| Retention (days)   | `0`         | Auto-delete old partitions (0 = keep forever)                |
+| Compression codec  | `lz4`       | On-disk WAL compression: `none`, `lz4`, or `zstd`            |
+| Compression level  | `3`         | ZSTD level 1-22 (only when codec is zstd)                    |
+
+## Data Storage
+
+QuestDB data is stored at `~/.signalk/plugin-config-data/signalk-questdb/` on the host, mounted into the container at `/var/lib/questdb`. Data survives container restarts, image upgrades, and plugin disable/enable cycles.
+
+## Grafana Integration
+
+Connect Grafana to QuestDB via the PostgreSQL data source:
+
+- Host: `localhost:8812`
+- User: `admin`
+- Password: `quest`
+- Database: `qdb`
+
+Example query:
+
+```sql
+SELECT ts AS time, avg(value) AS sog
+FROM signalk
+WHERE path = 'navigation.speedOverGround'
+  AND context = 'self'
+  AND ts BETWEEN $__timeFrom() AND $__timeTo()
+SAMPLE BY $__interval
+```
+
+## Requirements
+
+- Node.js >= 22
+- [signalk-container](https://github.com/dirkwa/signalk-container) plugin (for managed mode)
+- Signal K server
+
+## License
+
+MIT

package/dist/config/schema.d.ts

@@ -0,0 +1,20 @@
+import { Static } from "@sinclair/typebox";
+export declare const ConfigSchema: import("@sinclair/typebox").TObject<{
+    questdbHost: import("@sinclair/typebox").TString;
+    questdbIlpPort: import("@sinclair/typebox").TNumber;
+    questdbHttpPort: import("@sinclair/typebox").TNumber;
+    questdbPgPort: import("@sinclair/typebox").TNumber;
+    questdbVersion: import("@sinclair/typebox").TString;
+    managedContainer: import("@sinclair/typebox").TBoolean;
+    pathFilter: import("@sinclair/typebox").TObject<{
+        mode: import("@sinclair/typebox").TUnion<[import("@sinclair/typebox").TLiteral<"exclude">, import("@sinclair/typebox").TLiteral<"include">]>;
+        paths: import("@sinclair/typebox").TArray<import("@sinclair/typebox").TString>;
+    }>;
+    samplingRates: import("@sinclair/typebox").TRecord<import("@sinclair/typebox").TString, import("@sinclair/typebox").TNumber>;
+    recordSelf: import("@sinclair/typebox").TBoolean;
+    recordOthers: import("@sinclair/typebox").TBoolean;
+    retentionDays: import("@sinclair/typebox").TNumber;
+    compression: import("@sinclair/typebox").TUnion<[import("@sinclair/typebox").TLiteral<"none">, import("@sinclair/typebox").TLiteral<"lz4">, import("@sinclair/typebox").TLiteral<"zstd">]>;
+    compressionLevel: import("@sinclair/typebox").TNumber;
+}>;
+export type Config = Static<typeof ConfigSchema>;

package/dist/config/schema.js

@@ -0,0 +1,70 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.ConfigSchema = void 0;
+const typebox_1 = require("@sinclair/typebox");
+exports.ConfigSchema = typebox_1.Type.Object({
+    questdbHost: typebox_1.Type.String({
+        default: "127.0.0.1",
+        title: "QuestDB host",
+    }),
+    questdbIlpPort: typebox_1.Type.Number({
+        default: 9009,
+        title: "ILP port (writes)",
+    }),
+    questdbHttpPort: typebox_1.Type.Number({
+        default: 9000,
+        title: "HTTP port (queries)",
+    }),
+    questdbPgPort: typebox_1.Type.Number({
+        default: 8812,
+        title: "PostgreSQL wire port",
+    }),
+    questdbVersion: typebox_1.Type.String({
+        default: "9.2.0",
+        title: "QuestDB image version",
+    }),
+    managedContainer: typebox_1.Type.Boolean({
+        default: true,
+        title: "Manage QuestDB container via signalk-container",
+        description: "Disable to connect to an external QuestDB instance",
+    }),
+    pathFilter: typebox_1.Type.Object({
+        mode: typebox_1.Type.Union([typebox_1.Type.Literal("exclude"), typebox_1.Type.Literal("include")], {
+            default: "exclude",
+            title: "Filter mode",
+        }),
+        paths: typebox_1.Type.Array(typebox_1.Type.String(), {
+            default: [],
+            title: "Path patterns (glob supported)",
+            description: 'e.g. "notifications.*", "environment.wind.*"',
+        }),
+    }),
+    samplingRates: typebox_1.Type.Record(typebox_1.Type.String(), typebox_1.Type.Number(), {
+        default: {},
+        title: "Per-path sampling rates (ms)",
+        description: 'Minimum ms between writes per path. e.g. { "environment.wind.*": 1000 }',
+    }),
+    recordSelf: typebox_1.Type.Boolean({
+        default: true,
+        title: "Record own vessel",
+    }),
+    recordOthers: typebox_1.Type.Boolean({
+        default: false,
+        title: "Record AIS targets",
+    }),
+    retentionDays: typebox_1.Type.Number({
+        default: 0,
+        title: "Retention (days, 0 = keep forever)",
+    }),
+    compression: typebox_1.Type.Union([typebox_1.Type.Literal("none"), typebox_1.Type.Literal("lz4"), typebox_1.Type.Literal("zstd")], {
+        default: "lz4",
+        title: "Compression codec",
+        description: "Used for QuestDB on-disk WAL segments and Parquet exports (lz4 = fast, zstd = smaller)",
+    }),
+    compressionLevel: typebox_1.Type.Number({
+        default: 3,
+        title: "Compression level",
+        description: "zstd: 1-22 (default 3), lz4: ignored",
+    }),
+});
+//# sourceMappingURL=schema.js.map

package/dist/config/schema.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"schema.js","sourceRoot":"","sources":["../../src/config/schema.ts"],"names":[],"mappings":";;;AAAA,+CAAiD;AAEpC,QAAA,YAAY,GAAG,cAAI,CAAC,MAAM,CAAC;IACtC,WAAW,EAAE,cAAI,CAAC,MAAM,CAAC;QACvB,OAAO,EAAE,WAAW;QACpB,KAAK,EAAE,cAAc;KACtB,CAAC;IACF,cAAc,EAAE,cAAI,CAAC,MAAM,CAAC;QAC1B,OAAO,EAAE,IAAI;QACb,KAAK,EAAE,mBAAmB;KAC3B,CAAC;IACF,eAAe,EAAE,cAAI,CAAC,MAAM,CAAC;QAC3B,OAAO,EAAE,IAAI;QACb,KAAK,EAAE,qBAAqB;KAC7B,CAAC;IACF,aAAa,EAAE,cAAI,CAAC,MAAM,CAAC;QACzB,OAAO,EAAE,IAAI;QACb,KAAK,EAAE,sBAAsB;KAC9B,CAAC;IACF,cAAc,EAAE,cAAI,CAAC,MAAM,CAAC;QAC1B,OAAO,EAAE,OAAO;QAChB,KAAK,EAAE,uBAAuB;KAC/B,CAAC;IAEF,gBAAgB,EAAE,cAAI,CAAC,OAAO,CAAC;QAC7B,OAAO,EAAE,IAAI;QACb,KAAK,EAAE,gDAAgD;QACvD,WAAW,EAAE,oDAAoD;KAClE,CAAC;IAEF,UAAU,EAAE,cAAI,CAAC,MAAM,CAAC;QACtB,IAAI,EAAE,cAAI,CAAC,KAAK,CAAC,CAAC,cAAI,CAAC,OAAO,CAAC,SAAS,CAAC,EAAE,cAAI,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC,EAAE;YACnE,OAAO,EAAE,SAAS;YAClB,KAAK,EAAE,aAAa;SACrB,CAAC;QACF,KAAK,EAAE,cAAI,CAAC,KAAK,CAAC,cAAI,CAAC,MAAM,EAAE,EAAE;YAC/B,OAAO,EAAE,EAAE;YACX,KAAK,EAAE,gCAAgC;YACvC,WAAW,EAAE,8CAA8C;SAC5D,CAAC;KACH,CAAC;IAEF,aAAa,EAAE,cAAI,CAAC,MAAM,CAAC,cAAI,CAAC,MAAM,EAAE,EAAE,cAAI,CAAC,MAAM,EAAE,EAAE;QACvD,OAAO,EAAE,EAAE;QACX,KAAK,EAAE,8BAA8B;QACrC,WAAW,EACT,yEAAyE;KAC5E,CAAC;IAEF,UAAU,EAAE,cAAI,CAAC,OAAO,CAAC;QACvB,OAAO,EAAE,IAAI;QACb,KAAK,EAAE,mBAAmB;KAC3B,CAAC;IACF,YAAY,EAAE,cAAI,CAAC,OAAO,CAAC;QACzB,OAAO,EAAE,KAAK;QACd,KAAK,EAAE,oBAAoB;KAC5B,CAAC;IAEF,aAAa,EAAE,cAAI,CAAC,MAAM,CAAC;QACzB,OAAO,EAAE,CAAC;QACV,KAAK,EAAE,oCAAoC;KAC5C,CAAC;IAEF,WAAW,EAAE,cAAI,CAAC,KAAK,CACrB,CAAC,cAAI,CAAC,OAAO,CAAC,MAAM,CAAC,EAAE,cAAI,CAAC,OAAO,CAAC,KAAK,CAAC,EAAE,cAAI,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC,EACjE;QACE,OAAO,EAAE,KAAK;QACd,KAAK,EAAE,mBAAmB;QAC1B,WAAW,EACT,wFAAwF;KAC3F,CACF;IACD,gBAAgB,EAAE,cAAI,CAAC,MAAM,CAAC;QAC5B,OAAO,EAAE,CAAC;QACV,KAAK,EAAE,mBAAmB;QAC1B,WAAW,EAAE,sCAAsC;KACpD,CAAC;CACH,CAAC,CAAC"}

package/dist/history-v1.d.ts

@@ -0,0 +1,25 @@
+import { QueryClient } from "./query-client";
+interface HistoryOptions {
+    startTime: Date;
+    playbackRate: number;
+    subscribe?: string;
+}
+interface Delta {
+    context: string;
+    updates: {
+        timestamp: string;
+        values: {
+            path: string;
+            value: unknown;
+        }[];
+    }[];
+}
+export declare function createHistoryProviderV1(queryClient: QueryClient, selfContext: string, debug: (msg: string) => void): {
+    hasAnyData: (options: HistoryOptions, callback: (hasResults: boolean) => void) => void;
+    streamHistory: (spark: {
+        write: (data: unknown) => void;
+        on: (event: string, cb: (...args: unknown[]) => void) => void;
+    }, options: HistoryOptions, onChange: () => void) => () => void;
+    getHistory: (date: Date, path: string, callback: (deltas: Delta[]) => void) => void;
+};
+export {};

package/dist/history-v1.js

@@ -0,0 +1,116 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.createHistoryProviderV1 = createHistoryProviderV1;
+const query_client_1 = require("./query-client");
+function groupRowsIntoDeltas(rows) {
+    const byTimestamp = new Map();
+    for (const row of rows) {
+        const ts = row.ts;
+        const context = row.context || "self";
+        const path = row.path;
+        const value = row.value;
+        if (!byTimestamp.has(ts)) {
+            byTimestamp.set(ts, new Map());
+        }
+        const byContext = byTimestamp.get(ts);
+        if (!byContext.has(context)) {
+            byContext.set(context, []);
+        }
+        byContext.get(context).push({ path, value });
+    }
+    const deltas = [];
+    for (const [ts, byContext] of byTimestamp) {
+        for (const [context, values] of byContext) {
+            deltas.push({
+                context,
+                updates: [{ timestamp: ts, values }],
+            });
+        }
+    }
+    return deltas;
+}
+function createHistoryProviderV1(queryClient, selfContext, debug) {
+    function hasAnyData(options, callback) {
+        const startTime = (0, query_client_1.validateTimestamp)(options.startTime.toISOString());
+        queryClient
+            .exec(`SELECT count() as cnt FROM signalk WHERE ts >= '${startTime}' LIMIT 1`)
+            .then((result) => {
+            const count = result.dataset.length > 0 ? result.dataset[0][0] : 0;
+            callback(count > 0);
+        })
+            .catch(() => {
+            callback(false);
+        });
+    }
+    function streamHistory(spark, options, 
+    // eslint-disable-next-line @typescript-eslint/no-unused-vars
+    onChange) {
+        let stopped = false;
+        const startTime = (0, query_client_1.validateTimestamp)(options.startTime.toISOString());
+        const playbackRate = Math.max(1, options.playbackRate);
+        const CHUNK_SECONDS = 60;
+        let currentTime = new Date(startTime);
+        async function streamChunk() {
+            if (stopped)
+                return;
+            const from = (0, query_client_1.validateTimestamp)(currentTime.toISOString());
+            const chunkEnd = new Date(currentTime.getTime() + CHUNK_SECONDS * 1000);
+            const to = (0, query_client_1.validateTimestamp)(chunkEnd.toISOString());
+            try {
+                const result = await queryClient.exec(`SELECT ts, path, context, value FROM signalk WHERE ts >= '${from}' AND ts < '${to}' ORDER BY ts LIMIT 10000`);
+                if (result.dataset.length === 0) {
+                    currentTime = chunkEnd;
+                    if (!stopped) {
+                        setTimeout(streamChunk, 100);
+                    }
+                    return;
+                }
+                const rows = queryClient.toObjects(result);
+                const deltas = groupRowsIntoDeltas(rows);
+                for (const delta of deltas) {
+                    if (stopped)
+                        return;
+                    const resolvedContext = delta.context === "self" ? selfContext : delta.context;
+                    spark.write({
+                        ...delta,
+                        context: resolvedContext,
+                    });
+                }
+                currentTime = chunkEnd;
+                const wallDelay = (CHUNK_SECONDS * 1000) / playbackRate;
+                if (!stopped) {
+                    setTimeout(streamChunk, wallDelay);
+                }
+            }
+            catch (err) {
+                debug(`streamHistory error: ${err instanceof Error ? err.message : String(err)}`);
+                if (!stopped) {
+                    setTimeout(streamChunk, 1000);
+                }
+            }
+        }
+        streamChunk();
+        spark.on("end", () => {
+            stopped = true;
+        });
+        return () => {
+            stopped = true;
+        };
+    }
+    function getHistory(date, path, callback) {
+        const ts = (0, query_client_1.validateTimestamp)(date.toISOString());
+        queryClient
+            .exec(`SELECT path, value, ts, context FROM signalk WHERE ts <= '${ts}' LATEST ON ts PARTITION BY path`)
+            .then((result) => {
+            const rows = queryClient.toObjects(result);
+            const deltas = groupRowsIntoDeltas(rows);
+            callback(deltas);
+        })
+            .catch((err) => {
+            debug(`getHistory error: ${err instanceof Error ? err.message : String(err)}`);
+            callback([]);
+        });
+    }
+    return { hasAnyData, streamHistory, getHistory };
+}
+//# sourceMappingURL=history-v1.js.map

package/dist/history-v1.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"history-v1.js","sourceRoot":"","sources":["../src/history-v1.ts"],"names":[],"mappings":";;AAmDA,0DA6HC;AAhLD,iDAAgE;AAgBhE,SAAS,mBAAmB,CAAC,IAA+B;IAC1D,MAAM,WAAW,GAAG,IAAI,GAAG,EAGxB,CAAC;IAEJ,KAAK,MAAM,GAAG,IAAI,IAAI,EAAE,CAAC;QACvB,MAAM,EAAE,GAAG,GAAG,CAAC,EAAY,CAAC;QAC5B,MAAM,OAAO,GAAI,GAAG,CAAC,OAAkB,IAAI,MAAM,CAAC;QAClD,MAAM,IAAI,GAAG,GAAG,CAAC,IAAc,CAAC;QAChC,MAAM,KAAK,GAAG,GAAG,CAAC,KAAgB,CAAC;QAEnC,IAAI,CAAC,WAAW,CAAC,GAAG,CAAC,EAAE,CAAC,EAAE,CAAC;YACzB,WAAW,CAAC,GAAG,CAAC,EAAE,EAAE,IAAI,GAAG,EAAE,CAAC,CAAC;QACjC,CAAC;QACD,MAAM,SAAS,GAAG,WAAW,CAAC,GAAG,CAAC,EAAE,CAAE,CAAC;QACvC,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,OAAO,CAAC,EAAE,CAAC;YAC5B,SAAS,CAAC,GAAG,CAAC,OAAO,EAAE,EAAE,CAAC,CAAC;QAC7B,CAAC;QACD,SAAS,CAAC,GAAG,CAAC,OAAO,CAAE,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,KAAK,EAAE,CAAC,CAAC;IAChD,CAAC;IAED,MAAM,MAAM,GAAY,EAAE,CAAC;IAC3B,KAAK,MAAM,CAAC,EAAE,EAAE,SAAS,CAAC,IAAI,WAAW,EAAE,CAAC;QAC1C,KAAK,MAAM,CAAC,OAAO,EAAE,MAAM,CAAC,IAAI,SAAS,EAAE,CAAC;YAC1C,MAAM,CAAC,IAAI,CAAC;gBACV,OAAO;gBACP,OAAO,EAAE,CAAC,EAAE,SAAS,EAAE,EAAE,EAAE,MAAM,EAAE,CAAC;aACrC,CAAC,CAAC;QACL,CAAC;IACH,CAAC;IAED,OAAO,MAAM,CAAC;AAChB,CAAC;AAED,SAAgB,uBAAuB,CACrC,WAAwB,EACxB,WAAmB,EACnB,KAA4B;IAE5B,SAAS,UAAU,CACjB,OAAuB,EACvB,QAAuC;QAEvC,MAAM,SAAS,GAAG,IAAA,gCAAiB,EAAC,OAAO,CAAC,SAAS,CAAC,WAAW,EAAE,CAAC,CAAC;QACrE,WAAW;aACR,IAAI,CACH,mDAAmD,SAAS,WAAW,CACxE;aACA,IAAI,CAAC,CAAC,MAAM,EAAE,EAAE;YACf,MAAM,KAAK,GACT,MAAM,CAAC,OAAO,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAE,MAAM,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,CAAY,CAAC,CAAC,CAAC,CAAC,CAAC;YACnE,QAAQ,CAAC,KAAK,GAAG,CAAC,CAAC,CAAC;QACtB,CAAC,CAAC;aACD,KAAK,CAAC,GAAG,EAAE;YACV,QAAQ,CAAC,KAAK,CAAC,CAAC;QAClB,CAAC,CAAC,CAAC;IACP,CAAC;IAED,SAAS,aAAa,CACpB,KAGC,EACD,OAAuB;IACvB,6DAA6D;IAC7D,QAAoB;QAEpB,IAAI,OAAO,GAAG,KAAK,CAAC;QACpB,MAAM,SAAS,GAAG,IAAA,gCAAiB,EAAC,OAAO,CAAC,SAAS,CAAC,WAAW,EAAE,CAAC,CAAC;QACrE,MAAM,YAAY,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,OAAO,CAAC,YAAY,CAAC,CAAC;QAEvD,MAAM,aAAa,GAAG,EAAE,CAAC;QACzB,IAAI,WAAW,GAAG,IAAI,IAAI,CAAC,SAAS,CAAC,CAAC;QAEtC,KAAK,UAAU,WAAW;YACxB,IAAI,OAAO;gBAAE,OAAO;YAEpB,MAAM,IAAI,GAAG,IAAA,gCAAiB,EAAC,WAAW,CAAC,WAAW,EAAE,CAAC,CAAC;YAC1D,MAAM,QAAQ,GAAG,IAAI,IAAI,CAAC,WAAW,CAAC,OAAO,EAAE,GAAG,aAAa,GAAG,IAAI,CAAC,CAAC;YACxE,MAAM,EAAE,GAAG,IAAA,gCAAiB,EAAC,QAAQ,CAAC,WAAW,EAAE,CAAC,CAAC;YAErD,IAAI,CAAC;gBACH,MAAM,MAAM,GAAG,MAAM,WAAW,CAAC,IAAI,CACnC,6DAA6D,IAAI,eAAe,EAAE,2BAA2B,CAC9G,CAAC;gBAEF,IAAI,MAAM,CAAC,OAAO,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;oBAChC,WAAW,GAAG,QAAQ,CAAC;oBACvB,IAAI,CAAC,OAAO,EAAE,CAAC;wBACb,UAAU,CAAC,WAAW,EAAE,GAAG,CAAC,CAAC;oBAC/B,CAAC;oBACD,OAAO;gBACT,CAAC;gBAED,MAAM,IAAI,GAAG,WAAW,CAAC,SAAS,CAAC,MAAM,CAAC,CAAC;gBAC3C,MAAM,MAAM,GAAG,mBAAmB,CAAC,IAAI,CAAC,CAAC;gBAEzC,KAAK,MAAM,KAAK,IAAI,MAAM,EAAE,CAAC;oBAC3B,IAAI,OAAO;wBAAE,OAAO;oBAEpB,MAAM,eAAe,GACnB,KAAK,CAAC,OAAO,KAAK,MAAM,CAAC,CAAC,CAAC,WAAW,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC;oBACzD,KAAK,CAAC,KAAK,CAAC;wBACV,GAAG,KAAK;wBACR,OAAO,EAAE,eAAe;qBACzB,CAAC,CAAC;gBACL,CAAC;gBAED,WAAW,GAAG,QAAQ,CAAC;gBACvB,MAAM,SAAS,GAAG,CAAC,aAAa,GAAG,IAAI,CAAC,GAAG,YAAY,CAAC;gBACxD,IAAI,CAAC,OAAO,EAAE,CAAC;oBACb,UAAU,CAAC,WAAW,EAAE,SAAS,CAAC,CAAC;gBACrC,CAAC;YACH,CAAC;YAAC,OAAO,GAAG,EAAE,CAAC;gBACb,KAAK,CACH,wBAAwB,GAAG,YAAY,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,GAAG,CAAC,EAAE,CAC3E,CAAC;gBACF,IAAI,CAAC,OAAO,EAAE,CAAC;oBACb,UAAU,CAAC,WAAW,EAAE,IAAI,CAAC,CAAC;gBAChC,CAAC;YACH,CAAC;QACH,CAAC;QAED,WAAW,EAAE,CAAC;QAEd,KAAK,CAAC,EAAE,CAAC,KAAK,EAAE,GAAG,EAAE;YACnB,OAAO,GAAG,IAAI,CAAC;QACjB,CAAC,CAAC,CAAC;QAEH,OAAO,GAAG,EAAE;YACV,OAAO,GAAG,IAAI,CAAC;QACjB,CAAC,CAAC;IACJ,CAAC;IAED,SAAS,UAAU,CACjB,IAAU,EACV,IAAY,EACZ,QAAmC;QAEnC,MAAM,EAAE,GAAG,IAAA,gCAAiB,EAAC,IAAI,CAAC,WAAW,EAAE,CAAC,CAAC;QAEjD,WAAW;aACR,IAAI,CACH,6DAA6D,EAAE,kCAAkC,CAClG;aACA,IAAI,CAAC,CAAC,MAAM,EAAE,EAAE;YACf,MAAM,IAAI,GAAG,WAAW,CAAC,SAAS,CAAC,MAAM,CAAC,CAAC;YAC3C,MAAM,MAAM,GAAG,mBAAmB,CAAC,IAAI,CAAC,CAAC;YACzC,QAAQ,CAAC,MAAM,CAAC,CAAC;QACnB,CAAC,CAAC;aACD,KAAK,CAAC,CAAC,GAAG,EAAE,EAAE;YACb,KAAK,CACH,qBAAqB,GAAG,YAAY,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,GAAG,CAAC,EAAE,CACxE,CAAC;YACF,QAAQ,CAAC,EAAE,CAAC,CAAC;QACf,CAAC,CAAC,CAAC;IACP,CAAC;IAED,OAAO,EAAE,UAAU,EAAE,aAAa,EAAE,UAAU,EAAE,CAAC;AACnD,CAAC"}

package/dist/history-v2.d.ts

@@ -0,0 +1,50 @@
+import { QueryClient } from "./query-client";
+interface PathSpec {
+    path: string;
+    aggregate: string;
+    parameter: string[];
+}
+interface ValuesRequest {
+    from?: {
+        toString(): string;
+        add(d: unknown): unknown;
+        subtract?(d: unknown): unknown;
+    };
+    to?: {
+        toString(): string;
+        subtract?(d: unknown): unknown;
+    };
+    duration?: unknown;
+    context?: string;
+    resolution?: number;
+    pathSpecs: PathSpec[];
+}
+interface ValuesResponse {
+    context: string;
+    range: {
+        from: string;
+        to: string;
+    };
+    values: {
+        path: string;
+        method: string;
+    }[];
+    data: [string, ...unknown[]][];
+}
+type PathsRequest = {
+    from?: {
+        toString(): string;
+        add(d: unknown): unknown;
+    };
+    to?: {
+        toString(): string;
+    };
+    duration?: unknown;
+};
+type ContextsRequest = PathsRequest;
+export declare function createHistoryProviderV2(queryClient: QueryClient): {
+    getValues: (query: ValuesRequest) => Promise<ValuesResponse>;
+    getPaths: (query: PathsRequest) => Promise<string[]>;
+    getContexts: (query: ContextsRequest) => Promise<string[]>;
+};
+export {};

package/dist/history-v2.js

@@ -0,0 +1,192 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.createHistoryProviderV2 = createHistoryProviderV2;
+const query_client_1 = require("./query-client");
+const time_range_1 = require("./time-range");
+function aggregateToSql(method) {
+    switch (method) {
+        case "average":
+            return "avg(value)";
+        case "min":
+            return "min(value)";
+        case "max":
+            return "max(value)";
+        case "first":
+            return "first(value)";
+        case "last":
+            return "last(value)";
+        case "mid":
+            return "(min(value) + max(value)) / 2";
+        default:
+            return "avg(value)";
+    }
+}
+function needsClientSideAggregation(method) {
+    return method === "middle_index" || method === "sma" || method === "ema";
+}
+function computeSMA(values, n) {
+    const result = [];
+    const window = [];
+    for (const v of values) {
+        if (v === null) {
+            result.push(null);
+            continue;
+        }
+        window.push(v);
+        if (window.length > n)
+            window.shift();
+        result.push(window.reduce((a, b) => a + b, 0) / window.length);
+    }
+    return result;
+}
+function computeEMA(values, alpha) {
+    const result = [];
+    let prev = null;
+    for (const v of values) {
+        if (v === null) {
+            result.push(prev);
+            continue;
+        }
+        if (prev === null) {
+            prev = v;
+        }
+        else {
+            prev = alpha * v + (1 - alpha) * prev;
+        }
+        result.push(prev);
+    }
+    return result;
+}
+function buildRangeWhere(range, context) {
+    const from = (0, query_client_1.validateTimestamp)(range.from);
+    const to = (0, query_client_1.validateTimestamp)(range.to);
+    let where = `ts >= '${from}' AND ts <= '${to}'`;
+    if (context) {
+        where += ` AND context = '${(0, query_client_1.validateIdentifier)(context)}'`;
+    }
+    return where;
+}
+function createHistoryProviderV2(queryClient) {
+    async function getValues(query) {
+        const range = (0, time_range_1.resolveTimeRange)(query);
+        const context = query.context ?? "self";
+        const safeContext = (0, query_client_1.validateIdentifier)(context);
+        const valuesList = [];
+        const columnData = new Map();
+        for (const spec of query.pathSpecs) {
+            const safePath = (0, query_client_1.validateIdentifier)(spec.path);
+            valuesList.push({ path: spec.path, method: spec.aggregate });
+            const isPosition = spec.path === "navigation.position";
+            const table = isPosition ? "signalk_position" : "signalk";
+            if (isPosition) {
+                const where = buildRangeWhere(range, safeContext);
+                let sql;
+                if (query.resolution && query.resolution > 0) {
+                    sql = `SELECT ts, first(lat) as lat, first(lon) as lon FROM ${table} WHERE ${where} SAMPLE BY ${Math.floor(query.resolution)}s FILL(NULL) ORDER BY ts`;
+                }
+                else {
+                    sql = `SELECT ts, lat, lon FROM ${table} WHERE ${where} ORDER BY ts LIMIT 10000`;
+                }
+                const result = await queryClient.exec(sql);
+                const rows = result.dataset.map((row) => [
+                    row[0],
+                    row[1] !== null && row[2] !== null
+                        ? { latitude: row[1], longitude: row[2] }
+                        : null,
+                ]);
+                columnData.set(spec.path, rows);
+                continue;
+            }
+            const where = `${buildRangeWhere(range, safeContext)} AND path = '${safePath}'`;
+            if (needsClientSideAggregation(spec.aggregate)) {
+                const sql = `SELECT ts, value FROM ${table} WHERE ${where} ORDER BY ts LIMIT 50000`;
+                const result = await queryClient.exec(sql);
+                const timestamps = result.dataset.map((r) => r[0]);
+                const rawValues = result.dataset.map((r) => r[1]);
+                let computed;
+                if (spec.aggregate === "sma") {
+                    const n = parseInt(spec.parameter[0] ?? "5", 10);
+                    computed = computeSMA(rawValues, n);
+                }
+                else if (spec.aggregate === "ema") {
+                    const alpha = parseFloat(spec.parameter[0] ?? "0.2");
+                    computed = computeEMA(rawValues, alpha);
+                }
+                else {
+                    const mid = Math.floor(rawValues.length / 2);
+                    computed = rawValues.map((_, i) => (i === mid ? rawValues[i] : null));
+                }
+                const rows = timestamps.map((ts, i) => [
+                    ts,
+                    computed[i],
+                ]);
+                columnData.set(spec.path, rows);
+                continue;
+            }
+            const aggExpr = aggregateToSql(spec.aggregate);
+            let sql;
+            if (query.resolution && query.resolution > 0) {
+                sql = `SELECT ts, ${aggExpr} as agg_value FROM ${table} WHERE ${where} SAMPLE BY ${Math.floor(query.resolution)}s FILL(NULL) ORDER BY ts`;
+            }
+            else {
+                sql = `SELECT ts, value FROM ${table} WHERE ${where} ORDER BY ts LIMIT 10000`;
+            }
+            const result = await queryClient.exec(sql);
+            const rows = result.dataset.map((row) => [
+                row[0],
+                row[1],
+            ]);
+            columnData.set(spec.path, rows);
+        }
+        const allTimestamps = new Set();
+        for (const rows of columnData.values()) {
+            for (const [ts] of rows) {
+                allTimestamps.add(ts);
+            }
+        }
+        const sortedTimestamps = Array.from(allTimestamps).sort();
+        const pathOrder = query.pathSpecs.map((s) => s.path);
+        const indexMaps = new Map();
+        for (const [path, rows] of columnData) {
+            const m = new Map();
+            for (const [ts, val] of rows) {
+                m.set(ts, val);
+            }
+            indexMaps.set(path, m);
+        }
+        const data = sortedTimestamps.map((ts) => {
+            const row = [ts];
+            for (const path of pathOrder) {
+                const m = indexMaps.get(path);
+                row.push(m?.get(ts) ?? null);
+            }
+            return row;
+        });
+        return {
+            context,
+            range: { from: range.from, to: range.to },
+            values: valuesList,
+            data,
+        };
+    }
+    async function getPaths(query) {
+        const range = (0, time_range_1.resolveTimeRange)(query);
+        const where = buildRangeWhere(range);
+        const result = await queryClient.exec(`SELECT DISTINCT path FROM signalk WHERE ${where}
+       UNION
+       SELECT DISTINCT path FROM signalk_str WHERE ${where}
+       ORDER BY path`);
+        return result.dataset.map((row) => row[0]);
+    }
+    async function getContexts(query) {
+        const range = (0, time_range_1.resolveTimeRange)(query);
+        const where = buildRangeWhere(range);
+        const result = await queryClient.exec(`SELECT DISTINCT context FROM signalk WHERE ${where}
+       UNION
+       SELECT DISTINCT context FROM signalk_str WHERE ${where}
+       ORDER BY context`);
+        return result.dataset.map((row) => row[0]);
+    }
+    return { getValues, getPaths, getContexts };
+}
+//# sourceMappingURL=history-v2.js.map