signalk-questdb 1.1.0 → 1.1.2-beta.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -103,31 +103,58 @@ wants to slice it into kopia-dedup-friendly shards. Allowed tables:
103
103
 
104
104
  ## Configuration
105
105
 
106
- | Setting | Default | Description |
107
- | ------------------ | ------------ | ------------------------------------------------------------- |
108
- | QuestDB version | `latest` | Docker image tag (dropdown shows stable + pre-releases) |
109
- | Managed container | `true` | Let signalk-container manage QuestDB, or connect to external |
110
- | QuestDB host | `127.0.0.1` | Host (only used when managed=false) |
111
- | HTTP port | `9000` | QuestDB REST API port |
112
- | ILP port | `9009` | InfluxDB Line Protocol write port |
113
- | PostgreSQL port | `8812` | For Grafana connections |
114
- | Sampling rate (ms) | `1000` | Default min ms between writes per path (0 = every update) |
115
- | Record own vessel | `true` | Record self context |
116
- | Record AIS targets | `false` | Record other vessels |
117
- | Retention (days) | `0` | Auto-delete old partitions (0 = keep forever) |
118
- | Compression codec | `lz4` | On-disk WAL compression: `none`, `lz4`, or `zstd` |
119
- | Compression level | `3` | ZSTD level 1-22 (only when codec is zstd) |
120
- | Container network | `sk-network` | Shared Podman/Docker network for Grafana integration |
121
- | Bind to 0.0.0.0 | `false` | Bind ports to all interfaces instead of localhost (see below) |
106
+ | Setting | Default | Description |
107
+ | ------------------ | ------------ | ----------------------------------------------------------------- |
108
+ | QuestDB version | `latest` | Docker image tag (dropdown shows stable + pre-releases) |
109
+ | Managed container | `true` | Let signalk-container manage QuestDB, or connect to external |
110
+ | QuestDB host | `127.0.0.1` | External QuestDB host (only used when managed=false) |
111
+ | HTTP port | `9000` | External mode, or the host binding when "Bind to 0.0.0.0" is on |
112
+ | ILP port | `9009` | External mode, or the host binding when "Bind to 0.0.0.0" is on |
113
+ | PostgreSQL port | `8812` | Host binding for Grafana/psql when "Bind to 0.0.0.0" is on |
114
+ | Sampling rate (ms) | `2000` | Default min ms between writes per path (0 = every update) |
115
+ | Memory limit | `768m` | Hard cgroup cap on QuestDB container RAM (empty = unlimited) |
116
+ | CPU limit (cores) | `1.5` | Max CPU cores QuestDB can use (0 = unlimited) |
117
+ | Record own vessel | `true` | Record self context |
118
+ | Record AIS targets | `false` | Record other vessels |
119
+ | Retention (days) | `0` | Auto-delete old partitions (0 = keep forever) |
120
+ | Compression codec | `lz4` | On-disk WAL compression: `none`, `lz4`, or `zstd` |
121
+ | Compression level | `3` | ZSTD level 1-22 (only when codec is zstd) |
122
+ | Container network | `sk-network` | Shared network for QuestDB (only applied when binding to 0.0.0.0) |
123
+ | Bind to 0.0.0.0 | `false` | Expose QuestDB's ports on the LAN (see Connectivity below) |
124
+
125
+ ## Connectivity
126
+
127
+ In **managed mode** the plugin no longer needs `QuestDB host` to be correct for
128
+ your deployment — signalk-container resolves the right address automatically,
129
+ whether Signal K runs on bare metal or is itself containerized:
130
+
131
+ - **Bind to 0.0.0.0 = off (default).** QuestDB stays private. signalk-container
132
+ binds its ports to the host loopback (bare-metal Signal K) or attaches QuestDB
133
+ to Signal K's own container network (containerized Signal K), and the plugin
134
+ connects to whatever address it reports back. Nothing is exposed to the
135
+ network. QuestDB is also attached to the shared `Container network` so the
136
+ companion [signalk-grafana](https://github.com/dirkwa/signalk-grafana) plugin
137
+ still reaches it by container DNS. This is the recommended setup and fixes
138
+ connectivity for Signal K in a container.
139
+ - **Bind to 0.0.0.0 = on.** QuestDB's HTTP/ILP/PostgreSQL ports are published on
140
+ all interfaces using the configured port numbers, on the shared
141
+ `Container network`. Enable this only to reach QuestDB from another machine or
142
+ from a Grafana running in a separate Docker instance. When Signal K itself is
143
+ containerized it reaches the published ports via `host.containers.internal`.
144
+
145
+ In **external mode** (`Managed container` off) the plugin connects to the
146
+ QuestDB you point it at via `QuestDB host` + the HTTP/ILP ports.
122
147
 
123
148
  ## Performance (Pi / Low-Power Devices)
124
149
 
125
150
  The plugin is optimized for Raspberry Pi and similar low-power devices:
126
151
 
127
- - **Default sampling rate** of 1000ms limits each path to 1 write/sec, reducing total writes from ~500/s to ~238/s on a typical NMEA 2000 bus
152
+ - **Default sampling rate** of 2000ms limits each path to 1 write per 2 seconds, keeping write volume modest on busy NMEA 2000 buses
153
+ - **Resource caps** of 768 MB RAM and 1.5 CPU cores (cgroup limits via signalk-container) keep QuestDB from squeezing co-resident containers like Grafana, mayara, or signalk-backup. The JVM auto-sizes its heap to a fraction of the memory cap, so total footprint (heap + off-heap) is bounded
128
154
  - **QuestDB worker threads** reduced to 1 each (WAL, shared, ILP) to minimize CPU usage
129
155
  - **ILP batching** at 500ms intervals with 1000-row batches reduces TCP overhead
130
156
  - Per-path overrides allow faster rates for critical paths (e.g. `{ "environment.wind.*": 200 }`) while keeping slow-changing paths throttled
157
+ - Set the memory limit to empty or CPU limit to `0` to disable the cap entirely on roomier hosts
131
158
 
132
159
  ## History API Provider
133
160
 
@@ -139,16 +166,20 @@ QuestDB data is stored at `~/.signalk/plugin-config-data/signalk-questdb/` on th
139
166
 
140
167
  ## Grafana Integration
141
168
 
142
- Connect Grafana to QuestDB via the PostgreSQL data source:
169
+ Connect Grafana to QuestDB via the PostgreSQL data source (user `admin`,
170
+ password `quest`, database `qdb`).
143
171
 
144
- - Host: `localhost:8812`
145
- - User: `admin`
146
- - Password: `quest`
147
- - Database: `qdb`
172
+ The companion [signalk-grafana](https://github.com/dirkwa/signalk-grafana)
173
+ plugin wires this up for you: it runs Grafana as a managed container and
174
+ reaches QuestDB by its container DNS name on the shared `sk-network`, so no
175
+ host port needs to be exposed.
148
176
 
149
- If Grafana runs on the host or in Podman, `localhost:8812` works out of the box.
177
+ For a **self-hosted Grafana on the host or in Podman**, point it at
178
+ `localhost:<HTTP/PostgreSQL port>` — but note this only works when **"Bind to
179
+ 0.0.0.0"** is enabled (otherwise QuestDB is not published on a host port).
150
180
 
151
- If Grafana runs in a **separate Docker** container, it cannot reach the host's localhost. In that case, enable **"Bind to 0.0.0.0"** in the QuestDB plugin config and use your machine's LAN IP (e.g. `192.168.0.122:8812`) as the host in Grafana.
181
+ For Grafana in a **separate Docker instance**, enable **"Bind to 0.0.0.0"** and
182
+ use your machine's LAN IP (e.g. `192.168.0.122:8812`) as the host in Grafana.
152
183
 
153
184
  **Warning:** Binding to 0.0.0.0 exposes QuestDB's ports to your entire network. Only enable this if necessary, and ensure your firewall is configured appropriately.
154
185
 
@@ -166,7 +197,7 @@ SAMPLE BY $__interval
166
197
  ## Requirements
167
198
 
168
199
  - Node.js >= 22
169
- - [signalk-container](https://github.com/dirkwa/signalk-container) >= 1.6.0 plugin (for managed mode)
200
+ - [signalk-container](https://github.com/dirkwa/signalk-container) >= 1.14.0 plugin (for managed mode; older versions still work but fall back to loopback connectivity)
170
201
  - Signal K server
171
202
 
172
203
  ## License
@@ -6,18 +6,22 @@ exports.ConfigSchema = typebox_1.Type.Object({
6
6
  questdbHost: typebox_1.Type.String({
7
7
  default: "127.0.0.1",
8
8
  title: "QuestDB host",
9
+ description: "Only used when managed container is off (external QuestDB). In managed mode the address is resolved automatically.",
9
10
  }),
10
11
  questdbIlpPort: typebox_1.Type.Number({
11
12
  default: 9009,
12
13
  title: "ILP port (writes)",
14
+ description: "External mode, or the host binding when 'Bind to 0.0.0.0' is on. Ignored otherwise (signalk-container allocates the port).",
13
15
  }),
14
16
  questdbHttpPort: typebox_1.Type.Number({
15
17
  default: 9000,
16
18
  title: "HTTP port (queries)",
19
+ description: "External mode, or the host binding when 'Bind to 0.0.0.0' is on. Ignored otherwise (signalk-container allocates the port).",
17
20
  }),
18
21
  questdbPgPort: typebox_1.Type.Number({
19
22
  default: 8812,
20
23
  title: "PostgreSQL wire port",
24
+ description: "Host binding for Grafana/psql when 'Bind to 0.0.0.0' is on. Ignored otherwise (signalk-container allocates the port).",
21
25
  }),
22
26
  questdbVersion: typebox_1.Type.String({
23
27
  default: "9.2.0",
@@ -74,12 +78,12 @@ exports.ConfigSchema = typebox_1.Type.Object({
74
78
  networkName: typebox_1.Type.String({
75
79
  default: "sk-network",
76
80
  title: "Container network",
77
- description: "Shared Podman/Docker network so Grafana can connect via container DNS",
81
+ description: "Shared Podman/Docker network for QuestDB. Only applied when 'Bind to 0.0.0.0' is on (so a separate-Docker Grafana can reach QuestDB by DNS).",
78
82
  }),
79
83
  exposeToContainers: typebox_1.Type.Boolean({
80
84
  default: false,
81
85
  title: "Bind to 0.0.0.0",
82
- description: "Caution! This can expose your data to the internet. Only enable if Grafana runs in a separate Docker instance.",
86
+ description: "Publish QuestDB's ports on all interfaces so another machine or a separate-Docker Grafana can reach them. Off (default) keeps QuestDB private — Signal K still reaches it automatically. Caution! Enabling exposes your data to the network.",
83
87
  }),
84
88
  compression: typebox_1.Type.Union([typebox_1.Type.Literal("none"), typebox_1.Type.Literal("lz4"), typebox_1.Type.Literal("zstd")], {
85
89
  default: "lz4",
@@ -1 +1 @@
1
- {"version":3,"file":"schema.js","sourceRoot":"","sources":["../../src/config/schema.ts"],"names":[],"mappings":";;;AAAA,+CAAiD;AAEpC,QAAA,YAAY,GAAG,cAAI,CAAC,MAAM,CAAC;IACtC,WAAW,EAAE,cAAI,CAAC,MAAM,CAAC;QACvB,OAAO,EAAE,WAAW;QACpB,KAAK,EAAE,cAAc;KACtB,CAAC;IACF,cAAc,EAAE,cAAI,CAAC,MAAM,CAAC;QAC1B,OAAO,EAAE,IAAI;QACb,KAAK,EAAE,mBAAmB;KAC3B,CAAC;IACF,eAAe,EAAE,cAAI,CAAC,MAAM,CAAC;QAC3B,OAAO,EAAE,IAAI;QACb,KAAK,EAAE,qBAAqB;KAC7B,CAAC;IACF,aAAa,EAAE,cAAI,CAAC,MAAM,CAAC;QACzB,OAAO,EAAE,IAAI;QACb,KAAK,EAAE,sBAAsB;KAC9B,CAAC;IACF,cAAc,EAAE,cAAI,CAAC,MAAM,CAAC;QAC1B,OAAO,EAAE,OAAO;QAChB,KAAK,EAAE,uBAAuB;KAC/B,CAAC;IAEF,gBAAgB,EAAE,cAAI,CAAC,OAAO,CAAC;QAC7B,OAAO,EAAE,IAAI;QACb,KAAK,EAAE,gDAAgD;QACvD,WAAW,EAAE,oDAAoD;KAClE,CAAC;IAEF,kBAAkB,EAAE,cAAI,CAAC,MAAM,CAAC;QAC9B,OAAO,EAAE,MAAM;QACf,KAAK,EAAE,sBAAsB;QAC7B,WAAW,EACT,8KAA8K;KACjL,CAAC;IAEF,eAAe,EAAE,cAAI,CAAC,MAAM,CAAC;QAC3B,OAAO,EAAE,GAAG;QACZ,KAAK,EAAE,2BAA2B;QAClC,WAAW,EACT,sEAAsE;KACzE,CAAC;IAEF,UAAU,EAAE,cAAI,CAAC,MAAM,CAAC;QACtB,IAAI,EAAE,cAAI,CAAC,KAAK,CAAC,CAAC,cAAI,CAAC,OAAO,CAAC,SAAS,CAAC,EAAE,cAAI,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC,EAAE;YACnE,OAAO,EAAE,SAAS;YAClB,KAAK,EAAE,aAAa;SACrB,CAAC;QACF,KAAK,EAAE,cAAI,CAAC,KAAK,CAAC,cAAI,CAAC,MAAM,EAAE,EAAE;YAC/B,OAAO,EAAE,EAAE;YACX,KAAK,EAAE,gCAAgC;YACvC,WAAW,EAAE,8CAA8C;SAC5D,CAAC;KACH,CAAC;IAEF,mBAAmB,EAAE,cAAI,CAAC,MAAM,CAAC;QAC/B,OAAO,EAAE,IAAI;QACb,KAAK,EAAE,4BAA4B;QACnC,WAAW,EACT,gMAAgM;KACnM,CAAC;IAEF,aAAa,EAAE,cAAI,CAAC,MAAM,CAAC,cAAI,CAAC,MAAM,EAAE,EAAE,cAAI,CAAC,MAAM,EAAE,EAAE;QACvD,OAAO,EAAE,EAAE;QACX,KAAK,EAAE,8BAA8B;QACrC,WAAW,EACT,gGAAgG;KACnG,CAAC;IAEF,UAAU,EAAE,cAAI,CAAC,OAAO,CAAC;QACvB,OAAO,EAAE,IAAI;QACb,KAAK,EAAE,mBAAmB;KAC3B,CAAC;IACF,YAAY,EAAE,cAAI,CAAC,OAAO,CAAC;QACzB,OAAO,EAAE,KAAK;QACd,KAAK,EAAE,oBAAoB;KAC5B,CAAC;IAEF,aAAa,EAAE,cAAI,CAAC,MAAM,CAAC;QACzB,OAAO,EAAE,CAAC;QACV,KAAK,EAAE,oCAAoC;KAC5C,CAAC;IAEF,WAAW,EAAE,cAAI,CAAC,MAAM,CAAC;QACvB,OAAO,EAAE,YAAY;QACrB,KAAK,EAAE,mBAAmB;QAC1B,WAAW,EACT,uEAAuE;KAC1E,CAAC;IAEF,kBAAkB,EAAE,cAAI,CAAC,OAAO,CAAC;QAC/B,OAAO,EAAE,KAAK;QACd,KAAK,EAAE,iBAAiB;QACxB,WAAW,EACT,gHAAgH;KACnH,CAAC;IAEF,WAAW,EAAE,cAAI,CAAC,KAAK,CACrB,CAAC,cAAI,CAAC,OAAO,CAAC,MAAM,CAAC,EAAE,cAAI,CAAC,OAAO,CAAC,KAAK,CAAC,EAAE,cAAI,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC,EACjE;QACE,OAAO,EAAE,KAAK;QACd,KAAK,EAAE,mBAAmB;QAC1B,WAAW,EACT,wFAAwF;KAC3F,CACF;IACD,gBAAgB,EAAE,cAAI,CAAC,MAAM,CAAC;QAC5B,OAAO,EAAE,CAAC;QACV,KAAK,EAAE,mBAAmB;QAC1B,WAAW,EAAE,sCAAsC;KACpD,CAAC;CACH,CAAC,CAAC"}
1
+ {"version":3,"file":"schema.js","sourceRoot":"","sources":["../../src/config/schema.ts"],"names":[],"mappings":";;;AAAA,+CAAiD;AAEpC,QAAA,YAAY,GAAG,cAAI,CAAC,MAAM,CAAC;IACtC,WAAW,EAAE,cAAI,CAAC,MAAM,CAAC;QACvB,OAAO,EAAE,WAAW;QACpB,KAAK,EAAE,cAAc;QACrB,WAAW,EACT,oHAAoH;KACvH,CAAC;IACF,cAAc,EAAE,cAAI,CAAC,MAAM,CAAC;QAC1B,OAAO,EAAE,IAAI;QACb,KAAK,EAAE,mBAAmB;QAC1B,WAAW,EACT,4HAA4H;KAC/H,CAAC;IACF,eAAe,EAAE,cAAI,CAAC,MAAM,CAAC;QAC3B,OAAO,EAAE,IAAI;QACb,KAAK,EAAE,qBAAqB;QAC5B,WAAW,EACT,4HAA4H;KAC/H,CAAC;IACF,aAAa,EAAE,cAAI,CAAC,MAAM,CAAC;QACzB,OAAO,EAAE,IAAI;QACb,KAAK,EAAE,sBAAsB;QAC7B,WAAW,EACT,uHAAuH;KAC1H,CAAC;IACF,cAAc,EAAE,cAAI,CAAC,MAAM,CAAC;QAC1B,OAAO,EAAE,OAAO;QAChB,KAAK,EAAE,uBAAuB;KAC/B,CAAC;IAEF,gBAAgB,EAAE,cAAI,CAAC,OAAO,CAAC;QAC7B,OAAO,EAAE,IAAI;QACb,KAAK,EAAE,gDAAgD;QACvD,WAAW,EAAE,oDAAoD;KAClE,CAAC;IAEF,kBAAkB,EAAE,cAAI,CAAC,MAAM,CAAC;QAC9B,OAAO,EAAE,MAAM;QACf,KAAK,EAAE,sBAAsB;QAC7B,WAAW,EACT,8KAA8K;KACjL,CAAC;IAEF,eAAe,EAAE,cAAI,CAAC,MAAM,CAAC;QAC3B,OAAO,EAAE,GAAG;QACZ,KAAK,EAAE,2BAA2B;QAClC,WAAW,EACT,sEAAsE;KACzE,CAAC;IAEF,UAAU,EAAE,cAAI,CAAC,MAAM,CAAC;QACtB,IAAI,EAAE,cAAI,CAAC,KAAK,CAAC,CAAC,cAAI,CAAC,OAAO,CAAC,SAAS,CAAC,EAAE,cAAI,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC,EAAE;YACnE,OAAO,EAAE,SAAS;YAClB,KAAK,EAAE,aAAa;SACrB,CAAC;QACF,KAAK,EAAE,cAAI,CAAC,KAAK,CAAC,cAAI,CAAC,MAAM,EAAE,EAAE;YAC/B,OAAO,EAAE,EAAE;YACX,KAAK,EAAE,gCAAgC;YACvC,WAAW,EAAE,8CAA8C;SAC5D,CAAC;KACH,CAAC;IAEF,mBAAmB,EAAE,cAAI,CAAC,MAAM,CAAC;QAC/B,OAAO,EAAE,IAAI;QACb,KAAK,EAAE,4BAA4B;QACnC,WAAW,EACT,gMAAgM;KACnM,CAAC;IAEF,aAAa,EAAE,cAAI,CAAC,MAAM,CAAC,cAAI,CAAC,MAAM,EAAE,EAAE,cAAI,CAAC,MAAM,EAAE,EAAE;QACvD,OAAO,EAAE,EAAE;QACX,KAAK,EAAE,8BAA8B;QACrC,WAAW,EACT,gGAAgG;KACnG,CAAC;IAEF,UAAU,EAAE,cAAI,CAAC,OAAO,CAAC;QACvB,OAAO,EAAE,IAAI;QACb,KAAK,EAAE,mBAAmB;KAC3B,CAAC;IACF,YAAY,EAAE,cAAI,CAAC,OAAO,CAAC;QACzB,OAAO,EAAE,KAAK;QACd,KAAK,EAAE,oBAAoB;KAC5B,CAAC;IAEF,aAAa,EAAE,cAAI,CAAC,MAAM,CAAC;QACzB,OAAO,EAAE,CAAC;QACV,KAAK,EAAE,oCAAoC;KAC5C,CAAC;IAEF,WAAW,EAAE,cAAI,CAAC,MAAM,CAAC;QACvB,OAAO,EAAE,YAAY;QACrB,KAAK,EAAE,mBAAmB;QAC1B,WAAW,EACT,8IAA8I;KACjJ,CAAC;IAEF,kBAAkB,EAAE,cAAI,CAAC,OAAO,CAAC;QAC/B,OAAO,EAAE,KAAK;QACd,KAAK,EAAE,iBAAiB;QACxB,WAAW,EACT,8OAA8O;KACjP,CAAC;IAEF,WAAW,EAAE,cAAI,CAAC,KAAK,CACrB,CAAC,cAAI,CAAC,OAAO,CAAC,MAAM,CAAC,EAAE,cAAI,CAAC,OAAO,CAAC,KAAK,CAAC,EAAE,cAAI,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC,EACjE;QACE,OAAO,EAAE,KAAK;QACd,KAAK,EAAE,mBAAmB;QAC1B,WAAW,EACT,wFAAwF;KAC3F,CACF;IACD,gBAAgB,EAAE,cAAI,CAAC,MAAM,CAAC;QAC5B,OAAO,EAAE,CAAC;QACV,KAAK,EAAE,mBAAmB;QAC1B,WAAW,EAAE,sCAAsC;KACpD,CAAC;CACH,CAAC,CAAC"}
package/dist/index.js CHANGED
@@ -41,6 +41,10 @@ const history_v2_1 = require("./history-v2");
41
41
  const history_v1_1 = require("./history-v1");
42
42
  const retention_1 = require("./retention");
43
43
  const full_export_range_1 = require("./full-export-range");
44
+ const questdb_endpoint_1 = require("./questdb-endpoint");
45
+ // The managed QuestDB container's name. signalk-container prefixes it with
46
+ // `sk-` for the actual container and its DNS name (e.g. `sk-signalk-questdb`).
47
+ const QUESTDB_CONTAINER_NAME = "signalk-questdb";
44
48
  // Tables exposed by /api/full-export. Hardcoded (not introspected via
45
49
  // QuestDB's tables() function) because we DO know our schema — we
46
50
  // created it. Single source of truth for both the listing endpoint and
@@ -51,6 +55,27 @@ const FULL_EXPORT_TABLES = [
51
55
  "signalk_position",
52
56
  ];
53
57
  const FULL_EXPORT_TABLE_SET = new Set(FULL_EXPORT_TABLES);
58
+ // Healthcheck for the QuestDB container. The `questdb/questdb` image ships no
59
+ // HEALTHCHECK of its own, so under Podman the container would otherwise sit in
60
+ // `starting` forever (Podman reports a probeless container as perpetually
61
+ // starting, never healthy). We give it an explicit probe — `curl` is present
62
+ // in the image — hitting QuestDB's purpose-built `/ping` liveness endpoint,
63
+ // which returns an empty `204` immediately. (Probing `/` instead makes curl
64
+ // hang on the web-console `301` redirect until it times out.) signalk-container
65
+ // emits this as `--health-*` run flags (see its `healthcheck` ContainerConfig
66
+ // field).
67
+ const QUESTDB_HEALTHCHECK = {
68
+ test: [
69
+ "CMD",
70
+ "curl",
71
+ "-f",
72
+ `http://127.0.0.1:${questdb_endpoint_1.QUESTDB_INTERNAL_HTTP_PORT}/ping`,
73
+ ],
74
+ interval: "30s",
75
+ timeout: "5s",
76
+ startPeriod: "15s",
77
+ retries: 3,
78
+ };
54
79
  function buildResourceLimits(config) {
55
80
  return {
56
81
  memory: config.questdbMemoryLimit?.trim() || null,
@@ -64,8 +89,29 @@ module.exports = (app) => {
64
89
  let queryClient = null;
65
90
  let retentionTimer = null;
66
91
  let currentConfig = null;
92
+ // The HTTP/ILP endpoints the Signal K process uses to reach QuestDB. In
93
+ // managed mode signalk-container resolves these (loopback bare-metal,
94
+ // container DNS when SK is containerized); in external mode they come from
95
+ // config. The REST/export endpoints read `questdbEndpoints.http` so they
96
+ // stay correct in every topology, not just when questdbHost is loopback.
97
+ let questdbEndpoints = null;
67
98
  const unsubscribes = [];
68
99
  const throttleMap = new Map();
100
+ // The HTTP base URL for QuestDB's REST API (/exp, /exec). Used by the export
101
+ // endpoints, which talk to QuestDB directly rather than through QueryClient.
102
+ // Those callers all guard on `!queryClient` first, and `questdbEndpoints` is
103
+ // set before `queryClient` in asyncStart and cleared after it in stop(), so
104
+ // it is non-null whenever this runs. The config-derived branch is purely
105
+ // defensive (never exercised in practice).
106
+ function questdbHttpBaseUrl() {
107
+ if (questdbEndpoints) {
108
+ const { host, port } = questdbEndpoints.http;
109
+ return `http://${host}:${port}`;
110
+ }
111
+ const host = currentConfig?.questdbHost ?? "127.0.0.1";
112
+ const port = currentConfig?.questdbHttpPort ?? questdb_endpoint_1.QUESTDB_INTERNAL_HTTP_PORT;
113
+ return `http://${host}:${port}`;
114
+ }
69
115
  /**
70
116
  * Compute the bind-mount source for QuestDB's /var/lib/questdb volume.
71
117
  *
@@ -95,6 +141,77 @@ module.exports = (app) => {
95
141
  return dataPath;
96
142
  }
97
143
  }
144
+ /**
145
+ * Apply the chosen networking to a managed-QuestDB ContainerConfig and
146
+ * return the endpoints the Signal K process should use to reach it. Two
147
+ * paths, keyed on `exposeToContainers`:
148
+ *
149
+ * - off (default): signalkAccessiblePorts — signalk-container owns the
150
+ * networking and resolveContainerAddress() yields the endpoint. Secure;
151
+ * QuestDB is not published beyond loopback / the shared network.
152
+ * - on: publish the configured host ports on 0.0.0.0 (for LAN / separate-
153
+ * Docker Grafana) on the shared `networkName`; the Signal K process
154
+ * reaches them on the host loopback (bare-metal) or via the
155
+ * host.containers.internal gateway (containerized SK).
156
+ *
157
+ * `endpoints` is resolved AFTER ensureRunning() on the signalkAccessiblePorts
158
+ * path (the host port is allocated by then); on the LAN path it is already
159
+ * known, so the returned resolver just echoes it.
160
+ */
161
+ async function applyQuestdbNetworking(config, containers, name, containerConfig) {
162
+ const httpPort = config.questdbHttpPort ?? questdb_endpoint_1.QUESTDB_INTERNAL_HTTP_PORT;
163
+ const ilpPort = config.questdbIlpPort ?? questdb_endpoint_1.QUESTDB_INTERNAL_ILP_PORT;
164
+ const pgPort = config.questdbPgPort ?? questdb_endpoint_1.QUESTDB_INTERNAL_PG_PORT;
165
+ const fallbackHost = config.questdbHost ?? "127.0.0.1";
166
+ // signalkAccessiblePorts is the modern (1.14.0+) connectivity path. Without
167
+ // it, signalk-container ignores the field and would publish no host port,
168
+ // so on older versions we must keep the historical manual port bindings.
169
+ const hasAccessiblePorts = typeof containers.resolveContainerAddress === "function";
170
+ // The LAN-exposure path and the old-container fallback both publish ports
171
+ // and attach to networkName; only the host the SK process uses differs.
172
+ const publishOnHost = config.exposeToContainers || !hasAccessiblePorts;
173
+ if (publishOnHost) {
174
+ const bind = config.exposeToContainers ? "0.0.0.0" : "127.0.0.1";
175
+ containerConfig.ports = {
176
+ [`${questdb_endpoint_1.QUESTDB_INTERNAL_HTTP_PORT}/tcp`]: `${bind}:${httpPort}`,
177
+ [`${questdb_endpoint_1.QUESTDB_INTERNAL_ILP_PORT}/tcp`]: `${bind}:${ilpPort}`,
178
+ [`${questdb_endpoint_1.QUESTDB_INTERNAL_PG_PORT}/tcp`]: `${bind}:${pgPort}`,
179
+ };
180
+ if (config.networkName) {
181
+ await containers.ensureNetwork(config.networkName);
182
+ containerConfig.networkMode = config.networkName;
183
+ }
184
+ // A 0.0.0.0-published port is reachable from a containerized SK via the
185
+ // host gateway; a loopback-only one (or any bare-metal/old-container
186
+ // case) is reached on 127.0.0.1.
187
+ const skHost = config.exposeToContainers
188
+ ? await (0, questdb_endpoint_1.resolveLanExposureHost)(containers, (msg) => app.debug(msg))
189
+ : "127.0.0.1";
190
+ const endpoints = (0, questdb_endpoint_1.lanExposureEndpoints)(skHost, httpPort, ilpPort);
191
+ return async () => endpoints;
192
+ }
193
+ // Default path: signalk-container owns the networking. After the container
194
+ // is up we additionally attach it to `networkName` so the companion
195
+ // signalk-grafana (which joins that network and resolves QuestDB by its
196
+ // `sk-`-prefixed DNS name) keeps working, then resolve the SK->QuestDB
197
+ // endpoint from whatever address signalk-container reports.
198
+ containerConfig.signalkAccessiblePorts = questdb_endpoint_1.QUESTDB_ACCESSIBLE_PORTS;
199
+ return async () => {
200
+ if (config.networkName &&
201
+ typeof containers.connectToNetwork === "function") {
202
+ try {
203
+ await containers.ensureNetwork(config.networkName);
204
+ await containers.connectToNetwork(name, config.networkName);
205
+ }
206
+ catch (err) {
207
+ // Non-fatal: SK->QuestDB does not depend on this network; only the
208
+ // companion Grafana's DNS path does. Log and carry on.
209
+ app.debug(`connectToNetwork(${name}, ${config.networkName}) failed: ${String(err)}`);
210
+ }
211
+ }
212
+ return (0, questdb_endpoint_1.resolveManagedEndpoints)(containers, name, fallbackHost, (msg) => app.debug(msg));
213
+ };
214
+ }
98
215
  function shouldRecord(path, filter) {
99
216
  if (!filter.paths || filter.paths.length === 0)
100
217
  return true;
@@ -126,9 +243,18 @@ module.exports = (app) => {
126
243
  }
127
244
  async function asyncStart(config) {
128
245
  currentConfig = config;
129
- const host = config.questdbHost ?? "127.0.0.1";
130
- const ilpPort = config.questdbIlpPort ?? 9009;
131
- const httpPort = config.questdbHttpPort ?? 9000;
246
+ // External mode: the configured host/ports are authoritative. Managed mode
247
+ // overwrites this below with the endpoints signalk-container resolves.
248
+ questdbEndpoints = {
249
+ http: {
250
+ host: config.questdbHost ?? "127.0.0.1",
251
+ port: config.questdbHttpPort ?? questdb_endpoint_1.QUESTDB_INTERNAL_HTTP_PORT,
252
+ },
253
+ ilp: {
254
+ host: config.questdbHost ?? "127.0.0.1",
255
+ port: config.questdbIlpPort ?? questdb_endpoint_1.QUESTDB_INTERNAL_ILP_PORT,
256
+ },
257
+ };
132
258
  if (config.managedContainer !== false) {
133
259
  const containers = globalThis.__signalk_containerManager;
134
260
  if (!containers) {
@@ -174,30 +300,23 @@ module.exports = (app) => {
174
300
  }
175
301
  : {}),
176
302
  };
177
- const bind = config.exposeToContainers ? "0.0.0.0" : "127.0.0.1";
178
303
  const volumeSource = await resolveQuestdbVolumeSource(containers);
179
304
  const containerConfig = {
180
305
  image: "questdb/questdb",
181
306
  tag: config.questdbVersion ?? "latest",
182
- ports: {
183
- "9000/tcp": `${bind}:${httpPort}`,
184
- "9009/tcp": `${bind}:${ilpPort}`,
185
- "8812/tcp": `${bind}:${config.questdbPgPort ?? 8812}`,
186
- },
187
307
  volumes: {
188
308
  "/var/lib/questdb": volumeSource,
189
309
  },
190
310
  env: containerEnv,
191
311
  restart: "unless-stopped",
192
312
  resources: buildResourceLimits(config),
313
+ healthcheck: QUESTDB_HEALTHCHECK,
193
314
  };
194
- if (config.networkName) {
195
- await containers.ensureNetwork(config.networkName);
196
- containerConfig.networkMode = config.networkName;
197
- }
315
+ const resolveEndpoints = await applyQuestdbNetworking(config, containers, QUESTDB_CONTAINER_NAME, containerConfig);
198
316
  app.setPluginStatus("Starting QuestDB container...");
199
- await containers.ensureRunning("signalk-questdb", containerConfig);
317
+ await containers.ensureRunning(QUESTDB_CONTAINER_NAME, containerConfig);
200
318
  app.debug("QuestDB container ready");
319
+ questdbEndpoints = await resolveEndpoints();
201
320
  }
202
321
  catch (err) {
203
322
  app.debug("ensureRunning failed:", err);
@@ -205,8 +324,10 @@ module.exports = (app) => {
205
324
  return;
206
325
  }
207
326
  }
208
- app.debug("connecting to QuestDB at %s:%d", host, httpPort);
209
- queryClient = new query_client_1.QueryClient(host, httpPort);
327
+ const { host: httpHost, port: httpPort } = questdbEndpoints.http;
328
+ const { host: ilpHost, port: ilpPort } = questdbEndpoints.ilp;
329
+ app.debug("connecting to QuestDB at %s:%d", httpHost, httpPort);
330
+ queryClient = new query_client_1.QueryClient(httpHost, httpPort);
210
331
  app.setPluginStatus("Waiting for QuestDB to become ready...");
211
332
  const deadline = Date.now() + 30000;
212
333
  while (Date.now() < deadline) {
@@ -215,12 +336,12 @@ module.exports = (app) => {
215
336
  await new Promise((resolve) => setTimeout(resolve, 500));
216
337
  }
217
338
  if (!(await queryClient.isHealthy())) {
218
- app.setPluginError(`QuestDB not responding at ${host}:${httpPort}`);
339
+ app.setPluginError(`QuestDB not responding at ${httpHost}:${httpPort}`);
219
340
  return;
220
341
  }
221
342
  app.setPluginStatus("Creating tables...");
222
343
  await queryClient.ensureTables();
223
- writer = new ilp_writer_1.ILPWriter(host, ilpPort, (msg) => app.debug(msg));
344
+ writer = new ilp_writer_1.ILPWriter(ilpHost, ilpPort, (msg) => app.debug(msg));
224
345
  await writer.connect();
225
346
  const v2Provider = (0, history_v2_1.createHistoryProviderV2)(queryClient, app.selfContext);
226
347
  app.registerHistoryApiProvider(v2Provider);
@@ -273,7 +394,7 @@ module.exports = (app) => {
273
394
  if (config.retentionDays && config.retentionDays > 0) {
274
395
  retentionTimer = (0, retention_1.startRetention)(queryClient, config.retentionDays, (msg) => app.debug(msg));
275
396
  }
276
- app.setPluginStatus(`Recording to QuestDB at ${host}:${ilpPort}`);
397
+ app.setPluginStatus(`Recording to QuestDB at ${ilpHost}:${ilpPort}`);
277
398
  }
278
399
  const plugin = {
279
400
  id: "signalk-questdb",
@@ -306,12 +427,13 @@ module.exports = (app) => {
306
427
  }
307
428
  throttleMap.clear();
308
429
  queryClient = null;
430
+ questdbEndpoints = null;
309
431
  // Stop the managed container when plugin is disabled
310
432
  if (currentConfig?.managedContainer !== false) {
311
433
  const containers = globalThis.__signalk_containerManager;
312
434
  if (containers) {
313
435
  try {
314
- await containers.stop("signalk-questdb");
436
+ await containers.stop(QUESTDB_CONTAINER_NAME);
315
437
  }
316
438
  catch {
317
439
  // container may already be stopped
@@ -353,6 +475,9 @@ module.exports = (app) => {
353
475
  status: "running",
354
476
  totalRows,
355
477
  activePathsToday,
478
+ endpoint: questdbEndpoints
479
+ ? `${questdbEndpoints.http.host}:${questdbEndpoints.http.port}`
480
+ : null,
356
481
  });
357
482
  }
358
483
  catch (err) {
@@ -514,22 +639,11 @@ module.exports = (app) => {
514
639
  });
515
640
  });
516
641
  }
517
- const host = currentConfig?.questdbHost ?? "127.0.0.1";
518
- const httpPort = currentConfig?.questdbHttpPort ?? 9000;
519
- const ilpPort = currentConfig?.questdbIlpPort ?? 9009;
520
- const updateBind = currentConfig?.exposeToContainers
521
- ? "0.0.0.0"
522
- : "127.0.0.1";
523
642
  const updateVolumeSource = await resolveQuestdbVolumeSource(containers);
524
643
  app.setPluginStatus(`Starting QuestDB ${newTag}...`);
525
- await containers.ensureRunning("signalk-questdb", {
644
+ const updateConfig = {
526
645
  image: "questdb/questdb",
527
646
  tag: newTag,
528
- ports: {
529
- "9000/tcp": `${updateBind}:${httpPort}`,
530
- "9009/tcp": `${updateBind}:${ilpPort}`,
531
- "8812/tcp": `${updateBind}:${currentConfig?.questdbPgPort ?? 8812}`,
532
- },
533
647
  volumes: {
534
648
  "/var/lib/questdb": updateVolumeSource,
535
649
  },
@@ -554,8 +668,18 @@ module.exports = (app) => {
554
668
  resources: currentConfig
555
669
  ? buildResourceLimits(currentConfig)
556
670
  : undefined,
557
- });
558
- // Reconnect ILP and query client
671
+ healthcheck: QUESTDB_HEALTHCHECK,
672
+ };
673
+ const resolveUpdateEndpoints = await applyQuestdbNetworking(currentConfig ?? {}, containers, QUESTDB_CONTAINER_NAME, updateConfig);
674
+ await containers.ensureRunning(QUESTDB_CONTAINER_NAME, updateConfig);
675
+ // Re-resolve so the export endpoints and status line reflect the
676
+ // current endpoint. The version bump keeps the same container name
677
+ // and networking, so the endpoint is stable — the existing
678
+ // QueryClient/ILPWriter stay valid (and the registered history
679
+ // providers keep their reference).
680
+ questdbEndpoints = await resolveUpdateEndpoints();
681
+ const { host: ilpHost, port: ilpPort } = questdbEndpoints.ilp;
682
+ // Wait for the recreated container to answer, then reconnect ILP.
559
683
  if (queryClient) {
560
684
  const deadline = Date.now() + 30000;
561
685
  while (Date.now() < deadline) {
@@ -573,7 +697,7 @@ module.exports = (app) => {
573
697
  }
574
698
  await writer.connect();
575
699
  }
576
- app.setPluginStatus(`Recording to QuestDB ${newTag} at ${host}:${ilpPort}`);
700
+ app.setPluginStatus(`Recording to QuestDB ${newTag} at ${ilpHost}:${ilpPort}`);
577
701
  res.json({
578
702
  status: "updated",
579
703
  newVersion: newTag,
@@ -670,9 +794,7 @@ module.exports = (app) => {
670
794
  const safeTo = toDate.toISOString();
671
795
  const sql = `SELECT ts, path, context, value FROM signalk WHERE ts >= '${safeFrom}' AND ts <= '${safeTo}' ORDER BY ts`;
672
796
  const dateSlug = safeFrom.slice(0, 10);
673
- const host = currentConfig?.questdbHost ?? "127.0.0.1";
674
- const httpPort = currentConfig?.questdbHttpPort ?? 9000;
675
- const expUrl = new URL("/exp", `http://${host}:${httpPort}`);
797
+ const expUrl = new URL("/exp", questdbHttpBaseUrl());
676
798
  expUrl.searchParams.set("query", sql);
677
799
  if (format === "parquet") {
678
800
  expUrl.searchParams.set("fmt", "parquet");
@@ -772,9 +894,7 @@ module.exports = (app) => {
772
894
  // timestamp order, and adding ORDER BY forces a sort over the full
773
895
  // table that's slow on the Pi for the wide signalk table.
774
896
  const sql = `SELECT * FROM ${table}${rangeResult.where}`;
775
- const host = currentConfig?.questdbHost ?? "127.0.0.1";
776
- const httpPort = currentConfig?.questdbHttpPort ?? 9000;
777
- const expUrl = new URL("/exp", `http://${host}:${httpPort}`);
897
+ const expUrl = new URL("/exp", questdbHttpBaseUrl());
778
898
  expUrl.searchParams.set("query", sql);
779
899
  expUrl.searchParams.set("fmt", "parquet");
780
900
  const codec = currentConfig?.compression ?? "lz4";