influx4mqtt 0.0.1 → 2.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ The MIT License (MIT)
2
+
3
+ Copyright (c) Sebastian Raff <hobbyquaker@gmail.com> (https://hobbyquaker.github.io)
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in
13
+ all copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
21
+ THE SOFTWARE.
package/README.md CHANGED
@@ -1,17 +1,117 @@
1
1
  # influx4mqtt
2
2
 
3
- Insert incoming MQTT values into InfluxDB.
3
+ [![mqtt-smarthome](https://img.shields.io/badge/mqtt-smarthome-blue.svg)](https://github.com/mqtt-smarthome/mqtt-smarthome)
4
+ [![NPM version](https://badge.fury.io/js/influx4mqtt.svg)](http://badge.fury.io/js/influx4mqtt)
5
+ [![CI](https://github.com/hobbyquaker/influx4mqtt/actions/workflows/ci.yml/badge.svg)](https://github.com/hobbyquaker/influx4mqtt/actions/workflows/ci.yml)
6
+ [![License][mit-badge]][mit-url]
4
7
 
5
- see [https://github.com/mqtt-smarthome/mqtt-smarthome](https://github.com/mqtt-smarthome/mqtt-smarthome)
8
+ Record MQTT values in InfluxDB — the history behind a Grafana dashboard.
6
9
 
7
- ## Install & Usage
10
+ Subscribes to the topics you name, turns each message into one InfluxDB point and writes them in
11
+ batches. Numeric values only: booleans and `ON`/`OFF` become `1`/`0`, text is ignored, because a
12
+ time series with a string in it is a series InfluxDB will not let you graph.
13
+
14
+ Built on [mqtt-interfaces-core](https://github.com/hobbyquaker/mqtt-interfaces-core), so it has the
15
+ same options, `--install`, `<name>/info` and maintenance topics as the rest of the `xyz2mqtt` fleet.
16
+ Works with **InfluxDB 1 and 2**.
17
+
18
+ ## Install
19
+
20
+ ```
21
+ npm install -g influx4mqtt
22
+ influx4mqtt --mqtt-url mqtt://broker # foreground
23
+ sudo influx4mqtt --install -n influx -u mqtt://broker # systemd service influx4mqtt@influx
24
+ ```
25
+
26
+ `--install` writes the options to `/etc/influx4mqtt/<name>.env` and enables
27
+ `influx4mqtt@<name>.service`. Broker settings shared with the other adapters on the host live in
28
+ `/etc/mqtt-interfaces/broker.env`.
29
+
30
+ ### Docker
31
+
32
+ Multi-arch image (amd64, arm64, armv7):
8
33
 
9
34
  ```
10
- sudo npm install -g influx4mqtt
11
- influx4mqtt --help
35
+ docker run -d --name influx4mqtt --restart unless-stopped \
36
+ -e INFLUX4MQTT_MQTT_URL=mqtt://broker \
37
+ -e INFLUX4MQTT_INFLUX_URL=http://influxdb:8086 \
38
+ -e INFLUX4MQTT_SUBSCRIBE='+/status/#,$SYS/#' \
39
+ ghcr.io/hobbyquaker/influx4mqtt
12
40
  ```
13
41
 
42
+ ## What gets recorded
43
+
44
+ Every message on a subscribed topic becomes one point:
45
+
46
+ | topic | payload | series | value |
47
+ | -------------------------- | --------------------- | -------------------------- | --------------------------- |
48
+ | `hm/status/Licht/STATE` | `21.5` | `hm//Licht/STATE` | `21.5` |
49
+ | `hm/status/Licht/STATE` | `{"val":21.5,"ts":…}` | `hm//Licht/STATE` | `21.5` at the device's `ts` |
50
+ | `zigbee2mqtt/sensor/state` | `ON` | `zigbee2mqtt/sensor/state` | `1` |
51
+ | `hm/status/mode` | `auto` | — | not recorded |
52
+
53
+ - **The `status` level is collapsed**: `hm/status/lamp` is recorded as `hm//lamp`, the
54
+ mqtt-smarthome convention. `--no-shorten-status` records the full topic instead — but only do
55
+ that on a fresh database, since it renames every series you already have.
56
+ - **`{val, ts, lc}` payloads** are recorded at the device's own timestamp, not at arrival time.
57
+ - **Retained messages are ignored.** On every reconnect the broker replays the retained value of
58
+ every matching topic; recording those would pile duplicates at the reconnect time rather than
59
+ when the value was measured.
60
+ - **`$SYS/` is rewritten** to `$SYS/<hostname>/` by default, because `$SYS` topics are identical on
61
+ every broker and two of them in one database would interleave into nonsense. `--replace-sys`
62
+ chooses the prefix.
63
+
64
+ ## Options
65
+
66
+ | Option | Default | Meaning |
67
+ | ------------------- | ----------------------- | ------------------------------------------------------- |
68
+ | `-s, --subscribe` | `+/status/#` | topic to record, `+`/`#` wildcards; repeat for more |
69
+ | `-u, --mqtt-url` | `mqtt://localhost` | broker url |
70
+ | `-n, --name` | `influx` | instance name = topic prefix of its own topics |
71
+ | `--influx-url` | `http://127.0.0.1:8086` | InfluxDB base url |
72
+ | `--influx-version` | `1` | `1` (`/write?db=`) or `2` (`/api/v2/write`, token auth) |
73
+ | `-d, --influx-db` | `mqtt` | database (api 1) or bucket (api 2) |
74
+ | `--influx-org` | | organisation (api 2) |
75
+ | `--influx-token` | | api token (api 2; InfluxDB 1.8+ accepts it too) |
76
+ | `--influx-username` | | http basic auth (api 1) |
77
+ | `--influx-password` | | http basic auth (api 1) |
78
+ | `--shorten-status` | `true` | record `hm/status/lamp` as `hm//lamp` |
79
+ | `--replace-sys` | `$SYS/<hostname>/` | rewrite the `$SYS/` prefix |
80
+ | `--buf-length` | `1000` | write once this many points are buffered |
81
+ | `--buf-interval` | `30` | seconds between writes, however few points are buffered |
82
+ | `-v, --verbosity` | `info` | `error`, `warn`, `info`, `debug` |
83
+
84
+ Every option is also an environment variable (`INFLUX4MQTT_SUBSCRIBE`, `INFLUX4MQTT_INFLUX_URL`, …);
85
+ several subscriptions are comma separated there. `--help` lists the shared options too, and
86
+ `--config-schema` prints the JSON Schema a management UI reads.
87
+
88
+ ### InfluxDB 2
89
+
90
+ ```
91
+ influx4mqtt -u mqtt://broker \
92
+ --influx-version 2 --influx-url http://influxdb:8086 \
93
+ --influx-org home --influx-db mqtt --influx-token <token>
94
+ ```
95
+
96
+ `--influx-db` is the bucket. Put the token in the instance's env file rather than on the command
97
+ line, where a process list would show it — `--install` does that for you.
98
+
99
+ ## Topics of its own
100
+
101
+ | Topic | Meaning |
102
+ | --------------------------------- | ---------------------------------------------------------------- |
103
+ | `<name>/connected` | `2` InfluxDB is accepting writes · `1` broker only · `0` stopped |
104
+ | `<name>/info` | endpoint, subscriptions, points recorded and skipped |
105
+ | `<name>/maintenance/set/loglevel` | `error` \| `warn` \| `info` \| `debug` at runtime |
106
+ | `<name>/maintenance/set/restart` | graceful restart |
107
+ | `<name>/maintenance/stats` | memory, cpu, event loop lag, uptime |
108
+
109
+ A database that is unreachable shows up as `<name>/connected 1`, so a broken recorder is visible
110
+ without reading logs.
111
+
14
112
  ## License
15
113
 
16
- MIT
114
+ MIT © [Sebastian Raff](https://github.com/hobbyquaker)
17
115
 
116
+ [mit-badge]: https://img.shields.io/badge/License-MIT-blue.svg?style=flat
117
+ [mit-url]: LICENSE
package/config.js ADDED
@@ -0,0 +1,108 @@
1
+ /**
2
+ * Adapter options on top of the core's parseConfig(): the shared MQTT / name / maintenance
3
+ * options, INFLUX4MQTT_* environment variables and --config-schema come from
4
+ * mqtt-interfaces-core; only what is specific to recording into InfluxDB is defined here.
5
+ */
6
+
7
+ import os from 'node:os';
8
+ import {parseConfig} from 'mqtt-interfaces-core';
9
+ import pkg from './package.json' with {type: 'json'};
10
+
11
+ export const OPTIONS = {
12
+ subscribe: {
13
+ alias: 's',
14
+ type: 'array',
15
+ describe: 'mqtt topic to record, with + and # wildcards (repeatable)',
16
+ default: ['+/status/#'],
17
+ },
18
+ 'influx-url': {
19
+ type: 'string',
20
+ describe: 'influxdb base url',
21
+ default: 'http://127.0.0.1:8086',
22
+ },
23
+ 'influx-version': {
24
+ type: 'number',
25
+ describe: 'influxdb api: 1 (/write?db=) or 2 (/api/v2/write, token auth)',
26
+ choices: [1, 2],
27
+ default: 1,
28
+ },
29
+ 'influx-db': {
30
+ alias: 'd',
31
+ type: 'string',
32
+ describe: 'database (api 1) or bucket (api 2)',
33
+ default: 'mqtt',
34
+ },
35
+ 'influx-org': {
36
+ type: 'string',
37
+ describe: 'influxdb organisation (api 2)',
38
+ },
39
+ 'influx-token': {
40
+ type: 'string',
41
+ describe: 'influxdb api token (api 2; also accepted by influxdb 1.8+)',
42
+ secret: true,
43
+ },
44
+ 'influx-username': {
45
+ type: 'string',
46
+ describe: 'influxdb username (api 1, http basic auth)',
47
+ },
48
+ 'influx-password': {
49
+ type: 'string',
50
+ describe: 'influxdb password (api 1)',
51
+ secret: true,
52
+ },
53
+ 'shorten-status': {
54
+ type: 'boolean',
55
+ describe: 'record hm/status/lamp as hm//lamp — the mqtt-smarthome convention',
56
+ default: true,
57
+ },
58
+ 'replace-sys': {
59
+ type: 'string',
60
+ describe: '$SYS/ topics are the same on every broker: rewrite the prefix to tell them apart',
61
+ default: `$SYS/${os.hostname()}/`,
62
+ },
63
+ 'buf-length': {
64
+ type: 'number',
65
+ describe: 'write to influxdb once this many points are buffered',
66
+ default: 1000,
67
+ },
68
+ 'buf-interval': {
69
+ type: 'number',
70
+ describe: 'seconds between writes, however few points are buffered',
71
+ default: 30,
72
+ },
73
+ };
74
+
75
+ /** yargs .check(): value ranges the option types cannot express. */
76
+ export function check(argv) {
77
+ if (!(argv.bufLength >= 1)) {
78
+ throw new Error('--buf-length must be >= 1');
79
+ }
80
+ if (!(argv.bufInterval >= 1)) {
81
+ throw new Error('--buf-interval must be >= 1 second');
82
+ }
83
+ if (Number(argv.influxVersion) === 2 && !argv.influxToken) {
84
+ throw new Error('--influx-version 2 needs --influx-token');
85
+ }
86
+ if (!/^https?:\/\//.test(String(argv.influxUrl))) {
87
+ throw new Error('--influx-url must start with http:// or https://');
88
+ }
89
+ for (const topic of argv.subscribe || []) {
90
+ if (!topic || /#.+/.test(topic)) {
91
+ throw new Error(`--subscribe ${topic}: # is only allowed as the last level`);
92
+ }
93
+ }
94
+ return true;
95
+ }
96
+
97
+ export default parseConfig({
98
+ pkg,
99
+ options: OPTIONS,
100
+ defaults: {name: 'influx'},
101
+ check,
102
+ examples: [
103
+ ['$0 -u mqtt://broker', 'record every mqtt-smarthome status topic'],
104
+ ['$0 -u mqtt://broker -s "+/status/#" -s "zigbee2mqtt/+/+"', 'record two topic patterns'],
105
+ ['$0 --influx-version 2 --influx-org home --influx-db mqtt --influx-token …', 'influxdb 2'],
106
+ ['sudo $0 --install -n influx -u mqtt://broker', 'install as service influx4mqtt@influx'],
107
+ ],
108
+ });
package/index.js CHANGED
@@ -1,131 +1,87 @@
1
1
  #!/usr/bin/env node
2
- var pkg = require('./package.json');
3
- var config = require('yargs')
4
- .usage(pkg.name + ' ' + pkg.version + '\n' + pkg.description + '\n\nUsage: $0 [options]')
5
- .describe('v', 'possible values: "error", "warn", "info", "debug"')
6
- .describe('n', 'instance name. used as mqtt client id and as prefix for connection-state topic')
7
- .describe('u', 'mqtt broker url. See https://github.com/mqttjs/MQTT.js#connect-using-a-url')
8
- .describe('h', 'show help')
9
- .alias({
10
- 'c': 'config',
11
- 'h': 'help',
12
- 'n': 'name',
13
- 'u': 'url',
14
- 'v': 'verbosity',
15
- 'i': 'influx-host',
16
- 'p': 'influx-port',
17
- 'd': 'influx-db'
18
2
 
19
- })
20
- .default({
21
- 'u': 'mqtt://127.0.0.1',
22
- 'n': 'influx',
23
- 'v': 'info',
24
- 'influx': false,
25
- 'influx-host': '127.0.0.1',
26
- 'influx-port': 8086,
27
- 'influx-db': 'mqtt'
28
- })
29
- .config('config')
30
- .version(pkg.name + ' ' + pkg.version + '\n', 'version')
31
- .help('help')
32
- .argv;
33
-
34
- console.log('mqtt connecting', config.url);
35
- var mqtt = require('mqtt').connect(config.url, {will: {topic: config.name + '/connected', payload: '0'}});
36
- mqtt.publish(config.name + '/connected', '2');
37
-
38
- var subscriptions = [ // Todo command line param
39
- '+/status/#',
40
- '+/connected'
41
- ];
42
-
43
- console.log('connecting InfluxDB', config['influx-host']);
44
- var influx = require('influx')({
45
- host: config['influx-host'] || '127.0.0.1',
46
- port: config['influx-port'] || 8086, // optional, default 8086
47
- protocol: 'http', // optional, default 'http' // todo command line param
48
- //username: 'dbuser', // todo command line param
49
- //password: 'f4ncyp4ass', // todo command line param
50
- database: config['influx-db'] || 'mqtt'
51
- });
52
-
53
- var buffer = {};
54
- var bufferCount = 0;
55
-
56
- var connected;
57
- mqtt.on('connect', function () {
58
- connected = true;
59
- console.log('mqtt connected ' + config.url);
60
-
61
- subscriptions.forEach(function (subs) {
62
- console.log('mqtt subscribe ' + subs);
63
- mqtt.subscribe(subs);
64
- });
65
- });
66
-
67
-
68
- mqtt.on('close', function () {
69
- if (connected) {
70
- connected = false;
71
- console.log('mqtt closed ' + config.url);
3
+ /**
4
+ * influx4mqtt — record MQTT values in InfluxDB, on mqtt-interfaces-core.
5
+ *
6
+ * A *sink*: unlike a device adapter it has no device of its own, its subject is what every other
7
+ * adapter publishes. The core's `listen` gives it the subscriptions (absolute topics, not under
8
+ * `<name>/`); lib/line.js turns a message into a point and lib/influx.js writes them in batches.
9
+ *
10
+ * `<name>/connected` reports 2 while InfluxDB is accepting writes and 1 while it is not, so a
11
+ * database that is down is visible on the broker rather than only in the log.
12
+ */
13
+
14
+ import {createAdapter} from 'mqtt-interfaces-core';
15
+ import config from './config.js';
16
+ import pkg from './package.json' with {type: 'json'};
17
+ import {handle as handleInstall} from './lib/install.js';
18
+ import {seriesName, point} from './lib/line.js';
19
+ import {createWriter} from './lib/influx.js';
20
+
21
+ handleInstall(config); // --install / --uninstall never reach the rest
22
+
23
+ let writer;
24
+ let recorded = 0;
25
+ let skipped = 0;
26
+
27
+ /**
28
+ * One message -> at most one point.
29
+ *
30
+ * Retained messages are ignored: on every (re)connect the broker replays the retained value of
31
+ * every matching topic, and recording those would put a burst of duplicates at the reconnect time
32
+ * rather than at the time the value was actually measured.
33
+ */
34
+ function record(topic, value, raw, packet) {
35
+ if (packet.retain) {
36
+ return;
72
37
  }
73
- });
74
-
75
- mqtt.on('error', function () {
76
- console.error('mqtt error ' + config.url);
77
- });
78
-
79
-
80
- mqtt.on('message', function (topic, payload, msg) {
81
-
82
- if (msg.retain) return;
83
-
84
- var timestamp = (new Date()).getTime();
85
-
86
- payload = payload.toString();
87
-
88
- var seriesName = topic.replace(/^([^\/]+)\/status\/(.+)/, '$1//$2');
89
-
90
- var value;
91
-
92
- try {
93
- var tmp = JSON.parse(payload);
94
- value = tmp.val;
95
- timestamp = tmp.ts || timestamp;
96
- } catch (e) {
97
- value = payload;
38
+ // {val, ts, lc} payloads carry the device's own time; parsePayload gave us the value, the
39
+ // timestamp has to come out of the raw payload
40
+ let timestamp = Date.now();
41
+ if (raw.includes('{')) {
42
+ try {
43
+ const json = JSON.parse(raw);
44
+ if (json && typeof json === 'object' && Number(json.ts) > 0) {
45
+ timestamp = Number(json.ts);
46
+ }
47
+ } catch {
48
+ // not JSON after all; our clock it is
49
+ }
98
50
  }
99
- var valueFloat = parseFloat(value);
100
-
101
- if (value === true || value === 'true') {
102
- value = '1.0';
103
- } else if (value === false || value === 'false') {
104
- value = '0.0';
105
- } else if (isNaN(valueFloat)) {
106
- return; // FIXME do we need strings? Creating a field as string leads to errors when trying to write float on it. Can we expect topics to be of the same type always?
107
- value = '"' + value + '"';
108
- } else {
109
- value = '' + valueFloat;
110
- if (!value.match(/\./)) value = value + '.0';
51
+ const series = seriesName(topic, {shortenStatus: config.shortenStatus, replaceSys: config.replaceSys});
52
+ const line = point(series, value, timestamp);
53
+ if (!line) {
54
+ skipped++; // text, or a payload that is not a number - not a time series
55
+ return;
111
56
  }
57
+ recorded++;
58
+ writer.add(line);
59
+ }
112
60
 
113
- //console.log(seriesName, value, timestamp, tmp.ts);
114
- if (!buffer[seriesName]) buffer[seriesName] = [];
115
- buffer[seriesName].push([{value: value, time: timestamp}]);
116
- bufferCount += 1;
117
- if (bufferCount > 1000) write(); // todo command line param
118
-
61
+ const adapter = createAdapter({
62
+ pkg,
63
+ config,
64
+ deviceLabel: 'influx',
65
+ info: () => ({
66
+ influx: writer ? writer.url : undefined,
67
+ api: config.influxVersion,
68
+ db: config.influxDb,
69
+ subscribe: config.subscribe,
70
+ buffered: writer ? writer.size : 0,
71
+ recorded,
72
+ skipped,
73
+ }),
74
+ // a sink: the topics are the user's, anywhere on the broker, not under <name>/
75
+ listen: Object.fromEntries(config.subscribe.map((topic) => [topic, record])),
76
+ onShutdown: () => writer.stop(),
119
77
  });
120
78
 
121
- function write() {
122
- if (!bufferCount) return;
123
- //console.log('write', bufferCount);
124
- influx.writeSeries(buffer, {}, function (err, res) {
125
- if (err) console.error('error', err);
126
- });
127
- buffer = {};
128
- bufferCount = 0;
129
- }
79
+ writer = createWriter(config, {
80
+ log: adapter.log,
81
+ // <name>/connected 2 while influx accepts writes, 1 while it does not
82
+ onState: (up) => adapter.setDeviceConnected(up),
83
+ });
130
84
 
131
- setInterval(write, 30000); // todo command line param
85
+ adapter.log.info(`influxdb api ${config.influxVersion} at ${writer.url}`);
86
+ writer.start();
87
+ adapter.start();
package/lib/influx.js ADDED
@@ -0,0 +1,145 @@
1
+ /**
2
+ * The InfluxDB write client: where the points go and how the request is authenticated.
3
+ *
4
+ * InfluxDB 1 and 2 speak the same line protocol — only the endpoint and the auth differ, which is
5
+ * why one adapter can serve both and `--influx-version` is all it takes:
6
+ *
7
+ * v1 POST <url>/write?db=<db>&precision=ns optional HTTP basic auth
8
+ * v2 POST <url>/api/v2/write?org=<org>&bucket=<db>&precision=ns Authorization: Token <token>
9
+ *
10
+ * InfluxDB 2 also serves the v1 endpoint for compatibility, so `--influx-version 1` against a v2
11
+ * server works if a v1 mapping was set up there; `2` is the native path and the one to prefer.
12
+ *
13
+ * Points are buffered because one HTTP request per MQTT message would be absurd at the rate a
14
+ * smart home publishes. The buffer is flushed when it is full, on a timer, and on shutdown — the
15
+ * last one is why this takes the adapter's lifecycle rather than a bare `setInterval`.
16
+ */
17
+
18
+ /** The URL a write goes to, query string included. */
19
+ export function writeUrl({influxUrl, influxVersion = 1, influxDb, influxOrg}) {
20
+ const base = String(influxUrl).replace(/\/+$/, '');
21
+ const q = new URLSearchParams({precision: 'ns'});
22
+ if (Number(influxVersion) === 2) {
23
+ q.set('bucket', influxDb);
24
+ if (influxOrg) {
25
+ q.set('org', influxOrg);
26
+ }
27
+ return `${base}/api/v2/write?${q}`;
28
+ }
29
+ q.set('db', influxDb);
30
+ return `${base}/write?${q}`;
31
+ }
32
+
33
+ /** Auth and content headers for a write. The api version does not matter here — a token is
34
+ * accepted by both, so only which credential was given decides the scheme. */
35
+ export function writeHeaders({influxToken, influxUsername, influxPassword}) {
36
+ const headers = {'content-type': 'text/plain; charset=utf-8'};
37
+ if (influxToken) {
38
+ // v2's scheme, and accepted by v1.8+ as well
39
+ headers.authorization = `Token ${influxToken}`;
40
+ } else if (influxUsername) {
41
+ const basic = Buffer.from(`${influxUsername}:${influxPassword || ''}`).toString('base64');
42
+ headers.authorization = `Basic ${basic}`;
43
+ }
44
+ return headers;
45
+ }
46
+
47
+ /**
48
+ * A buffer of line-protocol points that writes itself out.
49
+ *
50
+ * @param {object} config parsed config (influxUrl, influxVersion, influxDb, influxOrg, credentials,
51
+ * bufLength, bufInterval)
52
+ * @param {{log?: object, fetchImpl?: Function, onState?: (up: boolean) => void}} [deps]
53
+ * `onState` reports reachability, so the adapter can put it in <name>/connected
54
+ */
55
+ export function createWriter(config, {log = console, fetchImpl = globalThis.fetch, onState} = {}) {
56
+ const url = writeUrl(config);
57
+ const headers = writeHeaders(config);
58
+ const max = Number(config.bufLength) > 0 ? Number(config.bufLength) : 1000;
59
+ let buffer = [];
60
+ let timer = null;
61
+ let up = null;
62
+ let inFlight = null;
63
+
64
+ function state(next) {
65
+ if (next !== up) {
66
+ up = next;
67
+ if (onState) {
68
+ onState(next);
69
+ }
70
+ }
71
+ }
72
+
73
+ async function send(lines) {
74
+ const body = lines.join('\n');
75
+ let response;
76
+ try {
77
+ response = await fetchImpl(url, {method: 'POST', headers, body});
78
+ } catch (error) {
79
+ // unreachable: the points are gone. Keeping them would grow without bound while a
80
+ // database is down for a day, and a gap is the honest outcome for a time series.
81
+ log.warn('influx unreachable —', error.message, `(${lines.length} points dropped)`);
82
+ state(false);
83
+ return false;
84
+ }
85
+ if (response.status === 204 || response.status === 200) {
86
+ log.debug('influx wrote', lines.length, 'points');
87
+ state(true);
88
+ return true;
89
+ }
90
+ let detail = '';
91
+ try {
92
+ detail = (await response.text()).slice(0, 300);
93
+ } catch {
94
+ // a body we cannot read changes nothing about the status
95
+ }
96
+ // 4xx is our own line protocol being wrong: the same points would fail again, so they go
97
+ log.error('influx write failed', response.status, detail);
98
+ state(response.status < 500);
99
+ return false;
100
+ }
101
+
102
+ /** Write everything buffered. Serialised: two overlapping writes could reorder points. */
103
+ function flush() {
104
+ if (buffer.length === 0) {
105
+ return inFlight || Promise.resolve(true);
106
+ }
107
+ const lines = buffer;
108
+ buffer = [];
109
+ inFlight = Promise.resolve(inFlight)
110
+ .catch(() => {})
111
+ .then(() => send(lines));
112
+ return inFlight;
113
+ }
114
+
115
+ return {
116
+ /** Buffer one point; writes early when the buffer is full. */
117
+ add(line) {
118
+ buffer.push(line);
119
+ if (buffer.length >= max) {
120
+ flush();
121
+ }
122
+ },
123
+ flush,
124
+ get size() {
125
+ return buffer.length;
126
+ },
127
+ get url() {
128
+ return url;
129
+ },
130
+ start() {
131
+ const seconds = Number(config.bufInterval) > 0 ? Number(config.bufInterval) : 30;
132
+ timer = setInterval(flush, seconds * 1000);
133
+ if (timer.unref) {
134
+ timer.unref(); // the mqtt connection keeps the process alive, not this
135
+ }
136
+ },
137
+ async stop() {
138
+ if (timer) {
139
+ clearInterval(timer);
140
+ timer = null;
141
+ }
142
+ await flush(); // whatever is buffered on SIGTERM would otherwise be lost
143
+ },
144
+ };
145
+ }
package/lib/install.js ADDED
@@ -0,0 +1,18 @@
1
+ /**
2
+ * --install / --uninstall: systemd template service influx4mqtt@<name> (mqtt-interfaces-core
3
+ * installer). Nothing privileged — it talks to two network services and keeps no state.
4
+ */
5
+
6
+ import {createInstaller} from 'mqtt-interfaces-core';
7
+
8
+ export const SERVICE = 'influx4mqtt';
9
+ export const ENV_PREFIX = 'INFLUX4MQTT';
10
+
11
+ const installer = createInstaller({
12
+ service: SERVICE,
13
+ envPrefix: ENV_PREFIX,
14
+ description: `${SERVICE} %i - MQTT to InfluxDB recorder`,
15
+ documentation: 'https://github.com/hobbyquaker/influx4mqtt',
16
+ });
17
+
18
+ export const {unitFile, envFile, installService, uninstallService, handle} = installer;
package/lib/line.js ADDED
@@ -0,0 +1,107 @@
1
+ /**
2
+ * Topic and payload → one InfluxDB line-protocol point. Pure: no sockets, no config object, so
3
+ * every rule below is testable on its own.
4
+ *
5
+ * The measurement is the topic, lightly rewritten (see `seriesName`); the only field is `value`,
6
+ * always a float, because a series whose type changes is a series Influx will refuse to write.
7
+ */
8
+
9
+ /** Line protocol escapes in a measurement name: spaces and commas separate tokens. */
10
+ export function escapeMeasurement(name) {
11
+ return name.replace(/,/g, '\\,').replace(/ /g, '\\ ');
12
+ }
13
+
14
+ /**
15
+ * The series a topic is recorded under.
16
+ *
17
+ * `shortenStatus` collapses the mqtt-smarthome `status` level — `hm/status/lamp` becomes
18
+ * `hm//lamp`. It has been the behaviour since 1.0 and stays the default: turning it off renames
19
+ * every existing series and splits the history in Grafana. New setups may prefer the full topic.
20
+ *
21
+ * `replaceSys` rewrites the leading `$SYS/`, whose topics are otherwise identical on every broker
22
+ * — recording two of them into one database would interleave them into nonsense.
23
+ *
24
+ * @param {string} topic
25
+ * @param {{shortenStatus?: boolean, replaceSys?: string}} [options]
26
+ */
27
+ export function seriesName(topic, {shortenStatus = true, replaceSys} = {}) {
28
+ let name = topic;
29
+ if (shortenStatus) {
30
+ name = name.replace(/^([^/]+)\/status\/(.+)$/, '$1//$2');
31
+ }
32
+ if (replaceSys) {
33
+ name = name.replace(/^\$SYS\//, replaceSys);
34
+ }
35
+ return name;
36
+ }
37
+
38
+ /** Payloads that mean true / false rather than a number. Case-insensitive since 2.0. */
39
+ const TRUE = new Set(['true', 'on', 'yes']);
40
+ const FALSE = new Set(['false', 'off', 'no']);
41
+
42
+ /**
43
+ * A value → the float Influx stores, as a string, or `null` for anything that is not a number.
44
+ *
45
+ * Booleans and the strings devices use for them become `1`/`0`; everything else has to parse as a
46
+ * number. Text is dropped rather than stored — this is a numeric time series, and a string field
47
+ * would fix the series' type to string forever.
48
+ *
49
+ * The trailing `.0` is deliberate: without it Influx infers an integer field, and a series that
50
+ * starts out integer rejects the first float that arrives.
51
+ *
52
+ * @returns {string|null}
53
+ */
54
+ export function toFloat(value) {
55
+ if (value === true) {
56
+ return '1.0';
57
+ }
58
+ if (value === false) {
59
+ return '0.0';
60
+ }
61
+ let number;
62
+ if (typeof value === 'number') {
63
+ number = value;
64
+ } else if (typeof value === 'string') {
65
+ const trimmed = value.trim();
66
+ const lower = trimmed.toLowerCase();
67
+ if (TRUE.has(lower)) {
68
+ return '1.0';
69
+ }
70
+ if (FALSE.has(lower)) {
71
+ return '0.0';
72
+ }
73
+ if (trimmed === '') {
74
+ return null;
75
+ }
76
+ /*
77
+ * Number(), not parseFloat(): parseFloat('21 °C') is 21, and recording that as a
78
+ * temperature is worse than recording nothing — it looks like a real measurement. The
79
+ * empty string is handled above because Number('') is 0, not NaN.
80
+ */
81
+ number = Number(trimmed);
82
+ } else {
83
+ // objects, null, undefined: Number(null) is 0 and Number([5]) is 5, so never fall through
84
+ return null;
85
+ }
86
+ if (!Number.isFinite(number)) {
87
+ return null;
88
+ }
89
+ const text = String(number);
90
+ return /[.e]/i.test(text) ? text : text + '.0';
91
+ }
92
+
93
+ /**
94
+ * One line-protocol point, or `null` when the value is not numeric.
95
+ *
96
+ * @param {string} series measurement name (already rewritten by `seriesName`)
97
+ * @param {*} value
98
+ * @param {number} timestampMs
99
+ */
100
+ export function point(series, value, timestampMs) {
101
+ const float = toFloat(value);
102
+ if (float === null || !series) {
103
+ return null;
104
+ }
105
+ // influx takes nanoseconds; the timestamp is milliseconds, from the payload or our clock
106
+ return `${escapeMeasurement(series)} value=${float} ${Math.round(timestampMs) * 1000000}`;
107
+ }
package/package.json CHANGED
@@ -1,34 +1,64 @@
1
1
  {
2
2
  "name": "influx4mqtt",
3
- "version": "0.0.1",
4
- "description": "Insert incoming MQTT values into InfluxDB. Follows mqtt-smarthome architecture.",
3
+ "version": "2.0.0",
4
+ "description": "Record MQTT values in InfluxDB. Follows the mqtt-smarthome architecture.",
5
+ "type": "module",
5
6
  "main": "index.js",
6
- "scripts": {
7
- "test": "echo \"Error: no test specified\" && exit 1"
8
- },
9
- "author": "Sebastian 'hobbyquaker' Raff <hq@ccu.io>",
10
- "license": "MIT",
11
- "dependencies": {
12
- "influx": "git://github.com/hobbyquaker/node-influx",
13
- "mqtt": "^1.3.3",
14
- "yargs": "^3.14.0"
15
- },
16
7
  "bin": {
17
- "influx4mqtt": "./index.js"
8
+ "influx4mqtt": "index.js"
18
9
  },
19
10
  "preferGlobal": true,
11
+ "files": [
12
+ "index.js",
13
+ "config.js",
14
+ "lib/"
15
+ ],
16
+ "engines": {
17
+ "node": "^20.19 || ^22.12 || >=24"
18
+ },
19
+ "scripts": {
20
+ "start": "node index.js",
21
+ "lint": "eslint . && prettier --check .",
22
+ "format": "prettier --write . && eslint --fix .",
23
+ "test": "node --test",
24
+ "deploy": "bash deploy.sh"
25
+ },
20
26
  "repository": {
21
27
  "type": "git",
22
- "url": "https://github.com/hobbyquaker/influx4mqtt"
28
+ "url": "git+https://github.com/hobbyquaker/influx4mqtt.git"
29
+ },
30
+ "homepage": "https://github.com/hobbyquaker/influx4mqtt",
31
+ "bugs": {
32
+ "url": "https://github.com/hobbyquaker/influx4mqtt/issues"
23
33
  },
34
+ "author": "Sebastian Raff <hobbyquaker@gmail.com> (https://github.com/hobbyquaker)",
35
+ "license": "MIT",
24
36
  "keywords": [
25
37
  "mqtt",
26
- "smarthome",
27
- "history",
38
+ "mqtt-smarthome",
39
+ "home-automation",
28
40
  "influxdb",
29
- "charts",
41
+ "influx",
30
42
  "grafana",
31
- "time",
32
- "series"
33
- ]
43
+ "time series",
44
+ "history"
45
+ ],
46
+ "mqttInterfaces": {
47
+ "spec": "2.0",
48
+ "envPrefix": "INFLUX4MQTT",
49
+ "needs": [
50
+ "network"
51
+ ],
52
+ "serviceExtra": []
53
+ },
54
+ "dependencies": {
55
+ "mqtt-interfaces-core": "^0.13.0"
56
+ },
57
+ "devDependencies": {
58
+ "@eslint/js": "^9",
59
+ "eslint": "^9",
60
+ "eslint-config-prettier": "^10",
61
+ "globals": "^16",
62
+ "prettier": "^3"
63
+ }
34
64
  }
package/.npmignore DELETED
@@ -1,3 +0,0 @@
1
- .idea
2
- node_modules
3
- .DS_Store