influx4mqtt 0.0.1 → 2.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +106 -6
- package/config.js +108 -0
- package/index.js +78 -122
- package/lib/influx.js +145 -0
- package/lib/install.js +18 -0
- package/lib/line.js +107 -0
- package/package.json +50 -20
- package/.npmignore +0 -3
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
The MIT License (MIT)
|
|
2
|
+
|
|
3
|
+
Copyright (c) Sebastian Raff <hobbyquaker@gmail.com> (https://hobbyquaker.github.io)
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in
|
|
13
|
+
all copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
|
21
|
+
THE SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,17 +1,117 @@
|
|
|
1
1
|
# influx4mqtt
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
[](https://github.com/mqtt-smarthome/mqtt-smarthome)
|
|
4
|
+
[](http://badge.fury.io/js/influx4mqtt)
|
|
5
|
+
[](https://github.com/hobbyquaker/influx4mqtt/actions/workflows/ci.yml)
|
|
6
|
+
[![License][mit-badge]][mit-url]
|
|
4
7
|
|
|
5
|
-
|
|
8
|
+
Record MQTT values in InfluxDB — the history behind a Grafana dashboard.
|
|
6
9
|
|
|
7
|
-
|
|
10
|
+
Subscribes to the topics you name, turns each message into one InfluxDB point and writes them in
|
|
11
|
+
batches. Numeric values only: booleans and `ON`/`OFF` become `1`/`0`, text is ignored, because a
|
|
12
|
+
time series with a string in it is a series InfluxDB will not let you graph.
|
|
13
|
+
|
|
14
|
+
Built on [mqtt-interfaces-core](https://github.com/hobbyquaker/mqtt-interfaces-core), so it has the
|
|
15
|
+
same options, `--install`, `<name>/info` and maintenance topics as the rest of the `xyz2mqtt` fleet.
|
|
16
|
+
Works with **InfluxDB 1 and 2**.
|
|
17
|
+
|
|
18
|
+
## Install
|
|
19
|
+
|
|
20
|
+
```
|
|
21
|
+
npm install -g influx4mqtt
|
|
22
|
+
influx4mqtt --mqtt-url mqtt://broker # foreground
|
|
23
|
+
sudo influx4mqtt --install -n influx -u mqtt://broker # systemd service influx4mqtt@influx
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
`--install` writes the options to `/etc/influx4mqtt/<name>.env` and enables
|
|
27
|
+
`influx4mqtt@<name>.service`. Broker settings shared with the other adapters on the host live in
|
|
28
|
+
`/etc/mqtt-interfaces/broker.env`.
|
|
29
|
+
|
|
30
|
+
### Docker
|
|
31
|
+
|
|
32
|
+
Multi-arch image (amd64, arm64, armv7):
|
|
8
33
|
|
|
9
34
|
```
|
|
10
|
-
|
|
11
|
-
|
|
35
|
+
docker run -d --name influx4mqtt --restart unless-stopped \
|
|
36
|
+
-e INFLUX4MQTT_MQTT_URL=mqtt://broker \
|
|
37
|
+
-e INFLUX4MQTT_INFLUX_URL=http://influxdb:8086 \
|
|
38
|
+
-e INFLUX4MQTT_SUBSCRIBE='+/status/#,$SYS/#' \
|
|
39
|
+
ghcr.io/hobbyquaker/influx4mqtt
|
|
12
40
|
```
|
|
13
41
|
|
|
42
|
+
## What gets recorded
|
|
43
|
+
|
|
44
|
+
Every message on a subscribed topic becomes one point:
|
|
45
|
+
|
|
46
|
+
| topic | payload | series | value |
|
|
47
|
+
| -------------------------- | --------------------- | -------------------------- | --------------------------- |
|
|
48
|
+
| `hm/status/Licht/STATE` | `21.5` | `hm//Licht/STATE` | `21.5` |
|
|
49
|
+
| `hm/status/Licht/STATE` | `{"val":21.5,"ts":…}` | `hm//Licht/STATE` | `21.5` at the device's `ts` |
|
|
50
|
+
| `zigbee2mqtt/sensor/state` | `ON` | `zigbee2mqtt/sensor/state` | `1` |
|
|
51
|
+
| `hm/status/mode` | `auto` | — | not recorded |
|
|
52
|
+
|
|
53
|
+
- **The `status` level is collapsed**: `hm/status/lamp` is recorded as `hm//lamp`, the
|
|
54
|
+
mqtt-smarthome convention. `--no-shorten-status` records the full topic instead — but only do
|
|
55
|
+
that on a fresh database, since it renames every series you already have.
|
|
56
|
+
- **`{val, ts, lc}` payloads** are recorded at the device's own timestamp, not at arrival time.
|
|
57
|
+
- **Retained messages are ignored.** On every reconnect the broker replays the retained value of
|
|
58
|
+
every matching topic; recording those would pile duplicates at the reconnect time rather than
|
|
59
|
+
when the value was measured.
|
|
60
|
+
- **`$SYS/` is rewritten** to `$SYS/<hostname>/` by default, because `$SYS` topics are identical on
|
|
61
|
+
every broker and two of them in one database would interleave into nonsense. `--replace-sys`
|
|
62
|
+
chooses the prefix.
|
|
63
|
+
|
|
64
|
+
## Options
|
|
65
|
+
|
|
66
|
+
| Option | Default | Meaning |
|
|
67
|
+
| ------------------- | ----------------------- | ------------------------------------------------------- |
|
|
68
|
+
| `-s, --subscribe` | `+/status/#` | topic to record, `+`/`#` wildcards; repeat for more |
|
|
69
|
+
| `-u, --mqtt-url` | `mqtt://localhost` | broker url |
|
|
70
|
+
| `-n, --name` | `influx` | instance name = topic prefix of its own topics |
|
|
71
|
+
| `--influx-url` | `http://127.0.0.1:8086` | InfluxDB base url |
|
|
72
|
+
| `--influx-version` | `1` | `1` (`/write?db=`) or `2` (`/api/v2/write`, token auth) |
|
|
73
|
+
| `-d, --influx-db` | `mqtt` | database (api 1) or bucket (api 2) |
|
|
74
|
+
| `--influx-org` | | organisation (api 2) |
|
|
75
|
+
| `--influx-token` | | api token (api 2; InfluxDB 1.8+ accepts it too) |
|
|
76
|
+
| `--influx-username` | | http basic auth (api 1) |
|
|
77
|
+
| `--influx-password` | | http basic auth (api 1) |
|
|
78
|
+
| `--shorten-status` | `true` | record `hm/status/lamp` as `hm//lamp` |
|
|
79
|
+
| `--replace-sys` | `$SYS/<hostname>/` | rewrite the `$SYS/` prefix |
|
|
80
|
+
| `--buf-length` | `1000` | write once this many points are buffered |
|
|
81
|
+
| `--buf-interval` | `30` | seconds between writes, however few points are buffered |
|
|
82
|
+
| `-v, --verbosity` | `info` | `error`, `warn`, `info`, `debug` |
|
|
83
|
+
|
|
84
|
+
Every option is also an environment variable (`INFLUX4MQTT_SUBSCRIBE`, `INFLUX4MQTT_INFLUX_URL`, …);
|
|
85
|
+
several subscriptions are comma separated there. `--help` lists the shared options too, and
|
|
86
|
+
`--config-schema` prints the JSON Schema a management UI reads.
|
|
87
|
+
|
|
88
|
+
### InfluxDB 2
|
|
89
|
+
|
|
90
|
+
```
|
|
91
|
+
influx4mqtt -u mqtt://broker \
|
|
92
|
+
--influx-version 2 --influx-url http://influxdb:8086 \
|
|
93
|
+
--influx-org home --influx-db mqtt --influx-token <token>
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
`--influx-db` is the bucket. Put the token in the instance's env file rather than on the command
|
|
97
|
+
line, where a process list would show it — `--install` does that for you.
|
|
98
|
+
|
|
99
|
+
## Topics of its own
|
|
100
|
+
|
|
101
|
+
| Topic | Meaning |
|
|
102
|
+
| --------------------------------- | ---------------------------------------------------------------- |
|
|
103
|
+
| `<name>/connected` | `2` InfluxDB is accepting writes · `1` broker only · `0` stopped |
|
|
104
|
+
| `<name>/info` | endpoint, subscriptions, points recorded and skipped |
|
|
105
|
+
| `<name>/maintenance/set/loglevel` | `error` \| `warn` \| `info` \| `debug` at runtime |
|
|
106
|
+
| `<name>/maintenance/set/restart` | graceful restart |
|
|
107
|
+
| `<name>/maintenance/stats` | memory, cpu, event loop lag, uptime |
|
|
108
|
+
|
|
109
|
+
A database that is unreachable shows up as `<name>/connected 1`, so a broken recorder is visible
|
|
110
|
+
without reading logs.
|
|
111
|
+
|
|
14
112
|
## License
|
|
15
113
|
|
|
16
|
-
MIT
|
|
114
|
+
MIT © [Sebastian Raff](https://github.com/hobbyquaker)
|
|
17
115
|
|
|
116
|
+
[mit-badge]: https://img.shields.io/badge/License-MIT-blue.svg?style=flat
|
|
117
|
+
[mit-url]: LICENSE
|
package/config.js
ADDED
|
@@ -0,0 +1,108 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Adapter options on top of the core's parseConfig(): the shared MQTT / name / maintenance
|
|
3
|
+
* options, INFLUX4MQTT_* environment variables and --config-schema come from
|
|
4
|
+
* mqtt-interfaces-core; only what is specific to recording into InfluxDB is defined here.
|
|
5
|
+
*/
|
|
6
|
+
|
|
7
|
+
import os from 'node:os';
|
|
8
|
+
import {parseConfig} from 'mqtt-interfaces-core';
|
|
9
|
+
import pkg from './package.json' with {type: 'json'};
|
|
10
|
+
|
|
11
|
+
export const OPTIONS = {
|
|
12
|
+
subscribe: {
|
|
13
|
+
alias: 's',
|
|
14
|
+
type: 'array',
|
|
15
|
+
describe: 'mqtt topic to record, with + and # wildcards (repeatable)',
|
|
16
|
+
default: ['+/status/#'],
|
|
17
|
+
},
|
|
18
|
+
'influx-url': {
|
|
19
|
+
type: 'string',
|
|
20
|
+
describe: 'influxdb base url',
|
|
21
|
+
default: 'http://127.0.0.1:8086',
|
|
22
|
+
},
|
|
23
|
+
'influx-version': {
|
|
24
|
+
type: 'number',
|
|
25
|
+
describe: 'influxdb api: 1 (/write?db=) or 2 (/api/v2/write, token auth)',
|
|
26
|
+
choices: [1, 2],
|
|
27
|
+
default: 1,
|
|
28
|
+
},
|
|
29
|
+
'influx-db': {
|
|
30
|
+
alias: 'd',
|
|
31
|
+
type: 'string',
|
|
32
|
+
describe: 'database (api 1) or bucket (api 2)',
|
|
33
|
+
default: 'mqtt',
|
|
34
|
+
},
|
|
35
|
+
'influx-org': {
|
|
36
|
+
type: 'string',
|
|
37
|
+
describe: 'influxdb organisation (api 2)',
|
|
38
|
+
},
|
|
39
|
+
'influx-token': {
|
|
40
|
+
type: 'string',
|
|
41
|
+
describe: 'influxdb api token (api 2; also accepted by influxdb 1.8+)',
|
|
42
|
+
secret: true,
|
|
43
|
+
},
|
|
44
|
+
'influx-username': {
|
|
45
|
+
type: 'string',
|
|
46
|
+
describe: 'influxdb username (api 1, http basic auth)',
|
|
47
|
+
},
|
|
48
|
+
'influx-password': {
|
|
49
|
+
type: 'string',
|
|
50
|
+
describe: 'influxdb password (api 1)',
|
|
51
|
+
secret: true,
|
|
52
|
+
},
|
|
53
|
+
'shorten-status': {
|
|
54
|
+
type: 'boolean',
|
|
55
|
+
describe: 'record hm/status/lamp as hm//lamp — the mqtt-smarthome convention',
|
|
56
|
+
default: true,
|
|
57
|
+
},
|
|
58
|
+
'replace-sys': {
|
|
59
|
+
type: 'string',
|
|
60
|
+
describe: '$SYS/ topics are the same on every broker: rewrite the prefix to tell them apart',
|
|
61
|
+
default: `$SYS/${os.hostname()}/`,
|
|
62
|
+
},
|
|
63
|
+
'buf-length': {
|
|
64
|
+
type: 'number',
|
|
65
|
+
describe: 'write to influxdb once this many points are buffered',
|
|
66
|
+
default: 1000,
|
|
67
|
+
},
|
|
68
|
+
'buf-interval': {
|
|
69
|
+
type: 'number',
|
|
70
|
+
describe: 'seconds between writes, however few points are buffered',
|
|
71
|
+
default: 30,
|
|
72
|
+
},
|
|
73
|
+
};
|
|
74
|
+
|
|
75
|
+
/** yargs .check(): value ranges the option types cannot express. */
|
|
76
|
+
export function check(argv) {
|
|
77
|
+
if (!(argv.bufLength >= 1)) {
|
|
78
|
+
throw new Error('--buf-length must be >= 1');
|
|
79
|
+
}
|
|
80
|
+
if (!(argv.bufInterval >= 1)) {
|
|
81
|
+
throw new Error('--buf-interval must be >= 1 second');
|
|
82
|
+
}
|
|
83
|
+
if (Number(argv.influxVersion) === 2 && !argv.influxToken) {
|
|
84
|
+
throw new Error('--influx-version 2 needs --influx-token');
|
|
85
|
+
}
|
|
86
|
+
if (!/^https?:\/\//.test(String(argv.influxUrl))) {
|
|
87
|
+
throw new Error('--influx-url must start with http:// or https://');
|
|
88
|
+
}
|
|
89
|
+
for (const topic of argv.subscribe || []) {
|
|
90
|
+
if (!topic || /#.+/.test(topic)) {
|
|
91
|
+
throw new Error(`--subscribe ${topic}: # is only allowed as the last level`);
|
|
92
|
+
}
|
|
93
|
+
}
|
|
94
|
+
return true;
|
|
95
|
+
}
|
|
96
|
+
|
|
97
|
+
export default parseConfig({
|
|
98
|
+
pkg,
|
|
99
|
+
options: OPTIONS,
|
|
100
|
+
defaults: {name: 'influx'},
|
|
101
|
+
check,
|
|
102
|
+
examples: [
|
|
103
|
+
['$0 -u mqtt://broker', 'record every mqtt-smarthome status topic'],
|
|
104
|
+
['$0 -u mqtt://broker -s "+/status/#" -s "zigbee2mqtt/+/+"', 'record two topic patterns'],
|
|
105
|
+
['$0 --influx-version 2 --influx-org home --influx-db mqtt --influx-token …', 'influxdb 2'],
|
|
106
|
+
['sudo $0 --install -n influx -u mqtt://broker', 'install as service influx4mqtt@influx'],
|
|
107
|
+
],
|
|
108
|
+
});
|
package/index.js
CHANGED
|
@@ -1,131 +1,87 @@
|
|
|
1
1
|
#!/usr/bin/env node
|
|
2
|
-
var pkg = require('./package.json');
|
|
3
|
-
var config = require('yargs')
|
|
4
|
-
.usage(pkg.name + ' ' + pkg.version + '\n' + pkg.description + '\n\nUsage: $0 [options]')
|
|
5
|
-
.describe('v', 'possible values: "error", "warn", "info", "debug"')
|
|
6
|
-
.describe('n', 'instance name. used as mqtt client id and as prefix for connection-state topic')
|
|
7
|
-
.describe('u', 'mqtt broker url. See https://github.com/mqttjs/MQTT.js#connect-using-a-url')
|
|
8
|
-
.describe('h', 'show help')
|
|
9
|
-
.alias({
|
|
10
|
-
'c': 'config',
|
|
11
|
-
'h': 'help',
|
|
12
|
-
'n': 'name',
|
|
13
|
-
'u': 'url',
|
|
14
|
-
'v': 'verbosity',
|
|
15
|
-
'i': 'influx-host',
|
|
16
|
-
'p': 'influx-port',
|
|
17
|
-
'd': 'influx-db'
|
|
18
2
|
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
var buffer = {};
|
|
54
|
-
var bufferCount = 0;
|
|
55
|
-
|
|
56
|
-
var connected;
|
|
57
|
-
mqtt.on('connect', function () {
|
|
58
|
-
connected = true;
|
|
59
|
-
console.log('mqtt connected ' + config.url);
|
|
60
|
-
|
|
61
|
-
subscriptions.forEach(function (subs) {
|
|
62
|
-
console.log('mqtt subscribe ' + subs);
|
|
63
|
-
mqtt.subscribe(subs);
|
|
64
|
-
});
|
|
65
|
-
});
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
mqtt.on('close', function () {
|
|
69
|
-
if (connected) {
|
|
70
|
-
connected = false;
|
|
71
|
-
console.log('mqtt closed ' + config.url);
|
|
3
|
+
/**
|
|
4
|
+
* influx4mqtt — record MQTT values in InfluxDB, on mqtt-interfaces-core.
|
|
5
|
+
*
|
|
6
|
+
* A *sink*: unlike a device adapter it has no device of its own, its subject is what every other
|
|
7
|
+
* adapter publishes. The core's `listen` gives it the subscriptions (absolute topics, not under
|
|
8
|
+
* `<name>/`); lib/line.js turns a message into a point and lib/influx.js writes them in batches.
|
|
9
|
+
*
|
|
10
|
+
* `<name>/connected` reports 2 while InfluxDB is accepting writes and 1 while it is not, so a
|
|
11
|
+
* database that is down is visible on the broker rather than only in the log.
|
|
12
|
+
*/
|
|
13
|
+
|
|
14
|
+
import {createAdapter} from 'mqtt-interfaces-core';
|
|
15
|
+
import config from './config.js';
|
|
16
|
+
import pkg from './package.json' with {type: 'json'};
|
|
17
|
+
import {handle as handleInstall} from './lib/install.js';
|
|
18
|
+
import {seriesName, point} from './lib/line.js';
|
|
19
|
+
import {createWriter} from './lib/influx.js';
|
|
20
|
+
|
|
21
|
+
handleInstall(config); // --install / --uninstall never reach the rest
|
|
22
|
+
|
|
23
|
+
let writer;
|
|
24
|
+
let recorded = 0;
|
|
25
|
+
let skipped = 0;
|
|
26
|
+
|
|
27
|
+
/**
|
|
28
|
+
* One message -> at most one point.
|
|
29
|
+
*
|
|
30
|
+
* Retained messages are ignored: on every (re)connect the broker replays the retained value of
|
|
31
|
+
* every matching topic, and recording those would put a burst of duplicates at the reconnect time
|
|
32
|
+
* rather than at the time the value was actually measured.
|
|
33
|
+
*/
|
|
34
|
+
function record(topic, value, raw, packet) {
|
|
35
|
+
if (packet.retain) {
|
|
36
|
+
return;
|
|
72
37
|
}
|
|
73
|
-
}
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
payload = payload.toString();
|
|
87
|
-
|
|
88
|
-
var seriesName = topic.replace(/^([^\/]+)\/status\/(.+)/, '$1//$2');
|
|
89
|
-
|
|
90
|
-
var value;
|
|
91
|
-
|
|
92
|
-
try {
|
|
93
|
-
var tmp = JSON.parse(payload);
|
|
94
|
-
value = tmp.val;
|
|
95
|
-
timestamp = tmp.ts || timestamp;
|
|
96
|
-
} catch (e) {
|
|
97
|
-
value = payload;
|
|
38
|
+
// {val, ts, lc} payloads carry the device's own time; parsePayload gave us the value, the
|
|
39
|
+
// timestamp has to come out of the raw payload
|
|
40
|
+
let timestamp = Date.now();
|
|
41
|
+
if (raw.includes('{')) {
|
|
42
|
+
try {
|
|
43
|
+
const json = JSON.parse(raw);
|
|
44
|
+
if (json && typeof json === 'object' && Number(json.ts) > 0) {
|
|
45
|
+
timestamp = Number(json.ts);
|
|
46
|
+
}
|
|
47
|
+
} catch {
|
|
48
|
+
// not JSON after all; our clock it is
|
|
49
|
+
}
|
|
98
50
|
}
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
if (
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
value = '0.0';
|
|
105
|
-
} else if (isNaN(valueFloat)) {
|
|
106
|
-
return; // FIXME do we need strings? Creating a field as string leads to errors when trying to write float on it. Can we expect topics to be of the same type always?
|
|
107
|
-
value = '"' + value + '"';
|
|
108
|
-
} else {
|
|
109
|
-
value = '' + valueFloat;
|
|
110
|
-
if (!value.match(/\./)) value = value + '.0';
|
|
51
|
+
const series = seriesName(topic, {shortenStatus: config.shortenStatus, replaceSys: config.replaceSys});
|
|
52
|
+
const line = point(series, value, timestamp);
|
|
53
|
+
if (!line) {
|
|
54
|
+
skipped++; // text, or a payload that is not a number - not a time series
|
|
55
|
+
return;
|
|
111
56
|
}
|
|
57
|
+
recorded++;
|
|
58
|
+
writer.add(line);
|
|
59
|
+
}
|
|
112
60
|
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
61
|
+
const adapter = createAdapter({
|
|
62
|
+
pkg,
|
|
63
|
+
config,
|
|
64
|
+
deviceLabel: 'influx',
|
|
65
|
+
info: () => ({
|
|
66
|
+
influx: writer ? writer.url : undefined,
|
|
67
|
+
api: config.influxVersion,
|
|
68
|
+
db: config.influxDb,
|
|
69
|
+
subscribe: config.subscribe,
|
|
70
|
+
buffered: writer ? writer.size : 0,
|
|
71
|
+
recorded,
|
|
72
|
+
skipped,
|
|
73
|
+
}),
|
|
74
|
+
// a sink: the topics are the user's, anywhere on the broker, not under <name>/
|
|
75
|
+
listen: Object.fromEntries(config.subscribe.map((topic) => [topic, record])),
|
|
76
|
+
onShutdown: () => writer.stop(),
|
|
119
77
|
});
|
|
120
78
|
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
//
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
});
|
|
127
|
-
buffer = {};
|
|
128
|
-
bufferCount = 0;
|
|
129
|
-
}
|
|
79
|
+
writer = createWriter(config, {
|
|
80
|
+
log: adapter.log,
|
|
81
|
+
// <name>/connected 2 while influx accepts writes, 1 while it does not
|
|
82
|
+
onState: (up) => adapter.setDeviceConnected(up),
|
|
83
|
+
});
|
|
130
84
|
|
|
131
|
-
|
|
85
|
+
adapter.log.info(`influxdb api ${config.influxVersion} at ${writer.url}`);
|
|
86
|
+
writer.start();
|
|
87
|
+
adapter.start();
|
package/lib/influx.js
ADDED
|
@@ -0,0 +1,145 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* The InfluxDB write client: where the points go and how the request is authenticated.
|
|
3
|
+
*
|
|
4
|
+
* InfluxDB 1 and 2 speak the same line protocol — only the endpoint and the auth differ, which is
|
|
5
|
+
* why one adapter can serve both and `--influx-version` is all it takes:
|
|
6
|
+
*
|
|
7
|
+
* v1 POST <url>/write?db=<db>&precision=ns optional HTTP basic auth
|
|
8
|
+
* v2 POST <url>/api/v2/write?org=<org>&bucket=<db>&precision=ns Authorization: Token <token>
|
|
9
|
+
*
|
|
10
|
+
* InfluxDB 2 also serves the v1 endpoint for compatibility, so `--influx-version 1` against a v2
|
|
11
|
+
* server works if a v1 mapping was set up there; `2` is the native path and the one to prefer.
|
|
12
|
+
*
|
|
13
|
+
* Points are buffered because one HTTP request per MQTT message would be absurd at the rate a
|
|
14
|
+
* smart home publishes. The buffer is flushed when it is full, on a timer, and on shutdown — the
|
|
15
|
+
* last one is why this takes the adapter's lifecycle rather than a bare `setInterval`.
|
|
16
|
+
*/
|
|
17
|
+
|
|
18
|
+
/** The URL a write goes to, query string included. */
|
|
19
|
+
export function writeUrl({influxUrl, influxVersion = 1, influxDb, influxOrg}) {
|
|
20
|
+
const base = String(influxUrl).replace(/\/+$/, '');
|
|
21
|
+
const q = new URLSearchParams({precision: 'ns'});
|
|
22
|
+
if (Number(influxVersion) === 2) {
|
|
23
|
+
q.set('bucket', influxDb);
|
|
24
|
+
if (influxOrg) {
|
|
25
|
+
q.set('org', influxOrg);
|
|
26
|
+
}
|
|
27
|
+
return `${base}/api/v2/write?${q}`;
|
|
28
|
+
}
|
|
29
|
+
q.set('db', influxDb);
|
|
30
|
+
return `${base}/write?${q}`;
|
|
31
|
+
}
|
|
32
|
+
|
|
33
|
+
/** Auth and content headers for a write. The api version does not matter here — a token is
|
|
34
|
+
* accepted by both, so only which credential was given decides the scheme. */
|
|
35
|
+
export function writeHeaders({influxToken, influxUsername, influxPassword}) {
|
|
36
|
+
const headers = {'content-type': 'text/plain; charset=utf-8'};
|
|
37
|
+
if (influxToken) {
|
|
38
|
+
// v2's scheme, and accepted by v1.8+ as well
|
|
39
|
+
headers.authorization = `Token ${influxToken}`;
|
|
40
|
+
} else if (influxUsername) {
|
|
41
|
+
const basic = Buffer.from(`${influxUsername}:${influxPassword || ''}`).toString('base64');
|
|
42
|
+
headers.authorization = `Basic ${basic}`;
|
|
43
|
+
}
|
|
44
|
+
return headers;
|
|
45
|
+
}
|
|
46
|
+
|
|
47
|
+
/**
|
|
48
|
+
* A buffer of line-protocol points that writes itself out.
|
|
49
|
+
*
|
|
50
|
+
* @param {object} config parsed config (influxUrl, influxVersion, influxDb, influxOrg, credentials,
|
|
51
|
+
* bufLength, bufInterval)
|
|
52
|
+
* @param {{log?: object, fetchImpl?: Function, onState?: (up: boolean) => void}} [deps]
|
|
53
|
+
* `onState` reports reachability, so the adapter can put it in <name>/connected
|
|
54
|
+
*/
|
|
55
|
+
export function createWriter(config, {log = console, fetchImpl = globalThis.fetch, onState} = {}) {
|
|
56
|
+
const url = writeUrl(config);
|
|
57
|
+
const headers = writeHeaders(config);
|
|
58
|
+
const max = Number(config.bufLength) > 0 ? Number(config.bufLength) : 1000;
|
|
59
|
+
let buffer = [];
|
|
60
|
+
let timer = null;
|
|
61
|
+
let up = null;
|
|
62
|
+
let inFlight = null;
|
|
63
|
+
|
|
64
|
+
function state(next) {
|
|
65
|
+
if (next !== up) {
|
|
66
|
+
up = next;
|
|
67
|
+
if (onState) {
|
|
68
|
+
onState(next);
|
|
69
|
+
}
|
|
70
|
+
}
|
|
71
|
+
}
|
|
72
|
+
|
|
73
|
+
async function send(lines) {
|
|
74
|
+
const body = lines.join('\n');
|
|
75
|
+
let response;
|
|
76
|
+
try {
|
|
77
|
+
response = await fetchImpl(url, {method: 'POST', headers, body});
|
|
78
|
+
} catch (error) {
|
|
79
|
+
// unreachable: the points are gone. Keeping them would grow without bound while a
|
|
80
|
+
// database is down for a day, and a gap is the honest outcome for a time series.
|
|
81
|
+
log.warn('influx unreachable —', error.message, `(${lines.length} points dropped)`);
|
|
82
|
+
state(false);
|
|
83
|
+
return false;
|
|
84
|
+
}
|
|
85
|
+
if (response.status === 204 || response.status === 200) {
|
|
86
|
+
log.debug('influx wrote', lines.length, 'points');
|
|
87
|
+
state(true);
|
|
88
|
+
return true;
|
|
89
|
+
}
|
|
90
|
+
let detail = '';
|
|
91
|
+
try {
|
|
92
|
+
detail = (await response.text()).slice(0, 300);
|
|
93
|
+
} catch {
|
|
94
|
+
// a body we cannot read changes nothing about the status
|
|
95
|
+
}
|
|
96
|
+
// 4xx is our own line protocol being wrong: the same points would fail again, so they go
|
|
97
|
+
log.error('influx write failed', response.status, detail);
|
|
98
|
+
state(response.status < 500);
|
|
99
|
+
return false;
|
|
100
|
+
}
|
|
101
|
+
|
|
102
|
+
/** Write everything buffered. Serialised: two overlapping writes could reorder points. */
|
|
103
|
+
function flush() {
|
|
104
|
+
if (buffer.length === 0) {
|
|
105
|
+
return inFlight || Promise.resolve(true);
|
|
106
|
+
}
|
|
107
|
+
const lines = buffer;
|
|
108
|
+
buffer = [];
|
|
109
|
+
inFlight = Promise.resolve(inFlight)
|
|
110
|
+
.catch(() => {})
|
|
111
|
+
.then(() => send(lines));
|
|
112
|
+
return inFlight;
|
|
113
|
+
}
|
|
114
|
+
|
|
115
|
+
return {
|
|
116
|
+
/** Buffer one point; writes early when the buffer is full. */
|
|
117
|
+
add(line) {
|
|
118
|
+
buffer.push(line);
|
|
119
|
+
if (buffer.length >= max) {
|
|
120
|
+
flush();
|
|
121
|
+
}
|
|
122
|
+
},
|
|
123
|
+
flush,
|
|
124
|
+
get size() {
|
|
125
|
+
return buffer.length;
|
|
126
|
+
},
|
|
127
|
+
get url() {
|
|
128
|
+
return url;
|
|
129
|
+
},
|
|
130
|
+
start() {
|
|
131
|
+
const seconds = Number(config.bufInterval) > 0 ? Number(config.bufInterval) : 30;
|
|
132
|
+
timer = setInterval(flush, seconds * 1000);
|
|
133
|
+
if (timer.unref) {
|
|
134
|
+
timer.unref(); // the mqtt connection keeps the process alive, not this
|
|
135
|
+
}
|
|
136
|
+
},
|
|
137
|
+
async stop() {
|
|
138
|
+
if (timer) {
|
|
139
|
+
clearInterval(timer);
|
|
140
|
+
timer = null;
|
|
141
|
+
}
|
|
142
|
+
await flush(); // whatever is buffered on SIGTERM would otherwise be lost
|
|
143
|
+
},
|
|
144
|
+
};
|
|
145
|
+
}
|
package/lib/install.js
ADDED
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* --install / --uninstall: systemd template service influx4mqtt@<name> (mqtt-interfaces-core
|
|
3
|
+
* installer). Nothing privileged — it talks to two network services and keeps no state.
|
|
4
|
+
*/
|
|
5
|
+
|
|
6
|
+
import {createInstaller} from 'mqtt-interfaces-core';
|
|
7
|
+
|
|
8
|
+
export const SERVICE = 'influx4mqtt';
|
|
9
|
+
export const ENV_PREFIX = 'INFLUX4MQTT';
|
|
10
|
+
|
|
11
|
+
const installer = createInstaller({
|
|
12
|
+
service: SERVICE,
|
|
13
|
+
envPrefix: ENV_PREFIX,
|
|
14
|
+
description: `${SERVICE} %i - MQTT to InfluxDB recorder`,
|
|
15
|
+
documentation: 'https://github.com/hobbyquaker/influx4mqtt',
|
|
16
|
+
});
|
|
17
|
+
|
|
18
|
+
export const {unitFile, envFile, installService, uninstallService, handle} = installer;
|
package/lib/line.js
ADDED
|
@@ -0,0 +1,107 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Topic and payload → one InfluxDB line-protocol point. Pure: no sockets, no config object, so
|
|
3
|
+
* every rule below is testable on its own.
|
|
4
|
+
*
|
|
5
|
+
* The measurement is the topic, lightly rewritten (see `seriesName`); the only field is `value`,
|
|
6
|
+
* always a float, because a series whose type changes is a series Influx will refuse to write.
|
|
7
|
+
*/
|
|
8
|
+
|
|
9
|
+
/** Line protocol escapes in a measurement name: spaces and commas separate tokens. */
|
|
10
|
+
export function escapeMeasurement(name) {
|
|
11
|
+
return name.replace(/,/g, '\\,').replace(/ /g, '\\ ');
|
|
12
|
+
}
|
|
13
|
+
|
|
14
|
+
/**
|
|
15
|
+
* The series a topic is recorded under.
|
|
16
|
+
*
|
|
17
|
+
* `shortenStatus` collapses the mqtt-smarthome `status` level — `hm/status/lamp` becomes
|
|
18
|
+
* `hm//lamp`. It has been the behaviour since 1.0 and stays the default: turning it off renames
|
|
19
|
+
* every existing series and splits the history in Grafana. New setups may prefer the full topic.
|
|
20
|
+
*
|
|
21
|
+
* `replaceSys` rewrites the leading `$SYS/`, whose topics are otherwise identical on every broker
|
|
22
|
+
* — recording two of them into one database would interleave them into nonsense.
|
|
23
|
+
*
|
|
24
|
+
* @param {string} topic
|
|
25
|
+
* @param {{shortenStatus?: boolean, replaceSys?: string}} [options]
|
|
26
|
+
*/
|
|
27
|
+
export function seriesName(topic, {shortenStatus = true, replaceSys} = {}) {
|
|
28
|
+
let name = topic;
|
|
29
|
+
if (shortenStatus) {
|
|
30
|
+
name = name.replace(/^([^/]+)\/status\/(.+)$/, '$1//$2');
|
|
31
|
+
}
|
|
32
|
+
if (replaceSys) {
|
|
33
|
+
name = name.replace(/^\$SYS\//, replaceSys);
|
|
34
|
+
}
|
|
35
|
+
return name;
|
|
36
|
+
}
|
|
37
|
+
|
|
38
|
+
/** Payloads that mean true / false rather than a number. Case-insensitive since 2.0. */
|
|
39
|
+
const TRUE = new Set(['true', 'on', 'yes']);
|
|
40
|
+
const FALSE = new Set(['false', 'off', 'no']);
|
|
41
|
+
|
|
42
|
+
/**
|
|
43
|
+
* A value → the float Influx stores, as a string, or `null` for anything that is not a number.
|
|
44
|
+
*
|
|
45
|
+
* Booleans and the strings devices use for them become `1`/`0`; everything else has to parse as a
|
|
46
|
+
* number. Text is dropped rather than stored — this is a numeric time series, and a string field
|
|
47
|
+
* would fix the series' type to string forever.
|
|
48
|
+
*
|
|
49
|
+
* The trailing `.0` is deliberate: without it Influx infers an integer field, and a series that
|
|
50
|
+
* starts out integer rejects the first float that arrives.
|
|
51
|
+
*
|
|
52
|
+
* @returns {string|null}
|
|
53
|
+
*/
|
|
54
|
+
export function toFloat(value) {
|
|
55
|
+
if (value === true) {
|
|
56
|
+
return '1.0';
|
|
57
|
+
}
|
|
58
|
+
if (value === false) {
|
|
59
|
+
return '0.0';
|
|
60
|
+
}
|
|
61
|
+
let number;
|
|
62
|
+
if (typeof value === 'number') {
|
|
63
|
+
number = value;
|
|
64
|
+
} else if (typeof value === 'string') {
|
|
65
|
+
const trimmed = value.trim();
|
|
66
|
+
const lower = trimmed.toLowerCase();
|
|
67
|
+
if (TRUE.has(lower)) {
|
|
68
|
+
return '1.0';
|
|
69
|
+
}
|
|
70
|
+
if (FALSE.has(lower)) {
|
|
71
|
+
return '0.0';
|
|
72
|
+
}
|
|
73
|
+
if (trimmed === '') {
|
|
74
|
+
return null;
|
|
75
|
+
}
|
|
76
|
+
/*
|
|
77
|
+
* Number(), not parseFloat(): parseFloat('21 °C') is 21, and recording that as a
|
|
78
|
+
* temperature is worse than recording nothing — it looks like a real measurement. The
|
|
79
|
+
* empty string is handled above because Number('') is 0, not NaN.
|
|
80
|
+
*/
|
|
81
|
+
number = Number(trimmed);
|
|
82
|
+
} else {
|
|
83
|
+
// objects, null, undefined: Number(null) is 0 and Number([5]) is 5, so never fall through
|
|
84
|
+
return null;
|
|
85
|
+
}
|
|
86
|
+
if (!Number.isFinite(number)) {
|
|
87
|
+
return null;
|
|
88
|
+
}
|
|
89
|
+
const text = String(number);
|
|
90
|
+
return /[.e]/i.test(text) ? text : text + '.0';
|
|
91
|
+
}
|
|
92
|
+
|
|
93
|
+
/**
|
|
94
|
+
* One line-protocol point, or `null` when the value is not numeric.
|
|
95
|
+
*
|
|
96
|
+
* @param {string} series measurement name (already rewritten by `seriesName`)
|
|
97
|
+
* @param {*} value
|
|
98
|
+
* @param {number} timestampMs
|
|
99
|
+
*/
|
|
100
|
+
export function point(series, value, timestampMs) {
|
|
101
|
+
const float = toFloat(value);
|
|
102
|
+
if (float === null || !series) {
|
|
103
|
+
return null;
|
|
104
|
+
}
|
|
105
|
+
// influx takes nanoseconds; the timestamp is milliseconds, from the payload or our clock
|
|
106
|
+
return `${escapeMeasurement(series)} value=${float} ${Math.round(timestampMs) * 1000000}`;
|
|
107
|
+
}
|
package/package.json
CHANGED
|
@@ -1,34 +1,64 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "influx4mqtt",
|
|
3
|
-
"version": "0.0
|
|
4
|
-
"description": "
|
|
3
|
+
"version": "2.0.0",
|
|
4
|
+
"description": "Record MQTT values in InfluxDB. Follows the mqtt-smarthome architecture.",
|
|
5
|
+
"type": "module",
|
|
5
6
|
"main": "index.js",
|
|
6
|
-
"scripts": {
|
|
7
|
-
"test": "echo \"Error: no test specified\" && exit 1"
|
|
8
|
-
},
|
|
9
|
-
"author": "Sebastian 'hobbyquaker' Raff <hq@ccu.io>",
|
|
10
|
-
"license": "MIT",
|
|
11
|
-
"dependencies": {
|
|
12
|
-
"influx": "git://github.com/hobbyquaker/node-influx",
|
|
13
|
-
"mqtt": "^1.3.3",
|
|
14
|
-
"yargs": "^3.14.0"
|
|
15
|
-
},
|
|
16
7
|
"bin": {
|
|
17
|
-
"influx4mqtt": "
|
|
8
|
+
"influx4mqtt": "index.js"
|
|
18
9
|
},
|
|
19
10
|
"preferGlobal": true,
|
|
11
|
+
"files": [
|
|
12
|
+
"index.js",
|
|
13
|
+
"config.js",
|
|
14
|
+
"lib/"
|
|
15
|
+
],
|
|
16
|
+
"engines": {
|
|
17
|
+
"node": "^20.19 || ^22.12 || >=24"
|
|
18
|
+
},
|
|
19
|
+
"scripts": {
|
|
20
|
+
"start": "node index.js",
|
|
21
|
+
"lint": "eslint . && prettier --check .",
|
|
22
|
+
"format": "prettier --write . && eslint --fix .",
|
|
23
|
+
"test": "node --test",
|
|
24
|
+
"deploy": "bash deploy.sh"
|
|
25
|
+
},
|
|
20
26
|
"repository": {
|
|
21
27
|
"type": "git",
|
|
22
|
-
"url": "https://github.com/hobbyquaker/influx4mqtt"
|
|
28
|
+
"url": "git+https://github.com/hobbyquaker/influx4mqtt.git"
|
|
29
|
+
},
|
|
30
|
+
"homepage": "https://github.com/hobbyquaker/influx4mqtt",
|
|
31
|
+
"bugs": {
|
|
32
|
+
"url": "https://github.com/hobbyquaker/influx4mqtt/issues"
|
|
23
33
|
},
|
|
34
|
+
"author": "Sebastian Raff <hobbyquaker@gmail.com> (https://github.com/hobbyquaker)",
|
|
35
|
+
"license": "MIT",
|
|
24
36
|
"keywords": [
|
|
25
37
|
"mqtt",
|
|
26
|
-
"smarthome",
|
|
27
|
-
"
|
|
38
|
+
"mqtt-smarthome",
|
|
39
|
+
"home-automation",
|
|
28
40
|
"influxdb",
|
|
29
|
-
"
|
|
41
|
+
"influx",
|
|
30
42
|
"grafana",
|
|
31
|
-
"time",
|
|
32
|
-
"
|
|
33
|
-
]
|
|
43
|
+
"time series",
|
|
44
|
+
"history"
|
|
45
|
+
],
|
|
46
|
+
"mqttInterfaces": {
|
|
47
|
+
"spec": "2.0",
|
|
48
|
+
"envPrefix": "INFLUX4MQTT",
|
|
49
|
+
"needs": [
|
|
50
|
+
"network"
|
|
51
|
+
],
|
|
52
|
+
"serviceExtra": []
|
|
53
|
+
},
|
|
54
|
+
"dependencies": {
|
|
55
|
+
"mqtt-interfaces-core": "^0.13.0"
|
|
56
|
+
},
|
|
57
|
+
"devDependencies": {
|
|
58
|
+
"@eslint/js": "^9",
|
|
59
|
+
"eslint": "^9",
|
|
60
|
+
"eslint-config-prettier": "^10",
|
|
61
|
+
"globals": "^16",
|
|
62
|
+
"prettier": "^3"
|
|
63
|
+
}
|
|
34
64
|
}
|
package/.npmignore
DELETED