kahu-signalk 0.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/.gitmodules ADDED
@@ -0,0 +1,3 @@
1
+ [submodule "data/protocol"]
2
+ path = data/protocol
3
+ url = https://github.com/KAHU-radar/radarhub-protocol.git
package/README.md ADDED
@@ -0,0 +1,195 @@
1
+ # KAHU Radar Hub
2
+
3
+ *A crowdsourcing [Signal K](https://signalk.org/) plugin for marine safety*
4
+
5
+ ---
6
+
7
+ ## What Does This Plugin Do?
8
+
9
+ This plugin takes **radar target data** from your boat and uploads it to a shared server so that other mariners can benefit from the information. Think of it like Waze for boats -- your radar sees nearby vessels and objects, and this plugin shares that data to help build a bigger picture of what's happening on the water.
10
+
11
+ ### The Big Picture
12
+
13
+ ```
14
+ Your Boat's Radar
15
+ |
16
+ | detects targets (other boats, obstacles)
17
+ v
18
+ Radar Unit ──NMEA sentences──> Signal K Server
19
+ |
20
+ | this plugin parses the data
21
+ v
22
+ KAHU Radar Hub Plugin
23
+ |
24
+ | uploads via TCP/Avro
25
+ v
26
+ KAHU Cloud Server
27
+ (crowdsource.kahu.earth)
28
+ ```
29
+
30
+ ---
31
+
32
+ ## Key Concepts Explained
33
+
34
+ ### What is NMEA?
35
+
36
+ **NMEA** (National Marine Electronics Association) is the standard "language" that marine electronics use to talk to each other. When your GPS, radar, depth sounder, or wind instrument sends data, it sends it as **NMEA sentences** -- short text messages with a specific format.
37
+
38
+ For example, a GPS position might look like: `$GPGGA,123519,4807.038,N,01131.000,E,...`
39
+
40
+ There are two main versions:
41
+ - **NMEA 0183** -- the older serial/text-based standard (what this plugin uses)
42
+ - **NMEA 2000** -- the newer CAN-bus based standard
43
+
44
+ ### What is ARPA?
45
+
46
+ **ARPA** (Automatic Radar Plotting Aid) is a feature built into marine radars. When your radar detects an object (another boat, a buoy, land), ARPA automatically:
47
+
48
+ 1. **Tracks** the object over time
49
+ 2. **Calculates** its speed and course
50
+ 3. **Predicts** its future position
51
+ 4. **Computes CPA** (Closest Point of Approach) -- how close it will get to you and when
52
+
53
+ ARPA is a critical safety tool for collision avoidance. Without ARPA, a radar just shows blips; with ARPA, those blips become tracked targets with speed, heading, and collision risk information.
54
+
55
+ ### What is $RATTM?
56
+
57
+ `$RATTM` is the specific NMEA 0183 sentence that a radar sends when reporting an ARPA-tracked target. The "RA" prefix means it comes from a **Radar**, and "TTM" stands for **Tracked Target Message**.
58
+
59
+ A `$RATTM` sentence contains:
60
+ | Field | Meaning |
61
+ |-------|---------|
62
+ | Target number | Which tracked target (00-99) |
63
+ | Distance | How far away the target is |
64
+ | Bearing | What direction the target is in (degrees) |
65
+ | Speed | How fast the target is moving |
66
+ | Course | What direction the target is heading |
67
+ | CPA distance | Closest the target will come to you |
68
+ | CPA time | When it will be closest |
69
+ | Target name | Optional identifier |
70
+ | Target status | Tracking status (e.g. lost, tracking) |
71
+
72
+ ### What is Signal K?
73
+
74
+ [Signal K](https://signalk.org/) is a modern, open-source data format and server for boats. It acts as a central hub that collects data from all your marine instruments (GPS, radar, AIS, wind, depth, etc.) and makes it available in a standard JSON format over your boat's network. Apps and plugins can then read and process this data.
75
+
76
+ ### What is Apache Avro?
77
+
78
+ [Apache Avro](https://avro.apache.org/) is a compact binary data format. This plugin uses it instead of JSON or plain text to send data to the server because it is much smaller -- important when you're on a boat with limited or expensive satellite/cellular internet.
79
+
80
+ ### What is AIS?
81
+
82
+ **AIS** (Automatic Identification System) is a tracking system used on ships. Vessels equipped with AIS transponders automatically broadcast their identity, position, course, and speed. Unlike radar (which detects objects passively), AIS requires the other vessel to be actively transmitting. This plugin currently does **not** collect AIS data -- only radar ARPA targets.
83
+
84
+ ---
85
+
86
+ ## How It Works (Step by Step)
87
+
88
+ 1. **Your radar** detects nearby objects and tracks them using ARPA
89
+ 2. **The radar sends** `$RATTM` NMEA sentences to Signal K (via a serial/network connection you configure)
90
+ 3. **This plugin** registers a custom NMEA parser inside Signal K that intercepts those sentences
91
+ 4. **The parser** extracts the target's bearing and distance (relative to your boat), then converts that to an absolute latitude/longitude using your boat's GPS position
92
+ 5. **Each target** is published into Signal K as a virtual vessel with its own position, speed, and course
93
+ 6. **Target positions** are cached locally in a SQLite database (so data is not lost if you lose internet)
94
+ 7. **A background connector** batches up cached track points and sends them to the KAHU server using the Avro protocol over TCP
95
+ 8. **If the connection drops**, data keeps accumulating locally and is sent when connectivity returns
96
+
97
+ ---
98
+
99
+ ## Installation
100
+
101
+ ```bash
102
+ # From your Signal K server's plugin directory
103
+ npm install radarhub-signalk
104
+
105
+ # Or clone and link for development
106
+ git clone --recurse-submodules https://github.com/KAHU-radar/radarhub-signalk.git
107
+ cd radarhub-signalk
108
+ npm install
109
+ npm link
110
+ # Then from your Signal K server directory:
111
+ signalk-server --install radarhub-signalk
112
+ ```
113
+
114
+ **Note:** The `data/protocol` folder is a git submodule. If you cloned without `--recurse-submodules`, run:
115
+ ```bash
116
+ git submodule update --init
117
+ ```
118
+
119
+ ---
120
+
121
+ ## Configuration
122
+
123
+ Enable the plugin in the Signal K admin UI under **Server > Plugin Config > KAHU Radar Hub**.
124
+
125
+ | Setting | Default | Description |
126
+ |---------|---------|-------------|
127
+ | `server` | `crowdsource.kahu.earth` | KAHU server hostname |
128
+ | `port` | `9900` | TCP port for the KAHU server |
129
+ | `api_key` | *(none)* | Your API key for authentication |
130
+ | `min_reconnect_time` | `100` | Minimum delay (ms) before reconnecting after a drop |
131
+ | `max_reconnect_time` | `600` | Maximum delay (ms) between reconnection attempts |
132
+
133
+ ### Prerequisites
134
+
135
+ - Your radar must be configured for **ARPA tracking** and set to output **$RATTM sentences**
136
+ - Signal K must have a **data connection** to your radar's NMEA output (serial port, TCP, or UDP)
137
+ - Your boat must have a **GPS** providing position data to Signal K
138
+
139
+ ---
140
+
141
+ ## Current Limitations
142
+
143
+ - Only supports `$RATTM` sentences (not `$RATTL` -- target list sentences)
144
+ - Does not collect AIS data, only radar ARPA targets
145
+ - The protocol is **NOT encrypted** (data is sent in plain text)
146
+ - The protocol is **NOT cryptographically signed** (no tamper protection)
147
+ - Relative bearings (`R`) are not supported -- only true bearings (`T`)
148
+ - The `uuid` npm package is used but not listed in `package.json` (relies on Signal K providing it)
149
+
150
+ ---
151
+
152
+ ## Project Structure
153
+
154
+ ```
155
+ radarhub-signalk/
156
+ ├── package.json # Plugin metadata and dependencies
157
+ ├── README.md # This file
158
+ ├── plugin/
159
+ │ ├── index.js # Main entry: NMEA parser, Signal K integration
160
+ │ ├── connector.js # Background TCP connection and track submission
161
+ │ ├── tcpclient.js # TCP socket client with auto-reconnect
162
+ │ ├── avroclient.js # Avro serialization/deserialization over TCP
163
+ │ ├── routecache.js # SQLite local cache for track points
164
+ │ └── utils.js # Utility helpers
165
+ └── data/protocol/ # Git submodule (radarhub-protocol)
166
+ ├── proto_avro.json # Avro schema defining the wire protocol
167
+ └── migrations/ # SQLite database migrations
168
+ ├── 0001-create-targets.sql
169
+ └── 0002-target-indices.sql
170
+ ```
171
+
172
+ ---
173
+
174
+ ## Server
175
+
176
+ An example server written in Python is provided
177
+ [here](https://github.com/KAHU-radar/radarhub-server). This server
178
+ implements the full protocol, but just dumps all received tracks to
179
+ disk in GeoJSON format. It can be used as a simple shore-based VDR
180
+ (Voyage Data Recorder), but mostly serves as an example base for anyone
181
+ wanting to build a more elaborate server-side setup.
182
+
183
+ ---
184
+
185
+ ## Runtime Requirements
186
+
187
+ - **Signal K Server:** v2.22.1+ (latest stable)
188
+ - **Node.js:** 20.x or later (required by Signal K server v2.22+)
189
+ - **Dependencies:** avro-js, promise-socket, sqlite/sqlite3, uuid
190
+
191
+ ---
192
+
193
+ ## License
194
+
195
+ ISC
package/package.json ADDED
@@ -0,0 +1,32 @@
1
+ {
2
+ "name": "kahu-signalk",
3
+ "version": "0.0.1",
4
+ "description": "Contribute AIS and ARPA targets from your vessel to crowdsourcing for marine safety!",
5
+ "keywords": [
6
+ "signalk-node-server-plugin",
7
+ "signalk-category-cloud",
8
+ "signalk-category-nmea-0183",
9
+ "kahu"
10
+ ],
11
+ "signalk-plugin-enabled-by-default": false,
12
+ "signalk": {
13
+ "appIcon": "./assets/icons/icon-72x72.png",
14
+ "displayName": "Kahu - Marine Safety Crowdsourcing"
15
+ },
16
+ "main": "plugin/index.js",
17
+ "engines": {
18
+ "node": ">=20"
19
+ },
20
+ "scripts": {
21
+ "test": "echo \"Error: no test specified\" && exit 1"
22
+ },
23
+ "author": "Kahu <info@kahu.earth>",
24
+ "license": "ISC",
25
+ "dependencies": {
26
+ "avro-js": "^1.12.0",
27
+ "promise-socket": "^8.0.0",
28
+ "sqlite": "^5.1.1",
29
+ "sqlite3": "^5.1.7",
30
+ "uuid": "^8.1.0"
31
+ }
32
+ }
@@ -0,0 +1,39 @@
1
+ const { TCPClient } = require('./tcpclient');
2
+ const avro = require('avro-js');
3
+
4
+ class AvroClient {
5
+ constructor({schema, ...props}) {
6
+ this.schema = schema;
7
+ this.type = avro.parse(schema);
8
+ this.tcpClient = new TCPClient(props);
9
+ }
10
+
11
+ async destroy() {
12
+ await this.tcpClient.destroy();
13
+ }
14
+
15
+ async ensureConnection() {
16
+ return await this.tcpClient.ensureConnection();
17
+ }
18
+
19
+ async send(data) {
20
+ this.tcpClient.send(this.type.toBuffer(data));
21
+ }
22
+
23
+ async read() {
24
+ while (true) {
25
+ const buffer = await this.tcpClient.read();
26
+ try {
27
+ const decoded = this.type.fromBuffer(buffer, undefined, true);
28
+ this.tcpClient.consume(this.type.toBuffer(decoded).length);
29
+ return decoded;
30
+ } catch (err) {
31
+ if (!(err.message == "truncated buffer")) {
32
+ this.tcpClient.connectionFailure(err, 'Failed to parse Avro document');
33
+ }
34
+ }
35
+ }
36
+ }
37
+ }
38
+
39
+ module.exports = { AvroClient };
@@ -0,0 +1,175 @@
1
+ const fs = require('fs').promises;
2
+ const { delay } = require('./utils');
3
+ const { AvroClient } = require('./avroclient');
4
+ const path = require('path');
5
+
6
+ // Connector acts as a cooperative thread: When created, it starts an
7
+ // async timeout function immediately
8
+
9
+ process.on("unhandledRejection", (reason, promise) => {
10
+ console.error("Unhandled Rejection at:", promise);
11
+ console.error("Reason:", reason.stack);
12
+ });
13
+
14
+ class Connector {
15
+ constructor({status_function, routecache, plugin_dir, config}) {
16
+ this.last_stats = null;
17
+ this.last_status = null;
18
+ this.status_function = status_function;
19
+ this.routecache = routecache;
20
+ this.config = config;
21
+ this.client = null;
22
+ this.schema = null;
23
+ this.callid = 0;
24
+ this.last_connection = Date(1970, 1, 1);
25
+ this.last_track_sent = Date(1970, 1, 1);
26
+ this.cancelled = false;
27
+ this.schema_file = path.join(
28
+ plugin_dir,
29
+ "data",
30
+ "protocol",
31
+ "proto_avro.json");
32
+ this.schema = null;
33
+ console.error("Connector created");
34
+ setTimeout(this.main.bind(this), 0);
35
+ }
36
+
37
+ async destroy() {
38
+ console.error("Connector destroyed");
39
+ this.cancelled = true;
40
+ if (this.client) await this.client.destroy();
41
+ this.client = null;
42
+ }
43
+
44
+ async updateStats() {
45
+ this.last_stats = await this.routecache.connectionStats();
46
+ this.updateStatus();
47
+ }
48
+
49
+ setStatus(status) {
50
+ this.last_status = status;
51
+ this.updateStatus();
52
+ }
53
+
54
+ updateStatus() {
55
+ const status = [];
56
+ if (this.last_status !== null) status.push(this.last_status);
57
+ if (this.last_stats !== null) {
58
+ status.push(`${this.last_stats.unsent_tracks} unsent tracks totalling ${this.last_stats.unsent_datapoints} unsent datapoints`);
59
+ }
60
+ this.status_function?.(status.join(", "));
61
+ }
62
+
63
+ async read(type) {
64
+ console.error("Connector parsing response");
65
+
66
+ const container = await this.client.read();
67
+ const message = container.Message;
68
+ const response = message["kahu.Response"].Response;
69
+ if (response.id != this.callid) {
70
+ throw "Received response with wrong callid";
71
+ }
72
+ const content = response.Response;
73
+
74
+ if (content["kahu.ErrorResponseMessage"] !== undefined) {
75
+ throw content.Error.exception;
76
+ } else if ( (type !== undefined)
77
+ && (content[type] === undefined)) {
78
+ throw "Received response for wrong method: expected " + type + " but got " + Object.keys(content)[0];
79
+ }
80
+ console.error("Connector response parsed");
81
+ return container;
82
+ }
83
+
84
+ async login() {
85
+ while (!this.config.api_key) await delay(500);
86
+
87
+ console.error("Connector logging in");
88
+
89
+ await this.client.send({
90
+ Message: {
91
+ "kahu.Call": {
92
+ Call: {
93
+ id: ++this.callid,
94
+ Call: {
95
+ "kahu.LoginMessage": {
96
+ Login: {
97
+ apikey: this.config.api_key
98
+ }
99
+ }
100
+ }
101
+ }
102
+ }
103
+ }
104
+ });
105
+
106
+ console.error("Send done, gonna parse response\n");
107
+ const response = await this.read("kahu.LoginResponseMessage");
108
+ console.error("Response parsed");
109
+ this.last_connection = new Date();
110
+ }
111
+
112
+ async sendTracks() {
113
+ console.error("Connector sending tracks");
114
+ const submit = await this.routecache.retrieve();
115
+ if (submit === null) return;
116
+ await this.client.send({
117
+ Message: {
118
+ "kahu.Call": {
119
+ Call: {
120
+ id: ++this.callid,
121
+ Call: {
122
+ "kahu.SubmitMessage": {
123
+ Submit: submit
124
+ }
125
+ }
126
+ }
127
+ }
128
+ }
129
+ });
130
+
131
+ await this.read("kahu.SubmitResponseMessage");
132
+ await this.routecache.markAsSent(submit);
133
+ this.last_track_sent = new Date();
134
+ console.log("Tracks sent");
135
+ await this.updateStats();
136
+ }
137
+
138
+ async main() {
139
+ try {
140
+ console.error("Connector running client is null: ", (this.client === null));
141
+
142
+ this.schema = await fs.readFile(this.schema_file, 'utf8');
143
+
144
+ this.client = new AvroClient({
145
+ schema: this.schema,
146
+ ip: this.config.server || "crowdsource.kahu.earth",
147
+ port: this.config.port || 9900,
148
+ min_reconnect_time: this.config.min_reconnect_time || 100.0,
149
+ max_reconnect_time: this.config.max_reconnect_time || 6000.0,
150
+ connect_function: this.login.bind(this),
151
+ status_function: this.setStatus.bind(this)
152
+ });
153
+
154
+ while (!this.cancelled) {
155
+ try {
156
+ console.error("Connector connecting...");
157
+ await this.client.ensureConnection();
158
+ if (this.cancelled) break;
159
+ await this.sendTracks();
160
+ if (this.cancelled) break;
161
+ await delay(500);
162
+ } catch (e) {
163
+ console.error(e.toString(), " in Connector");
164
+ console.error(e.stack);
165
+ }
166
+ }
167
+ console.error("Connector exiting");
168
+ } catch (e) {
169
+ console.error(e.toString(), " in Connector, exiting");
170
+ console.error(e.stack);
171
+ }
172
+ }
173
+ };
174
+
175
+ module.exports = { Connector };
@@ -0,0 +1,165 @@
1
+ const { v4: uuidv4 } = require('uuid');
2
+ const { Routecache } = require("./routecache");
3
+ const { Connector } = require("./connector");
4
+ const path = require('path');
5
+ const fs = require('fs');
6
+
7
+ let packageDir = __dirname;
8
+ while (packageDir !== path.dirname(packageDir)
9
+ && !fs.existsSync(path.join(packageDir, 'package.json'))) {
10
+ packageDir = path.dirname(packageDir);
11
+ }
12
+
13
+ const nmeaRattmRegex = /\$RATTM,(\d{2}),([\d\.\-]+),([\d\.\-]+),([^,]*),([\d\.\-]+),([\d\.\-]+),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),(..*)\*([A-Fa-f0-9]{2})\s*/;
14
+
15
+ const deg2rad = (degrees) => degrees * (Math.PI / 180);
16
+
17
+ const polar2Pos = (ownPos, bearing, distance) => {
18
+ return {
19
+ latitude: ownPos.latitude + distance * Math.cos(deg2rad(bearing)) / 60. / 1852.,
20
+ longitude: ownPos.longitude + distance * Math.sin(deg2rad(bearing)) / Math.cos(deg2rad(ownPos.latitude)) / 60. / 1852.};
21
+ }
22
+
23
+ module.exports = (app) => {
24
+
25
+ const plugin = {
26
+ id: 'radarhub',
27
+ name: 'KAHU Radar Hub',
28
+ start: async (settings, restartPlugin) => {
29
+ console.log("Started KAHU radar Hub")
30
+
31
+ plugin.cache = new Routecache(
32
+ path.join(packageDir, "data", "protocol", "migrations"),
33
+ path.join(app.getDataDirPath(), "routecache.sqlite3"));
34
+ await plugin.cache.init();
35
+
36
+ plugin.connector = new Connector({
37
+ routecache: plugin.cache,
38
+ plugin_dir: packageDir,
39
+ config: settings,
40
+ status_function: app.setPluginStatus.bind(app)});
41
+
42
+ const now = new Date(1970, 1, 1);
43
+ plugin.route_updates = Array.from(Array(100)).map(() => now);
44
+ plugin.route_ids = Array.from(Array(100));
45
+
46
+ app.emitPropertyValue('nmea0183sentenceParser', {
47
+ sentence: 'TTM',
48
+ parser: ({ id, sentence, parts, tags }, session) => {
49
+ if (sentence.startsWith("$RATTL")) {
50
+ } else if (sentence.startsWith("$RATTM")) {
51
+ const match = nmeaRattmRegex.exec(sentence);
52
+
53
+ if (!match) {
54
+ console.error("Failed to parse RATTM NMEA sentence: [", sentence, "]");
55
+ return;
56
+ }
57
+ if (match.length - 1 != 14) {
58
+ console.log("Only parsed ", (match.length - 1), " fields of RATTM NMEA sentence: [", sentence, "]");
59
+ return;
60
+ }
61
+
62
+ const target_id = parseInt(match[1]);
63
+ const target_distance = parseFloat(match[2]);
64
+ const target_bearing = parseFloat(match[3]);
65
+ const target_bearing_unit = match[4];
66
+
67
+ if (target_bearing_unit === 'R') {
68
+ console.warn("Relative bearings not yet supported, skipping RATTM sentence");
69
+ return;
70
+ }
71
+
72
+ const ownPos = app.getSelfPath("navigation.position")?.value;
73
+ if (!ownPos) {
74
+ console.warn("No own-ship position available, skipping RATTM sentence");
75
+ return;
76
+ }
77
+ const targetPos = polar2Pos(ownPos, target_bearing, target_distance);
78
+
79
+ const relative = {
80
+ position: ownPos,
81
+ distance: target_distance,
82
+ bearing: target_bearing,
83
+ bearing_unit: target_bearing_unit,
84
+ distance_unit: match[10],
85
+ }
86
+
87
+ const target_speed = parseFloat(match[5]);
88
+ const target_course = parseFloat(match[6]);
89
+ const target_course_unit = match[7];
90
+ // target_distance_closes_point_of_approac: parseInt(match[8]),
91
+ // target_time_closes_point_of_approac: parseInt(match[9]),
92
+ const target_name = match[11];
93
+ const target_status = match[12];
94
+
95
+ const now = new Date();
96
+ if (now - plugin.route_updates[target_id] > 60000) {
97
+ plugin.route_updates[target_id] = now;
98
+ plugin.route_ids[target_id] = uuidv4();
99
+ }
100
+
101
+ return {
102
+ context: 'vessels.urn:mrn:signalk:uuid:' + plugin.route_ids[target_id],
103
+ updates: [
104
+ {
105
+ values: [
106
+ { path: 'name',
107
+ value: target_name
108
+ },
109
+ { path: 'navigation.speedOverGround',
110
+ value: target_speed
111
+ },
112
+ { path: 'navigation.courseOverGroundTrue',
113
+ value: target_course
114
+ },
115
+ { path: 'navigation.position',
116
+ value: {...targetPos, relative}
117
+ },
118
+ ]
119
+ }
120
+ ]
121
+ };
122
+ }
123
+ }
124
+ });
125
+
126
+ app.streambundle
127
+ .getBus('navigation.position')
128
+ .forEach(plugin.updatePosition);
129
+
130
+ },
131
+ stop: async () => {
132
+ await plugin.connector?.destroy?.();
133
+ await plugin.routecache?.destroy?.();
134
+ console.log("Stopped KAHU radar Hub")
135
+ },
136
+ schema: () => {
137
+ return {
138
+ properties: {
139
+ server: {type: "string", default: "crowdsource.kahu.earth"},
140
+ port: {type: "number", default: 9900},
141
+ api_key: {type: "string"},
142
+ min_reconnect_time: {type: "number", default: 100.0},
143
+ max_reconnect_time: {type: "number", default: 600.0}
144
+ }
145
+ };
146
+ },
147
+ updatePosition: (pos) => {
148
+ if (pos.source.sentence != "TTM") return;
149
+
150
+ const rest = app.getPath(pos.context);
151
+
152
+ const target_id = pos.context.split("vessels.urn:mrn:signalk:uuid:")[1];
153
+
154
+ plugin.cache.insert({
155
+ target_id: target_id,
156
+ position: pos.value,
157
+ speedOverGround: rest?.navigation?.speedOverGround?.value,
158
+ courseOverGroundTrue: rest?.navigation?.courseOverGroundTrue?.value,
159
+ name: rest?.name?.value,
160
+ });
161
+ }
162
+ };
163
+
164
+ return plugin;
165
+ };
@@ -0,0 +1,266 @@
1
+ const sqlite3 = require('sqlite3').verbose();
2
+ const sqlite = require('sqlite');
3
+ const fs = require('fs').promises;
4
+ const path = require('path');
5
+
6
+ class Routecache {
7
+ constructor(migrations_dir, db_name) {
8
+ this.db = null;
9
+ this.migrations_dir = migrations_dir;
10
+ this.db_name = db_name;
11
+ console.log("Routecache created for " + db_name + " with migrations " + migrations_dir);
12
+ }
13
+
14
+ async init() {
15
+ try {
16
+ await this.openDB();
17
+ await this.createEmpty();
18
+ await this.migrate();
19
+ return;
20
+ } catch (e) {
21
+ console.error(e, ". Deleting route cache.");
22
+ try {
23
+ await fs.unlink(this.db_name);
24
+ this.closeDB();
25
+ throw e;
26
+ } catch (err) {
27
+ console.error("Unable to delete route cache: ", this.db_name);
28
+ throw err;
29
+ }
30
+ }
31
+ }
32
+
33
+ async doesTableExist(tableName) {
34
+ const query = await this.db.all(
35
+ "SELECT count(*) as count FROM sqlite_master WHERE type='table' AND name=?",
36
+ [tableName]);
37
+ return query[0].count > 0;
38
+ }
39
+
40
+ async destroy() {
41
+ if (this.db != null) await this.db.close();
42
+ this.db = null;
43
+ }
44
+
45
+ async openDB() {
46
+ this.db = await sqlite.open({
47
+ filename: this.db_name,
48
+ driver: sqlite3.Database
49
+ });
50
+ }
51
+
52
+ async closeDB() {
53
+ if (this.db != null) await this.db.close();
54
+ this.db = null;
55
+ }
56
+
57
+ async createEmpty() {
58
+ if (await this.doesTableExist("migrations")) return;
59
+ const res = await this.db.run(`
60
+ CREATE TABLE IF NOT EXISTS migrations (
61
+ id integer,
62
+ name text,
63
+ applied datetime default current_timestamp
64
+ )
65
+ `);
66
+ }
67
+
68
+ async migrate() {
69
+ const rows = await this.db.all(
70
+ "SELECT max(id) as maxid FROM migrations");
71
+ const maxId = rows[0].maxid;
72
+
73
+ try {
74
+ const files = (
75
+ await fs.readdir(
76
+ this.migrations_dir, { withFileTypes: true }
77
+ )
78
+ ).filter(
79
+ (file) => file.isFile()
80
+ ).map(
81
+ (file) => file.name);
82
+ files.sort();
83
+
84
+ for (const filename of files) {
85
+ const migrationId = parseInt(filename, 10);
86
+ if (migrationId > maxId) {
87
+ const migrationPath = path.join(this.migrations_dir, filename);
88
+ await this.runMigration(migrationId, migrationPath);
89
+ }
90
+ }
91
+ } catch (error) {
92
+ throw new Error(`Unable to process migrations directory: ${error.message}`);
93
+ }
94
+ }
95
+
96
+ async runMigration(i, name) {
97
+ console.error("Running migration ", i, ": ", name);
98
+
99
+ const sql = await fs.readFile(name, 'utf8');
100
+ await this.db.exec(sql);
101
+ await this.db.run(
102
+ "insert into migrations (id, name) values (?, ?)",
103
+ [i, name]);
104
+ }
105
+
106
+ async insert({...props}) {
107
+ const target_count = await this.db.all(`
108
+ select count(*) as count from target where uuid = ?;
109
+ `, [props.target_id]);
110
+ if (target_count[0].count == 0) {
111
+ await this.db.run(`
112
+ insert into target (uuid) values (?);
113
+ `, [props.target_id]);
114
+ }
115
+ const target = await this.db.all(`
116
+ select target_id from target where uuid = ?;
117
+ `, [props.target_id]);
118
+
119
+ await this.db.run(`
120
+ insert into target_position (
121
+ target_id,
122
+ target_distance,
123
+ target_bearing,
124
+ target_bearing_unit,
125
+ target_speed,
126
+ target_course,
127
+ target_course_unit,
128
+ target_distance_unit,
129
+ target_name,
130
+ target_status,
131
+ latitude,
132
+ longitude,
133
+ target_latitude,
134
+ target_longitude
135
+ ) values (
136
+ ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
137
+ `,
138
+ [target[0].target_id,
139
+ props.position?.relative?.distance,
140
+ props.position?.relative?.bearing,
141
+ props.position?.relative?.bearing_unit,
142
+ props.speedOverGround,
143
+ props.courseOverGroundTrue,
144
+ 'T', // target_course_unit
145
+ props.position?.relative?.distance_unit,
146
+ props.name,
147
+ 'T', //props.target_status,
148
+ props.position?.relative?.position?.latitude,
149
+ props.position?.relative?.position?.longitude,
150
+ props.position?.latitude,
151
+ props.position?.longitude]);
152
+ }
153
+
154
+ async connectionStats() {
155
+ const query1 = await this.db.all(`
156
+ select
157
+ count(*) as unsent_datapoints
158
+ from
159
+ target_position
160
+ where
161
+ not sent;
162
+ `);
163
+ const query2 = await this.db.all(`
164
+ select
165
+ count(*) as unsent_tracks
166
+ from
167
+ (select distinct
168
+ target_id
169
+ from
170
+ target_position
171
+ where
172
+ not sent
173
+ )
174
+ `);
175
+ return {unsent_datapoints: query1[0].unsent_datapoints,
176
+ unsent_tracks: query2[0].unsent_tracks};
177
+ }
178
+
179
+ async retrieve() {
180
+ const query = await this.db.all(`
181
+ select
182
+ target.uuid,
183
+ target_position.timestamp,
184
+ (strftime('%s', timestamp) + strftime('%f', timestamp) - strftime('%S', timestamp)) * 1000
185
+ as timestamp_epoch,
186
+ target_position.target_latitude,
187
+ target_position.target_longitude
188
+ from
189
+ target_position,
190
+ target
191
+ where
192
+ target_position.target_id = (
193
+ select
194
+ target_id
195
+ from
196
+ target_position
197
+ where
198
+ not sent
199
+ and target_id in (
200
+ select
201
+ target_id
202
+ from
203
+ target_position
204
+ group by
205
+ target_id
206
+ having
207
+ count(*) > 1
208
+ )
209
+ order by
210
+ timestamp ASC
211
+ limit 1)
212
+ and target.target_id = target_position.target_id
213
+ and not target_position.sent
214
+ order by timestamp asc
215
+ limit 100;
216
+ `);
217
+
218
+ if (!query.length) return null;
219
+
220
+ const res = {
221
+ uuid: {"string": query[0].uuid},
222
+ route: [],
223
+ nmea: null,
224
+ start: query[0].timestamp_epoch};
225
+
226
+ let isfirst = true;
227
+ let start;
228
+
229
+ for (const row of query) {
230
+ res.route.push({
231
+ lat: row.target_latitude,
232
+ lon: row.target_longitude,
233
+ timestamp: row.timestamp_epoch - res.start
234
+ });
235
+ }
236
+
237
+ return res;
238
+ }
239
+
240
+ async markAsSent(route_message) {
241
+ const end = route_message.route[route_message.route.length - 1].timestamp + route_message.start;
242
+
243
+ const uuid = route_message.uuid.string;
244
+
245
+ const query = await this.db.run(`
246
+ update
247
+ target_position
248
+ set
249
+ sent = 1
250
+ where
251
+ target_id = (select target_id from target where uuid = ?)
252
+ and timestamp <= datetime(? / 1000, 'unixepoch');
253
+ `, [uuid, end]);
254
+
255
+ console.error(
256
+ "Updated "
257
+ + query.changes
258
+ + " rows for "
259
+ + uuid
260
+ + " @ "
261
+ + end
262
+ + ".");
263
+ }
264
+ }
265
+
266
+ module.exports = { Routecache };
@@ -0,0 +1,155 @@
1
+ const net = require("net");
2
+ const { delay } = require('./utils');
3
+
4
+ let _PromiseSocket = null;
5
+ async function getPromiseSocket() {
6
+ if (_PromiseSocket === null) {
7
+ const { PromiseSocket } = await import('promise-socket');
8
+ _PromiseSocket = PromiseSocket;
9
+ }
10
+ return _PromiseSocket;
11
+ }
12
+
13
+ class SocketException extends Error {
14
+ constructor(exc, attempt) {
15
+ const msg = attempt + ": " + exc.toString();
16
+ super(msg);
17
+ this.exc = exc;
18
+ this.attempt = attempt;
19
+ }
20
+ }
21
+
22
+ class TCPClient {
23
+ constructor({ip, port, min_reconnect_time, max_reconnect_time, connect_function, status_function}) {
24
+ this.ip = ip;
25
+ this.port = port;
26
+ this.reconnect_time = 0;
27
+ this.min_reconnect_time = min_reconnect_time;
28
+ this.max_reconnect_time = max_reconnect_time;
29
+ this.connect_function = connect_function;
30
+ this.status_function = status_function;
31
+ this.sock = null;
32
+ this.reconnect_time;
33
+ this.cancelled = false;
34
+ this.buffer = Buffer.alloc(0);
35
+ this.setStatus("Not yet connected");
36
+ }
37
+
38
+ async destroy() {
39
+ this.cancelled = true;
40
+ this.close();
41
+ }
42
+
43
+ close() {
44
+ if (this.sock) {
45
+ console.error("Deleting socket");
46
+ this.sock.destroy();
47
+ this.sock = null;
48
+ }
49
+ }
50
+
51
+ setStatus(status) {
52
+ this.status = status;
53
+ this.status_function?.(status);
54
+ }
55
+
56
+ connectionFailure(error, attempt) {
57
+ this.close();
58
+ this.buffer = Buffer.alloc(0);
59
+ if (this.reconnect_time == 0) {
60
+ this.reconnect_time = this.min_reconnect_time;
61
+ } else if (this.reconnect_time < this.max_reconnect_time) {
62
+ this.reconnect_time = this.reconnect_time * 2;
63
+ }
64
+ const exc = new SocketException(error, attempt);
65
+ if (error.stack !== undefined) exc.stack = error.stack;
66
+ this.setStatus(exc.toString());
67
+ throw exc;
68
+ }
69
+
70
+ async connect() {
71
+ try {
72
+ if (this.cancelled) return;
73
+ console.log("Reconnecting in " + this.reconnect_time + "ms");
74
+ await delay(this.reconnect_time);
75
+ if (this.cancelled) return;
76
+
77
+ // If any exception is thrown from inside Connect that isn't from
78
+ // a ConnectionFailure(), we might have an old socket here that
79
+ // needs cleanup.
80
+ this.close();
81
+
82
+ const PromiseSocket = await getPromiseSocket();
83
+ this.sock = new PromiseSocket(new net.Socket());
84
+ await this.sock.connect(this.port, this.ip);
85
+
86
+ this.setStatus("Connected");
87
+ await this.connect_function?.();
88
+
89
+ this.setStatus("Logged in");
90
+ this.reconnect_time = this.min_reconnect_time;
91
+ } catch (e) {
92
+ this.connectionFailure(e, "Unable to connect");
93
+ }
94
+ }
95
+
96
+ async ensureConnection() {
97
+ while (!this.cancelled && !this.sock) {
98
+ try {
99
+ await this.connect();
100
+ } catch (e) {
101
+ console.error(e.toString());
102
+ console.error(e.stack);
103
+ if (this.cancelled) throw e;
104
+ }
105
+ }
106
+ }
107
+
108
+ async send(data) {
109
+ if (!this.sock) {
110
+ this.connectionFailure("No socket", "Socket disconnected when trying to send");
111
+ }
112
+
113
+ try {
114
+ await this.sock.write(data);
115
+ } catch (e) {
116
+ this.connectionFailure(e, "Send failed");
117
+ }
118
+ }
119
+
120
+ async waitAndSendInitial(data) {
121
+ while (!this.cancelled) {
122
+ await this.ensureConnection();
123
+ try {
124
+ await this.send(data);
125
+ return;
126
+ } catch (e) {
127
+ console.error(e.toString());
128
+ console.error(e.stack);
129
+ }
130
+ }
131
+ }
132
+
133
+ async read() {
134
+ if (this.cancelled) {
135
+ this.connectionFailure("Cancelled", "Interrupted by thread termination");
136
+ }
137
+ try {
138
+ this.buffer = Buffer.concat([
139
+ this.buffer,
140
+ await this.sock.read()]);
141
+ } catch (e) {
142
+ this.connectionFailure(e, "Socket error while reading");
143
+ }
144
+ return this.buffer;
145
+ }
146
+
147
+ consume(size) {
148
+ if (this.buffer.length < size) {
149
+ throw new Error("Not enough data in buffer");
150
+ }
151
+ this.buffer = this.buffer.subarray(size);
152
+ }
153
+ }
154
+
155
+ module.exports = { TCPClient }
@@ -0,0 +1,11 @@
1
+ function delay(ms) {
2
+ return new Promise((resolve) => {
3
+ if (ms <= 0) {
4
+ resolve();
5
+ } else {
6
+ setTimeout(resolve, ms)
7
+ }
8
+ });
9
+ }
10
+
11
+ module.exports = { delay };