signalk-to-noforeignland 0.1.24 → 0.1.25

package/CHANGELOG.md

@@ -1,5 +1,25 @@
+0.1.25
+* CHANGE: Minimum boat move default increased from 50m to 80m
+* CHANGE: Updated the README.md
+* CHANGE: Use public ipv4 DNS instead of local with cache for testInternet()
+* Final version after successful testing SV MOIN and SV KIAPA NUI
+
+0.1.25-beta.3
+* NEW: Check if boat key is set on startup, else report error to dashboard
+* CHANGE: Changing the order and label of the Plugin Settings to make it more clear for unexpierienced users and grouped to Mandatory, Advanced and Expert.
+* CHANGE: Migration of < 0.1.25 Plugin settings to new structure.
+* CHANGE: PluginStatus last track sent "Never" changed to "Not transfered since plugin start" to avoid confusions.
+
+
+0.1.25-beta.2
+* CHANGE: Typo in pluginName fixed
+* CHANGE: Dates for SetPlugin now ISO8601 formated (https://github.com/noforeignland/nfl-signalk/issues/9)
+
+0.1.25-beta.1
+* CHANGE: User mattzilla470 reported timout Issues on VE Cerbo with a small CPU and using 4G (https://github.com/noforeignland/nfl-signalk/issues/7). So added a timout option in the plugin config and a tripple retry while increasing the timeout for the API call.
+
 0.1.24
-* CHANGE: testInternet only uses ipv4 now, some users don't have ipv6 configured properly and where unable to reach the API, when testInternet returned false
+* CHANGE: testInternet() only uses ipv4 now, some users don't have ipv6 configured properly and where unable to reach the API, when testInternet returned false
 * NEW: PluginStatus on SK dashboard now shows last savePoint and last API transfer, so a user has more feedback what the app is doing without enabling the debug log and crawling though it.
 * CHANGE: Renamed CHANGELOG to CHANGELOG.md
 * CHANGE: CHANGELOG ORDER - newest on top.

package/README.md

@@ -1,23 +1,26 @@
 # SignalK To NFL
 Effortlessly log your boat's movement to **noforeignland.com**
 
-## Testing Notes
-- This version includes experimental features
-- Report issues on GitHub
-- Not recommended for production use
-
-
 ## Features
 * Automatically log your position to NFL
 * Send detailed tracks to log your entire trip and not just your final position
 * Can be used in near real time or cache and upload when stopped and data-connection is available.
+* Can sent a 24h keepalive, when off the boat for a while.
 
-An internet connection is required in order to update NFL.
+## Issues
+* Report issues on GitHub (https://github.com/noforeignland/nfl-signalk/issues)
 
 ## Requirements
+* An internet connection is required in order to update NFL.
+* A navigation.position data path inside Signal K for self, which is your current GPS position
 * A **noforeignland.com** account
 * Your Boat API Key from the **noforeignland.com** website: 
   * Account > Settings > Boat tracking > API Key
 
 > Note your Boat API Key is not available in the app. 
-> You must sign in to the **noforeignland.com** website (using the same authentication method you use for the app: Google. Facebook, Email).
+> You must sign in to the **noforeignland.com** website (using the same authentication method you use for the app: Google. Facebook, Email).
+
+## Configuration
+1. Add your boat's API Key into the Server > Plugin Config > Signal K to Noforeignland > Boat API Key
+2. Hit "Submit"
+3. Restart the Signal K server 

package/index.js

@@ -17,7 +17,7 @@ class SignalkToNoforeignland {
   constructor(app) {
     this.app = app;
     this.pluginId = 'signalk-to-noforeignland';
-    this.pluginName = 'SignalK to Noforeland';
+    this.pluginName = 'Signal K to Noforeignland';
     this.creator = 'signalk-track-logger';
 
     // runtime state
@@ -30,79 +30,110 @@ class SignalkToNoforeignland {
     this.lastSuccessfulTransfer = null;
   }
 
-  getSchema() {
-    return {
-      title: this.pluginName,
-      description: 'Some parameters need for use',
-      type: 'object',
-      required: ['apiCron', 'boatApiKey'],
-      properties: {
-        trackFrequency: {
+getSchema() {
+  return {
+    title: this.pluginName,
+    description: 'Some parameters need for use',
+    type: 'object',
+    required: ['boatApiKey', 'apiCron'],
+    properties: {
+      // Mandatory Settings Group
+      mandatory: {
+        type: 'object',
+        title: 'Mandatory Settings',
+        properties: {
+          boatApiKey: {
+            type: 'string',
+            title: 'Boat API Key',
+            description: 'Boat API Key from noforeignland.com. Can be found in Account > Settings > Boat tracking > API Key.'
+          }
+        }
+      },
+      
+      // Advanced Settings Group
+      advanced: {
+        type: 'object',
+        title: 'Advanced Settings',
+        properties: {
+          minMove: {
+            type: 'number',
+            title: 'Minimum boat move to log in meters',
+            description: 'To keep file sizes small we only log positions if a move larger than this size (if set to 0 will log every move)',
+            default: 80
+          },
+          minSpeed: {
+            type: 'number',
+            title: 'Minimum boat speed to log in knots',
+            description: 'To keep file sizes small we only log positions if boat speed goes above this value to minimize recording position on anchor or mooring (if set to 0 will log every move)',
+            default: 1.5
+          },
+          sendWhileMoving: {
+            type: 'boolean',
+            title: 'Attempt sending location while moving',
+            description: 'Should the plugin attempt to send tracking data to NFL while detecting the vessel is moving or only when stopped?',
+            default: true
+          },
+          ping_api_every_24h: {
+            type: 'boolean',
+            title: 'Force a send every 24 hours',
+            description: 'Keeps your boat active on NFL in your current location even if you do not move',
+            default: true
+          }
+        }
+      },
+      
+      // Expert Settings Group
+      expert: {
+        type: 'object',
+        title: 'Expert Settings',
+        properties: {
+          filterSource: {
+            type: 'string',
+            title: 'Position source device',
+            description: 'EMPTY DEFAULT IS FINE - Set this value to the name of a source if you want to only use the position given by that source.'
+          },
+          trackDir: {
+            type: 'string',
+            title: 'Directory to cache tracks',
+            description: 'EMPTY DEFAULT IS FINE - Path in server filesystem, absolute or from plugin directory.\noptional param (only used to keep file cache).'
+          },
+          keepFiles: {
+            type: 'boolean',
+            title: 'Keep track files on disk',
+            description: 'If you have a lot of hard drive space you can keep the track files for logging purposes.',
+            default: false
+          },
+          trackFrequency: {
+            type: 'integer',
+            title: 'Position tracking frequency in seconds',
+            description: 'To keep file sizes small we only log positions once in a while (unless you set this value to 0)',
+            default: 60
+          },
+          apiCron: {
+            type: 'string',
+            title: 'Send attempt CRON',
+            description: 'We send the tracking data to NFL once in a while, you can set the schedule with this setting.\nCRON format: https://crontab.guru/',
+            default: '*/10 * * * *'
+          },
+          internetTestTimeout: {
+            type: 'number',
+            title: 'Timeout for testing internet connection in ms',
+            description: 'Set this number higher for slower computers and internet connections',
+            default: 2000
+          },
+          apiTimeout: {
           type: 'integer',
-          title: 'Position tracking frequency in seconds.',
-          description: 'To keep file sizes small we only log positions once in a while (unless you set this value to 0)',
-          default: 60
-        },
-        minMove: {
-          type: 'number',
-          title: 'Minimum boat move to log in meters',
-          description: 'To keep file sizes small we only log positions if a move larger than this size (if set to 0 will log every move)',
-          default: 50
-        },
-        minSpeed: {
-          type: 'number',
-          title: 'Minimum boat speed to log in knots',
-          description: 'To keep file sizes small we only log positions if boat speed goes above this value to minimize recording position on anchor or mooring (if set to 0 will log every move)',
-          default: 1.5
-        },
-        apiCron: {
-          type: 'string',
-          title: 'Send attempt CRON',
-          description: 'We send the tracking data to NFL once in a while, you can set the schedule with this setting.\nCRON format: https://crontab.guru/',
-          default: '*/10 * * * *'
-        },
-        boatApiKey: {
-          type: 'string',
-          title: 'Boat API key',
-          description: 'Boat API key from noforeignland.com. Can be found in Account > Settings > Boat tracking > API Key.\n*required only in API method is set*'
-        },
-        internetTestTimeout: {
-          type: 'number',
-          title: 'Timeout for testing internet connection in ms',
-          description: 'Set this number higher for slower computers and internet connections',
-          default: 2000
-        },
-        sendWhileMoving: {
-          type: 'boolean',
-          title: 'Attempt sending location while moving',
-          description: 'Should the plugin attempt to send tracking data to NFL while detecting the vessel is moving or only when stopped?',
-          default: true
-        },
-        filterSource: {
-          type: 'string',
-          title: 'Position source device',
-          description: 'Set this value to the name of a source if you want to only use the position given by that source.'
-        },
-        trackDir: {
-          type: 'string',
-          title: 'Directory to cache tracks.',
-          description: 'Path in server filesystem, absolute or from plugin directory.\noptional param (only used to keep file cache).'
-        },
-        keepFiles: {
-          type: 'boolean',
-          title: 'Should keep track files on disk?',
-          description: 'If you have a lot of hard drive space you can keep the track files for logging purposes.',
-          default: false
-        },
-        ping_api_every_24h: {
-          type: 'boolean',
-          title: 'Should I force a send every 24 hours',
-          description: 'Keeps your boat active on NFL in your current location even if you do not move',
-          default: true
+          title: 'API request timeout in seconds',
+          description: 'Timeout for sending data to NFL API. Increase for slow connections.',
+          default: 30,
+          minimum: 10,
+          maximum: 180
+        }
         }
       }
-    };
-  }
+    }
+  };
+}
 
   getPluginObject() {
     return {
@@ -116,38 +147,107 @@ class SignalkToNoforeignland {
   }
 
   async start(options = {}, restartPlugin) {
-    // normalize options
-    this.options = Object.assign({}, options);
-    if (!this.options.trackDir) this.options.trackDir = defaultTracksDir;
-    if (!path.isAbsolute(this.options.trackDir)) {
-      this.options.trackDir = path.join(__dirname, this.options.trackDir);
+  
+  // Backward compatibility: migrate old flat structure to new nested structure
+  let needsSave = false;
+  if (options.boatApiKey && !options.mandatory) {
+    // Old config detected, migrate to new structure
+    this.app.debug('Migrating old configuration to new grouped structure');
+    needsSave = true;
+    
+    options = {
+      mandatory: {
+        boatApiKey: options.boatApiKey
+      },
+      advanced: {
+        minMove: options.minMove !== undefined ? options.minMove : 50,
+        minSpeed: options.minSpeed !== undefined ? options.minSpeed : 1.5,
+        sendWhileMoving: options.sendWhileMoving !== undefined ? options.sendWhileMoving : true,
+        ping_api_every_24h: options.ping_api_every_24h !== undefined ? options.ping_api_every_24h : true
+      },
+      expert: {
+        filterSource: options.filterSource,
+        trackDir: options.trackDir,
+        keepFiles: options.keepFiles !== undefined ? options.keepFiles : false,
+        trackFrequency: options.trackFrequency !== undefined ? options.trackFrequency : 60,
+        internetTestTimeout: options.internetTestTimeout !== undefined ? options.internetTestTimeout : 2000,
+        apiCron: options.apiCron || '*/10 * * * *',
+        apiTimeout: options.apiTimeout !== undefined ? options.apiTimeout : 30
+      }
+    };
+    
+    // Save the migrated configuration
+    try {
+      this.app.debug('Saving migrated configuration...');
+      await this.app.savePluginOptions(options, () => {
+        this.app.debug('Configuration successfully migrated and saved');
+      });
+    } catch (err) {
+      this.app.debug('Failed to save migrated configuration:', err.message);
+      // Continue anyway - the migration will work in memory
     }
+  }
 
-    if (!this.createDir(this.options.trackDir)) {
-      this.stop();
-      return;
-    }
+  // Flatten the nested structure for easier access and apply defaults
+  this.options = {
+    // Mandatory defaults
+    boatApiKey: options.mandatory?.boatApiKey,
+    
+    // Advanced defaults
+    minMove: options.advanced?.minMove !== undefined ? options.advanced.minMove : 80,
+    minSpeed: options.advanced?.minSpeed !== undefined ? options.advanced.minSpeed : 1.5,
+    sendWhileMoving: options.advanced?.sendWhileMoving !== undefined ? options.advanced.sendWhileMoving : true,
+    ping_api_every_24h: options.advanced?.ping_api_every_24h !== undefined ? options.advanced.ping_api_every_24h : true,
+    
+    // Expert defaults
+    filterSource: options.expert?.filterSource,
+    trackDir: options.expert?.trackDir || defaultTracksDir,
+    keepFiles: options.expert?.keepFiles !== undefined ? options.expert.keepFiles : false,
+    trackFrequency: options.expert?.trackFrequency !== undefined ? options.expert.trackFrequency : 60,
+    internetTestTimeout: options.expert?.internetTestTimeout !== undefined ? options.expert.internetTestTimeout : 2000,
+    apiCron: options.expert?.apiCron || '*/10 * * * *',
+    apiTimeout: options.expert?.apiTimeout !== undefined ? options.expert.apiTimeout : 30
+  };
+  
+  // Validate that boatApiKey is set
+  if (!this.options.boatApiKey || this.options.boatApiKey.trim() === '') {
+    const errorMsg = 'No boat API key configured. Please set your API key in plugin settings (Mandatory Settings > Boat API key). You can find your API key at noforeignland.com under Account > Settings > Boat tracking > API Key.';
+    this.app.debug(errorMsg);
+    this.app.setPluginError(errorMsg);
+    this.stop();
+    return;
+  }
+  
+  if (!path.isAbsolute(this.options.trackDir)) {
+    this.options.trackDir = path.join(__dirname, this.options.trackDir);
+  }
+
+  if (!this.createDir(this.options.trackDir)) {
+    this.stop();
+    return;
+  }
 
-    this.app.debug('track logger started, now logging to', this.options.trackDir);
-    this.app.setPluginStatus(`Started`);
-    this.upSince = new Date().getTime();
+  this.app.debug('track logger started, now logging to', this.options.trackDir);
+  this.app.setPluginStatus(`Started${needsSave ? ' (config migrated)' : ''}`);
+  this.upSince = new Date().getTime();
 
-    // adjust default CRON if unchanged
-    if (!this.options.apiCron || this.options.apiCron === '*/10 * * * *') {
-      const startMinute = Math.floor(Math.random() * 10);
-      const startSecond = Math.floor(Math.random() * 60);
-      this.options.apiCron = `${startSecond} ${startMinute}/10 * * * *`;
-    }
+  // adjust default CRON if unchanged
+  if (!this.options.apiCron || this.options.apiCron === '*/10 * * * *') {
+    const startMinute = Math.floor(Math.random() * 10);
+    const startSecond = Math.floor(Math.random() * 60);
+    this.options.apiCron = `${startSecond} ${startMinute}/10 * * * *`;
+  }
 
-    this.app.debug('Setting CRON to ', this.options.apiCron);
+  this.app.debug('Setting CRON to ', this.options.apiCron);
+  this.app.debug('trackFrequency is set to', this.options.trackFrequency, 'seconds');
 
-    // subscribe and logging
-    this.doLogging();
+  // subscribe and logging
+  this.doLogging();
 
-    // start cron job
-    this.cron = new CronJob(this.options.apiCron, this.interval.bind(this));
-    this.cron.start();
-  }
+  // start cron job
+  this.cron = new CronJob(this.options.apiCron, this.interval.bind(this));
+  this.cron.start();
+}
 
   stop() {
     this.app.debug('plugin stopped');
@@ -280,11 +380,10 @@ class SignalkToNoforeignland {
   this.app.debug(`save data point:`, obj);
   await fs.appendFile(path.join(this.options.trackDir, routeSaveName), JSON.stringify(obj) + EOL);
   
-  // Inform user about last saved point
-  const lastSaveTime = new Date().toLocaleString();
-  const lastTransferTime = this.lastSuccessfulTransfer ? this.lastSuccessfulTransfer.toLocaleString() : 'Never';
+  const lastSaveTime = new Date().toISOString();
+  const lastTransferTime = this.lastSuccessfulTransfer ? this.lastSuccessfulTransfer.toISOString() : 'Not transfered since plugin start';
   this.app.setPluginStatus(`Last save: ${lastSaveTime} | Last transfer: ${lastTransferTime}`);
- }
+  }
 
   isValidLatitude(obj) {
     return this.isDefinedNumber(obj) && obj > -90 && obj < 90;
@@ -367,33 +466,41 @@ class SignalkToNoforeignland {
       } 
   }
 
- async testInternet() {
+  async testInternet() {
   const dns = require('dns').promises;
   
   this.app.debug('testing internet connection');
   
-  try {
-    // Force IPv4 DNS lookup with timeout
-    const timeoutMs = this.options.internetTestTimeout || internetTestTimeout;
-    const addresses = await Promise.race([
-      dns.resolve4(internetTestAddress),
-      new Promise((_, reject) => 
-        setTimeout(() => reject(new Error('DNS timeout')), timeoutMs)
-      )
-    ]);
-    
-    if (addresses && addresses.length > 0) {
-      this.app.debug('internet connection = true, resolved IPv4:', addresses[0]);
+  const timeoutMs = this.options.internetTestTimeout || 2000;
+  
+  // Prüfe mehrere öffentliche DNS-Server
+  const dnsServers = [
+    { name: 'Google DNS', ip: '8.8.8.8' },
+    { name: 'Cloudflare DNS', ip: '1.1.1.1' }
+  ];
+  
+  for (const server of dnsServers) {
+    try {
+      // Versuche, den DNS-Server direkt zu erreichen
+      // Wir machen einen reverse lookup auf die IP selbst
+      const result = await Promise.race([
+        dns.reverse(server.ip),
+        new Promise((_, reject) => 
+          setTimeout(() => reject(new Error('DNS timeout')), timeoutMs)
+        )
+      ]);
+      
+      this.app.debug(`internet connection = true, ${server.name} (${server.ip}) is reachable`);
       return true;
-    } else {
-      this.app.debug('internet connection = false, no IPv4 addresses found');
-      return false;
+    } catch (err) {
+      this.app.debug(`${server.name} (${server.ip}) not reachable:`, err.message);
+      // Weiter zum nächsten Server
     }
-  } catch (err) {
-    this.app.debug('internet connection = false, error:', err.message);
-    return false;
   }
- }
+  
+  this.app.debug('internet connection = false, no public DNS servers reachable');
+  return false;
+}
 
   async checkTrack() {
     const trackFile = path.join(this.options.trackDir, routeSaveName);
@@ -429,36 +536,75 @@ class SignalkToNoforeignland {
     const headers = { 'X-NFL-API-Key': pluginApiKey };
     this.app.debug('sending track to API');
 
-    try {
-      const response = await fetch(apiUrl, { method: 'POST', body: params, headers: new fetch.Headers(headers) });
-      if (response.ok) {
-        const responseBody = await response.json();
-        if (responseBody.status === 'ok') {
-          this.lastSuccessfulTransfer = new Date();
-          this.app.debug('Track successfully sent to API');
-          this.app.setPluginStatus(`Started - last Track sent successfully at ${new Date().toLocaleString()}`);
-          if (this.options.keepFiles) {
-            const filename = new Date().toJSON().slice(0, 19).replace(/:/g, '') + '-nfl-track.jsonl';
-            this.app.debug('moving and keeping track file: ', filename);
-            await fs.move(path.join(this.options.trackDir, routeSaveName), path.join(this.options.trackDir, filename));
+    // Retry-Logik mit exponentiell steigendem Timeout
+    const maxRetries = 3;
+    const baseTimeout = (this.options.apiTimeout || 30) * 1000; // Konfigurierbarer Basis-Timeout in ms
+    
+    for (let attempt = 1; attempt <= maxRetries; attempt++) {
+      try {
+        const currentTimeout = baseTimeout * attempt; // 30s, 60s, 90s
+        this.app.debug(`Attempt ${attempt}/${maxRetries} with ${currentTimeout}ms timeout`);
+        
+        // AbortController für Timeout
+        const controller = new AbortController();
+        const timeoutId = setTimeout(() => controller.abort(), currentTimeout);
+        
+        const response = await fetch(apiUrl, { 
+          method: 'POST', 
+          body: params, 
+          headers: new fetch.Headers(headers),
+          signal: controller.signal
+        });
+        
+        clearTimeout(timeoutId);
+        
+        if (response.ok) {
+          const responseBody = await response.json();
+          if (responseBody.status === 'ok') {
+            this.lastSuccessfulTransfer = new Date();
+            this.app.debug('Track successfully sent to API');
+            this.app.setPluginStatus(`Started - last Track sent successfully at ${new Date().toISOString()}`);
+            if (this.options.keepFiles) {
+              const filename = new Date().toJSON().slice(0, 19).replace(/:/g, '') + '-nfl-track.jsonl';
+              this.app.debug('moving and keeping track file: ', filename);
+              await fs.move(path.join(this.options.trackDir, routeSaveName), path.join(this.options.trackDir, filename));
+            } else {
+              this.app.debug('Deleting track file');
+              await fs.remove(path.join(this.options.trackDir, routeSaveName));
+            }
+            return; // Erfolg - beende Funktion
           } else {
-            this.app.debug('Deleting track file');
-            await fs.remove(path.join(this.options.trackDir, routeSaveName));
+            this.app.debug('Could not send track to API, returned response json:', responseBody);
+            // Bei API-Fehler nicht erneut versuchen
+            this.app.setPluginError(`Failed to send track - API returned error.`);
+            return;
+          }
+        } else {
+          this.app.debug('Could not send track to API, returned response code:', response.status, response.statusText);
+          // Bei 4xx Fehler nicht erneut versuchen
+          if (response.status >= 400 && response.status < 500) {
+            this.app.setPluginError(`Failed to send track - HTTP ${response.status}.`);
+            return;
           }
+          // Bei 5xx Fehler retry
+          throw new Error(`HTTP ${response.status}`);
+        }
+      } catch (err) {
+        this.app.debug(`Attempt ${attempt} failed:`, err.message);
+        
+        // Bei letztem Versuch Fehler setzen
+        if (attempt === maxRetries) {
+          this.app.debug('Could not send track to API after', maxRetries, 'attempts:', err);
+          this.app.setPluginError(`Failed to send track after ${maxRetries} attempts - check logs for details.`);
         } else {
-          this.app.debug('Could not send track to API, returned response json:', responseBody);
-          this.app.setPluginError(`Failed to send track - check logs for details.`);
+          // Kurze Pause vor nächstem Versuch
+          const waitTime = 2000 * attempt; // 2s, 4s
+          this.app.debug(`Waiting ${waitTime}ms before retry...`);
+          await new Promise(resolve => setTimeout(resolve, waitTime));
         }
-      } else {
-        this.app.debug('Could not send track to API, returned response code:', response.status, response.statusText);
-        this.app.setPluginError(`Failed to send track - check logs for details.`);
       }
-    } catch (err) {
-      this.app.debug('Could not send track to API due to error:', err);
-      this.app.setPluginError(`Failed to send track - check logs for details.`);
     }
   }
-
   async createTrack(inputPath) {
     const fileStream = fs.createReadStream(inputPath);
     const rl = readline.createInterface({ input: fileStream, crlfDelay: Infinity });

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "signalk-to-noforeignland",
-  "version": "0.1.24",
+  "version": "0.1.25",
   "description": "SignalK track logger to noforeignland.com ",
   "main": "index.js",
   "keywords": [