signalk-chiplog 2.3.0 → 2.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -6,6 +6,47 @@ All notable changes to Chiplog are documented here. The format follows
6
6
 
7
7
  ## [Unreleased]
8
8
 
9
+ ## [2.4.0] - 2026-09-18
10
+
11
+ ### Added
12
+
13
+ - **Previous and next passage links at the top of a passage page**, to step through the log without going back to it.
14
+ `GET /entries/:id` gives them as `previousEntryId` and `nextEntryId`, in the log's order. Alt+← and Alt+→ do the same
15
+ from the keyboard.
16
+ - **A Statistics page** sums the logbook up over a period: number of passages, first and last dates, total distance,
17
+ time under way, top speed, strongest wind, the longest passage without a stop (a passage is cut at its stopovers, each
18
+ stretch measured on its own), the flags of the countries visited, and the top 5 passages by duration, distance,
19
+ average speed, top speed and wind. The period is a shortcut — all time, this month, last month, the last 12 months,
20
+ this year, last year — or a range picked in a calendar with two clicks, the first and the last day. It reads
21
+ `GET /statistics`.
22
+ - **One range picker for every date range** in the webapp: the Animation, Export and Retrospective pages use the same
23
+ calendar — two clicks, the first and the last day — instead of a pair of date fields, so a range with its end before
24
+ its start can no longer be entered.
25
+ - **Places now know their country** (`countryCode` on `GET /places`), from the same geocoding lookup that names them;
26
+ places saved earlier, or added by hand, are asked about in the background once pending names are done. Database
27
+ migration 16.
28
+
29
+ ## [2.3.1] - 2026-09-18
30
+
31
+ ### Added
32
+
33
+ - **Retrospective analysis now shows what has been reconstructed so far while it is still running**, not only once it
34
+ finishes: passages, distance, engine/sail time, track points and events update after every committed slice instead of
35
+ only the clock position. A run that fails partway — an InfluxDB query timing out on a slow host — keeps that same
36
+ summary next to the error, instead of leaving a bare error message with no way to tell what was saved.
37
+ - **A retrospective replay retries an InfluxDB query that times out** up to 3 times, 5 seconds apart, instead of failing
38
+ the whole run on what is often just a Raspberry Pi momentarily busy sharing its InfluxDB with Signal K itself; the
39
+ webapp shows which attempt is under way while it waits. The timeout covers the whole round trip, including a chunk's
40
+ JSON still streaming in after the connection answered, not just getting a connection in the first place. Each window
41
+ of history is also fetched in smaller, two-hour chunks instead of six, so a slow host has less to answer per request.
42
+
43
+ ### Fixed
44
+
45
+ - **A retrospective replay running alongside live tracking** could have a live detection tick, track sample or event
46
+ check land on the passage the replay was reconstructing — mistaking it for the current one, since both read the same
47
+ `active` row — and close it early or splice live position and instrument data into a past passage. Live detection,
48
+ track sampling and event watching now pause for as long as a replay is running.
49
+
9
50
  ## [2.3.0] - 2026-09-18
10
51
 
11
52
  ### Added
@@ -266,7 +307,9 @@ First release.
266
307
  - REST API under `/plugins/signalk-chiplog/api`, documented in [docs/API.md](docs/API.md).
267
308
  - Single SQLite database through Node's built-in `node:sqlite`: no native module to build.
268
309
 
269
- [Unreleased]: https://github.com/ricard33/signalk-chiplog/compare/v2.3.0...HEAD
310
+ [Unreleased]: https://github.com/ricard33/signalk-chiplog/compare/v2.4.0...HEAD
311
+ [2.4.0]: https://github.com/ricard33/signalk-chiplog/compare/v2.3.1...v2.4.0
312
+ [2.3.1]: https://github.com/ricard33/signalk-chiplog/compare/v2.3.0...v2.3.1
270
313
  [2.3.0]: https://github.com/ricard33/signalk-chiplog/compare/v2.2.0...v2.3.0
271
314
  [2.2.0]: https://github.com/ricard33/signalk-chiplog/compare/v2.1.0...v2.2.0
272
315
  [2.1.0]: https://github.com/ricard33/signalk-chiplog/compare/v2.0.0...v2.1.0
package/README.md CHANGED
@@ -201,36 +201,48 @@ Open **Chiplog** from the Signal K webapps, or `/signalk-chiplog/`. Reading need
201
201
  not just what is loaded), then passages grouped by day, newest first, with times, departure and arrival, distance,
202
202
  duration and an engine/sail bar. A passage across midnight appears on both days. Provisional place names are shown as
203
203
  such.
204
- - **Passage page** — summary (distance, duration, average speed, the highest speed and wind seen, and the crew aboard),
205
- map of the track (OpenStreetMap with OpenSeaMap seamarks, which can be hidden) with a small boat marker at the
206
- selected point, a scrubber under the map to step back and forth through its history (defaulting to the latest point,
207
- so it shows the current position on a passage in progress) with a band of that point's time, SOG, COG, STW, TWS, TWD,
208
- TWA and AWA, the marine weather forecast every 3 hours from departure, the tide forecast near the departure (place,
209
- high/low times and heights, height curve) when one was fetched, the engine and sail periods, the boat's status (each
210
- engine's hour counter at departure and arrival and the hours run, and the tank levels and battery charge, voltage and
211
- current noted at departure), and the log: every reading and event in order, including handwritten notes. A passage in
212
- progress refreshes every minute. Each line's comment can be edited (read/write access); a manoeuvre or note the crew
213
- logged themselves can also be deleted automatic lines (alarms, autopilot, weather, corrections) can only be
214
- annotated. Under each position, in grey, its bearing and distance from the nearest landmark.
204
+ - **Passage page** — links at the top to the previous and the next passage (or **Alt+←** and **Alt+→**), then a summary
205
+ (distance, duration, average speed, the highest speed and wind seen, and the crew aboard), map of the track
206
+ (OpenStreetMap with OpenSeaMap seamarks, which can be hidden) with a small boat marker at the selected point, a
207
+ scrubber under the map to step back and forth through its history (defaulting to the latest point, so it shows the
208
+ current position on a passage in progress) with a band of that point's time, SOG, COG, STW, TWS, TWD, TWA and AWA, the
209
+ marine weather forecast every 3 hours from departure, the tide forecast near the departure (place, high/low times and
210
+ heights, height curve) when one was fetched, the engine and sail periods, the boat's status (each engine's hour
211
+ counter at departure and arrival and the hours run, and the tank levels and battery charge, voltage and current noted
212
+ at departure), and the log: every reading and event in order, including handwritten notes. A passage in progress
213
+ refreshes every minute. Each line's comment can be edited (read/write access); a manoeuvre or note the crew logged
214
+ themselves can also be deleted — automatic lines (alarms, autopilot, weather, corrections) can only be annotated.
215
+ Under each position, in grey, its bearing and distance from the nearest landmark.
215
216
  - **Corrections** (read/write access):
216
217
  - rename the departure, or the arrival once the passage is closed — a passage in progress has none yet to rename;
217
218
  - switch an engine period to sail or back;
218
219
  - close a passage in progress, e.g. to confirm an arrival;
219
220
  - merge with the previous or next passage;
220
221
  - delete a passage (admin).
221
- - **Animation** — pick two dates and every passage between them replays on the map, one after another, the port time
222
- skipped. The map follows the boat at a scale chosen for each passage a short hop kept readable rather than
223
- magnified, a long crossing allowed a wider view but never so wide the boat crawls across empty water while a bubble
224
- shows the speed, the distance covered since the start and the date. Play, pause and a slider over the animation's own
225
- time, at ×0,5, ×1, ×2 or ×4. **Export MP4** saves it as a video in one of five shapes (Mobile 9:16, Portrait 3:4,
226
- Square 1:1, Landscape 4:3, Widescreen 16:9). Everything happens in the browser: nothing is rendered or encoded on the
227
- Signal K server, and the map tiles are the only thing downloaded.
222
+ - **Statistics** — the logbook summed up over a period: a shortcut (all time, this month, last month, the last 12
223
+ months, this year, last year) or a range picked in a calendar with two clicks, the first and the last day. It gives
224
+ the number of passages, the dates of the first and the last, total distance, time under way, top speed and strongest
225
+ wind, the longest passage without a stop (a passage with a stopover counts as its stretches between stops) with its
226
+ distance and time, the flags of the countries visited, and the top 5 passages by duration, distance, average speed,
227
+ top speed and wind. Countries come from the place names, so they need geocoding enabled and the boat online now and
228
+ then; places named before this existed fill in by themselves.
229
+ - **Animation** — pick a period and every passage in it replays on the map, one after another, the port time skipped.
230
+ The map follows the boat at a scale chosen for each passage — a short hop kept readable rather than magnified, a long
231
+ crossing allowed a wider view but never so wide the boat crawls across empty water — while a bubble shows the speed,
232
+ the distance covered since the start and the date. Play, pause and a slider over the animation's own time, at ×0,5,
233
+ ×1, ×2 or ×4. **Export MP4** saves it as a video in one of five shapes (Mobile 9:16, Portrait 3:4, Square 1:1,
234
+ Landscape 4:3, Widescreen 16:9). Everything happens in the browser: nothing is rendered or encoded on the Signal K
235
+ server, and the map tiles are the only thing downloaded.
228
236
  - **Export** — download the whole logbook or a date range as a PDF logbook to print, JSON, CSV or GPX, and write the
229
237
  abandon-ship copy to the USB drive now (admin). The PDF is written in the webapp's language and the device's time
230
238
  zone.
231
239
  - **Retrospective** (admin) — reconstruct past passages for a date range from an InfluxDB history (see
232
240
  [Retrospective analysis](#retrospective-analysis-)).
233
241
 
242
+ Every date range in the webapp — Statistics, Animation, Export and Retrospective — is picked the same way: one button
243
+ showing the period, and a calendar under it where two clicks, the first and the last day, make the range. Statistics has
244
+ shortcuts beside it; where any date is allowed, the calendar has an _Any date_ button to clear the range.
245
+
234
246
  **Helm entry** in the top bar opens the tablet entry app.
235
247
 
236
248
  ## The tablet entry app 📱
@@ -410,6 +422,8 @@ at the time.
410
422
  - **Place names.** With geocoding on, the position of each departure and arrival that matches no known place is sent to
411
423
  the geocoding service — OpenStreetMap's public Nominatim by default. Nothing else is sent, and nothing at all when it
412
424
  is off.
425
+ - **Countries.** With geocoding on, a place whose country is not known yet has its position sent to the geocoding
426
+ service once, to ask which country it is in. Nothing at all when it is off.
413
427
  - **Landmarks.** With them on, the area a passage sailed through — a half-degree box, not its track — is sent to the
414
428
  Overpass service, OpenStreetMap's public instance by default, once per area ever. Nothing at all when it is off.
415
429
  - **Tide forecast.** With it on, the departure position of each passage is sent to the tide service — the public
package/index.js CHANGED
@@ -274,6 +274,20 @@ module.exports = function (app) {
274
274
  };
275
275
 
276
276
  function runDetection() {
277
+ // A retrospective replay drives the same detector against a past window,
278
+ // holding its own passage 'active' in log_entries for as long as it takes
279
+ // to close it there — live detection, track sampling and event watching
280
+ // must not touch that row in the meantime, or a live tick would treat a
281
+ // reconstructed passage as the current one and close it early or splice
282
+ // live data into it.
283
+ if (replayJob?.status().running) {
284
+ const status = 'Retrospective replay in progress — live tracking paused';
285
+ if (status !== lastStatus) {
286
+ app.setPluginStatus(status);
287
+ lastStatus = status;
288
+ }
289
+ return;
290
+ }
277
291
  try {
278
292
  const outcome = detector.tick();
279
293
  usbExport.afterDetection(outcome);
@@ -471,11 +485,19 @@ module.exports = function (app) {
471
485
  timers = [
472
486
  setInterval(runDetection, TICK_INTERVAL_MS),
473
487
  setInterval(
474
- guarded('Track recording', () => recorder.sample()),
488
+ guarded('Track recording', () => {
489
+ if (!replayJob?.status().running) {
490
+ recorder.sample();
491
+ }
492
+ }),
475
493
  SAMPLE_INTERVAL_MS
476
494
  ),
477
495
  setInterval(
478
- guarded('Event watching', () => watcher.check()),
496
+ guarded('Event watching', () => {
497
+ if (!replayJob?.status().running) {
498
+ watcher.check();
499
+ }
500
+ }),
479
501
  CHECK_INTERVAL_MS
480
502
  ),
481
503
  setInterval(
package/lib/api.js CHANGED
@@ -12,6 +12,7 @@ const places = require('./places');
12
12
  const propulsion = require('./propulsion');
13
13
  const { getTideForecast } = require('./tide-forecaster');
14
14
  const { getWeatherForecast } = require('./weather-forecaster');
15
+ const { getStatistics } = require('./statistics');
15
16
  const track = require('./track');
16
17
  const v = require('./validation');
17
18
 
@@ -132,6 +133,11 @@ function registerRoutes(router, { getContext, logError }) {
132
133
  handle(({ db, now }, req) => entries.getStats(db, { ...parseRange(req.query), now: now() }))
133
134
  );
134
135
 
136
+ readonly.get(
137
+ '/api/statistics',
138
+ handle(({ db, now }, req) => getStatistics(db, { ...parseRange(req.query), now: now() }))
139
+ );
140
+
135
141
  readonly.get(
136
142
  '/api/entries/:id',
137
143
  handle(({ db }, req) => entries.getEntry(db, entryId(req)))
package/lib/database.js CHANGED
@@ -358,6 +358,19 @@ const MIGRATIONS = [
358
358
  -- passages default to pending, so their areas are fetched too.
359
359
  ALTER TABLE log_entries ADD COLUMN landmarks_pending INTEGER NOT NULL DEFAULT 1
360
360
  CHECK (landmarks_pending IN (0, 1));
361
+ `,
362
+ `
363
+ -- The country a place is in (SPEC §4.14), as an upper-case ISO 3166-1
364
+ -- alpha-2 code, for the statistics page's visited countries. It lives on the
365
+ -- place rather than on each entry: a passage reaches its country through
366
+ -- start_place_id / end_place_id. country_checked says the lookup has been
367
+ -- made, so a place whose position has no country (open water, or geocoding
368
+ -- switched off after the fact) is not asked about forever. Existing places
369
+ -- start unchecked and are filled in by the same background chain that names
370
+ -- pending departures.
371
+ ALTER TABLE places ADD COLUMN country_code TEXT;
372
+ ALTER TABLE places ADD COLUMN country_checked INTEGER NOT NULL DEFAULT 0
373
+ CHECK (country_checked IN (0, 1));
361
374
  `
362
375
  ];
363
376
 
package/lib/entries.js CHANGED
@@ -48,7 +48,7 @@ function requireEntryRow(db, id) {
48
48
  return row;
49
49
  }
50
50
 
51
- // Shared by listEntries and getStats, so a date range means the same thing
51
+ // Shared by listEntries, getStats and the statistics page, so a date range means the same thing
52
52
  // to both: matched on startTime, from inclusive and to exclusive.
53
53
  function rangeWhere({ from, to }) {
54
54
  const conditions = [];
@@ -94,8 +94,27 @@ function getStats(db, { from, to, now }) {
94
94
  return { count: row.count, distance: row.distance, duration: Math.round(row.duration) };
95
95
  }
96
96
 
97
+ // The passages either side of this one in the log's order (start time, then
98
+ // id), for a page to step to its neighbours.
99
+ function neighbourIds(db, row) {
100
+ const before = db
101
+ .prepare(
102
+ `SELECT id FROM log_entries WHERE start_time < ?1 OR (start_time = ?1 AND id < ?2)
103
+ ORDER BY start_time DESC, id DESC LIMIT 1`
104
+ )
105
+ .get(row.start_time, row.id);
106
+ const after = db
107
+ .prepare(
108
+ `SELECT id FROM log_entries WHERE start_time > ?1 OR (start_time = ?1 AND id > ?2)
109
+ ORDER BY start_time, id LIMIT 1`
110
+ )
111
+ .get(row.start_time, row.id);
112
+ return { previousEntryId: before?.id ?? null, nextEntryId: after?.id ?? null };
113
+ }
114
+
97
115
  function getEntry(db, id) {
98
- const entry = toEntry(requireEntryRow(db, id));
116
+ const row = requireEntryRow(db, id);
117
+ const entry = toEntry(row);
99
118
  const count = (table) =>
100
119
  db.prepare(`SELECT COUNT(*) AS n FROM ${table} WHERE entry_id = ?`).get(id).n;
101
120
  const maxSpeed = db
@@ -116,6 +135,7 @@ function getEntry(db, id) {
116
135
  .get(id, id);
117
136
  return {
118
137
  ...entry,
138
+ ...neighbourIds(db, row),
119
139
  counts: {
120
140
  trackPoints: count('track_points'),
121
141
  observations: count('observations'),
@@ -488,6 +508,7 @@ function deleteEntry(db, id) {
488
508
  module.exports = {
489
509
  toEntry,
490
510
  requireEntryRow,
511
+ rangeWhere,
491
512
  listEntries,
492
513
  getStats,
493
514
  getEntry,
@@ -18,13 +18,19 @@ const DEFAULT_QUERY_TIMEOUT_SECONDS = 30;
18
18
 
19
19
  // A resource-constrained host (a Raspberry Pi running both Signal K and
20
20
  // InfluxDB) can be brought down by one query spanning weeks across every
21
- // path at once; six hours at a time keeps each request's result small
21
+ // path at once; two hours at a time keeps each request's result small
22
22
  // regardless of how long the requested range is, with a short pause between
23
23
  // them so the database is never asked for the next one while still
24
24
  // recovering from the last.
25
- const CHUNK_MS = 6 * 60 * 60 * 1000;
25
+ const CHUNK_MS = 2 * 60 * 60 * 1000;
26
26
  const CHUNK_PAUSE_MS = 200;
27
27
 
28
+ // A query that times out is retried rather than failing the whole replay
29
+ // outright -- a Raspberry Pi sharing its InfluxDB with Signal K itself is
30
+ // often just busy for a moment, not actually unreachable.
31
+ const QUERY_MAX_RETRIES = 3;
32
+ const QUERY_RETRY_DELAY_MS = 5 * 1000;
33
+
28
34
  // The motion scan asks for one mean per minute, light enough to cover a week
29
35
  // per request.
30
36
  const SCAN_BUCKET_MS = 60 * 1000;
@@ -112,7 +118,9 @@ function createInfluxHistory({
112
118
  selfContext,
113
119
  queryTimeoutSeconds = INFLUX_DEFAULTS.influxQueryTimeoutSeconds,
114
120
  fetch = globalThis.fetch,
115
- signal
121
+ signal,
122
+ onRetry = () => {},
123
+ retryDelayMs = QUERY_RETRY_DELAY_MS
116
124
  }) {
117
125
  const queryTimeoutMs = queryTimeoutSeconds * 1000;
118
126
  // path -> [{ time, node }], ascending by time; `node` is what readSelfPath
@@ -136,36 +144,59 @@ function createInfluxHistory({
136
144
  epoch: 'ms',
137
145
  q: statements.join(';')
138
146
  });
139
- const timeout = AbortSignal.timeout(queryTimeoutMs);
140
- let response;
141
- try {
142
- response = await fetch(url, {
143
- method: 'POST',
144
- headers,
145
- body,
146
- signal: signal ? AbortSignal.any([timeout, signal]) : timeout
147
- });
148
- } catch (err) {
149
- if (signal?.aborted) {
150
- throw err;
151
- }
152
- if (err.name === 'TimeoutError') {
147
+
148
+ // One full round trip: the timeout signal covers reading the response
149
+ // body too, not just getting the headers back, so a database that is slow
150
+ // to stream a large chunk's JSON times out here just as it would waiting
151
+ // for the connection -- both must retry the same way.
152
+ async function attempt() {
153
+ const timeout = AbortSignal.timeout(queryTimeoutMs);
154
+ let response;
155
+ try {
156
+ response = await fetch(url, {
157
+ method: 'POST',
158
+ headers,
159
+ body,
160
+ signal: signal ? AbortSignal.any([timeout, signal]) : timeout
161
+ });
162
+ } catch (err) {
163
+ if (signal?.aborted || err.name === 'TimeoutError') {
164
+ throw err;
165
+ }
166
+ // Node's fetch wraps a connection failure (wrong host, refused,
167
+ // certificate…) as a bare "fetch failed"; the actual reason is here.
153
168
  throw new Error(
154
- `InfluxDB at ${protocol}://${host}:${port} did not answer within ${queryTimeoutSeconds}s`,
169
+ `Could not reach InfluxDB at ${protocol}://${host}:${port}: ${err.cause?.message ?? err.message}`,
155
170
  { cause: err }
156
171
  );
157
172
  }
158
- // Node's fetch wraps a connection failure (wrong host, refused,
159
- // certificate…) as a bare "fetch failed"; the actual reason is here.
160
- throw new Error(
161
- `Could not reach InfluxDB at ${protocol}://${host}:${port}: ${err.cause?.message ?? err.message}`,
162
- { cause: err }
163
- );
173
+ if (!response.ok) {
174
+ throw new Error(`InfluxDB query failed: ${response.status} ${await response.text()}`);
175
+ }
176
+ return response.json();
164
177
  }
165
- if (!response.ok) {
166
- throw new Error(`InfluxDB query failed: ${response.status} ${await response.text()}`);
178
+
179
+ let payload;
180
+ // Each attempt gets its own full timeout budget, not a shared one left
181
+ // over from the last -- a query that timed out once is retried outright,
182
+ // not with less time to answer than before.
183
+ for (let retries = 0; ; retries += 1) {
184
+ try {
185
+ payload = await attempt();
186
+ onRetry(null);
187
+ break;
188
+ } catch (err) {
189
+ if (signal?.aborted || err.name !== 'TimeoutError') {
190
+ throw err;
191
+ }
192
+ const message = `InfluxDB at ${protocol}://${host}:${port} did not answer within ${queryTimeoutSeconds}s`;
193
+ if (retries >= QUERY_MAX_RETRIES) {
194
+ throw new Error(message, { cause: err });
195
+ }
196
+ onRetry(retries + 1, QUERY_MAX_RETRIES, message);
197
+ await sleep(retryDelayMs);
198
+ }
167
199
  }
168
- const payload = await response.json();
169
200
  return payload.results.map((result) => {
170
201
  if (result.error) {
171
202
  throw new Error(`InfluxDB query error: ${result.error}`);
@@ -482,4 +513,4 @@ function createInfluxHistory({
482
513
  return { preload, clear, scanMotion, readSelfPath };
483
514
  }
484
515
 
485
- module.exports = { createInfluxHistory, INFLUX_DEFAULTS };
516
+ module.exports = { createInfluxHistory, INFLUX_DEFAULTS, CHUNK_MS, QUERY_MAX_RETRIES };
@@ -61,6 +61,19 @@ function pickName(result) {
61
61
  return district && district !== settlement ? `${settlement} (${district})` : settlement;
62
62
  }
63
63
 
64
+ // The country as an upper-case ISO 3166-1 alpha-2 code, which Nominatim gives in
65
+ // lower case. Null when the position has none, as at sea.
66
+ function pickCountry(result) {
67
+ const code = result?.address?.country_code;
68
+ return typeof code === 'string' && /^[a-z]{2}$/i.test(code) ? code.toUpperCase() : null;
69
+ }
70
+
71
+ // Country level for a place asked about only for its country: the answer to a
72
+ // position in a marina at street level would be the same, at more cost to
73
+ // Nominatim's own lookup.
74
+ const COUNTRY_ZOOM = 3;
75
+ const NAME_ZOOM = 17;
76
+
64
77
  function createPlaceNamer({ db, settings, userAgent, fetch = globalThis.fetch }) {
65
78
  const stopping = new AbortController();
66
79
  let failures = 0;
@@ -101,18 +114,35 @@ function createPlaceNamer({ db, settings, userAgent, fetch = globalThis.fetch })
101
114
 
102
115
  // The places table is the geocoding cache: the next departure or arrival
103
116
  // within the radius is named from it, with no request.
104
- function rememberGeocoded(name, { lat, lon }) {
117
+ function rememberGeocoded(name, { lat, lon }, countryCode) {
105
118
  const now = new Date().toISOString();
106
119
  const { lastInsertRowid } = db
107
120
  .prepare(
108
- `INSERT INTO places (name, lat, lon, source, created_at, updated_at)
109
- VALUES (?, ?, ?, 'geocoding', ?, ?)`
121
+ `INSERT INTO places (name, lat, lon, source, country_code, country_checked, created_at, updated_at)
122
+ VALUES (?, ?, ?, 'geocoding', ?, 1, ?, ?)`
110
123
  )
111
- .run(name, lat, lon, now, now);
124
+ .run(name, lat, lon, countryCode, now, now);
112
125
  return { id: Number(lastInsertRowid), name };
113
126
  }
114
127
 
115
- async function lookup({ lat, lon }) {
128
+ // A place added by hand, or one saved before countries were recorded, has
129
+ // not been asked about yet.
130
+ function nextPlaceWithoutCountry() {
131
+ return db
132
+ .prepare('SELECT id, lat, lon FROM places WHERE country_checked = 0 ORDER BY id DESC LIMIT 1')
133
+ .get();
134
+ }
135
+
136
+ // Only a place whose country has not been set since is filled in, so a
137
+ // lookup that was out while the place was deleted or corrected changes nothing.
138
+ function settleCountry({ id }, countryCode) {
139
+ db.prepare(
140
+ 'UPDATE places SET country_code = ?, country_checked = 1 WHERE id = ? AND country_checked = 0'
141
+ ).run(countryCode, id);
142
+ }
143
+
144
+ // The raw answer, or null when Nominatim has nothing for the position.
145
+ async function lookup({ lat, lon }, zoom) {
116
146
  const base = settings.geocodingUrl.endsWith('/')
117
147
  ? settings.geocodingUrl
118
148
  : `${settings.geocodingUrl}/`;
@@ -120,7 +150,7 @@ function createPlaceNamer({ db, settings, userAgent, fetch = globalThis.fetch })
120
150
  url.searchParams.set('format', 'jsonv2');
121
151
  url.searchParams.set('lat', String(lat));
122
152
  url.searchParams.set('lon', String(lon));
123
- url.searchParams.set('zoom', '17');
153
+ url.searchParams.set('zoom', String(zoom));
124
154
  url.searchParams.set('addressdetails', '1');
125
155
 
126
156
  const response = await fetch(url, {
@@ -141,7 +171,36 @@ function createPlaceNamer({ db, settings, userAgent, fetch = globalThis.fetch })
141
171
  return null;
142
172
  }
143
173
  const result = await response.json();
144
- return result.error ? null : pickName(result);
174
+ return result.error ? null : result;
175
+ }
176
+
177
+ // What the caller does with a lookup that could not be made or was
178
+ // interrupted; shared by naming and country lookups.
179
+ function failure(error) {
180
+ if (stopping.signal.aborted) {
181
+ return { outcome: 'stopped' };
182
+ }
183
+ failures += 1;
184
+ return {
185
+ outcome: 'failed',
186
+ error,
187
+ retryInMs: Math.min(FIRST_RETRY_MS * 2 ** (failures - 1), MAX_RETRY_MS)
188
+ };
189
+ }
190
+
191
+ async function resolveCountry(place) {
192
+ let result;
193
+ try {
194
+ result = await lookup(place, COUNTRY_ZOOM);
195
+ } catch (error) {
196
+ return failure(error);
197
+ }
198
+ if (stopping.signal.aborted) {
199
+ return { outcome: 'stopped' };
200
+ }
201
+ failures = 0;
202
+ settleCountry(place, pickCountry(result));
203
+ return { outcome: 'country', retryInMs: NEXT_LOOKUP_MS };
145
204
  }
146
205
 
147
206
  return {
@@ -151,7 +210,8 @@ function createPlaceNamer({ db, settings, userAgent, fetch = globalThis.fetch })
151
210
  }
152
211
  const pending = nextPending();
153
212
  if (!pending) {
154
- return { outcome: 'idle', retryInMs: IDLE_MS };
213
+ const unchecked = nextPlaceWithoutCountry();
214
+ return unchecked ? resolveCountry(unchecked) : { outcome: 'idle', retryInMs: IDLE_MS };
155
215
  }
156
216
 
157
217
  // A lookup for a nearby departure or arrival may already have named it.
@@ -161,25 +221,18 @@ function createPlaceNamer({ db, settings, userAgent, fetch = globalThis.fetch })
161
221
  return { outcome: 'known', retryInMs: 0 };
162
222
  }
163
223
 
164
- let name;
224
+ let result;
165
225
  try {
166
- name = await lookup(pending);
226
+ result = await lookup(pending, NAME_ZOOM);
167
227
  } catch (error) {
168
- if (stopping.signal.aborted) {
169
- return { outcome: 'stopped' };
170
- }
171
- failures += 1;
172
- return {
173
- outcome: 'failed',
174
- error,
175
- retryInMs: Math.min(FIRST_RETRY_MS * 2 ** (failures - 1), MAX_RETRY_MS)
176
- };
228
+ return failure(error);
177
229
  }
178
230
  if (stopping.signal.aborted) {
179
231
  return { outcome: 'stopped' };
180
232
  }
181
233
  failures = 0;
182
234
 
235
+ const name = result ? pickName(result) : null;
183
236
  if (!name) {
184
237
  settle(pending, null);
185
238
  return { outcome: 'no_result', retryInMs: NEXT_LOOKUP_MS };
@@ -187,7 +240,7 @@ function createPlaceNamer({ db, settings, userAgent, fetch = globalThis.fetch })
187
240
  withTransaction(db, () => {
188
241
  const place =
189
242
  findNearestPlace(db, pending, settings.placeMatchRadius) ??
190
- rememberGeocoded(name, pending);
243
+ rememberGeocoded(name, pending, pickCountry(result));
191
244
  settle(pending, place);
192
245
  });
193
246
  return { outcome: 'resolved', name, retryInMs: NEXT_LOOKUP_MS };
package/lib/places.js CHANGED
@@ -54,6 +54,7 @@ function toPlace(row) {
54
54
  name: row.name,
55
55
  position: { lat: row.lat, lon: row.lon },
56
56
  source: row.source,
57
+ countryCode: row.country_code,
57
58
  createdAt: row.created_at,
58
59
  updatedAt: row.updated_at
59
60
  };
package/lib/replay-job.js CHANGED
@@ -13,7 +13,8 @@ function createReplayJob({
13
13
  clock = Date.now,
14
14
  log = () => {},
15
15
  fetch = globalThis.fetch,
16
- onDone = () => {}
16
+ onDone = () => {},
17
+ retryDelayMs
17
18
  }) {
18
19
  let current = null;
19
20
  let lastResult = null;
@@ -73,7 +74,19 @@ function createReplayJob({
73
74
  selfContext: settings.influxSelfContext || app.selfContext,
74
75
  queryTimeoutSeconds: settings.influxQueryTimeoutSeconds,
75
76
  fetch,
76
- signal
77
+ signal,
78
+ retryDelayMs,
79
+ onRetry: (attempt, of, message) => {
80
+ if (attempt !== null) {
81
+ log(
82
+ 'error',
83
+ `Retrospective replay: InfluxDB query timed out, retrying (attempt ${attempt}/${of}): ${message}`
84
+ );
85
+ }
86
+ if (current) {
87
+ current.retry = attempt === null ? null : { attempt, of, message };
88
+ }
89
+ }
77
90
  });
78
91
  try {
79
92
  await runWindowedReplay({
@@ -91,6 +104,11 @@ function createReplayJob({
91
104
  onProgress: (nowMs) => {
92
105
  if (current) {
93
106
  current.now = iso(nowMs);
107
+ // Recomputed each slice so the webapp can show what has actually
108
+ // been committed so far, not just how far the clock has got --
109
+ // the only way to know anything was saved if the run then times
110
+ // out or otherwise fails.
111
+ current.summary = summarise(afterId, to);
94
112
  }
95
113
  }
96
114
  });
@@ -105,7 +123,13 @@ function createReplayJob({
105
123
  summary: summarise(afterId, to)
106
124
  };
107
125
  } else {
108
- lastError = { at: iso(clock()), from, to, message: err.message };
126
+ lastError = {
127
+ at: iso(clock()),
128
+ from,
129
+ to,
130
+ message: err.message,
131
+ summary: summarise(afterId, to)
132
+ };
109
133
  log('error', `Retrospective replay failed (${from} to ${to}): ${err.message}`);
110
134
  }
111
135
  } finally {
@@ -141,7 +165,15 @@ function createReplayJob({
141
165
  );
142
166
  }
143
167
  const controller = new AbortController();
144
- current = { from, to, startedAt: iso(clock()), now: from, phase: 'scanning', controller };
168
+ current = {
169
+ from,
170
+ to,
171
+ startedAt: iso(clock()),
172
+ now: from,
173
+ phase: 'scanning',
174
+ retry: null,
175
+ controller
176
+ };
145
177
  lastError = null;
146
178
  perform(from, to, controller.signal);
147
179
  return { from, to };
@@ -163,7 +195,9 @@ function createReplayJob({
163
195
  from: current.from,
164
196
  to: current.to,
165
197
  now: current.now,
166
- phase: current.phase
198
+ phase: current.phase,
199
+ summary: current.summary ?? null,
200
+ retry: current.retry
167
201
  },
168
202
  lastResult,
169
203
  lastError