@meri-imperiumi/signalk-passage-briefing 0.5.0 → 0.5.2

package/CHANGELOG.md

@@ -2,6 +2,27 @@
 
 ## [Unreleased]
 
+## [0.5.2] - 2026-10-04
+
+### Fixed
+
+- Cached briefings kept serving bulletin blocks computed at compile time, so filtering and parsing fixes only reached the crew after a re-fetch — on the Tactical route view the console still showed the pre-fix boilerplate while the freshly compiled conditions-here view was already clean. The `/api/briefing` serve path now recomputes the merged bulletin console from the bulletin cache against the payload's track on every serve (route and here mode alike, subsuming the old splice-when-missing logic): parsing and filtering improvements apply to cached briefings on the next page load, a payload compiled before a fresher bulletin arrived picks the newer merged view, and a payload whose bulletin cache is gone keeps its own compile-time console.
+- Residual bulletin-console noise after the boilerplate rule went live: the EP1 seasonal coverage note ("FROM OCTOBER 15 TO APRIL 1, DUE TO THE CLIMATOLOGY…") survived by enumerating the newly added severity keywords (freezing spray, dense fog, volcanic ash), and the NWS area-header line ("NORTH PACIFIC EQUATOR TO 30N BETWEEN 140W AND 180W") parsed to a pole-to-pole box on its longitude band, intersecting every water on it — visible as far away as Fiji. Fixed both ways: the fixed NWS disclaimer paragraphs (seasonal note, sea-state definition, 1-minute-winds note, tropical-cyclone pointers, ice-free note, outreach appeal) are now stripped whole by their stable opening lines at cruft-cut time, and "EQUATOR" parses as a real southern bound (lat 0) in the cardinal-bounds reader, so "EQUATOR TO 30N" clips to the bulletin's actual hemisphere.
+- Severity keyword false positives on product and office names: "HIGH SEAS FORECAST" (the masthead) no longer trips the HIGH SEAS keyword and "NATIONAL HURRICANE CENTER" no longer trips the HURRICANE keyword in the boilerplate classifier, so mastheads and forecaster footers cannot masquerade as warnings. HURRICANE and CYCLONE are now severity keywords (kept in sync with the webapp highlighter, which also gained freezing spray, dense fog and volcanic ash) — hurricane and tropical-cyclone paragraphs highlight and survive the console filter, as the crew intended: a cyclone near our waters is on our radar even when the exact bounding phrase parses loosely.
+- The "Warnings On Your Waters" console drowned in bulletin boilerplate (day-one finding from the data-source checklist): the conservative keep for geometry-less blocks meant every NWS masthead, disclaimer ("SEAS GIVEN AS…", "SUPERSEDED BY…", "SECURITE"), section heading and forecaster footer from all four merged bulletins rendered as "warnings" while the actual warning paragraphs were correctly filtered out as off-waters. The block filter now drops blocks that carry neither parseable geography nor a severity keyword — transmission boilerplate is not a warning — with the product name "HIGH SEAS FORECAST" excluded from the severity match so the masthead cannot masquerade as a HIGH SEAS warning. Structured UKHO warnings are unaffected (they are warnings by definition). The severity keyword list gains freezing spray, dense fog and volcanic ash, synced to the webapp highlighter; a warning whose geography failed to parse still survives on its severity alone.
+- NWS High Seas Forecast texts via the api.weather.gov product feed began with a `000` frame line and a bare WMO routing header (`FZPN03 KNHC 030838`), which became the bulletin header ("000") and a junk block in the console. The cruft cutter strips digit-only frame lines and routing headers without the trailing `Z`; the product code line (`HSFEP2`) survives and now serves as the bulletin header.
+- The NWS api.weather.gov product resolver (configured HSF bulletin feeds) failed with `400 Bad Request`: the API started rejecting the `?limit=1` query parameter the resolver appended ("Query parameter limit is not recognized"), and the second resolution step fetched the graph's `id` field — a bare UUID, not a URL. The resolver now requests the bare locations URL, picks the newest iteration by `issuanceTime`, and follows `@id` (falling back to the UUID in the products URL). Fetch failures now carry the failing URL in the message so the source status checklist row detail names the exact request. Verified live: the NP feed returns the FZPN40 High Seas Forecast text.
+- The UKHO structured-warnings source was dead twice over: the `/api/Warnings/Area/{XIV}` JSON endpoint no longer exists (the MSI site is server-rendered HTML now), and it was being queried for every resolved zone although the UKHO only coordinates NAVAREA I — other zones 404 forever. `ukhoWarningsUrl` now returns the Radio Navigational Warnings page for zone 1 and null otherwise, and the page's warning sections (reference, date-time group, full ANMB text in `<pre class="warning-description">`) parse into the same structured warning shape the old JSON fed, so the serve-time filtering pipeline is unchanged. Verified live: 28 in-force warnings parsed with references and timestamps.
+
+### Added
+
+- Data source status checklist (work doc #23): every ingest path — the weather fetch, the per-zone bulletin and UKHO pulls, the configured extra bulletin feeds, the synoptic chart rasters, the GDACS hazard feed and the local celestial ephemeris — now records the outcome of its latest cycle in a persistent per-source registry (`plugin/source-status.js`, survives restarts in `source-status.json`), and the webapp renders it as a collapsed-by-default checklist card in the strategic outlook. Error classification is coarse and stable (`http-404`, `http-429`, `http-4xx`, `http-5xx`, `timeout`, `network`, `parse`, `unavailable`) so a moved URL reads differently from a dead host; cycles skipped while the boat is offline record as `offline-skipped` and never count as failures — an offline boat shows a wall of `[ SKIP ]`, which is correct information, not an error state. The checklist also covers the Signal K side: the subscribed paths the state machine and payload compile consume (internet state, navigation state, house SoC required; active route and the energy outlook optional) and the resource sources (route resources, polar, ship's time, logbook) record availability every ticker minute — a required path with no value since startup reads `[ FAIL ]`, an absent optional source renders muted `[ N/A ]`. Brackets: `[ OK ]` green, `[ WARN ]` orange (429/parse failures, or success stale beyond the source's expected refresh interval), `[ FAIL ]` red, `[ SKIP ]`/`[ N/A ]` muted; the summary line ("12 sources, 1 FAIL") stays visible while collapsed, rows expand to URL, last success, classified error and consecutive-failure count. Served at `GET /plugins/signalk-passage-briefing/sources` for external diagnostics; polling at a minute rate, rows update in place.
+
+## [0.5.1] - 2026-10-03
+
+### Fixed
+- Duplicate satellite-pass lines (ISS ×2, Tiangong ×3): CelesTrak's stations group lists several catalog entries per crewed station — the docked modules ISS (ZARYA), ISS (NAUKA) and CSS (TIANHE), CSS (WENTIAN), CSS (MENGTIAN) — all mapped to the same display name, so every pass was propagated and reported once per module. `parseTLEs` now keeps one element set per station, preferring the core module's (ZARYA, TIANHE).
+
 ## [0.5.0] - 2026-10-03
 
 ### Added

package/SPEC.md

@@ -164,6 +164,12 @@ The plugin monitors `network.internet.state`, `navigation.state`, and `electrica
 * **Cron Schedule Rules:** Runs at `02:15`, `08:15`, `14:15`, and `20:15` UTC (15 minutes after major global ensemble publication windows).
 * **Execution Guard:** Checks if `navigation.state === 'moored' | 'anchored'`. If `navigation.state === 'sailing'`, cron timers are disabled and data fetches are strictly tied to single explicit transitions of `network.internet.state` to `online` or `metered`.
 
+### 2.3 Data Source Status Registry (work doc #23)
+
+Every ingest path records the outcome of its latest cycle in a persistent per-source registry (`plugin/source-status.js`, persisted in `source-status.json`) so a moved URL or dead host is diagnosable mid-passage instead of surfacing only as absence of data. Covered sources: the weather fetch, per-zone bulletin/UKHO pulls, the configured extra bulletin feeds, the synoptic chart rasters, the GDACS hazard feed, the local celestial ephemeris, and — checked on the one-minute ticker — the Signal K sources themselves (required subscribed paths: internet state, navigation state, house SoC; optional: active route, energy outlook, route resources, polar, ship's time, logbook).
+
+Each entry carries `id`, `label`, `kind`, `url`, `lastAttemptAt`, `lastSuccessAt`, `lastError` (`{class, message}`), `consecutiveFailures`, `expectedRefreshMs` and `lastStatus` (`ok` / `fail` / `skip` / `absent`). Error classes are coarse and stable: `http-404`, `http-429`, `http-4xx`, `http-5xx`, `timeout`, `network`, `parse`, `unavailable`. Cycles the online gate skips record `offline-skipped` behavior via `lastStatus: "skip"` — never counted as failures. The registry serves at `GET /plugins/signalk-passage-briefing/sources` (diagnostics, not briefing content) and survives restarts.
+
 ---
 
 ## 3. Data Schemas & Structural Interfaces
@@ -563,6 +569,7 @@ Displays:
 2. Macro Sea State warnings block.
 3. Convective Warning block (CAPE / K-Index).
 4. Raw METAREA text bulletin container.
+5. `<source-status>` checklist (work doc #23): collapsed by default, summary line always visible ("12 sources, 1 FAIL"). Rows: last-attempt stamp (muted) | source label + kind | right-aligned monospace status bracket mapped by the pure `statusVerdict` view model — `[ OK ]` green, `[ WARN ]` orange (429/parse failures, or success stale beyond the source's expected refresh interval), `[ FAIL ]` red, `[ SKIP ]`/`[ N/A ]` muted. Tap expands the row's detail (URL, last success, classified error, consecutive failures). The component polls `GET /plugins/signalk-passage-briefing/sources` at a minute rate and mutates only `textContent`/class attributes; expanded state persists in `localStorage`.
 
 #### `<horizon-sparkline>` (SVG Renderer)
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@meri-imperiumi/signalk-passage-briefing",
-  "version": "0.5.0",
+  "version": "0.5.2",
   "description": "Offshore passage daily briefing webapp for Signal K",
   "main": "plugin/index.js",
   "scripts": {

package/plugin/bulletin-engine.js

@@ -26,15 +26,40 @@
 const SEVERE_KEYWORDS = [
   "GALE",
   "STORM",
+  "HURRICANE",
   "HURRICANE FORCE",
+  "CYCLONE",
   "SQUALL",
   "VIOLENT STORM",
   "ROUGH SEAS",
   "VERY ROUGH SEAS",
   "HIGH SEAS",
   "PHENOMENAL SEAS",
+  "FREEZING SPRAY",
+  "DENSE FOG",
+  "VOLCANIC ASH",
 ];
 
+/**
+ * Whether a block carries a severity keyword (kept in sync with the
+ * webapp's highlighter copy in `public/components/models.mjs`).
+ *
+ * Product and office names must not trip the keywords, or every NWS
+ * masthead survives the boilerplate filter: "HIGH SEAS FORECAST"
+ * would read as a HIGH SEAS warning, "NATIONAL HURRICANE CENTER"
+ * as a hurricane warning.
+ *
+ * @param {string} text
+ * @returns {boolean}
+ */
+function hasSevereKeyword(text) {
+  const upper = String(text ?? "")
+    .toUpperCase()
+    .replace(/HIGH SEAS FORECAST/g, "")
+    .replace(/HURRICANE CENTER/g, "");
+  return SEVERE_KEYWORDS.some((keyword) => upper.includes(keyword));
+}
+
 /**
  * Retained NAVTEX B_2 subject indicators (work doc #4): A navigational
  * warnings, B gale warnings, E meteorological forecasts. The rest
@@ -61,19 +86,46 @@ const SECTION_ANCHORS =
  * @param {string} text - Raw bulletin text
  * @returns {string} Cleaned text
  */
+/**
+ * NWS fixed disclaimer paragraphs, stripped whole by their stable
+ * opening line. These frame every High Seas Forecast (seasonal
+ * coverage note, sea-state definitions, tropical-cyclone pointers);
+ * they are transmission framing, not bulletin content — and with
+ * HURRICANE/CYCLONE in the severity keywords they would otherwise
+ * survive the boilerplate filter by enumeration.
+ */
+const DISCLAIMER_START =
+  /^(SEAS GIVEN AS SIGNIFICANT|SUPERSEDED BY NEXT ISSUANCE|THIS HIGH SEAS FORECAST USES|FORECAST WINDS IN AND NEAR ACTIVE TROPICAL CYCLONES|ONLY YOU KNOW THE WEATHER|FOR ANY TROPICAL CYCLONE INFORMATION|ALL FORECASTS VALID OVER ICE FREE|FROM\s+[A-Z]+\s+\d+\s+TO\s+[A-Z]+\s+\d+\s*,?\s+DUE TO THE CLIMATOLOGY)/i;
+
 function stripBoilerplate(text) {
-  return text
-    .split(/\r?\n/)
-    .filter((line) => {
-      const trimmed = line.trim();
-      if (/^NNNN$/.test(trimmed)) {
-        return false; // End-of-message frame
-      }
-      if (/^[A-Z]{4}\d{2}\s+[A-Z]{4}\s+\d{6}Z/.test(trimmed)) {
-        return false; // Routing header (FQPS01 NFFN 011200Z AUG 26)
-      }
-      return true;
-    })
+  const kept = [];
+  let skipping = false;
+  for (const line of text.split(/\r?\n/)) {
+    const trimmed = line.trim();
+    if (trimmed.length === 0) {
+      skipping = false;
+      kept.push(line);
+      continue;
+    }
+    if (/^NNNN$/.test(trimmed)) {
+      continue; // End-of-message frame
+    }
+    if (/^\d{3,4}$/.test(trimmed)) {
+      continue; // NWS product frame line ("000")
+    }
+    if (/^[A-Z]{4}\d{2}\s+[A-Z]{4}\s+\d{6}Z?/.test(trimmed)) {
+      continue; // Routing header (FQPS01 NFFN 011200Z AUG 26,
+      // or the NWS API's bare "FZPN03 KNHC 030838")
+    }
+    if (!skipping && DISCLAIMER_START.test(trimmed)) {
+      skipping = true; // Fixed disclaimer paragraph: drop to the blank line
+      continue;
+    }
+    if (!skipping) {
+      kept.push(line);
+    }
+  }
+  return kept
     .join("\n")
     .replace(/\bGULF OF AMERICA\b/gi, "GULF OF MEXICO")
     .replace(/\n{3,}/g, "\n\n")
@@ -280,6 +332,22 @@ function parseCardinalBounds(text) {
     minLat = hemisphereDegrees(northOf[1], northOf[2].toUpperCase());
     found = true;
   }
+  // The equator is a real bound in NWS area headers ("NORTH PACIFIC
+  // EQUATOR TO 30N BETWEEN 140W AND 180W"), not a poleward default —
+  // without this the area line parses to a pole-to-pole box that
+  // intersects every water on the longitude band
+  const equatorTo = text.match(/EQUATOR\s+TO\s+(\d+(?:\.\d+)?)\s*([NS])/i);
+  if (equatorTo) {
+    const deg = hemisphereDegrees(equatorTo[1], equatorTo[2].toUpperCase());
+    if (deg >= 0) {
+      minLat = Math.max(minLat, 0);
+      maxLat = Math.min(maxLat, deg);
+    } else {
+      minLat = Math.max(minLat, deg);
+      maxLat = Math.min(maxLat, 0);
+    }
+    found = true;
+  }
 
   // EAST OF x means lon ≥ x; WEST OF y means lon ≤ y. Both present:
   // the box runs from x eastwards to y (unwrapping y across the seam
@@ -829,6 +897,13 @@ function filterBulletin({
       if (!intersectsTrack(geometry, track)) {
         return null; // Discard rule: not on our waters
       }
+      // The warnings console is for relevant paragraphs only: a block
+      // with neither geography nor severity is transmission
+      // boilerplate (preamble, disclaimers, footers) — dropping it
+      // keeps real warnings from drowning in NWS preamble text
+      if (!geometry && !hasSevereKeyword(blockText)) {
+        return null;
+      }
       return {
         text: blockText,
         subject: subject ?? null,
@@ -914,7 +989,11 @@ async function resolveBulletinSource(
         headers: { Accept: "text/plain" },
       });
       if (!response.ok) {
-        throw new Error(`${response.status} ${response.statusText}`);
+        // The URL rides in the message: the source status checklist
+        // shows it verbatim in the row detail (work doc #23)
+        throw new Error(
+          `${url} returned ${response.status} ${response.statusText}`,
+        );
       }
       return { text: await response.text(), source: "api" };
     } finally {
@@ -922,8 +1001,11 @@ async function resolveBulletinSource(
     }
   }
 
-  // api.weather.gov: latest iteration of the product type/location
-  const listUrl = `${apiMatch[0]}?limit=1`;
+  // api.weather.gov: latest iteration of the product type/location.
+  // No query parameters — the API now rejects unknown ones (limit
+  // included) with a 400, so "latest" is resolved client-side by
+  // sorting the returned graph on issuanceTime
+  const listUrl = apiMatch[0];
   const get = async (u) => {
     const controller = new AbortController();
     const timer = setTimeout(() => controller.abort(), timeoutMs ?? 15000);
@@ -933,7 +1015,9 @@ async function resolveBulletinSource(
         headers: { Accept: "application/geo+json, application/json" },
       });
       if (!response.ok) {
-        throw new Error(`${response.status} ${response.statusText}`);
+        throw new Error(
+          `${u} returned ${response.status} ${response.statusText}`,
+        );
       }
       return response.json();
     } finally {
@@ -941,11 +1025,18 @@ async function resolveBulletinSource(
     }
   };
   const list = await get(listUrl);
-  const latest = (list?.["@graph"] ?? [])[0];
-  if (!latest?.id) {
+  // "id" is a bare UUID; the fetchable product URL lives in "@id"
+  const latest = [...(list?.["@graph"] ?? [])]
+    .filter((product) => product?.id)
+    .sort((a, b) =>
+      String(b.issuanceTime ?? "").localeCompare(String(a.issuanceTime ?? "")),
+    )[0];
+  if (!latest) {
     throw new Error("No product iterations available");
   }
-  const product = await get(latest.id);
+  const productUrl =
+    latest["@id"] ?? `https://api.weather.gov/products/${latest.id}`;
+  const product = await get(productUrl);
   const text = product?.productText;
   if (typeof text !== "string" || text.length === 0) {
     throw new Error("Product has no text");
@@ -1018,6 +1109,7 @@ module.exports = {
   SEVERE_KEYWORDS,
   SECTION_ANCHORS,
   stripBoilerplate,
+  hasSevereKeyword,
   navtexSubject,
   shouldRetainSubject,
   segmentBlocks,

package/plugin/bulletin-source.js

@@ -95,6 +95,9 @@ async function saveBulletinCache(dataDir, entries) {
  *   [params.zoneBulletins] - Already-fetched zone bulletins
  * @param {typeof fetch} [params.fetchImpl]
  * @param {number} [params.timeoutMs]
+ * @param {Function} [params.onFailure] - Called with `(url, error)`
+ *   for every URL that failed this cycle (source status registry,
+ *   work doc #23)
  * @returns {Promise<{fetched: string[], failed: string[], entries: BulletinEntry[]}>}
  *   Newly fetched URLs (failures listed separately) and the merged
  *   cache
@@ -105,6 +108,7 @@ async function refreshBulletins({
   zoneBulletins = [],
   fetchImpl = fetch,
   timeoutMs,
+  onFailure,
 }) {
   const cached = await loadBulletinCache(dataDir);
   const fetched = [];
@@ -133,7 +137,8 @@ async function refreshBulletins({
         source,
       });
       fetched.push(url);
-    } catch {
+    } catch (error) {
+      onFailure?.(url, error);
       failed.push(url);
     }
   }

package/plugin/hazard-source.js

@@ -225,12 +225,16 @@ async function saveHazards(dataDir, events) {
  * @param {string} params.dataDir - Plugin data directory
  * @param {typeof fetch} [params.fetchImpl]
  * @param {number} [params.timeoutMs]
+ * @param {Function} [params.onFailure] - Called with the failure
+ *   (transport error or HTTP status error) when the feed could not
+ *   be fetched this cycle (source status registry, work doc #23)
  * @returns {Promise<{fetched: boolean, events: Array<object>}>}
  */
 async function refreshHazards({
   dataDir,
   fetchImpl = fetch,
   timeoutMs = 15000,
+  onFailure,
 }) {
   const cached = await loadHazards(dataDir);
   let incoming = null;
@@ -242,8 +246,13 @@ async function refreshHazards({
     });
     if (response.ok) {
       incoming = parseHazardsXml(await response.text());
+    } else {
+      onFailure?.(
+        new Error(`${response.status} ${response.statusText}`.trim()),
+      );
     }
-  } catch {
+  } catch (error) {
+    onFailure?.(error);
     incoming = null;
   } finally {
     clearTimeout(timer);