@mxtommy/kip 4.4.0 → 4.5.0

package/.github/copilot-instructions.md

@@ -1,6 +1,6 @@
 # KIP – Copilot Instructions (for AI coding agents)
 
-Use this quick-start map to be productive in this repo. Prefer these concrete patterns over generic Angular tips. For depth, see COPILOT.md (root) and .github/instructions/angular.instructions.md.
+Use this quick-start map to be productive in this repo. Prefer these concrete patterns over generic Angular tips. For depth, see `COPILOT.md` (root entrypoint), this file, and `.github/instructions/angular.instructions.md`.
 
 ## Big picture
 - Angular v20+ PWA served under base path /@mxtommy/kip/ (angular.json baseHref, package.json scripts).
@@ -8,6 +8,18 @@ Use this quick-start map to be productive in this repo. Prefer these concrete pa
 - UI: Dashboard(s) with draggable/resizable widgets (gridstack). Themes: light/dark/night via SCSS roles + CSS variables.
 - Storage: Config lives in Signal K when logged in, else local (StorageService). App init via APP_INITIALIZER (AppNetworkInitService).
 
+## Final architecture (2026 Q1)
+- Historical-series orchestration path: `DashboardService` → `DashboardHistorySeriesSyncService` → `KipSeriesApiClientService` → plugin `/plugins/kip/series/reconcile`.
+- Dataset write ownership is centralized in `WidgetDatasetOrchestratorService`; avoid direct dataset create/edit/remove calls from widget/dashboard flows.
+- Shared history mapping path: `HistoryToChartMapperService` performs history-values → chart datapoint adaptation; `DatasetStreamService` delegates to it.
+- Widget delete cleanup uses owner UUID matching (`ownerUuid` and `ownerUuid-*`) through lifecycle service, replacing selector-specific cleanup.
+
+### Migration guardrails
+1. For chart/trend widgets, use lifecycle sync helpers (`syncDataChartDataset`, `syncNumericMiniChartDataset`, `syncWindTrendsDatasets`).
+2. Keep widget UUIDs stable and unique; ownership drives both dataset cleanup and history-series reconciliation.
+3. Route history response mapping changes through `HistoryToChartMapperService` only.
+4. Do not reintroduce legacy selector-branch dataset cleanup in dashboard/widget code.
+
 ## Daily workflows
 - Dev: npm run dev, then open http://localhost:4200/@mxtommy/kip/ (needs a running Signal K server).
 - Build KIP app: npm run build:dev | npm run build:prod (outputs KIP to public/ and respects baseHref).
@@ -121,8 +133,9 @@ Template:
 - SCSS: use variables from src/themes/_m3*.scss; avoid hardcoded hex.
 
 ## Datasets & charts
-- Historical/trend data: DataSetService (src/app/core/services/data-set.service.ts). Create/update/remove in widget lifecycle.
-- Example: src/app/widgets/widget-windtrends-chart uses Chart.js + date-fns and DataSetService for batch-then-live streams.
+- Historical/trend data: DatasetStreamService (src/app/core/services/dataset-stream.service.ts). Create/update/remove in widget lifecycle.
+- Example: src/app/widgets/widget-windtrends-chart uses Chart.js + date-fns and DatasetStreamService for batch-then-live streams.
+- Preferred write path: `WidgetDatasetOrchestratorService` (centralized dataset orchestration for Data Chart / Numeric minichart / Windtrends and owner-based cleanup).
 
 ## Signal K PUT/requests
 - Read via DataService; write via SignalKRequestsService. UI filters PUT-enabled paths (see src/assets/help-docs/putcontrols.md).
@@ -132,6 +145,7 @@ Template:
 - CommonJS deps are explicitly allowed (js-quantities). Avoid introducing new CJS without adding to allowedCommonJsDependencies.
 - Use standalone components, signals, @if/@for; follow .github/instructions/angular.instructions.md for style.
 - Widget config UIs live under src/app/widget-config; path controls use custom validators (no Validators.required). Respect isPathConfigurable and pathRequired.
+- Documentation standard: Every public property and public method in TypeScript code MUST include full JSDoc with: purpose, parameters, return value, and at least one usage example. Apply this by default for all generated/edited code unless a file explicitly cannot use comments.
 
 ## Widgets: do this, not that (Host2)
 - Do: Provide a complete `DEFAULT_CONFIG` with all paths & options. Don’t: Scatter defaults across lifecycle hooks.
@@ -142,16 +156,18 @@ Template:
 - Do: Use widget-embedded or inline directives for composites. Don’t: Reintroduce legacy host wrappers.
 
 ## Key files/dirs
-- Core services: `src/app/core/services/` (DataService, SignalKConnectionService, SignalKDeltaService, AppNetworkInitService, UnitsService, DataSetService, NotificationsService)
+- Core services: `src/app/core/services/` (DataService, SignalKConnectionService, SignalKDeltaService, AppNetworkInitService, UnitsService, DatasetStreamService, NotificationsService)
+- Plugin config foundation: `src/app/core/services/signalk-plugin-config.service.ts` (plugin-only detection, dependency validation, schema normalization metadata, and config persistence via `/plugins` endpoints)
 - Directives: `src/app/core/directives/` (widget-runtime, widget-streams, widget-metadata)
 - Widgets: `src/app/widgets/` (e.g., widget-numeric, widget-gauge-ng-*, widget-data-chart, widget-windtrends-chart, widget-autopilot)
 - Embedded host: `src/app/core/components/widget-embedded/`
 - Config UI: `src/app/widget-config/`
+- Plugin management (server plugins) is handled separately through `PluginConfigClientService` and `/plugins` REST endpoints. Keep install/uninstall out of scope unless explicitly added.
 - Build: `angular.json`, `package.json` scripts
 
 ## Debugging
 - Use Data Inspector (src/app/core/components/data-inspector) to verify live paths/metadata.
-- Dev with source maps: npm run dev. Watch console from DataService/DataSetService for timeouts/lifecycle logs.
+- Dev with source maps: npm run dev. Watch console from DataService/DatasetStreamService for timeouts/lifecycle logs.
 - Embeds (widget-iframe): prefer same-origin or relative URLs to avoid CORS and input-injection limits (see embedwidget.md).
 
 ## SVG Animation Helpers (rAF)

package/.github/instructions/best-practices.instructions.md

@@ -53,3 +53,7 @@ You are an expert in TypeScript, Angular, and scalable web application developme
 - Design services around a single responsibility
 - Use the `providedIn: 'root'` option for singleton services
 - Use the `inject()` function instead of constructor injection
+
+## Documentation
+
+- Every public TypeScript property and public method MUST include full JSDoc with: purpose, parameters, return value, and at least one usage example.

package/.github/instructions/project.instructions.md

@@ -22,6 +22,18 @@ KIP Instrument MFD is an advanced and versatile marine instrumentation package d
   - The main configuration form logic is in `src/app/widget-config/modal-widget-config/`. For unique widget config needs, you may add a new config component (e.g., `modal-widget-<name>-config`).
   - Widget logic/UI and widget configuration are separate concepts that work together.
 
+### Finalized architecture (2026 Q1)
+- **History series reconciliation pipeline:** Dashboard widget topology is normalized by `DashboardHistorySeriesSyncService` and reconciled through `KipSeriesApiClientService` into plugin-backed series definitions.
+- **Dataset lifecycle centralization:** `WidgetDatasetOrchestratorService` is the write-owner for widget datasets (create/edit/remove and owner-based cleanup).
+- **Shared history adaptation:** `HistoryToChartMapperService` centralizes History API values-to-chart datapoint mapping, including average aliases and circular-angle summary stats.
+- **Delete-path cleanup simplification:** Dashboard delete flow now performs owner UUID dataset cleanup via lifecycle service instead of selector-specific branches.
+
+### Architecture guardrails for contributors
+- Replace direct widget/dashboard `DatasetStreamService` write calls with `WidgetDatasetOrchestratorService` sync/remove helpers.
+- Preserve stable widget UUID ownership contracts; these IDs are used for dataset and series reconciliation behavior.
+- Keep history response transformation logic in `HistoryToChartMapperService` to avoid divergence across chart consumers.
+- Avoid reintroducing legacy widget-selector cleanup branches.
+
 ---
 
 ## 4. Conventions & Patterns
@@ -53,6 +65,7 @@ KIP Instrument MFD is an advanced and versatile marine instrumentation package d
 ## 6. Documentation & Comments
 - **Document all custom validators and business rules.**
 - **Update this file and the README with any major changes or new patterns.**
+- **JSDoc requirement:** Every public TypeScript property and public method must include full JSDoc containing: purpose, parameters, return value, and at least one usage example.
 
 ---
 
@@ -101,7 +114,7 @@ All major services in `src/app/core/services/` are summarized below for Copilot
   - Key methods: Event emitters, hotkey handlers.
   - Dependencies: Angular core, browser APIs.
 
-- **AppSettingsService (`app-settings.service.ts`)**
+- **SettingsService (`settings.service.ts`)**
   - Purpose: Manages persistent app settings, user preferences, and configuration storage.
   - Key methods: `getSetting()`, `setSetting()`, config file management.
   - Dependencies: StorageService.
@@ -113,12 +126,12 @@ All major services in `src/app/core/services/` are summarized below for Copilot
 
 - **NotificationsService (`notifications.service.ts`)**
   - Key methods: Notification state management, audio/visual alerts, muting.
-  - Dependencies: AppSettingsService, DataService, SignalkRequestsService.
+  - Dependencies: SettingsService, DataService, SignalkRequestsService.
 
 - **ToastService (`toast.service.ts`)**
   - Purpose: In-app snackbars using Angular Material.
   - Key methods: `show(message, duration = 1500, silent = true, action = 'Dismiss', severity = 'message')`.
-  - Dependencies: MatSnackBar, AppSettingsService.
+  - Dependencies: MatSnackBar, SettingsService.
   - Usage: Use for short, user-visible feedback (login errors, PUT failures, success confirmations). Sound is suppressed when `silent=true` or user sound settings disable audio.
 
 - **CanvasService (`canvas.service.ts`)**
@@ -131,16 +144,40 @@ All major services in `src/app/core/services/` are summarized below for Copilot
   - Key methods: Layout management, widget arrangement, dashboard state.
   - Dependencies: StorageService, WidgetService.
 
-- **DataSetService (`data-set.service.ts`)**
+- **DatasetStreamService (`dataset-stream.service.ts`)**
   - Purpose: Manages data sets, including loading, saving, and updating widget data sources.
   - Key methods: Data set CRUD, data source updates.
   - Dependencies: DataService, StorageService.
 
-- **SignalKPluginsService (`signalk-plugins.service.ts`)**
-  - Purpose: Manages Signal K plugin discovery, configuration, and state.
-  - Key methods: Plugin list management, config updates.
-  - Dependencies: SignalKConnectionService, DataService.
-  - Usage: Used to manage plugins and their configuration.
+- **WidgetDatasetOrchestratorService (`widget-dataset-orchestrator.service.ts`)**
+  - Purpose: Centralized dataset orchestration for widget lifecycle operations.
+  - Key methods: `syncDataChartDataset()`, `syncNumericMiniChartDataset()`, `syncWindTrendsDatasets()`, `removeOwnedDatasets()`.
+  - Dependencies: DatasetStreamService.
+  - Usage: Preferred write path for widget/dash dataset lifecycle actions.
+
+- **HistoryToChartMapperService (`history-to-chart-mapper.service.ts`)**
+  - Purpose: Shared adapter for converting History API value responses into chart datapoints.
+  - Key methods: `mapValuesToChartDatapoints()`.
+  - Dependencies: None (core mapping service).
+  - Usage: Keeps chart history behavior consistent across all consumers.
+
+- **DashboardHistorySeriesSyncService (`dashboard-history-series-sync.service.ts`)**
+  - Purpose: Derives desired historical-series definitions from current dashboard/widget state and reconciles with plugin backend.
+  - Key methods: Internal extraction + debounced reconcile scheduling.
+  - Dependencies: DashboardService, KipSeriesApiClientService, SignalKConnectionService.
+  - Usage: Single orchestration path for add/edit/delete/copy/paste/duplicate widget series convergence.
+
+- **KipSeriesApiClientService (`kip-series-api-client.service.ts`)**
+  - Purpose: Frontend bridge to KIP plugin series reconcile endpoint.
+  - Key methods: `reconcileSeries()`.
+  - Dependencies: HttpClient, SignalKConnectionService.
+  - Usage: Posts full desired series definitions to plugin for canonical reconciliation.
+
+- **PluginConfigClientService (`plugin-config-client.service.ts`)**
+  - Purpose: Plugin configuration foundation service for dependency checks and plugin state/config persistence via Signal K `/plugins` endpoints.
+  - Key methods: `listPlugins()`, `getPlugin()`, `getPluginConfig()`, `savePluginConfig()`, `setPluginEnabled()`, `validateDependency()`, `normalizePluginSchema()`.
+  - Dependencies: HttpClient, SignalKConnectionService.
+  - Usage: Source of truth for plugin detection and config save flows; no plugin install/uninstall support.
 
 - **SignalKRequestsService (`signalk-requests.service.ts`)**
   - Purpose: Handles requests to the Signal K server, such as PUT/POST operations and custom actions.
@@ -155,7 +192,7 @@ All major services in `src/app/core/services/` are summarized below for Copilot
 - **UnitsService (`units.service.ts`)**
   - Purpose: Handles unit conversion and formatting for all displayed data.
   - Key methods: `convert()`, `format()`, unit preference management.
-  - Dependencies: AppSettingsService.
+  - Dependencies: SettingsService.
 
 - **WidgetService (`widget.service.ts`)**
   - Purpose: Manages widget registration, configuration, and lifecycle.

package/CHANGELOG.md

@@ -1,3 +1,16 @@
+# v4.5.0
+## New Features
+* Effortlessly review your vessel’s history with the new Widget Historical Charts—automatically track, store, and visualize key data. Instantly access up to the last full day of performance: just two-finger tap or right-click any widget to open a seamless history dialog—no setup, no clutter, just the trends you need. (Requires Signal K v2.22.1)
+* Dashboards now start with fully populated Data Charts, powered by KIP’s managed Time-Series History-API provider or other compatible history providers. (Requires Signal K v2.22.1)
+* Added internet availability detection for remote queries.
+## Improvements
+* Added "Days" as a selectable time scale in the Data Chart widget.
+* Improved integration by validating server plugin presence, plugin state, and configuration.
+* Added a Node-RED introduction guide to the Help section.
+## Fixes
+* Improved KIP plugin OpenAPI compatibility.
+* Resolved slow Data Inspector performance caused by high resource usage in deep loop logic.
+* Remote Control feature should not require Admin permission. Fixes #940
 # v4.4.0
 ## New Features
 * New AIS Radar widget: Turn AIS traffic into an instant tactical view with live targets, dynamic range rings, fast zoom controls, and smart filters—so you can spot nearby vessels quicker and make confident navigation decisions at a glance.

package/README.md

@@ -24,7 +24,7 @@ With KIP, you get the **familiar feel of professional Multi-Function Displays**
 - [Installations Showcase](#installations-showcase)
 - [Design Goal](#design-goal)
 - [User Experience](#user-experience)
-- [Dashboards and Configuration](#dashboards-and-configuration) & [Widget Library](#widget-library)
+- [Dashboards and Configuration](#dashboards-and-configuration), [Widget Library](#widget-library) & [Historical Data](#historycal-data)
 - [Night Modes](#night-modes)
 - [Chartplotter Mode](#chartplotter-mode)
 - [Remote Control](#remote-control-other-kip-displays)
@@ -54,12 +54,12 @@ The goal is to replicate and enhance the functionality of modern marine instrume
 
 ## User Experience
 
-### Interractions
+### Interactions
 - Swipe up and down to navigate through your dashboards effortlessly.
 - Swipe left and right to access notifications and other system features quickly.
 - Use keyboard shortcuts for essential features, ensuring fast and efficient navigation across devices types.
 
-### Customyze
+### Customize
 - Effortlessly create and customize dashboards using an intuitive grid layout system.
 - Add, resize, and align widgets to design tailored displays for your specific needs.
 - Duplicate widgets or entire dashboards, including their configurations, with a single click.
@@ -137,6 +137,9 @@ Freeboard-SK Chartplotter integration with Autopilot widget
 Grafana integration with other widgets
 ![Embedded Webpage Concept Image](./images/KipGaugeSample3-1024x508.png)
 
+## Historical Data
+Experience effortless insight into your vessel’s past with KIP’s Widget Historical Charts—automatically track, store, and visualize key data, unlocking instant access charts showing up to the last full day of performance. Whether you’re sailing or docked, simply tap or right-click widgets to reveal a seamless history dialog—no setup, no clutter, just the trends you need. With full support for Data Driven widgets, live-to-history transitions, KIP puts your boat’s story at your fingertips—so you can make smarter decisions, spot patterns, and sail with confidence.
+
 ## Night Modes
 Keep your night vision with automatic or manual day and night switching to a color preserving dim mode or an all Red theme. The images below look very dark, but at night... they are perfect!
 
@@ -258,16 +261,17 @@ Once done with your work, from your fork's working branch, make a GitHub pull re
 
 For comprehensive development guidance, please refer to these instruction files:
 
-### **Primary Instructions:**
+### Primary Instructions
 - **[COPILOT.md](./COPILOT.md)**: Complete KIP project guidelines including architecture, services, widget development patterns, theming, and Signal K integration.
-- **[Angular Instructions](./.github/instructions/angular.instructions.md)**: Modern Angular v20+ coding standards, component patterns, and framework best practices.
+- **[Angular Instructions](./.github/instructions/angular.instructions.md)**: Modern Angular v21+ coding standards, component patterns, and framework best practices.
+- **[Copilot Agent Instructions](./.github/copilot-instructions.md)**: Architecture details and coding-agent guardrails for this repository.
 
-### **Development Workflow:**
+### Development Workflow
 1. **Start Here**: Read `COPILOT.md` for KIP-specific architecture and patterns.
 2. **Angular Standards**: Follow `.github/instructions/angular.instructions.md` for modern Angular development.
 3. **Setup & Build**: Use this README for project setup and build commands.
 
-### **Key Priorities:**
+### Key Priorities
 - **Widget Development**: Use the Host2 widget pattern (signals + directives) and scaffold new widgets with the `create-host2-widget` schematic (see `COPILOT.md`).
 - **Angular Patterns**: Use signals, standalone components, and modern control flow.
 - **Theming**: Follow KIP's theme system for consistent UI.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@mxtommy/kip",
-  "version": "4.4.0",
+  "version": "4.5.0",
   "description": "An advanced and versatile marine instrumentation package to display Signal K data.",
   "license": "MIT",
   "author": {
@@ -26,6 +26,7 @@
     "signalk-webapp",
     "signalk-category-instruments",
     "signalk-category-notifications",
+    "signalk-category-ais",
     "signalk-node-server-plugin",
     "signal k",
     "signalk",
@@ -42,6 +43,7 @@
   "signalk-plugin-enabled-by-default": true,
   "scripts": {
     "test": "CI=1 ng test --watch=false",
+    "test:plugin": "npm run build:plugin && node --test --test-concurrency=1 kip-plugin/tests/index.test.cjs",
     "test:interactive": "ng test",
     "test:headless": "CI=1 ng test --browsers=ChromeHeadless --watch=false",
     "lint": "ng lint",
@@ -62,12 +64,12 @@
     "@angular/compiler-cli": "21.1.4",
     "@angular/language-service": "21.1.4",
     "@types/canvas-gauges": "^2.1.8",
+    "@types/d3": "^7.4.3",
     "@types/jasmine": "~3.6.0",
     "@types/jasminewd2": "^2.0.9",
     "@types/js-quantities": "^1.6.6",
     "@types/lodash-es": "^4.17.9",
     "@types/node": "^24.1.0",
-    "@types/d3": "^7.4.3",
     "angular-eslint": "21.2.0",
     "codelyzer": "^6.0.0",
     "eslint": "^9.29.0",
@@ -100,6 +102,7 @@
     "@angular/router": "21.1.4",
     "@aziham/chartjs-plugin-streaming": "^3.5.1",
     "@godind/ng-canvas-gauges": "^6.2.1",
+    "@signalk/server-api": "^2.22.0",
     "@zakj/no-sleep": "^0.13.5",
     "chart.js": "^4.5.1",
     "chartjs-adapter-date-fns": "^3.0.0",
@@ -109,6 +112,7 @@
     "core-js": "^3.13.1",
     "d3": "^7.9.0",
     "date-fns": "^2.30.0",
+    "duckdb": "^1.4.4",
     "gridstack": "^12.3.3",
     "js-quantities": "^1.8.0",
     "lodash-es": "^4.17.23",
@@ -116,10 +120,9 @@
     "prismjs": "^1.30.0",
     "rxjs": "^7.8.2",
     "screenfull": "^6.0.2",
+    "sk-ais-status-plugin": "^1.0.0",
     "steelseries": "^2.0.9",
     "tslib": "^2.6.2",
-    "zone.js": "^0.15.1",
-    "@signalk/server-api": "^2.7.2",
-    "sk-ais-status-plugin": "^1.0.0"
+    "zone.js": "^0.15.1"
   }
 }