@fxi/zartigl 0.3.0 → 0.4.0

package/README.md

@@ -14,9 +14,9 @@ Zartigl renders scalar rasters and vector particle fields from multidimensional
 - Vector rendering with GPU particle advection and optional raster magnitude display
 - Time and vertical-dimension controls for depth, elevation, level, or pressure-like axes
 - Point queries for time series and vertical profiles when a point-series store is available
-- Catalog-backed datasets with automatic Zarr/WMTS backend selection for scalar layers
+- UUID-addressed catalog entries with independently selectable Zarr, WMTS, and GeoVideo sources
 - MapLibre custom layer integration, including Mercator and globe rendering paths
-- Pre-rendered GeoVideo backends with true polar globe draping and portable masks
+- Pre-rendered GeoVideo sources with true polar globe draping and portable masks
 - No server component required for public CORS-enabled stores
 
 ## Quick Start
@@ -28,7 +28,7 @@ npm install @fxi/zartigl maplibre-gl
 ```ts
 import maplibregl from "maplibre-gl";
 import { Zartigl } from "@fxi/zartigl";
-import { catalog } from "@fxi/zartigl/catalog";
+import { catalog, searchCatalog } from "@fxi/zartigl/catalog";
 
 const map = new maplibregl.Map({
   container: "map",
@@ -41,13 +41,14 @@ const z = new Zartigl({
   id: "zarr-layer",
   map,
   catalog,
-  backend: "auto",
+  source: "auto",
   // Optional: expose only the month ending at the latest available timestamp.
   timeRange: { trailing: "P1M" },
   geoVideo: { autoplay: false, loop: true, playbackRate: 1 },
 });
 
-await z.setLayer("ocean-current-velocity");
+const currents = searchCatalog("ocean-current-velocity")[0];
+await z.setLayer(currents.id);
 z.setTimeAndDepth(new Date("2025-01-01T00:00:00Z"), 0);
 z.updateSettings({
   palette: "rdylbu",
@@ -55,6 +56,10 @@ z.updateSettings({
 });
 ```
 
+Pass a source UUID (or source type) as the optional second argument when entry
+and source should switch atomically. `setSource()` changes the source of the
+active entry.
+
 `setLayer()` resolves after the field store's consolidated metadata and coordinate axes have loaded. Metadata getters are therefore authoritative after the awaited call, including exact values for irregular time axes.
 
 The root import does not bundle the catalog presets. Import catalog data from `@fxi/zartigl/catalog` only when you want the built-in catalog.
@@ -87,13 +92,7 @@ npm run dev:story
 npm run build:story
 ```
 
-For the smaller public API demo:
-
-```bash
-npm run dev:minimal
-```
-
-The demos read the built-in catalog, which points to public Copernicus Marine ARCO stores. No local data download is needed; visible chunks are fetched directly from cloud object storage or, for some scalar layers, from WMTS when that is the safer backend.
+The demos read the built-in catalog, which points to public browser-accessible Zarr, WMTS, and GeoVideo sources. No local data download is needed; data is fetched directly from the selected source.
 
 ## Public API
 
@@ -139,17 +138,17 @@ Missing remote chunks are interpreted as Zarr fill data. A wholly unavailable
 frame is not cached, so selecting it again can recover after an upstream store
 finishes publishing it; a renderable sparse frame is cached normally.
 
-`VectorLayer`, `ScalarLayer`, `ArcoLayer`, and `ZarrSource` remain exported for advanced use cases that need direct control over renderer internals. Ordinary MapLibre integrations should prefer `Zartigl`.
+`VectorLayer`, `ScalarLayer`, `GeoVideoLayer`, and `ZarrSource` remain exported for advanced use cases that need direct control over renderer internals. Ordinary MapLibre integrations should prefer `Zartigl`.
 
 ## Alternatives And Project Fit
 
-Zartigl overlaps with several useful projects. The goal is not to replace them, but to cover a specific MapX/ARCO workflow: public cloud Zarr stores, MapLibre integration, animated vector particles, scalar rasters, time/depth controls, point inspection, and reproducible widget configuration without a dedicated tile server.
+Zartigl overlaps with several useful projects. The goal is not to replace them, but to cover a specific cloud-native geoscience workflow: public Zarr stores and compatible WMTS/GeoVideo sources, MapLibre integration, animated vector particles, scalar rasters, time/depth controls, point inspection, and reproducible widget configuration without a dedicated tile server.
 
 The table below is a fit matrix based on documented project scope. "Not specified" means the feature is not a stated focus in the referenced project, not that it is impossible.
 
 | Project | Best fit | Zarr raster | Vector particles | Time/depth UI | Query API | Polar/globe rendering | Notes |
 | --- | --- | --- | --- | --- | --- | --- | --- |
-| zartigl | MapX-ready ARCO-like ocean layers | Yes, Zarr v2 | Yes | Built in | Time series and vertical profiles when a point-series store is available | MapLibre globe/ECEF path for lon/lat scalar and vector layers, including over-pole views | Catalog defaults, WMTS fallback for selected scalar layers, MapX snippet generation |
+| zartigl | MapX-ready cloud geoscience layers | Yes, Zarr v2 | Yes | Built in | Time series and vertical profiles when a point-series store is available | MapLibre globe/ECEF path for lon/lat scalar and vector layers, including over-pole views | Provider-neutral catalog, selectable sources, MapX snippet generation |
 | [Copernicus Marine MyOcean Pro](https://data.marine.copernicus.eu/viewer/expert) | Copernicus end-user exploration | Service-managed | Product-dependent | Application UI | Product UI, not a reusable library API | Service-managed | Reference viewer, not a MapLibre/WebGL library to embed in MapX |
 | [@carbonplan/zarr-layer](https://github.com/carbonplan/zarr-layer) | General Zarr raster layers for MapLibre/Mapbox | Yes, Zarr v2/v3 | No, raster-focused | Selector API | GeoJSON queries | Full polar coverage documented for MapLibre with suitable untiled datasets | Strong closest alternative for raster Zarr; supports custom stores, CRS reprojection, custom fragment shaders |
 | [@carbonplan/maps](https://github.com/carbonplan/maps) | React maps for prepared multidimensional rasters | Yes, prepared Zarr pyramids | No | React selector workflow | Not its main focus | Web Mercator-focused | Higher-level React framework; its core MapLibre path is documented as Web Mercator-only |
@@ -160,7 +159,7 @@ The closest general alternative is `@carbonplan/zarr-layer`. It has a broader do
 
 Zartigl remains useful because it combines pieces that are not currently covered together by those projects:
 
-- ARCO-oriented catalog entries and defaults for Copernicus Marine products.
+- Provider-neutral catalog entries with native dataset/variable provenance and source-specific defaults.
 - U/V vector products rendered as animated particles, with optional raster magnitude in the same layer.
 - Time and vertical-axis metadata exposed as application controls.
 - Depth/elevation labeling, ordering, and point inspection tuned for ocean products.
@@ -168,7 +167,7 @@ Zartigl remains useful because it combines pieces that are not currently covered
 - GPU particle-state fallbacks for browser and hardware differences, including float, half-float, and `RGBA8` compatibility mode.
 - A facade API that can generate MapX widget snippets and keep widget state reproducible.
 
-deck.gl, Three.js, or `@carbonplan/zarr-layer` could still be useful integration targets. They would not remove the ARCO catalog, time/depth metadata, vector particle simulation, point-query workflow, or MapX widget-state work that zartigl currently owns.
+deck.gl, Three.js, or `@carbonplan/zarr-layer` could still be useful integration targets. They would not remove the catalog identity/provenance model, time/depth metadata, vector particle simulation, point-query workflow, or MapX widget-state work that zartigl currently owns.
 
 ## Catalog Metadata
 
@@ -183,20 +182,25 @@ To add or update catalog entries:
 ```bash
 uv run scripts/catalog_builder/skills/list_layers.py
 uv run scripts/catalog_builder/skills/search_products.py wave
-uv run scripts/catalog_builder/skills/query_dataset.py <dataset_id>
+uv run scripts/catalog_builder/skills/query_dataset.py <dataset_id> --variable <scalar_id>
 uv run scripts/catalog_builder/skills/validate_catalog.py
+uv run scripts/catalog_builder/skills/validate_remote.py --entry <uuid-or-alias>
 ```
 
+Copernicus scalar defaults are derived from the provider's WMTS JSON legend.
+Use the maintained `analyze_variable.py` fallback only when upstream
+visualization statistics are unavailable.
+
 This requires Python >= 3.12, [uv](https://docs.astral.sh/uv/), and a free [Copernicus Marine](https://data.marine.copernicus.eu/register) account.
 
 ## GeoVideo
 
 GeoVideo pre-renders an expensive scalar Zarr timeline into a short, streamable
 H.264 MP4 while retaining spatial bounds, scientific dates, provenance, units,
-palette, and color domain in a sidecar manifest. Version 2 transports quantized
+palette, and color domain in a sidecar manifest. Version 3 transports quantized
 scalar values in luminance and a static lossless mask as a separate PNG, so the
 browser can apply palette and scalar styling in WebGL. It is a visualization
-backend only: point queries always use the authoritative Zarr point-series store.
+render source only: point queries use a separately configured Zarr point-series source when the entry provides one.
 
 Unlike MapLibre's native four-corner video source, `GeoVideoLayer` maps an
 equirectangular scalar-luma texture through zartigl's subdivided lon/lat mesh.
@@ -206,13 +210,13 @@ static validity mask without doubling the video width.
 ### Displaying a GeoVideo
 
 The built-in catalog includes an Arctic sea-ice thickness animation. Select the
-GeoVideo backend explicitly and enable autoplay when creating the `Zartigl`
+GeoVideo source explicitly and enable autoplay when creating the `Zartigl`
 instance:
 
 ```ts
 import maplibregl from "maplibre-gl";
 import { Zartigl } from "@fxi/zartigl";
-import { catalog } from "@fxi/zartigl/catalog";
+import { catalog, searchCatalog } from "@fxi/zartigl/catalog";
 
 const map = new maplibregl.Map({
   container: "map",
@@ -226,7 +230,7 @@ map.setProjection({ type: "globe" });
 const z = new Zartigl({
   map,
   catalog,
-  backend: "geovideo",
+  source: "auto",
   geoVideo: {
     autoplay: true,
     loop: true,
@@ -234,17 +238,17 @@ const z = new Zartigl({
   },
 });
 
-await z.setLayer("sea-ice-thickness");
+const ice = searchCatalog("sea-ice-thickness")[0];
+await z.setLayer(ice.id);
+await z.setSource(ice.sources.find((source) => source.type === "geovideo")!.id);
 ```
 
-`backend: "geovideo"` selects the published GeoVideo manifest from the
-catalog's `sea-ice-thickness` entry. Globe projection demonstrates its polar
+`setSource()` selects the entry's published GeoVideo manifest by its stable
+source UUID. Globe projection demonstrates its polar
 coverage. `loop` and `playbackRate` are optional playback settings. Browsers
 may restrict autoplay, so embedding applications should provide a user-gesture
 fallback when necessary.
 
-Try the configured instance directly in the [Arctic sea-ice GeoVideo example](https://fxi.io/zartigl/#eyJkIjoxMSwidCI6MTc4MTAwNDMyNTY3OC40OTY4LCJ2IjowLCJwIjoiaWNlIiwicyI6Imdlb3ZpZGVvIiwicHIiOiJnbG9iZSIsImMiOlstMTMyLjQ1NDgyNiw4NS4wNTExMjldLCJ6IjowLjUxMSwiYiI6MCwicGkiOjAsInBkIjowLjA1LCJzcCI6MSwiZiI6MC43LCJybSI6InBhcnRpY2xlcyIsIm9wIjowLjksImxzIjp0cnVlLCJ2YiI6LTEsImNkIjpudWxsLCJnYSI6dHJ1ZSwiZ2wiOnRydWUsImdyIjoxMH0=).
-
 ```bash
 uv run scripts/geovideo/render.py scripts/geovideo/examples/sst-anomaly.json --dry-run
 uv run scripts/geovideo/render.py scripts/geovideo/examples/sst-anomaly.json
@@ -255,6 +259,11 @@ The Arctic sea-ice configuration covers June 2022 through August 2026 and is
 available at
 [`scripts/geovideo/examples/sea-ice-thickness-arctic.json`](scripts/geovideo/examples/sea-ice-thickness-arctic.json).
 
+For exact scientific timelines, GeoVideo also supports discrete sample
+sequences. The Baltic bottom-oxygen example shows every real monthly mean from
+September 1993 through September 2025 without interpolating values; it is available
+at [`scripts/geovideo/examples/baltic-bottom-oxygen-monthly.json`](scripts/geovideo/examples/baltic-bottom-oxygen-monthly.json).
+
 Generation requires FFmpeg. Upload credentials remain in `.env`; no private
 key is bundled into the demo or written to a manifest. See
 [`scripts/geovideo/README.md`](scripts/geovideo/README.md) for the configuration
@@ -262,7 +271,7 @@ and smoke-test workflow.
 
 ## Dataset Scope
 
-The built-in catalog focuses on Copernicus Marine ARCO products, including ocean currents and scalar ocean/ice variables. Other public cloud-native Zarr v2 stores can be used when their store URLs and variables are described in a compatible catalog entry and runtime compatibility is verified. Time, vertical, spatial, and variable metadata are loaded live from consolidated Zarr metadata and coordinate chunks.
+The built-in catalog currently focuses on Copernicus Marine products, but catalog v2 treats Zarr, WMTS, and GeoVideo as independent provider-neutral sources. Public cloud-native Zarr v2 stores can be used when their URLs and variables match the runtime contract; standalone WMTS sources load dimensions from GetCapabilities, and standalone GeoVideo sources load their manifest metadata. Live source metadata remains authoritative.
 
 Good fits include:
 
@@ -274,7 +283,7 @@ Good fits include:
 
 Vector layers need either U/V component variables or a catalog derivation from direction and magnitude. Scalar layers need one numeric variable and enough coordinate metadata to locate chunks by longitude, latitude, time, and optional vertical dimension.
 
-Catalog validation checks the JSON contract, not remote service behavior. For built-in catalog additions, use the ARCO-oriented catalog-builder scripts first. WMTS is only an optional scalar shortcut; for polar products, prefer Zarr unless WMTS coverage is verified to reach the poles.
+Catalog validation checks the JSON contract, not remote service behavior. For built-in catalog additions, use the ARCO-oriented discovery scripts first. WMTS sources must expose a compatible Web Mercator tile matrix; for polar products, prefer Zarr unless WMTS coverage is verified to reach the poles.
 
 ## How It Works
 

package/dist/catalog.d.ts

@@ -1,53 +1,23 @@
 export declare interface Catalog {
-    schemaVersion: 1;
-    layers: CatalogLayer[];
+    schemaVersion: 2;
+    defaultLocale: string;
+    layers: CatalogEntry[];
 }
 
 export declare const catalog: Catalog;
 
-export declare interface CatalogGeoVideo {
+export declare interface CatalogEntry {
     id: string;
-    label?: string;
-    manifestUrl: string;
-}
-
-export declare type CatalogLayer = (CatalogLayerBase & {
-    kind: "scalar";
-    variables: Extract<CatalogVariables, {
-        kind: "scalar";
-    }>;
-}) | (CatalogLayerBase & {
-    kind: "vector";
-    variables: Extract<CatalogVariables, {
-        kind: "vector";
-    }>;
-});
-
-declare interface CatalogLayerBase {
-    id: string;
-    label: string;
-    description?: string;
-    category?: string;
+    aliases?: string[];
+    title: LocalizedText;
+    description?: LocalizedText;
+    category: string;
     tags?: string[];
-    dataset: {
-        id: string;
-        provider?: "copernicus" | string;
-        productId?: string;
-    };
-    stores: {
-        field: {
-            url: string;
-        };
-        pointSeries?: {
-            url: string;
-        };
-        wmts?: CatalogWmts;
-    };
-    derived?: {
-        geoVideos?: CatalogGeoVideo[];
-    };
-    defaults?: {
-        backend?: "zarr" | "geovideo" | "wmts";
+    kind: "scalar" | "vector";
+    sources: CatalogSource[];
+    defaults: {
+        sourceId: string;
+        querySourceId?: string;
         palette?: string;
         renderMode?: "particles" | "raster" | "raster+particles";
         particles?: {
@@ -59,12 +29,53 @@ declare interface CatalogLayerBase {
             opacity?: number;
             logScale?: boolean;
             vibrance?: number;
-            /** Fixed physical-value domain for scalar color normalization. */
             colorDomain?: [number, number];
         };
     };
 }
 
+export declare interface CatalogEntrySelector extends CatalogSearchOptions {
+    identifiers?: Record<string, string>;
+    variableId?: string;
+}
+
+export declare interface CatalogGeoVideoSource extends CatalogSourceBase {
+    type: "geovideo";
+    manifestUrl: string;
+}
+
+export declare interface CatalogSearchOptions {
+    locale?: string;
+    category?: string;
+    sourceType?: CatalogSourceType;
+    provider?: string;
+}
+
+export declare type CatalogSource = CatalogZarrSource | CatalogWmtsSource | CatalogGeoVideoSource;
+
+declare interface CatalogSourceBase {
+    id: string;
+    title: LocalizedText;
+    provenance?: CatalogSourceProvenance;
+    temporal?: CatalogTemporalSummary;
+}
+
+export declare type CatalogSourcePreference = "auto" | CatalogSourceType | string;
+
+export declare interface CatalogSourceProvenance {
+    provider: string;
+    identifiers?: Record<string, string>;
+}
+
+export declare type CatalogSourceType = CatalogSource["type"];
+
+export declare interface CatalogTemporalSummary {
+    cadence?: string;
+    mode: "historical" | "near-real-time" | "analysis-forecast" | "fixed";
+    start?: string;
+    end?: string;
+}
+
 export declare type CatalogVariables = {
     kind: "scalar";
     value: string;
@@ -83,21 +94,42 @@ export declare interface CatalogVectorDerivation {
     output_direction: "from" | "toward";
 }
 
-export declare interface CatalogWmts {
-    capabilities_url: string;
-    base_url: string;
+export declare interface CatalogWmtsSource extends CatalogSourceBase {
+    type: "wmts";
+    capabilitiesUrl: string;
+    baseUrl?: string;
     layer: string;
-    tileMatrixSet: string;
-    format: string;
+    tileMatrixSet?: string;
+    format?: string;
     style?: string;
+    tileUrlTemplate?: string;
 }
 
+export declare interface CatalogZarrSource extends CatalogSourceBase {
+    type: "zarr";
+    endpoints: {
+        field: string;
+        pointSeries?: string;
+    };
+    variables: CatalogVariables;
+}
+
+export declare function findCatalogEntries(selector: CatalogEntrySelector, data?: Catalog): CatalogEntry[];
+
 export declare function formatTime(ms: number): string;
 
 export declare function formatVertical(v: number, label: string, units?: string): string;
 
-export declare function getCatalogLayer(id: string, data?: Catalog): CatalogLayer | undefined;
+export declare function getCatalogEntry(id: string, data?: Catalog): CatalogEntry | undefined;
+
+export declare type LocalizedText = Record<string, string>;
+
+export declare function pickPreferredSource(entry: CatalogEntry): CatalogSource;
+
+export declare function requireCatalogEntry(id: string, data?: Catalog): CatalogEntry;
+
+export declare function resolveLocalizedText(value: LocalizedText | undefined, locale: string, defaultLocale?: string): string;
 
-export declare function requireCatalogLayer(id: string, data?: Catalog): CatalogLayer;
+export declare function searchCatalog(query: string, options?: CatalogSearchOptions, data?: Catalog): CatalogEntry[];
 
 export { }