@fxi/zartigl 0.5.0 → 0.7.0

package/README.md

@@ -1,11 +1,31 @@
 # zartigl
 
-A MapLibre GL JS plugin for exploring cloud-hosted Zarr geoscience data directly in the browser.
+Cloud-native ocean and climate data on a MapLibre map: a browser runtime, a
+curated catalog, and a pipeline that pre-renders long timelines as streamable
+video.
 
 [![Demo](https://img.shields.io/badge/demo-live-blue)](https://fxi.io/zartigl/)
 [![Story](https://img.shields.io/badge/story-explore-8b5cf6)](https://fxi.io/zartigl/story/)
 
-Zartigl renders scalar rasters and vector particle fields from multidimensional Zarr stores without a dedicated tile server. It is aimed at expert analysis workflows where time, depth or pressure level, point inspection, and reproducible map state matter as much as the visual layer.
+Zartigl started as a MapLibre plugin that renders scalar rasters and vector
+particle fields straight from multidimensional Zarr stores, without a tile
+server. It now also maintains the data side that makes those layers fast and
+reproducible: a provider-neutral catalog of Copernicus Marine products, and
+GeoVideo archives that turn years of daily or hourly fields into chunked H.264
+videos played back continuously in the browser. It is aimed at expert
+workflows where time, depth or pressure level, point inspection, and
+reproducible map state matter as much as the visual layer, and it powers the
+ARCO layers of [MapX](https://app.mapx.org).
+
+## What Is In This Repository
+
+| Part | Location | Role |
+| --- | --- | --- |
+| Runtime library | `src/lib/` | `@fxi/zartigl`: the `Zartigl` facade, Zarr/WMTS/GeoVideo layers, shaders, particle simulation, point queries |
+| Catalog | `src/catalog/`, `scripts/catalog_builder/` | `@fxi/zartigl/catalog` presets and the tools that discover, describe, and validate entries |
+| GeoVideo pipeline | `scripts/geovideo/`, `.github/workflows/geovideo.yml` | Single renders and incremental archives published to S3, rendered daily on a self-hosted runner |
+| Demos | `src/demo-prod/`, `src/demo-story/` | Technical demo and narrative story, deployed to GitHub Pages |
+| MapX integration | `src/mapx/` | Widget and standalone snippet generation for reproducible MapX views |
 
 ## Features
 
@@ -16,7 +36,9 @@ Zartigl renders scalar rasters and vector particle fields from multidimensional
 - Point queries for time series and vertical profiles when a point-series store is available
 - UUID-addressed catalog entries with independently selectable Zarr, WMTS, and GeoVideo sources
 - MapLibre custom layer integration, including Mercator and globe rendering paths
-- Pre-rendered GeoVideo sources with true polar globe draping and portable masks
+- Pre-rendered scalar and vector GeoVideo with true polar globe draping and portable masks
+- GeoVideo archives split into calendar chunks, with the next chunk preloaded so
+  playback runs across the whole record without gaps
 - No server component required for public CORS-enabled stores
 
 ## Quick Start
@@ -48,8 +70,8 @@ const z = new Zartigl({
   timeRange: { trailing: "P1M" },
   time: new Date("2025-01-01T00:00:00Z"),
   depth: 0,
-  settings: { palette: "rdylbu", opacity: 0.9 },
-  geoVideo: { autoplay: false, loop: true, playbackRate: 1 },
+  style: { palette: "rdylbu", raster: { opacity: 0.9 } },
+  geoVideo: { autoplay: false, loop: true, stepsPerSecond: 4 },
 });
 
 z.on("status", (status) => console.log(status.phase));
@@ -59,10 +81,76 @@ await z.init();
 Use `await z.update({ ... })` to change any subset of the active configuration.
 Pass `layer` and `source` together when both should switch atomically.
 
+`source` takes a source id or a preference:
+
+- `"geovideo"`: fast; surface level and GeoVideo dates only.
+- `"zarr"`: all dates and depths; heavier on the client.
+- `"auto"`: GeoVideo, falling back to Zarr for other dates and depths; the
+  layer style always applies. WMTS only when the entry has neither.
+- `"wmts"`: rendered by the server, palette and legend included; no
+  particles, and the layer style has no effect.
+
 `init()` and layer/source updates resolve after consolidated metadata and coordinate axes have loaded. Metadata getters are therefore authoritative afterward, including exact values for irregular time axes.
 
 The root import does not bundle the catalog presets. Import catalog data from `@fxi/zartigl/catalog` only when you want the built-in catalog.
 
+## Layer style
+
+A layer style has the same shape everywhere: catalog `defaults`, the
+`Zartigl` `style` option and `update({ style })`, story views, projversor,
+the MapX `ArcoMapLegend` and the snippets the demo exports.
+
+```jsonc
+{
+  "palette": "thermal",
+  "renderMode": "particles", // or "raster", "raster+particles"
+  "particles": {
+    "density": 0.03,
+    "speed": 1,
+    "fade": 0.7,
+    "grade": "vibrant",
+    "backdrop": { "enabled": true, "color": "#000000", "opacity": 0.3 }
+  },
+  "raster": { "opacity": 0.9, "logScale": false, "vibrance": 0.2, "colorDomain": null }
+}
+```
+
+`particles.grade` is `"off"`, `"vibrant"`, or a tuned vibrant grade
+`{ exposure, contrast, saturation, highlights }`; it is replaced as a whole.
+
+`particles.backdrop` paints a flat color under the particle trails, on the
+valid cells of the field only (the ocean, not the land), so particles stay
+readable over a light basemap. It is off by default; an object is on unless
+`enabled` is false, and its keys left out take the defaults (`color:
+"#000000"`, `#rgb` or `#rrggbb`; `opacity: 0.5`). Like the grade, it is
+replaced as a whole.
+
+Keys merge one by one: engine defaults (`LAYER_STYLE_DEFAULTS`), then the
+catalog entry `defaults`, then the engine's own layer style, then the view.
+From `@fxi/zartigl/catalog`: `resolveEntryStyle(entry, ...styles)` gives the
+style a layer is drawn with, `mergeStyles` patches an override,
+`diffStyle` keeps what differs from a resolved style, and `parseLayerStyle`
+validates, with the path of the offending key. `z.getStyle()` returns the
+style drawn, `z.getStyleOverride()` the override.
+
+The view style is an override that replaces the previous one:
+`update({ style })` and projversor `apply({ style })` return every style key
+to the layer's own, then apply it (`{}` clears it). Without `style`, the
+override stays, across layers too. Scalar layers take no `particles` and only
+the `"raster"` render mode.
+
+`particles.speed` is always the particle speed; playback speed lives
+elsewhere (`geoVideo.stepsPerSecond`, or a projversor view `playback.speed`).
+In projversor, playback also scales the particles by `particleSpeedFactor`:
+0.325 at 1×, 1 at 5×.
+
+`settings` carries engine tuning only (`particleState`,
+`rgba8MaxParticleZoom`). Flat style keys in it (`particleDensity`,
+`opacity`, `colorGrade`…) are deprecated until 1.0: they patch the style
+override, `style` winning, with one warning per key
+(`splitLegacySettings`, `LEGACY_STYLE_SETTINGS`). A custom palette ramp
+stays a setting.
+
 ## Demo
 
 The production demo is the default Vite app and is published at
@@ -114,7 +202,8 @@ z.on("status", (status) => {
 // GeoVideo uses native media playback and reports scientific time updates.
 z.on("timeChange", (time) => console.log(new Date(time)));
 z.on("playbackChange", (playing) => console.log({ playing }));
-await z.update({ geoVideo: { playbackRate: 2, loop: false } });
+// Speed in time steps per second means the same on every artifact.
+await z.update({ geoVideo: { stepsPerSecond: 16, loop: false } });
 
 // Embedding applications can stop rendering and abort field requests while
 // retaining the latest requested time/depth for one reload on resume.
@@ -138,6 +227,24 @@ finishes publishing it; a renderable sparse frame is cached normally.
 
 `VectorLayer`, `ScalarLayer`, `GeoVideoLayer`, and `ZarrSource` remain exported for advanced use cases that need direct control over renderer internals. Ordinary MapLibre integrations should prefer `Zartigl`.
 
+### Entries without MapLibre
+
+Three entries need no MapLibre (`maplibre-gl` is an optional peer):
+
+- `@fxi/zartigl/engine`: GeoVideo manifests, indexes, decoders and archive
+  timelines, palettes and color ramps, color grade, backdrop and trail fade,
+  WebGL helpers and the shared GLSL chunks as strings (`VIBRANCE_GLSL`,
+  `COLOR_GRADE_GLSL`, `RANDOM_GLSL`).
+- `@fxi/zartigl/catalog`: the built-in catalog, named palettes and styles.
+- `@fxi/zartigl/story`: the story runtime, with JSON documents, localized
+  copy and scroll activation.
+
+[projversor](https://github.com/fxi/projversor) (`@fxi/projversor`), the
+renderer of GeoVideo archives on a sphere that morphs between world
+projections, is built on these entries in its own repository, with its demos.
+zartigl never imports it. It replaces the former `@fxi/zartigl/projversor`
+entry: the same names are exported by `@fxi/projversor/zartigl`.
+
 ## Alternatives And Project Fit
 
 Zartigl overlaps with several useful projects. The goal is not to replace them, but to cover a specific cloud-native geoscience workflow: public Zarr stores and compatible WMTS/GeoVideo sources, MapLibre integration, animated vector particles, scalar rasters, time/depth controls, point inspection, and reproducible widget configuration without a dedicated tile server.
@@ -235,17 +342,53 @@ const z = new Zartigl({
   geoVideo: {
     autoplay: true,
     loop: true,
-    playbackRate: 1,
+    stepsPerSecond: 8,
   },
 });
 
 await z.init();
 ```
 
-The source UUID selects the entry's published GeoVideo manifest. Globe projection demonstrates its polar
-coverage. `loop` and `playbackRate` are optional playback settings. Browsers
-may restrict autoplay, so embedding applications should provide a user-gesture
-fallback when necessary.
+The source UUID selects the entry's published GeoVideo. Globe projection
+demonstrates its polar coverage. `loop` and `stepsPerSecond` are optional
+playback settings. Browsers may restrict autoplay, so embedding applications
+should provide a user-gesture fallback when necessary.
+
+### Archives
+
+Most catalog entries carry a GeoVideo archive source: an `indexUrl` pointing to
+an `index.json` that lists immutable calendar chunks (monthly, yearly, or per
+decade, depending on cadence). zartigl reads the index at runtime, revalidates
+it every few minutes, and expands it into one source per chunk, so newly
+published chunks appear without a catalog release. Times outside the published
+chunks, deeper levels, and point queries fall back to Zarr.
+
+During playback the next chunk's manifest, media, and mask load while the
+current one plays; at its end the layer swaps media in place and keeps playing,
+so a whole archive plays as one continuous timeline.
+
+Archives are produced by `scripts/geovideo/archive.py` from the policies in
+[`scripts/geovideo/archive.json`](scripts/geovideo/archive.json). Every product
+is archived over its full upstream time axis; only event scenes, such as the
+regional Cyclone Chido archive, keep a fixed window. The
+[`GeoVideo archive`](.github/workflows/geovideo.yml) workflow renders pending
+chunks daily, and on demand with a longer budget, on the self-hosted `bigproc`
+runner. Published chunks are never dropped from an index, so an archive keeps
+periods that upstream later removes.
+
+Follow progress against the published indexes. The command reads the upstream
+time axes and the indexes with the S3 credentials in `.env`, writes nothing,
+and can run while the workflow renders:
+
+```bash
+npm run geovideo:progress
+#   80/500   Mediterranean Significant Wave Height
+# Total 971/5347 chunks (18.2%); 32/61 archives complete
+```
+
+`npm run geovideo:plan` prints the same plan as JSON, chunk by chunk.
+
+### Single Renders
 
 ```bash
 uv run scripts/geovideo/render.py scripts/geovideo/examples/sst-anomaly.json --dry-run
@@ -253,7 +396,9 @@ uv run scripts/geovideo/render.py scripts/geovideo/examples/sst-anomaly.json
 uv run scripts/geovideo/render.py scripts/geovideo/examples/sst-anomaly.json --upload
 ```
 
-The Arctic sea-ice configuration covers June 2022 through August 2026 and is
+`render.py` (scalar) and `render_vector.py` (vector) produce one standalone
+artifact from a configuration in `scripts/geovideo/examples/`, such as the
+hourly Cyclone Chido wind scene used by the story. The Arctic sea-ice example is
 available at
 [`scripts/geovideo/examples/sea-ice-thickness-arctic.json`](scripts/geovideo/examples/sea-ice-thickness-arctic.json).
 
@@ -320,6 +465,7 @@ npm test
 npm run build:prod
 npm run build:lib
 npm run catalog:validate
+npm run test:geovideo
 ```
 
 `npm run release:check` runs the main validation path before publishing.

package/dist/backdrop-DctkJ45Q.js

@@ -0,0 +1,79 @@
+const i = {
+  exposure: 0.8,
+  contrast: 0.6,
+  saturation: 0.4,
+  highlights: 0.6
+}, l = Object.freeze({
+  enabled: !1,
+  ...i
+}), s = Object.freeze({
+  enabled: !0,
+  ...i
+}), u = {
+  exposure: [0, 4],
+  contrast: [0, 1],
+  saturation: [-1, 1],
+  highlights: [0, 1]
+}, a = u;
+function f(e) {
+  if (!e)
+    return { ...l };
+  const o = {
+    ...s,
+    enabled: e.enabled ?? !0
+  };
+  for (const n of Object.keys(a)) {
+    const t = e[n];
+    if (typeof t == "number" && Number.isFinite(t)) {
+      const [r, b] = a[n];
+      o[n] = Math.min(b, Math.max(r, t));
+    }
+  }
+  return o;
+}
+function h(e) {
+  return [e.exposure, e.contrast, e.saturation, e.highlights];
+}
+function m(e) {
+  return e === void 0 || e === "off" ? { ...l } : e === "vibrant" ? { ...s } : f({ ...e, enabled: !0 });
+}
+function y(e) {
+  const o = f(e);
+  if (!o.enabled)
+    return "off";
+  const n = Object.keys(a);
+  return n.every((t) => o[t] === s[t]) ? "vibrant" : Object.fromEntries(n.map((t) => [t, o[t]]));
+}
+const c = Object.freeze({
+  enabled: !1,
+  color: "#000000",
+  opacity: 0.5
+}), d = /^#(?:[0-9a-f]{3}|[0-9a-f]{6})$/i, O = (e) => typeof e == "string" && d.test(e);
+function R(e) {
+  if (!e)
+    return { ...c };
+  const o = { ...c, enabled: !0 };
+  return typeof e.enabled == "boolean" && (o.enabled = e.enabled), O(e.color) && (o.color = e.color), typeof e.opacity == "number" && Number.isFinite(e.opacity) && (o.opacity = Math.min(1, Math.max(0, e.opacity))), o;
+}
+function p(e) {
+  const o = e.color.slice(1), n = o.length === 3 ? o.split("").map((r) => r + r).join("") : o, t = (r) => parseInt(n.slice(r, r + 2), 16) / 255;
+  return [
+    t(0),
+    t(2),
+    t(4),
+    e.enabled ? e.opacity : 0
+  ];
+}
+export {
+  c as B,
+  l as C,
+  R as a,
+  p as b,
+  h as c,
+  y as d,
+  s as e,
+  u as f,
+  m as g,
+  O as i,
+  f as r
+};

package/dist/catalog.d.ts

@@ -1,3 +1,15 @@
+/**
+ * Solid color under the particle trails, drawn on valid data cells only: it
+ * keeps particles readable over a light basemap. Off by default; an object
+ * without `enabled` is on.
+ */
+export declare interface BackdropStyle {
+    enabled: boolean;
+    /** Hex color, `#rgb` or `#rrggbb`. */
+    color: string;
+    opacity: number;
+}
+
 export declare interface Catalog {
     schemaVersion: 2;
     defaultLocale: string;
@@ -15,22 +27,10 @@ export declare interface CatalogEntry {
     tags?: string[];
     kind: "scalar" | "vector";
     sources: CatalogSource[];
-    defaults: {
+    /** Default source, plus the layer's base style. */
+    defaults: LayerStyleInput & {
         sourceId: string;
         querySourceId?: string;
-        palette?: string;
-        renderMode?: "particles" | "raster" | "raster+particles";
-        particles?: {
-            density?: number;
-            speed?: number;
-            fade?: number;
-        };
-        raster?: {
-            opacity?: number;
-            logScale?: boolean;
-            vibrance?: number;
-            colorDomain?: [number, number];
-        };
     };
 }
 
@@ -39,9 +39,21 @@ export declare interface CatalogEntrySelector extends CatalogSearchOptions {
     variableId?: string;
 }
 
+/**
+ * An incremental archive of calendar-aligned artifacts listed by a mutable
+ * index; zartigl resolves it to the chunk covering the requested time.
+ */
+declare interface CatalogGeoVideoArchiveSource extends CatalogSourceBase {
+    type: "geovideo";
+    indexUrl: string;
+    manifestUrl?: never;
+}
+
+/** One published artifact. */
 export declare interface CatalogGeoVideoSource extends CatalogSourceBase {
     type: "geovideo";
     manifestUrl: string;
+    indexUrl?: never;
 }
 
 export declare interface CatalogSearchOptions {
@@ -51,7 +63,7 @@ export declare interface CatalogSearchOptions {
     provider?: string;
 }
 
-export declare type CatalogSource = CatalogZarrSource | CatalogWmtsSource | CatalogGeoVideoSource;
+export declare type CatalogSource = CatalogZarrSource | CatalogWmtsSource | CatalogGeoVideoSource | CatalogGeoVideoArchiveSource;
 
 declare interface CatalogSourceBase {
     id: string;
@@ -114,6 +126,9 @@ export declare interface CatalogZarrSource extends CatalogSourceBase {
     variables: CatalogVariables;
 }
 
+/** The keys of `style` whose value differs from `base`'s. */
+export declare function diffStyle(style: LayerStyleInput, base: LayerStyle): LayerStyleInput;
+
 export declare function findCatalogEntries(selector: CatalogEntrySelector, data?: Catalog): CatalogEntry[];
 
 export declare function formatTime(ms: number): string;
@@ -122,14 +137,158 @@ export declare function formatVertical(v: number, label: string, units?: string)
 
 export declare function getCatalogEntry(id: string, data?: Catalog): CatalogEntry | undefined;
 
+/** Vibrant grade values; each one left out takes the vibrant preset's. */
+export declare interface GradeTuning {
+    /** Mid-tone lift [0, 4]. */
+    exposure?: number;
+    /** S-curve strength [0, 1]. */
+    contrast?: number;
+    /** Vibrance-weighted saturation [-1, 1]. */
+    saturation?: number;
+    /** Push of the bright half toward white [0, 1]. */
+    highlights?: number;
+}
+
+/** Engine fallbacks, under the catalog `defaults`. */
+export declare const LAYER_STYLE_DEFAULTS: Readonly<LayerStyle>;
+
+export declare interface LayerStyle {
+    /** Palette id; undefined lets the engine pick its own. */
+    palette?: string;
+    renderMode: StyleRenderMode;
+    particles: ParticleStyle;
+    raster: RasterStyle;
+}
+
+/** A partial style: any key left out takes the value below it. */
+export declare interface LayerStyleInput {
+    palette?: string;
+    renderMode?: StyleRenderMode;
+    particles?: ParticleStyleInput;
+    raster?: Partial<RasterStyle>;
+}
+
 export declare type LocalizedText = Record<string, string>;
 
-export declare function pickPreferredSource(entry: CatalogEntry): CatalogSource;
+/**
+ * One partial style from several, later inputs winning key by key: patch an
+ * override without resolving defaults.
+ */
+export declare function mergeStyles(...inputs: (LayerStyleInput | undefined)[]): LayerStyleInput;
+
+/**
+ * Validate a style, throwing an error that names the offending path, e.g.
+ * `views.swell.config.style.particles.density must be a number in (0, 0.2]`.
+ */
+export declare function parseLayerStyle(value: unknown, path: string, options?: ParseLayerStyleOptions): LayerStyleInput;
+
+/** Which keys a caller supports, and for which layer kind. */
+export declare interface ParseLayerStyleOptions {
+    kind?: "scalar" | "vector";
+    /** Keys left out are rejected as unsupported; all when omitted. */
+    supports?: {
+        palette?: boolean;
+        renderMode?: boolean;
+        particles?: readonly (keyof ParticleStyle)[];
+        raster?: readonly (keyof RasterStyle)[];
+    };
+}
+
+/** Style keys, for checks against other definitions of the same shape. */
+export declare const PARTICLE_STYLE_KEYS: (keyof ParticleStyle)[];
+
+/** Particle trail color grade: a preset, or the vibrant one tuned. */
+export declare type ParticleGrade = "off" | "vibrant" | GradeTuning;
+
+export declare interface ParticleStyle {
+    /** Particles per device pixel, (0, 0.2]. */
+    density: number;
+    /** Css px per frame for the fastest value of the field, (0, 10]. */
+    speed: number;
+    /** Trail length, [0, 1] (longer when higher). */
+    fade: number;
+    /** Trail color grade; replaced as a whole, not merged. */
+    grade: ParticleGrade;
+    /** Solid color under the trails; replaced as a whole, not merged. */
+    backdrop: BackdropStyle;
+}
+
+/** Partial particle style; a partial backdrop takes the defaults' keys. */
+export declare type ParticleStyleInput = Partial<Omit<ParticleStyle, "backdrop">> & {
+    backdrop?: Partial<BackdropStyle>;
+};
+
+/**
+ * Preferred render source: GeoVideo, then Zarr, then WMTS (scalar), skipping
+ * fixed-period sources that cannot show the requested time. Several GeoVideo
+ * periods may coexist; the latest covering one wins. GeoVideo is a
+ * lightweight display transport; Zarr stays the fallback and query source.
+ */
+export declare function pickPreferredSource(entry: CatalogEntry, request?: SourceTimeRequest): CatalogSource;
+
+/**
+ * First source type in `priority` that the entry provides, preferring sources
+ * covering the request, then the latest period. Undefined when no type matches.
+ */
+export declare function pickSourceByPriority(entry: CatalogEntry, priority: readonly CatalogSource["type"][], request?: SourceTimeRequest): CatalogSource | undefined;
+
+export declare const RASTER_STYLE_KEYS: (keyof RasterStyle)[];
+
+export declare interface RasterStyle {
+    opacity: number;
+    logScale: boolean;
+    /** Vibrance adjustment, [-1, 1]. */
+    vibrance: number;
+    /** Fixed physical-value domain; null follows the data. */
+    colorDomain: [number, number] | null;
+}
 
 export declare function requireCatalogEntry(id: string, data?: Catalog): CatalogEntry;
 
+/**
+ * The style a catalog layer is drawn with: its own style, then `inputs`.
+ * Scalar layers draw a raster.
+ */
+export declare function resolveEntryStyle(entry: StyledEntry | undefined, ...inputs: (LayerStyleInput | undefined)[]): LayerStyle;
+
+/**
+ * Merge styles over {@link LAYER_STYLE_DEFAULTS}, later inputs winning key by
+ * key. Usual order: catalog `defaults`, engine style, view.
+ */
+export declare function resolveLayerStyle(...inputs: (LayerStyleInput | undefined)[]): LayerStyle;
+
 export declare function resolveLocalizedText(value: LocalizedText | undefined, locale: string, defaultLocale?: string): string;
 
 export declare function searchCatalog(query: string, options?: CatalogSearchOptions, data?: Catalog): CatalogEntry[];
 
+/**
+ * Whether a source can show the requested time. Only fixed-period sources
+ * (rendered artifacts such as GeoVideo) declare hard bounds; live sources are
+ * assumed to cover any request and are bounded by their own metadata.
+ */
+export declare function sourceCoversTime(source: CatalogSource, request?: SourceTimeRequest): boolean;
+
+/** Requested instant and/or window, in epoch milliseconds. */
+export declare interface SourceTimeRequest {
+    time?: number;
+    start?: number;
+    end?: number;
+}
+
+/** A catalog entry, or anything with its kind and default style. */
+export declare interface StyledEntry {
+    kind: "scalar" | "vector";
+    defaults?: LayerStyleInput;
+}
+
+/** `style` without the keys a layer of `kind` cannot draw. */
+export declare function styleForKind(style: LayerStyleInput, kind: StyledEntry["kind"] | undefined): LayerStyleInput;
+
+/**
+ * Layer style: how a layer looks, with the same keys in the catalog
+ * `defaults`, the engines and story views. Playback (how time moves) is not
+ * part of it, so `particles.speed` is always the particle speed.
+ */
+export declare type StyleRenderMode = "particles" | "raster" | "raster+particles";
+
 export { }