@dclimate/zarr-map 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +374 -0
- package/dist/core/index.cjs +1603 -0
- package/dist/core/index.cjs.map +1 -0
- package/dist/core/index.d.cts +50 -0
- package/dist/core/index.d.ts +50 -0
- package/dist/core/index.js +1575 -0
- package/dist/core/index.js.map +1 -0
- package/dist/dclimate/index.cjs +1859 -0
- package/dist/dclimate/index.cjs.map +1 -0
- package/dist/dclimate/index.d.cts +80 -0
- package/dist/dclimate/index.d.ts +80 -0
- package/dist/dclimate/index.js +1856 -0
- package/dist/dclimate/index.js.map +1 -0
- package/dist/index.cjs +3071 -0
- package/dist/index.cjs.map +1 -0
- package/dist/index.d.cts +5 -0
- package/dist/index.d.ts +5 -0
- package/dist/index.js +3038 -0
- package/dist/index.js.map +1 -0
- package/dist/jaxray-CZWT_ZgD.d.ts +57 -0
- package/dist/jaxray-D_mmLPHk.d.cts +57 -0
- package/dist/react/index.cjs +3953 -0
- package/dist/react/index.cjs.map +1 -0
- package/dist/react/index.d.cts +71 -0
- package/dist/react/index.d.ts +71 -0
- package/dist/react/index.js +3945 -0
- package/dist/react/index.js.map +1 -0
- package/dist/renderers/index.cjs +903 -0
- package/dist/renderers/index.cjs.map +1 -0
- package/dist/renderers/index.d.cts +115 -0
- package/dist/renderers/index.d.ts +115 -0
- package/dist/renderers/index.js +899 -0
- package/dist/renderers/index.js.map +1 -0
- package/dist/types-DEZwfJNY.d.cts +210 -0
- package/dist/types-DEZwfJNY.d.ts +210 -0
- package/docs/README.md +12 -0
- package/docs/api-design.md +185 -0
- package/docs/architecture.md +246 -0
- package/docs/decision-record-renderer.md +144 -0
- package/docs/package-boundaries.md +50 -0
- package/docs/release-checklist.md +31 -0
- package/package.json +121 -0
|
@@ -0,0 +1,185 @@
|
|
|
1
|
+
# API Design
|
|
2
|
+
|
|
3
|
+
The package is built around a normalized `GridDataSource`. Renderers, adapters,
|
|
4
|
+
selectors, query helpers, and React components all consume the same contract.
|
|
5
|
+
|
|
6
|
+
## Core Contract
|
|
7
|
+
|
|
8
|
+
`GridDataSource` contains:
|
|
9
|
+
|
|
10
|
+
- `variables`: numeric Zarr/Jaxray variables with dtype, dimensions, shape,
|
|
11
|
+
chunks, units, fill values, and scale/offset metadata.
|
|
12
|
+
- `dimensions`: named axes with size, kind, coordinate values, units, calendar,
|
|
13
|
+
coordinate orientation, and optional bounds.
|
|
14
|
+
- `spatialDimensions`: explicit `{ x, y }` mapping when aliases such as `lon`
|
|
15
|
+
and `lat` are not enough.
|
|
16
|
+
- `bounds`, `crs`, and optional `proj4` for renderer placement.
|
|
17
|
+
- `source` or `store`: an HTTP Zarr URL or a structural Jaxray/Zarrita readable
|
|
18
|
+
store.
|
|
19
|
+
- Optional `readSlice` and `queryGeometry` hooks for query helpers.
|
|
20
|
+
|
|
21
|
+
Validation is available through `validateGridDataSource` and
|
|
22
|
+
`assertValidGridDataSource`. Validation errors use `GridError` codes such as
|
|
23
|
+
`MISSING_SPATIAL_DIMENSIONS`, `MISSING_COORDINATES`, and `UNSUPPORTED_DTYPE` so
|
|
24
|
+
applications can show actionable failures.
|
|
25
|
+
|
|
26
|
+
## Selectors
|
|
27
|
+
|
|
28
|
+
Selectors preserve intent:
|
|
29
|
+
|
|
30
|
+
```ts
|
|
31
|
+
import { indexSelector, isoTimeSelector } from "@dclimate/zarr-map/core";
|
|
32
|
+
|
|
33
|
+
const selectors = {
|
|
34
|
+
time: isoTimeSelector("2024-01-01T00:00:00.000Z"),
|
|
35
|
+
month: 1,
|
|
36
|
+
band: indexSelector(0)
|
|
37
|
+
};
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
Bare numbers are coordinate values. Numeric indexes must use
|
|
41
|
+
`indexSelector(index)`. Before rendering, selectors are converted to
|
|
42
|
+
`@carbonplan/zarr-layer` selector objects with `type: "value"` or
|
|
43
|
+
`type: "index"`.
|
|
44
|
+
|
|
45
|
+
## Generic Jaxray Example
|
|
46
|
+
|
|
47
|
+
```ts
|
|
48
|
+
import { createJaxraySource } from "@dclimate/zarr-map/core";
|
|
49
|
+
|
|
50
|
+
const source = createJaxraySource(dataset, {
|
|
51
|
+
bounds: [-10, 35, 5, 45],
|
|
52
|
+
spatialDimensions: { x: "lon", y: "lat" }
|
|
53
|
+
});
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
The adapter accepts Jaxray-like datasets or data arrays with `data_vars`,
|
|
57
|
+
`coords`, `dims`, `shape`, `dtype`, and `attrs`. Users can override inferred
|
|
58
|
+
dimension or variable metadata when datasets omit required renderer fields.
|
|
59
|
+
|
|
60
|
+
## dClimate Example
|
|
61
|
+
|
|
62
|
+
```ts
|
|
63
|
+
import { createDClimateSource } from "@dclimate/zarr-map/dclimate";
|
|
64
|
+
|
|
65
|
+
const source = await createDClimateSource(
|
|
66
|
+
{
|
|
67
|
+
collection: "ecmwf_era5",
|
|
68
|
+
dataset: "temperature_2m",
|
|
69
|
+
variant: "finalized"
|
|
70
|
+
},
|
|
71
|
+
{
|
|
72
|
+
gatewayUrl: "https://ipfs-gateway.dclimate.net"
|
|
73
|
+
}
|
|
74
|
+
);
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
React callers should prefer the `DClimateZarrMap` convenience component when
|
|
78
|
+
they already know the dataset, variable, bounds, and time window:
|
|
79
|
+
|
|
80
|
+
```tsx
|
|
81
|
+
<DClimateZarrMap
|
|
82
|
+
bounds={[-12, 35, 16, 60]}
|
|
83
|
+
dataset={{
|
|
84
|
+
collection: "ecmwf_era5",
|
|
85
|
+
dataset: "temperature_2m",
|
|
86
|
+
variant: "finalized"
|
|
87
|
+
}}
|
|
88
|
+
map={mapConfig}
|
|
89
|
+
timeRange={{
|
|
90
|
+
start: "2024-01-01T00:00:00Z",
|
|
91
|
+
end: "2024-01-07T23:00:00Z"
|
|
92
|
+
}}
|
|
93
|
+
variable="2m_temperature"
|
|
94
|
+
/>
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
`bounds` and `timeRange` normalize into `sourceOptions.selection`; lower-level
|
|
98
|
+
callers can still pass `sourceOptions.selection` directly.
|
|
99
|
+
|
|
100
|
+
The validated dClimate fixture uses variable `2m_temperature` and CID
|
|
101
|
+
`bafyr4ifr5jtjeommsulg7srirdkskybj5pay3n4qyehecyzsesas5k2kd4`. The dClimate
|
|
102
|
+
adapter should call `DClimateClient.loadDataset({ request, options })` and
|
|
103
|
+
unwrap the returned `[Dataset, metadata]` tuple before normalizing the source.
|
|
104
|
+
|
|
105
|
+
`@dclimate/dclimate-client-js` and `@dclimate/jaxray` are optional peer
|
|
106
|
+
dependencies. Generic users can use `@dclimate/zarr-map/core` without installing
|
|
107
|
+
the dClimate client stack.
|
|
108
|
+
|
|
109
|
+
## Renderer Example
|
|
110
|
+
|
|
111
|
+
```ts
|
|
112
|
+
import { createMapLibreGridLayer } from "@dclimate/zarr-map/renderer";
|
|
113
|
+
|
|
114
|
+
const controller = createMapLibreGridLayer({
|
|
115
|
+
map,
|
|
116
|
+
source,
|
|
117
|
+
variable: "temperature",
|
|
118
|
+
selectors: { time: { kind: "index", index: 0 } },
|
|
119
|
+
colorScale: { palette: "viridis", domain: [260, 320], unit: "K" }
|
|
120
|
+
});
|
|
121
|
+
|
|
122
|
+
await controller.mount();
|
|
123
|
+
await controller.update({ selectors: { time: { kind: "index", index: 1 } } });
|
|
124
|
+
controller.remove();
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
The renderer can attach to a caller-owned MapLibre map or create a map from
|
|
128
|
+
`mapConfig`. Selector and color updates call runtime layer methods instead of
|
|
129
|
+
recreating the map.
|
|
130
|
+
|
|
131
|
+
`colorScale.palette` accepts built-in palette names or caller-provided color
|
|
132
|
+
stop arrays. Built-ins include semantic weather palettes such as `temperature`,
|
|
133
|
+
`precipitation`, `humidity`, `vegetation`, and `wind`; `magma`,
|
|
134
|
+
`precipitation`, and `vegetation` begin transparent at the lower bound for
|
|
135
|
+
overlay rendering. Renderer domains remain in source units. Legend and inspect
|
|
136
|
+
display can convert supported temperature and precipitation units when `unit`,
|
|
137
|
+
`displayUnit`, and `quantity` are provided.
|
|
138
|
+
|
|
139
|
+
React inspect controls support the legacy click panel via
|
|
140
|
+
`controls={{ inspect: true }}` and a pointer-following hover tooltip via
|
|
141
|
+
`controls={{ inspect: { mode: "hover" } }}`.
|
|
142
|
+
|
|
143
|
+
## Query Example
|
|
144
|
+
|
|
145
|
+
```ts
|
|
146
|
+
import { queryGrid, queryPoint } from "@dclimate/zarr-map/core";
|
|
147
|
+
|
|
148
|
+
const point = await queryPoint(source, "temperature", [-8.6, 39.5], {
|
|
149
|
+
time: "2024-01-01T00:00:00.000Z"
|
|
150
|
+
});
|
|
151
|
+
|
|
152
|
+
const box = await queryGrid({
|
|
153
|
+
source,
|
|
154
|
+
variable: "temperature",
|
|
155
|
+
geometry: { type: "BoundingBox", bounds: [-9, 38, -8, 40] },
|
|
156
|
+
maxCells: 5000
|
|
157
|
+
});
|
|
158
|
+
```
|
|
159
|
+
|
|
160
|
+
Bounded and polygon queries enforce a `maxCells` limit. Polygon queries scan the
|
|
161
|
+
polygon bounding box and return a warning in the result.
|
|
162
|
+
|
|
163
|
+
## Preflight Example
|
|
164
|
+
|
|
165
|
+
```ts
|
|
166
|
+
import { preflightGridRequest } from "@dclimate/zarr-map/core";
|
|
167
|
+
|
|
168
|
+
const preflight = preflightGridRequest({
|
|
169
|
+
bounds: [-12, 35, 16, 60],
|
|
170
|
+
limits: {
|
|
171
|
+
maxBytes: 256 * 1024 * 1024,
|
|
172
|
+
maxCells: 100_000
|
|
173
|
+
},
|
|
174
|
+
source,
|
|
175
|
+
timeRange: {
|
|
176
|
+
start: "2024-01-01T00:00:00Z",
|
|
177
|
+
end: "2024-01-07T23:00:00Z"
|
|
178
|
+
},
|
|
179
|
+
variable: "2m_temperature"
|
|
180
|
+
});
|
|
181
|
+
```
|
|
182
|
+
|
|
183
|
+
Preflight uses only source metadata. It estimates selected dimension sizes,
|
|
184
|
+
cell count, and uncompressed bytes when dtype size is known; it does not call
|
|
185
|
+
`readSlice`, `queryGeometry`, `sel`, or `compute`.
|
|
@@ -0,0 +1,246 @@
|
|
|
1
|
+
# Architecture: Renderer Compatibility
|
|
2
|
+
|
|
3
|
+
Date: 2026-06-09
|
|
4
|
+
|
|
5
|
+
## Recommendation
|
|
6
|
+
|
|
7
|
+
Use MapLibre GL JS with `@carbonplan/zarr-layer` as the primary MVP renderer. Build the package around a normalized `GridDataSource` contract, then adapt dClimate/Jaxray-backed datasets into the `zarrita.Readable` store shape accepted by `ZarrLayer`.
|
|
8
|
+
|
|
9
|
+
Do not pass a dClimate `GeoTemporalDataset` or Jaxray `Dataset` directly to the renderer. The renderer bridge should pass either:
|
|
10
|
+
|
|
11
|
+
- `source: "https://.../dataset.zarr"` for conventional HTTP/S3 Zarr stores.
|
|
12
|
+
- `store: jaxrayStore` for dClimate IPFS datasets opened with `openIpfsStore(cid, options)`.
|
|
13
|
+
|
|
14
|
+
The dClimate gateway is not generally down. A current STAC ERA5 fixture opens through `https://ipfs-gateway.dclimate.net` using dClimate client plus Jaxray. The static CIDs that timed out during the first spike should be treated as stale or unretrievable fixtures, not proof that the gateway or Jaxray are broken. Jaxray `ShardedStore` and `HamtStore` expose the `get(key)` method, and sometimes `getRange(key, range)`, that `@carbonplan/zarr-layer` requires for custom stores.
|
|
15
|
+
|
|
16
|
+
## Renderer Path
|
|
17
|
+
|
|
18
|
+
The MVP data flow should be:
|
|
19
|
+
|
|
20
|
+
```txt
|
|
21
|
+
dClimate request or direct CID
|
|
22
|
+
-> DClimateClient.loadDataset({ request, options }) resolves [Dataset, metadata]
|
|
23
|
+
-> adapter unwraps the Dataset and metadata tuple
|
|
24
|
+
-> openIpfsStore(cid, { gatewayUrl | ipfsElements })
|
|
25
|
+
-> validate and normalize GridDataSource metadata
|
|
26
|
+
-> new ZarrLayer({
|
|
27
|
+
id,
|
|
28
|
+
store,
|
|
29
|
+
variable,
|
|
30
|
+
selector,
|
|
31
|
+
spatialDimensions: { lon, lat },
|
|
32
|
+
zarrVersion,
|
|
33
|
+
crs | proj4,
|
|
34
|
+
bounds,
|
|
35
|
+
fillValue,
|
|
36
|
+
colormap,
|
|
37
|
+
clim
|
|
38
|
+
})
|
|
39
|
+
-> map.addLayer(layer)
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
For non-dClimate HTTP stores, skip the dClimate/Jaxray step and pass a URL as `source`.
|
|
43
|
+
|
|
44
|
+
## Compatibility Findings
|
|
45
|
+
|
|
46
|
+
### `@carbonplan/zarr-layer`
|
|
47
|
+
|
|
48
|
+
`@carbonplan/zarr-layer` 0.5.0 is the best default fit because it is already a MapLibre/Mapbox custom layer, supports Zarr v2 and v3 through `zarrita`, accepts custom stores, supports selectors, has runtime color/selector updates, exposes loading callbacks, and includes `queryData` for point and polygon inspection.
|
|
49
|
+
|
|
50
|
+
Important constraints:
|
|
51
|
+
|
|
52
|
+
- High-resolution datasets need multiscale metadata or practical chunking. Single-level arrays may work only for small datasets.
|
|
53
|
+
- Tiled data must be in EPSG:4326 or EPSG:3857. Untiled data can use `proj4`, but explicit `bounds` are strongly recommended.
|
|
54
|
+
- Selectors are exact by value unless explicitly passed as `{ selected, type: "index" }`.
|
|
55
|
+
- `setSelector` can trigger a fetch per call, so time slider updates must be debounced.
|
|
56
|
+
- The package is explicitly described as experimental, so pin exact versions for the MVP.
|
|
57
|
+
|
|
58
|
+
### `@dclimate/dclimate-client-js`
|
|
59
|
+
|
|
60
|
+
`@dclimate/dclimate-client-js` 0.5.5 resolves dClimate dataset requests, supports direct CID loading, has browser and Node builds, and uses `@dclimate/jaxray` for IPFS-backed Zarr access. It should be optional for users who only use generic grid-source mode.
|
|
61
|
+
|
|
62
|
+
The adapter should call the real `DClimateClient.loadDataset({ request, options })` API, pass gateway and Jaxray options through `options`, request a Jaxray dataset when supported, and unwrap the returned `[Dataset, metadata]` tuple. For rendering, the useful output is the resolved CID, the Jaxray dataset metadata, and the underlying Jaxray/IPFS store, not the higher-level geotemporal selection wrapper.
|
|
63
|
+
|
|
64
|
+
### `@dclimate/jaxray`
|
|
65
|
+
|
|
66
|
+
`@dclimate/jaxray` 0.6.9 can open IPFS-hosted Zarr datasets from one CID and auto-detect `sharded` or `hamt` stores. Its stores are structurally compatible with `zarrita.Readable`:
|
|
67
|
+
|
|
68
|
+
- `ShardedStore.get(key): Promise<Uint8Array | undefined>`
|
|
69
|
+
- `HamtStore.get(key): Promise<Uint8Array | undefined>`
|
|
70
|
+
- `HamtStore.getRange(key, range): Promise<Uint8Array | undefined>`
|
|
71
|
+
- `listMetadataKeys()` for metadata discovery in Jaxray's own backend
|
|
72
|
+
|
|
73
|
+
The adapter should not assume TypeScript package identity between Jaxray's `zarrita` dependency and `zarr-layer`'s `zarrita` dependency. Treat store compatibility as structural and validate at runtime.
|
|
74
|
+
|
|
75
|
+
### MapLibre GL JS
|
|
76
|
+
|
|
77
|
+
MapLibre GL JS is still the correct base map. `ZarrLayer` implements MapLibre's custom-layer model, so it can be inserted into the normal map layer stack and render directly into the map WebGL context.
|
|
78
|
+
|
|
79
|
+
### deck.gl Alternatives
|
|
80
|
+
|
|
81
|
+
Core deck.gl provides `BitmapLayer` and `TileLayer`, which can be composed into a raster renderer but do not by themselves understand Zarr metadata.
|
|
82
|
+
|
|
83
|
+
`@developmentseed/deck.gl-zarr` 0.7.0 is a real Zarr option. It renders GeoZarr datasets with deck.gl, uses `zarrita`, and builds on `@developmentseed/deck.gl-raster`. It is a credible fallback, especially if the project later needs deck.gl's broader layer ecosystem. It is not the MVP default because it requires a deck.gl runtime, caller-supplied `getTileData` and `renderTile` callbacks, and stricter GeoZarr metadata/selection constraints than `@carbonplan/zarr-layer`.
|
|
84
|
+
|
|
85
|
+
## Proof Of Concept Results
|
|
86
|
+
|
|
87
|
+
### dClimate Gateway Validation
|
|
88
|
+
|
|
89
|
+
Validated on 2026-06-09 with the sibling `../dclimate-client-js-autoresearch`
|
|
90
|
+
project. Its smoke benchmark still works through `@dclimate/dclimate-client-js`
|
|
91
|
+
and `@dclimate/jaxray` against the current STAC CID.
|
|
92
|
+
|
|
93
|
+
Current validated fixture:
|
|
94
|
+
|
|
95
|
+
- collection: `ecmwf_era5`
|
|
96
|
+
- dataset: `temperature_2m`
|
|
97
|
+
- variant: `finalized`
|
|
98
|
+
- variable: `2m_temperature`
|
|
99
|
+
- CID: `bafyr4ifr5jtjeommsulg7srirdkskybj5pay3n4qyehecyzsesas5k2kd4`
|
|
100
|
+
- gateway: `https://ipfs-gateway.dclimate.net`
|
|
101
|
+
|
|
102
|
+
The current CID returned successful gateway root and range probes, opened as a
|
|
103
|
+
Jaxray store, and loaded through `DClimateClient.loadDataset({ request, options })`.
|
|
104
|
+
|
|
105
|
+
### Stale dClimate CID attempts
|
|
106
|
+
|
|
107
|
+
Tested the static CIDs bundled in `@dclimate/dclimate-client-js`:
|
|
108
|
+
|
|
109
|
+
- `cpc-precip-conus`: `bafyr4iff6vuvpkb4zexkx7exoafsjvfnno4ofidyc37gsto2aaxixg5zia`
|
|
110
|
+
- `cpc-precip-global`: `bafyr4ihe35o55qz47y2dbau6ikvfkbrthqn57pdhem5jdxhvqf5u7qlrbm`
|
|
111
|
+
- `chirps-final-p05`: `bafyr4ic2n63fkxdvwgh2uzealcrbeubncivcqaptttfrr4vp5ecbe5e65a`
|
|
112
|
+
|
|
113
|
+
Observed results:
|
|
114
|
+
|
|
115
|
+
- `openIpfsStore` stalled on the first CID through `https://ipfs-gateway.dclimate.net`.
|
|
116
|
+
- Bounded `curl` GET/HEAD requests to the same gateway timed out with zero bytes for tested root CIDs.
|
|
117
|
+
- `w3s.link` returned CORS headers on redirect and preflight, but followed requests ended in timeout or `504`.
|
|
118
|
+
- `https://ipfs-gateway.dclimate.net` returned permissive CORS preflight headers, including `Range`, `Content-Range`, and IPFS headers, but data fetches still timed out.
|
|
119
|
+
|
|
120
|
+
Conclusion: these static CIDs are stale or unretrievable through the tested
|
|
121
|
+
gateways. They should not be used as proof that the gateway or Jaxray are
|
|
122
|
+
broken, and they should not be used as the first public dClimate fixture.
|
|
123
|
+
|
|
124
|
+
### Control Zarr read
|
|
125
|
+
|
|
126
|
+
The CarbonPlan demo store at `https://carbonplan-maps.s3.us-west-2.amazonaws.com/v2/demo/4d/tavg-prec-month` was readable with `zarrita`:
|
|
127
|
+
|
|
128
|
+
- `.zmetadata`: HTTP 200 with browser CORS when an `Origin` header is sent.
|
|
129
|
+
- Chunk range request: HTTP 206 with browser CORS.
|
|
130
|
+
- Opened array `2/climate`:
|
|
131
|
+
- shape: `[2, 12, 512, 512]`
|
|
132
|
+
- chunks: `[2, 12, 128, 128]`
|
|
133
|
+
- dimensions: `["band", "month", "y", "x"]`
|
|
134
|
+
- fill value: `9.969209968386869e+36`
|
|
135
|
+
- month coordinate values: `1` through `12`
|
|
136
|
+
|
|
137
|
+
This validates the `zarr-layer`/`zarrita` renderer path for browser-compatible Zarr stores.
|
|
138
|
+
|
|
139
|
+
## Required Dataset Metadata
|
|
140
|
+
|
|
141
|
+
The normalized source contract must require or derive the following before creating a renderer layer.
|
|
142
|
+
|
|
143
|
+
### Variable
|
|
144
|
+
|
|
145
|
+
- Variable name or path inside the Zarr hierarchy.
|
|
146
|
+
- Numeric dtype supported by `zarrita` and WebGL texture upload path.
|
|
147
|
+
- Shape, chunks, and dimension order.
|
|
148
|
+
- Fill value from Zarr metadata, `_FillValue`, or explicit config.
|
|
149
|
+
- Optional `scale_factor` and `add_offset`; if present, query and render paths should apply them consistently.
|
|
150
|
+
- Units, long name, and standard name when available for legends and tooltips.
|
|
151
|
+
|
|
152
|
+
### Dimensions
|
|
153
|
+
|
|
154
|
+
- Ordered dimension names from Zarr v3 `dimension_names` or Zarr v2 `_ARRAY_DIMENSIONS`.
|
|
155
|
+
- Core spatial dimensions mapped to `{ x, y }`, accepting common aliases such
|
|
156
|
+
as `lat`, `latitude`, `y`, `lon`, `longitude`, `x`, and `lng`. The renderer
|
|
157
|
+
adapter converts this to CarbonPlan's `spatialDimensions: { lon, lat }`
|
|
158
|
+
option.
|
|
159
|
+
- Non-spatial dimensions such as `time`, `band`, `member`, `scenario`, or `level`.
|
|
160
|
+
- Size for each dimension and whether the dimension has a coordinate array.
|
|
161
|
+
|
|
162
|
+
### Coordinates
|
|
163
|
+
|
|
164
|
+
- Coordinate arrays for all selector dimensions when value-based selection is exposed.
|
|
165
|
+
- Spatial coordinate arrays, bounds, or affine metadata sufficient to locate pixels.
|
|
166
|
+
- Latitude orientation (`latIsAscending`) when it cannot be inferred safely.
|
|
167
|
+
- Longitude convention, especially `[-180, 180]` versus `[0, 360]`.
|
|
168
|
+
- Time coordinate raw values plus decoded ISO strings when decoding is possible.
|
|
169
|
+
|
|
170
|
+
### CRS And Bounds
|
|
171
|
+
|
|
172
|
+
- CRS as `EPSG:4326`, `EPSG:3857`, or a full `proj4` definition.
|
|
173
|
+
- Explicit edge bounds `[xMin, yMin, xMax, yMax]`, especially for projected or untiled datasets.
|
|
174
|
+
- For tiled datasets, confirm the CRS is EPSG:4326 or EPSG:3857.
|
|
175
|
+
- For nonstandard projections, require `proj4` and explicit bounds.
|
|
176
|
+
|
|
177
|
+
### Zarr Format And Codecs
|
|
178
|
+
|
|
179
|
+
- Zarr format version, preferably explicit as `2` or `3`.
|
|
180
|
+
- Consolidated metadata availability when applicable.
|
|
181
|
+
- Codec list. Register aliases for `numcodecs.*` names before opening the renderer if a dataset uses them.
|
|
182
|
+
|
|
183
|
+
### Network
|
|
184
|
+
|
|
185
|
+
- Browser CORS support for metadata and chunk URLs.
|
|
186
|
+
- Range request support and exposed `Content-Range` headers when range reads are used.
|
|
187
|
+
- Gateway timeout and retry behavior.
|
|
188
|
+
- Authentication/header behavior through `transformRequest` or custom store injection.
|
|
189
|
+
|
|
190
|
+
## Time Selector Representation
|
|
191
|
+
|
|
192
|
+
Use a normalized internal selector model that preserves intent:
|
|
193
|
+
|
|
194
|
+
```ts
|
|
195
|
+
type NormalizedSelector =
|
|
196
|
+
| { kind: "coordinate"; value: number | string }
|
|
197
|
+
| { kind: "index"; index: number }
|
|
198
|
+
| { kind: "isoTime"; iso: string };
|
|
199
|
+
```
|
|
200
|
+
|
|
201
|
+
Rules:
|
|
202
|
+
|
|
203
|
+
- Public API should prefer coordinate values and ISO strings for time.
|
|
204
|
+
- Numeric indexes are supported only when explicitly marked as indexes.
|
|
205
|
+
- Bare numeric selector values are treated as coordinate values first, not indexes.
|
|
206
|
+
- Before passing selectors to `ZarrLayer`, convert:
|
|
207
|
+
- coordinate values to `{ selected: value, type: "value" }`
|
|
208
|
+
- numeric indexes to `{ selected: index, type: "index" }`
|
|
209
|
+
- ISO strings to the dataset's actual time coordinate value when possible
|
|
210
|
+
- If the dataset time coordinate is already an ISO/string coordinate, pass the ISO value as a value selector.
|
|
211
|
+
- If the dataset time coordinate is numeric CF time, decode enough metadata to map ISO input to the numeric coordinate value. Exact matching is the MVP behavior.
|
|
212
|
+
- Arrays are supported for query/time-series selectors and custom-shader bands, but normal map rendering should use one selected value per non-spatial dimension unless a renderer-specific multi-band shader is configured.
|
|
213
|
+
|
|
214
|
+
This avoids the ambiguity in `zarr-layer` where a simple numeric selector may match a coordinate value first and fall back to an index.
|
|
215
|
+
|
|
216
|
+
## Follow-Up Work
|
|
217
|
+
|
|
218
|
+
- `todo/011-validate-dclimate-browser-gateway-fixture.md`: keep the current STAC dClimate fixture and gateway validation record up to date.
|
|
219
|
+
- `todo/002-core-grid-data-source-contract.md`: encode the metadata requirements and selector model above.
|
|
220
|
+
- `todo/003-renderer-adapter.md`: implement the MapLibre plus `ZarrLayer` bridge, including debounced selector updates and typed load errors.
|
|
221
|
+
|
|
222
|
+
## Ticket 011 Revalidation
|
|
223
|
+
|
|
224
|
+
Ticket 011 revalidation found that the dClimate gateway works for the current
|
|
225
|
+
STAC ERA5 fixture:
|
|
226
|
+
`bafyr4ifr5jtjeommsulg7srirdkskybj5pay3n4qyehecyzsesas5k2kd4`. The sibling
|
|
227
|
+
`../dclimate-client-js-autoresearch` project still works with dClimate client
|
|
228
|
+
plus Jaxray, and its smoke benchmark succeeded on that CID.
|
|
229
|
+
|
|
230
|
+
The static CIDs tested during the renderer research still timed out or returned
|
|
231
|
+
gateway errors and should be documented as stale or unretrievable. The adapter
|
|
232
|
+
should use `DClimateClient.loadDataset({ request, options })`, unwrap
|
|
233
|
+
`[Dataset, metadata]`, and use the metadata CID with
|
|
234
|
+
`openIpfsStore(cid, { gatewayUrl })` when a renderer store is needed. The
|
|
235
|
+
current validation record is checked in at
|
|
236
|
+
`fixtures/dclimate-gateway-validation.json`.
|
|
237
|
+
|
|
238
|
+
## Sources Inspected
|
|
239
|
+
|
|
240
|
+
- `@carbonplan/zarr-layer` 0.5.0 README and type definitions: https://github.com/carbonplan/zarr-layer
|
|
241
|
+
- `@dclimate/dclimate-client-js` 0.5.5 README and type definitions: https://github.com/dClimate/dclimate-client-js
|
|
242
|
+
- `@dclimate/jaxray` 0.6.9 README and type definitions: https://github.com/dClimate/jaxray
|
|
243
|
+
- MapLibre GL JS custom layer documentation: https://maplibre.org/maplibre-gl-js/docs/API/interfaces/CustomLayerInterface/
|
|
244
|
+
- deck.gl `BitmapLayer` and `TileLayer` documentation: https://deck.gl/docs/api-reference/layers/bitmap-layer and https://deck.gl/docs/api-reference/geo-layers/tile-layer
|
|
245
|
+
- `@developmentseed/deck.gl-zarr` 0.7.0 package README and type definitions: https://github.com/developmentseed/deck.gl-raster
|
|
246
|
+
- Zarr v2 and v3 specs: https://zarr.readthedocs.io/en/v2.13.6/spec/v2.html and https://zarr-specs.readthedocs.io/en/latest/v3/core/index.html
|
|
@@ -0,0 +1,144 @@
|
|
|
1
|
+
# Decision Record: Renderer Selection
|
|
2
|
+
|
|
3
|
+
Date: 2026-06-09
|
|
4
|
+
|
|
5
|
+
## Status
|
|
6
|
+
|
|
7
|
+
Accepted for MVP. Ticket `011` validated a current STAC dClimate fixture through
|
|
8
|
+
the dClimate gateway; old static CIDs from the first spike remain stale or
|
|
9
|
+
unretrievable and should not be used as public fixtures.
|
|
10
|
+
|
|
11
|
+
## Context
|
|
12
|
+
|
|
13
|
+
`@dclimate/zarr-map` needs to render dClimate and generic Zarr/Jaxray gridded climate datasets on an interactive map. The package direction already prefers MapLibre GL JS and wants to avoid a custom WebGL renderer unless existing renderers cannot support the target datasets.
|
|
14
|
+
|
|
15
|
+
The renderer must support:
|
|
16
|
+
|
|
17
|
+
- Browser-compatible Zarr chunk and metadata loading.
|
|
18
|
+
- Time and other non-spatial selectors.
|
|
19
|
+
- Latitude/longitude grids and projected grids.
|
|
20
|
+
- Color scale, opacity, loading, error, cleanup, and query behavior.
|
|
21
|
+
- dClimate IPFS stores opened through Jaxray.
|
|
22
|
+
|
|
23
|
+
## Decision
|
|
24
|
+
|
|
25
|
+
Use `@carbonplan/zarr-layer` as the primary renderer behind a renderer adapter. Keep deck.gl Zarr support as a fallback path, and do not build a custom raster renderer for the MVP.
|
|
26
|
+
|
|
27
|
+
For dClimate datasets, do not rely on `source: "https://gateway/ipfs/<cid>"`
|
|
28
|
+
alone unless the CID is a conventional directory-style Zarr store. The realistic
|
|
29
|
+
path is:
|
|
30
|
+
|
|
31
|
+
```txt
|
|
32
|
+
DClimateClient.loadDataset({ request, options })
|
|
33
|
+
-> [Dataset, metadata]
|
|
34
|
+
-> metadata.cid
|
|
35
|
+
-> openIpfsStore
|
|
36
|
+
-> Jaxray store
|
|
37
|
+
-> ZarrLayer({ store, spatialDimensions: { lon, lat }, ... })
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
`@carbonplan/zarr-layer` can render target dClimate datasets through a realistic
|
|
41
|
+
adapter when the dataset exposes renderer-compatible metadata and chunks through
|
|
42
|
+
a responsive browser gateway. On 2026-06-09, the current STAC ERA5
|
|
43
|
+
`ecmwf_era5/temperature_2m/finalized` fixture loaded through
|
|
44
|
+
`https://ipfs-gateway.dclimate.net`; its validated CID is
|
|
45
|
+
`bafyr4ifr5jtjeommsulg7srirdkskybj5pay3n4qyehecyzsesas5k2kd4`. The bundled
|
|
46
|
+
static CIDs tested during the first spike timed out or returned `504`, so treat
|
|
47
|
+
those as stale fixture choices rather than proof that Jaxray or the gateway are
|
|
48
|
+
broken.
|
|
49
|
+
|
|
50
|
+
## Options Compared
|
|
51
|
+
|
|
52
|
+
| Option | Fit | Pros | Constraints | Decision |
|
|
53
|
+
| ----------------------------------------- | ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------- |
|
|
54
|
+
| MapLibre GL JS + `@carbonplan/zarr-layer` | Strong | Native MapLibre/Mapbox custom layer; Zarr v2/v3 via `zarrita`; accepts custom `Readable` stores; selectors, color updates, loading state, queries, CRS/proj4 support | Experimental package; high-res data needs multiscales or practical chunking; exact selector matching; dClimate CID path needs Jaxray custom store and a current pinned fixture | Primary MVP renderer |
|
|
55
|
+
| deck.gl core raster path | Medium | Mature visualization stack; `TileLayer` and `BitmapLayer` can render tiled raster products; broad layer ecosystem | Core layers do not understand Zarr metadata; requires building tile/chunk loading, pixel conversion, color mapping, and MapLibre overlay integration | Not MVP default |
|
|
56
|
+
| `@developmentseed/deck.gl-zarr` | Medium to strong | Current package for GeoZarr rendering; uses `zarrita`; handles tiling and reprojection through `@developmentseed/deck.gl-raster`; caller-owned store | Adds deck.gl/luma runtime; requires caller-supplied `getTileData` and `renderTile`; expects GeoZarr metadata and strict selection/spatial-dim layout | Keep as fallback or future adapter |
|
|
57
|
+
| Custom renderer | Weak for MVP | Full control over Jaxray, time selection, shader logic, and query APIs | Highest implementation and testing cost; must solve projection, tiling, caching, WebGL lifecycle, color mapping, and queries from scratch | Defer unless existing renderers fail after gateway/metadata validation |
|
|
58
|
+
|
|
59
|
+
## Why `@carbonplan/zarr-layer`
|
|
60
|
+
|
|
61
|
+
The package directly matches the intended MapLibre architecture. It implements a custom layer, can be added with `map.addLayer`, accepts either `source` or a custom `store`, and exposes runtime methods such as `setSelector`, `setVariable`, `setClim`, `setColormap`, and `queryData`.
|
|
62
|
+
|
|
63
|
+
The custom-store requirement is the key compatibility point. `ZarrLayerOptions.store` requires a `zarrita.Readable` with at least `get(key)`. Jaxray's IPFS stores expose that shape, so the bridge can adapt dClimate CIDs without repackaging datasets.
|
|
64
|
+
|
|
65
|
+
## Required Adapter Behavior
|
|
66
|
+
|
|
67
|
+
The renderer adapter must:
|
|
68
|
+
|
|
69
|
+
- Open dClimate CIDs with `openIpfsStore` rather than passing gateway CID URLs directly to `ZarrLayer`.
|
|
70
|
+
- Validate that the selected variable is numeric and has exactly two spatial dimensions.
|
|
71
|
+
- Normalize core `{ x, y }` spatial dimension names into CarbonPlan's
|
|
72
|
+
`spatialDimensions: { lon, lat }` option when aliases are not enough.
|
|
73
|
+
- Pass only non-spatial selectors to `ZarrLayer`; latitude and longitude remain
|
|
74
|
+
render axes.
|
|
75
|
+
- Pass explicit `zarrVersion` when known.
|
|
76
|
+
- Pass `fillValue`, `crs`, `proj4`, `bounds`, and `latIsAscending` when inferred metadata is missing or ambiguous.
|
|
77
|
+
- Debounce rapid selector changes before calling `setSelector`.
|
|
78
|
+
- Surface metadata, chunk, CORS, timeout, codec, and projection failures as typed load errors.
|
|
79
|
+
- Keep renderer lifecycle separate from React and map ownership.
|
|
80
|
+
|
|
81
|
+
## Time Selector Decision
|
|
82
|
+
|
|
83
|
+
Support coordinate values, ISO strings, and numeric indexes, but keep them distinct internally.
|
|
84
|
+
|
|
85
|
+
Use this internal intent model:
|
|
86
|
+
|
|
87
|
+
```ts
|
|
88
|
+
type NormalizedSelector =
|
|
89
|
+
| { kind: "coordinate"; value: number | string }
|
|
90
|
+
| { kind: "index"; index: number }
|
|
91
|
+
| { kind: "isoTime"; iso: string };
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
For `@carbonplan/zarr-layer`:
|
|
95
|
+
|
|
96
|
+
- Coordinate values become `{ selected: value, type: "value" }`.
|
|
97
|
+
- Numeric indexes become `{ selected: index, type: "index" }`.
|
|
98
|
+
- ISO strings become exact coordinate values after time normalization.
|
|
99
|
+
|
|
100
|
+
Bare numbers in public APIs should not silently mean indexes. They should mean numeric coordinate values unless the user uses an explicit index selector. This avoids ambiguity for dimensions where coordinate values are also integers, such as `month: 1`.
|
|
101
|
+
|
|
102
|
+
## Metadata Contract Required Before Rendering
|
|
103
|
+
|
|
104
|
+
The core source contract must require:
|
|
105
|
+
|
|
106
|
+
- Variable name/path, dtype, shape, chunks, dimensions, units, fill value, optional `scale_factor`, and optional `add_offset`.
|
|
107
|
+
- Dimension names from v2 `_ARRAY_DIMENSIONS` or v3 `dimension_names`.
|
|
108
|
+
- Coordinate arrays for time and every value-selectable non-spatial dimension.
|
|
109
|
+
- Spatial dimension mapping, coordinate orientation, and longitude convention.
|
|
110
|
+
- CRS as EPSG code or `proj4`, plus explicit edge bounds when projected or untiled.
|
|
111
|
+
- Zarr format version and codec requirements.
|
|
112
|
+
- Browser-reachable metadata/chunk URLs or a custom store with equivalent access.
|
|
113
|
+
|
|
114
|
+
## Consequences
|
|
115
|
+
|
|
116
|
+
Positive:
|
|
117
|
+
|
|
118
|
+
- MVP implementation can stay focused on adapters and validation rather than low-level raster rendering.
|
|
119
|
+
- The renderer remains MapLibre-first and can attach to existing user maps.
|
|
120
|
+
- The same normalized source can support both dClimate and generic Zarr/Jaxray use cases.
|
|
121
|
+
|
|
122
|
+
Negative:
|
|
123
|
+
|
|
124
|
+
- The package inherits `@carbonplan/zarr-layer`'s experimental status and exact selector semantics.
|
|
125
|
+
- Direct dClimate CID rendering depends on gateway responsiveness, CORS, IPFS layout, and using a current pinned CID.
|
|
126
|
+
- Some dClimate datasets may need metadata overrides or preprocessing before they are renderer-compatible.
|
|
127
|
+
|
|
128
|
+
## Follow-Up Tickets
|
|
129
|
+
|
|
130
|
+
- `todo/011-validate-dclimate-browser-gateway-fixture.md`
|
|
131
|
+
- `todo/002-core-grid-data-source-contract.md`
|
|
132
|
+
- `todo/003-renderer-adapter.md`
|
|
133
|
+
|
|
134
|
+
## References
|
|
135
|
+
|
|
136
|
+
- `@carbonplan/zarr-layer`: https://github.com/carbonplan/zarr-layer
|
|
137
|
+
- `@dclimate/dclimate-client-js`: https://github.com/dClimate/dclimate-client-js
|
|
138
|
+
- `@dclimate/jaxray`: https://github.com/dClimate/jaxray
|
|
139
|
+
- MapLibre custom layers: https://maplibre.org/maplibre-gl-js/docs/API/interfaces/CustomLayerInterface/
|
|
140
|
+
- deck.gl `BitmapLayer`: https://deck.gl/docs/api-reference/layers/bitmap-layer
|
|
141
|
+
- deck.gl `TileLayer`: https://deck.gl/docs/api-reference/geo-layers/tile-layer
|
|
142
|
+
- `@developmentseed/deck.gl-zarr`: https://github.com/developmentseed/deck.gl-raster
|
|
143
|
+
- Zarr v2 spec: https://zarr.readthedocs.io/en/v2.13.6/spec/v2.html
|
|
144
|
+
- Zarr v3 spec: https://zarr-specs.readthedocs.io/en/latest/v3/core/index.html
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
# Package Boundaries
|
|
2
|
+
|
|
3
|
+
`@dclimate/zarr-map` is a light map-rendering wrapper. It should make a known
|
|
4
|
+
dClimate-compatible Zarr dataset easy to render on MapLibre, without becoming a
|
|
5
|
+
marketplace, dashboard, or explorer application.
|
|
6
|
+
|
|
7
|
+
## Belongs Here
|
|
8
|
+
|
|
9
|
+
- React and non-React APIs for rendering normalized gridded sources on maps.
|
|
10
|
+
- dClimate dataset request adaptation into `GridDataSource`.
|
|
11
|
+
- Passing caller-provided bounds and time ranges into dClimate loading.
|
|
12
|
+
- Jaxray/Zarr metadata normalization needed by the renderer.
|
|
13
|
+
- MapLibre layer setup, updates, loading state, errors, and teardown.
|
|
14
|
+
- Selector normalization for time, band, and other non-spatial dimensions.
|
|
15
|
+
- Light visual customization: palette or gradient, color domain, opacity,
|
|
16
|
+
display unit, legend, time selector, and inspect callbacks.
|
|
17
|
+
- Request preflight that estimates cells and bytes before unsuitable render jobs
|
|
18
|
+
begin expensive bounded loading.
|
|
19
|
+
|
|
20
|
+
## Belongs Upstream
|
|
21
|
+
|
|
22
|
+
Prefer `@dclimate/dclimate-client-js` or another dClimate data package for:
|
|
23
|
+
|
|
24
|
+
- Catalog lookup and dataset discovery.
|
|
25
|
+
- Authentication or access-gated dataset retrieval.
|
|
26
|
+
- Decryption codec registration and key handling.
|
|
27
|
+
- IPFS and gateway store opening improvements.
|
|
28
|
+
- Slicing APIs that are generally useful outside map rendering.
|
|
29
|
+
- Dataset metadata repair or standardization.
|
|
30
|
+
|
|
31
|
+
## Belongs In Consuming Apps
|
|
32
|
+
|
|
33
|
+
- Marketplace purchase flow.
|
|
34
|
+
- Wallet or order state.
|
|
35
|
+
- User-facing decryption UX.
|
|
36
|
+
- CSV or JSON export workflows.
|
|
37
|
+
- 2D charts and aggregation dashboards.
|
|
38
|
+
- Modal layouts and dashboard pages.
|
|
39
|
+
- Coordinate entry forms and calendars.
|
|
40
|
+
- Map drawing or editing workflows unless accepted later as a small optional
|
|
41
|
+
primitive.
|
|
42
|
+
|
|
43
|
+
## Dependency Guardrails
|
|
44
|
+
|
|
45
|
+
Package source must not import app-level dependencies such as `@apollo/client`,
|
|
46
|
+
wallet or ethers packages, `eciesjs`, `libsodium`, `next`, dClimate frontend
|
|
47
|
+
theme/component packages, `mapbox-gl`, `streamsaver`, or `chart.js`.
|
|
48
|
+
|
|
49
|
+
The test suite includes a lightweight boundary check for `package.json` and
|
|
50
|
+
`src/**` imports.
|
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
# Release Checklist
|
|
2
|
+
|
|
3
|
+
- `npm run typecheck`
|
|
4
|
+
- `npm run format:check`
|
|
5
|
+
- `npm run lint`
|
|
6
|
+
- `npm run test`
|
|
7
|
+
- `npm run build`
|
|
8
|
+
- Confirm package output is ESM/CJS with declaration files in `dist`.
|
|
9
|
+
- Confirm the separate demo app builds against the package:
|
|
10
|
+
`cd ../zarr-map-demo && npm ci && npm run build`.
|
|
11
|
+
- Confirm peer dependencies are installed by the consuming app:
|
|
12
|
+
- `maplibre-gl`
|
|
13
|
+
- `@carbonplan/zarr-layer`
|
|
14
|
+
- `react` and `react-dom` for React components
|
|
15
|
+
- `@dclimate/dclimate-client-js` and `@dclimate/jaxray` for dClimate mode
|
|
16
|
+
- Confirm dClimate mode still loads the validated current STAC fixture through
|
|
17
|
+
`https://ipfs-gateway.dclimate.net`:
|
|
18
|
+
`ecmwf_era5/temperature_2m/finalized`, variable `2m_temperature`, CID
|
|
19
|
+
`bafyr4ifr5jtjeommsulg7srirdkskybj5pay3n4qyehecyzsesas5k2kd4`.
|
|
20
|
+
- Confirm the adapter uses `DClimateClient.loadDataset({ request, options })`
|
|
21
|
+
and unwraps the returned `[Dataset, metadata]` tuple.
|
|
22
|
+
- `package.json` is publishable and does not set `private: true`.
|
|
23
|
+
|
|
24
|
+
Known MVP renderer limitations:
|
|
25
|
+
|
|
26
|
+
- `@carbonplan/zarr-layer` is experimental and should remain pinned at `0.5.0`.
|
|
27
|
+
- High-resolution datasets need practical chunking or multiscale metadata.
|
|
28
|
+
- Selector values require exact coordinate matches after normalization.
|
|
29
|
+
- The dClimate gateway is not generally down; the current STAC ERA5 fixture
|
|
30
|
+
works. Static CIDs from the first spike timed out and should be described as
|
|
31
|
+
stale or unretrievable fixtures.
|