deckgl-marimo 0.2.0__tar.gz → 0.4.0__tar.gz

{deckgl_marimo-0.2.0 → deckgl_marimo-0.4.0}/.gitignore

@@ -20,6 +20,7 @@ js/node_modules/
 .venv/
 venv/
 
+__marimo__/
 
 # IDE
 .idea/
@@ -39,3 +40,6 @@ htmlcov/
 
 # mypy / type checking
 .mypy_cache/
+
+# Large test data files
+examples/data/*.npz

{deckgl_marimo-0.2.0 → deckgl_marimo-0.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: deckgl-marimo
-Version: 0.2.0
+Version: 0.4.0
 Summary: deck.gl visualization library for marimo notebooks — interactive maps with 33 layer types
 Project-URL: Homepage, https://github.com/kihaji/deckgl-marimo
 Project-URL: Documentation, https://kihaji.github.io/deckgl-marimo
@@ -22,9 +22,12 @@ Classifier: Topic :: Scientific/Engineering :: GIS
 Classifier: Topic :: Scientific/Engineering :: Visualization
 Requires-Python: >=3.10
 Requires-Dist: anywidget>=0.9.0
-Requires-Dist: marimo>=0.21.1
+Requires-Dist: marimo>=0.22
 Requires-Dist: narwhals>=1.0.0
+Requires-Dist: pandas>=2.3.3
 Requires-Dist: traitlets>=5.0.0
+Provides-Extra: binary
+Requires-Dist: numpy; extra == 'binary'
 Provides-Extra: dev
 Requires-Dist: duckdb; extra == 'dev'
 Requires-Dist: geopandas; extra == 'dev'
@@ -50,11 +53,14 @@ Interactive [deck.gl](https://deck.gl) visualization library for [marimo](https:
 
 ## Features
 
-- **33 deck.gl layer types** — scatter plots, hexagonal bins, heatmaps, arcs, paths, polygons, GeoJSON, 3D columns, and more
+- **12 deck.gl layer types** — scatter plots, hexagonal bins, heatmaps, arcs, paths, polygons, GeoJSON, 3D columns, lines, point clouds, and more
+- **Binary data transfer** — bypass JSON serialization for large datasets (30x faster, 4x smaller payloads)
 - **Multi-layer maps** — compose multiple layers on a single map
 - **Standalone layers** — display any layer directly without explicit map setup
 - **Marimo-native reactivity** — bind layer properties to sliders, dropdowns, and other widgets
+- **Performance metrics** — built-in FPS counter and frame time tracking via `perf_metrics` traitlet
 - **Multiple data sources** — pandas, polars, geopandas, DuckDB, GeoJSON dicts, and URLs
+- **Authenticated data loading** — pass HTTP headers, API keys, or credentials for remote data sources
 - **Fully offline** — all JavaScript bundled in the package, no CDN dependencies
 - **Viewport readback** — read the current map center, zoom, pitch, and bearing from Python
 - **Click & hover events** — inspect picked objects reactively in downstream cells
@@ -160,30 +166,143 @@ rel = duckdb.sql("SELECT lon, lat, value FROM 'data.parquet' WHERE value > 100")
 layer = dgl.ScatterplotLayer(data=rel, get_position=["lon", "lat"])
 ```
 
+### Binary data for large datasets
+
+For datasets with 100k+ rows, binary data transfer bypasses JSON serialization entirely, sending typed arrays directly to the GPU via deck.gl's native binary format.
+
+**Using `use_binary=True` (automatic packing from list-of-dicts):**
+
+```python
+layer = dgl.ScatterplotLayer(
+    data=large_df.to_dict("records"),
+    get_position=["lon", "lat"],
+    get_fill_color="color",
+    get_radius="radius",
+    use_binary=True,
+)
+```
+
+**Using pre-built numpy arrays (fastest — zero dict iteration):**
+
+```python
+import numpy as np
+from deckgl_marimo._binary import pack_binary
+
+# Prepare arrays
+positions = np.column_stack([lons, lats]).astype(np.float32)
+colors = np.array(color_data, dtype=np.uint8)  # (n, 4)
+
+# Create layer spec (no data — binary provides it)
+layer = dgl.ScatterplotLayer(
+    get_fill_color=[255, 140, 0],  # constant props still go in spec
+    radius_min_pixels=2,
+    use_binary=True,
+)
+spec = layer.to_spec()
+
+# Pack binary buffer
+meta, buf = pack_binary(
+    n=len(positions),
+    attributes={
+        "getPosition": (positions, "float32", 2),
+        "getFillColor": (colors, "uint8", 4),
+    },
+)
+meta["id"] = spec["id"]
+
+# Send to map
+map_widget.binary_metadata = {"layers": [meta]}
+map_widget.binary_data = buf
+map_widget.layer_specs = [spec]
+```
+
+**Performance at 200k polygons:**
+
+| Mode | Serialization | Payload | Speedup |
+|------|--------------|---------|---------|
+| JSON | 1,167 ms | 62 MB | — |
+| Binary | 39 ms | 14.5 MB | 30x faster, 4.3x smaller |
+
+Binary data is supported on: `ScatterplotLayer`, `PolygonLayer`, `PathLayer`, `ArcLayer`, `LineLayer`, `ColumnLayer`, and `PointCloudLayer`.
+
+### Performance metrics
+
+The `Map` widget includes a built-in FPS counter that reports metrics back to Python:
+
+```python
+map_widget = dgl.Map(basemap="dark-matter", center=(0, 0), zoom=1)
+widget = mo.ui.anywidget(map_widget)
+
+# Read performance metrics (updated every 500ms)
+perf = widget.value.get("perf_metrics", {})
+fps = perf.get("fps")           # frames per second
+frame_time = perf.get("frameTimeAvg")  # ms per frame
+```
+
+### Authenticated remote data
+
+Any layer that loads data from a URL supports custom HTTP headers via
+`fetch_headers`, or full control over the fetch request via `load_options`.
+
+```python
+# Bearer token
+layer = dgl.GeoJsonLayer(
+    data="https://secure-api.example.com/data.geojson",
+    fetch_headers={"Authorization": "Bearer my-token"},
+    get_fill_color=[0, 180, 230, 160],
+)
+
+# API key
+layer = dgl.GeoJsonLayer(
+    data="https://api.example.com/features",
+    fetch_headers={"X-API-Key": "abc123"},
+)
+
+# Full fetch control (mTLS / CORS / custom options)
+layer = dgl.GeoJsonLayer(
+    data="https://internal.example.com/data.geojson",
+    load_options={
+        "fetch": {
+            "credentials": "include",
+            "mode": "cors",
+            "headers": {"Authorization": "Bearer token"},
+        }
+    },
+)
+```
+
+Both parameters are available on all layer types via `BaseLayer`. When
+`fetch_headers` and `load_options` both specify headers, the `load_options`
+headers take precedence.
+
 ## Available layers
 
-### Fully tested (10)
+### Fully tested (12)
+
+| Layer | Use case | Binary support |
+|-------|----------|----------------|
+| `ScatterplotLayer` | Point data | Yes |
+| `GeoJsonLayer` | Polygons, lines, points from GeoJSON/GeoDataFrame | — |
+| `ArcLayer` | Origin-destination flows | Yes |
+| `PathLayer` | Routes, trajectories | Yes |
+| `PolygonLayer` | Filled regions | Yes |
+| `IconLayer` | Marker icons | — |
+| `TextLayer` | Labels | — |
+| `ColumnLayer` | 3D bars on map | Yes |
+| `HexagonLayer` | Hexagonal binning | — |
+| `HeatmapLayer` | Density visualization | — |
+| `LineLayer` | Straight lines between point pairs | Yes |
+| `PointCloudLayer` | 3D point clouds (LiDAR, etc.) | Yes |
 
-| Layer | Use case |
-|-------|----------|
-| `ScatterplotLayer` | Point data |
-| `GeoJsonLayer` | Polygons, lines, points from GeoJSON/GeoDataFrame |
-| `ArcLayer` | Origin-destination flows |
-| `PathLayer` | Routes, trajectories |
-| `PolygonLayer` | Filled regions |
-| `IconLayer` | Marker icons |
-| `TextLayer` | Labels |
-| `ColumnLayer` | 3D bars on map |
-| `HexagonLayer` | Hexagonal binning |
-| `HeatmapLayer` | Density visualization |
+`LineLayer` and `PointCloudLayer` are newly exported experimental layers.
 
-### Experimental (23)
+### Experimental (21+)
 
 All additional deck.gl layers are available as experimental stubs via `deckgl_marimo.layers`:
 
 ```python
 from deckgl_marimo.layers import TripsLayer, MVTLayer, H3HexagonLayer, ContourLayer
-# ... and 19 more
+# ... and more
 ```
 
 ## Basemaps
@@ -200,18 +319,8 @@ dgl.Map(basemap="https://my-tileserver.example.com/style.json")
 
 ### `Content-Length` errors in the marimo console
 
-You may see `h11._util.LocalProtocolError: Too little data for declared Content-Length`
-in the server console when a notebook loads. This is a known issue with uvicorn's
-default `h11` HTTP parser serving the large JS bundle. The map still works correctly.
-
-To suppress it, install `httptools` so uvicorn uses a faster HTTP parser, then
-**fully restart marimo** (kill and relaunch):
+This is fixed in Marimo >= 0.22.0 please update to that.
 
-```bash
-uv add httptools
-# then restart marimo
-uv run marimo edit your_notebook.py
-```
 
 ## License
 

{deckgl_marimo-0.2.0 → deckgl_marimo-0.4.0}/README.md

@@ -10,11 +10,14 @@ Interactive [deck.gl](https://deck.gl) visualization library for [marimo](https:
 
 ## Features
 
-- **33 deck.gl layer types** — scatter plots, hexagonal bins, heatmaps, arcs, paths, polygons, GeoJSON, 3D columns, and more
+- **12 deck.gl layer types** — scatter plots, hexagonal bins, heatmaps, arcs, paths, polygons, GeoJSON, 3D columns, lines, point clouds, and more
+- **Binary data transfer** — bypass JSON serialization for large datasets (30x faster, 4x smaller payloads)
 - **Multi-layer maps** — compose multiple layers on a single map
 - **Standalone layers** — display any layer directly without explicit map setup
 - **Marimo-native reactivity** — bind layer properties to sliders, dropdowns, and other widgets
+- **Performance metrics** — built-in FPS counter and frame time tracking via `perf_metrics` traitlet
 - **Multiple data sources** — pandas, polars, geopandas, DuckDB, GeoJSON dicts, and URLs
+- **Authenticated data loading** — pass HTTP headers, API keys, or credentials for remote data sources
 - **Fully offline** — all JavaScript bundled in the package, no CDN dependencies
 - **Viewport readback** — read the current map center, zoom, pitch, and bearing from Python
 - **Click & hover events** — inspect picked objects reactively in downstream cells
@@ -120,30 +123,143 @@ rel = duckdb.sql("SELECT lon, lat, value FROM 'data.parquet' WHERE value > 100")
 layer = dgl.ScatterplotLayer(data=rel, get_position=["lon", "lat"])
 ```
 
+### Binary data for large datasets
+
+For datasets with 100k+ rows, binary data transfer bypasses JSON serialization entirely, sending typed arrays directly to the GPU via deck.gl's native binary format.
+
+**Using `use_binary=True` (automatic packing from list-of-dicts):**
+
+```python
+layer = dgl.ScatterplotLayer(
+    data=large_df.to_dict("records"),
+    get_position=["lon", "lat"],
+    get_fill_color="color",
+    get_radius="radius",
+    use_binary=True,
+)
+```
+
+**Using pre-built numpy arrays (fastest — zero dict iteration):**
+
+```python
+import numpy as np
+from deckgl_marimo._binary import pack_binary
+
+# Prepare arrays
+positions = np.column_stack([lons, lats]).astype(np.float32)
+colors = np.array(color_data, dtype=np.uint8)  # (n, 4)
+
+# Create layer spec (no data — binary provides it)
+layer = dgl.ScatterplotLayer(
+    get_fill_color=[255, 140, 0],  # constant props still go in spec
+    radius_min_pixels=2,
+    use_binary=True,
+)
+spec = layer.to_spec()
+
+# Pack binary buffer
+meta, buf = pack_binary(
+    n=len(positions),
+    attributes={
+        "getPosition": (positions, "float32", 2),
+        "getFillColor": (colors, "uint8", 4),
+    },
+)
+meta["id"] = spec["id"]
+
+# Send to map
+map_widget.binary_metadata = {"layers": [meta]}
+map_widget.binary_data = buf
+map_widget.layer_specs = [spec]
+```
+
+**Performance at 200k polygons:**
+
+| Mode | Serialization | Payload | Speedup |
+|------|--------------|---------|---------|
+| JSON | 1,167 ms | 62 MB | — |
+| Binary | 39 ms | 14.5 MB | 30x faster, 4.3x smaller |
+
+Binary data is supported on: `ScatterplotLayer`, `PolygonLayer`, `PathLayer`, `ArcLayer`, `LineLayer`, `ColumnLayer`, and `PointCloudLayer`.
+
+### Performance metrics
+
+The `Map` widget includes a built-in FPS counter that reports metrics back to Python:
+
+```python
+map_widget = dgl.Map(basemap="dark-matter", center=(0, 0), zoom=1)
+widget = mo.ui.anywidget(map_widget)
+
+# Read performance metrics (updated every 500ms)
+perf = widget.value.get("perf_metrics", {})
+fps = perf.get("fps")           # frames per second
+frame_time = perf.get("frameTimeAvg")  # ms per frame
+```
+
+### Authenticated remote data
+
+Any layer that loads data from a URL supports custom HTTP headers via
+`fetch_headers`, or full control over the fetch request via `load_options`.
+
+```python
+# Bearer token
+layer = dgl.GeoJsonLayer(
+    data="https://secure-api.example.com/data.geojson",
+    fetch_headers={"Authorization": "Bearer my-token"},
+    get_fill_color=[0, 180, 230, 160],
+)
+
+# API key
+layer = dgl.GeoJsonLayer(
+    data="https://api.example.com/features",
+    fetch_headers={"X-API-Key": "abc123"},
+)
+
+# Full fetch control (mTLS / CORS / custom options)
+layer = dgl.GeoJsonLayer(
+    data="https://internal.example.com/data.geojson",
+    load_options={
+        "fetch": {
+            "credentials": "include",
+            "mode": "cors",
+            "headers": {"Authorization": "Bearer token"},
+        }
+    },
+)
+```
+
+Both parameters are available on all layer types via `BaseLayer`. When
+`fetch_headers` and `load_options` both specify headers, the `load_options`
+headers take precedence.
+
 ## Available layers
 
-### Fully tested (10)
+### Fully tested (12)
+
+| Layer | Use case | Binary support |
+|-------|----------|----------------|
+| `ScatterplotLayer` | Point data | Yes |
+| `GeoJsonLayer` | Polygons, lines, points from GeoJSON/GeoDataFrame | — |
+| `ArcLayer` | Origin-destination flows | Yes |
+| `PathLayer` | Routes, trajectories | Yes |
+| `PolygonLayer` | Filled regions | Yes |
+| `IconLayer` | Marker icons | — |
+| `TextLayer` | Labels | — |
+| `ColumnLayer` | 3D bars on map | Yes |
+| `HexagonLayer` | Hexagonal binning | — |
+| `HeatmapLayer` | Density visualization | — |
+| `LineLayer` | Straight lines between point pairs | Yes |
+| `PointCloudLayer` | 3D point clouds (LiDAR, etc.) | Yes |
 
-| Layer | Use case |
-|-------|----------|
-| `ScatterplotLayer` | Point data |
-| `GeoJsonLayer` | Polygons, lines, points from GeoJSON/GeoDataFrame |
-| `ArcLayer` | Origin-destination flows |
-| `PathLayer` | Routes, trajectories |
-| `PolygonLayer` | Filled regions |
-| `IconLayer` | Marker icons |
-| `TextLayer` | Labels |
-| `ColumnLayer` | 3D bars on map |
-| `HexagonLayer` | Hexagonal binning |
-| `HeatmapLayer` | Density visualization |
+`LineLayer` and `PointCloudLayer` are newly exported experimental layers.
 
-### Experimental (23)
+### Experimental (21+)
 
 All additional deck.gl layers are available as experimental stubs via `deckgl_marimo.layers`:
 
 ```python
 from deckgl_marimo.layers import TripsLayer, MVTLayer, H3HexagonLayer, ContourLayer
-# ... and 19 more
+# ... and more
 ```
 
 ## Basemaps
@@ -160,18 +276,8 @@ dgl.Map(basemap="https://my-tileserver.example.com/style.json")
 
 ### `Content-Length` errors in the marimo console
 
-You may see `h11._util.LocalProtocolError: Too little data for declared Content-Length`
-in the server console when a notebook loads. This is a known issue with uvicorn's
-default `h11` HTTP parser serving the large JS bundle. The map still works correctly.
-
-To suppress it, install `httptools` so uvicorn uses a faster HTTP parser, then
-**fully restart marimo** (kill and relaunch):
+This is fixed in Marimo >= 0.22.0 please update to that.
 
-```bash
-uv add httptools
-# then restart marimo
-uv run marimo edit your_notebook.py
-```
 
 ## License