geolibre-wasm 0.4.0__tar.gz

geolibre_wasm-0.4.0/.gitignore

@@ -0,0 +1,20 @@
+/target
+node_modules
+# Local-only drafts and notes (e.g. LinkedIn announcements)
+/private/
+# Build artifacts staged into npm/ by build.sh (regenerated on every build)
+npm/geolibre-cli.wasm
+npm/geolibre_wasm.js
+npm/geolibre_wasm_bg.wasm
+npm/geolibre_wasm.d.ts
+npm/geolibre_wasm_bg.wasm.d.ts
+npm/snippets/
+*.wasm-opt.wasm
+# Python wrapper build artifacts
+__pycache__/
+*.egg-info/
+python/build/
+python/dist/
+.pytest_cache/
+# Cargo.lock is committed: this workspace ships a binary, and the lock pins the
+# exact whitebox_next_gen fork commit for reproducible WASI builds.

geolibre_wasm-0.4.0/PKG-INFO

@@ -0,0 +1,108 @@
+Metadata-Version: 2.4
+Name: geolibre-wasm
+Version: 0.4.0
+Summary: Run the GeoLibre / whitebox_next_gen geospatial tool suite (WASM/WASI) from Python.
+Project-URL: Homepage, https://github.com/opengeos/geolibre-rust
+Project-URL: Repository, https://github.com/opengeos/geolibre-rust
+Project-URL: Issues, https://github.com/opengeos/geolibre-rust/issues
+Author-email: Qiusheng Wu <giswqs@gmail.com>
+License-Expression: MIT
+Keywords: geolibre,geoparquet,geospatial,gis,raster,vector,wasi,wasm,whitebox
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Scientific/Engineering :: GIS
+Requires-Python: >=3.9
+Requires-Dist: wasmtime>=20
+Description-Content-Type: text/markdown
+
+# geolibre-wasm (Python)
+
+Run the [`geolibre-rust`](https://github.com/opengeos/geolibre-rust) geospatial
+tool suite (the `whitebox_next_gen` tools plus GeoLibre's own) from Python. The
+tools are a single WebAssembly (WASI) module executed in-process via
+[`wasmtime`](https://github.com/bytecodealliance/wasmtime-py), so there is **no
+native install, no GDAL, and no server** — just `pip install`.
+
+This mirrors the JavaScript `geolibre-wasm/tools` API (`list_tools`,
+`list_manifests`, `run_tool`), so the two stay in sync.
+
+> The import package is `geolibre_wasm` (the distribution is `geolibre-wasm`),
+> matching the npm package name and avoiding a clash with the separate
+> `geolibre` application package.
+
+## Install
+
+```bash
+pip install geolibre-wasm
+```
+
+On first use the runtime (`geolibre-cli.wasm`, ~20 MB) is downloaded from the
+matching GitHub release and cached under `~/.cache/geolibre/`. To use a local
+copy instead, set `GEOLIBRE_WASM=/path/to/geolibre-cli.wasm` or pass
+`wasm_path=` to any call.
+
+## Usage
+
+Inputs are passed as `bytes` under `/work`; the tool reads/writes there and any
+new files come back as `bytes`.
+
+```python
+import geolibre_wasm as gl
+
+# Discover tools (each manifest carries a "source": "geolibre" | "whitebox")
+tools = gl.list_tools()
+manifests = gl.list_manifests()
+
+# Raster: compute slope from a DEM
+dem = open("dem.tif", "rb").read()
+res = gl.run_tool(
+    "slope",
+    args=["--input=/work/dem.tif", "--output=/work/slope.tif", "--units=degrees"],
+    input={"dem.tif": dem},
+)
+assert res.exit_code == 0
+open("slope.tif", "wb").write(res.files["slope.tif"])
+
+# Reproject (warp) to a target EPSG
+res = gl.run_tool(
+    "reproject_raster",
+    args=["--input=/work/dem.tif", "--output=/work/wgs84.tif", "--epsg=4326"],
+    input={"dem.tif": dem},
+)
+
+# Vector: GeoJSON -> GeoParquet (Hilbert-sorted, bbox covering, ZSTD by default)
+gj = open("cities.geojson", "rb").read()
+res = gl.run_tool(
+    "write_geoparquet",
+    args=["--input=/work/in.geojson", "--output=/work/out.parquet"],
+    input={"in.geojson": gj},
+)
+open("cities.parquet", "wb").write(res.files["out.parquet"])
+```
+
+Tools that write a directory tree (e.g. `raster_to_tiles`) return nested keys:
+
+```python
+res = gl.run_tool(
+    "raster_to_tiles",
+    args=["--input=/work/dem.tif", "--output_dir=/work/tiles", "--min_zoom=16", "--max_zoom=18"],
+    input={"dem.tif": dem},
+)
+for path, data in res.files.items():
+    # e.g. "tiles/16/9559/32767.png"
+    ...
+```
+
+## API
+
+- `list_tools(wasm_path=None) -> list[str]`
+- `list_manifests(wasm_path=None) -> list[dict]`
+- `run_tool(tool, args=None, input=None, wasm_path=None) -> ToolResult`
+- `ToolResult(exit_code: int, stdout: list[str], files: dict[str, bytes])`
+- `runtime_path(wasm_path=None) -> str` — resolve the runtime (explicit > `GEOLIBRE_WASM` > cached download)
+- `download_runtime(dest=None) -> str` — fetch the runtime ahead of time
+
+## License
+
+MIT

geolibre_wasm-0.4.0/README.md

@@ -0,0 +1,90 @@
+# geolibre-wasm (Python)
+
+Run the [`geolibre-rust`](https://github.com/opengeos/geolibre-rust) geospatial
+tool suite (the `whitebox_next_gen` tools plus GeoLibre's own) from Python. The
+tools are a single WebAssembly (WASI) module executed in-process via
+[`wasmtime`](https://github.com/bytecodealliance/wasmtime-py), so there is **no
+native install, no GDAL, and no server** — just `pip install`.
+
+This mirrors the JavaScript `geolibre-wasm/tools` API (`list_tools`,
+`list_manifests`, `run_tool`), so the two stay in sync.
+
+> The import package is `geolibre_wasm` (the distribution is `geolibre-wasm`),
+> matching the npm package name and avoiding a clash with the separate
+> `geolibre` application package.
+
+## Install
+
+```bash
+pip install geolibre-wasm
+```
+
+On first use the runtime (`geolibre-cli.wasm`, ~20 MB) is downloaded from the
+matching GitHub release and cached under `~/.cache/geolibre/`. To use a local
+copy instead, set `GEOLIBRE_WASM=/path/to/geolibre-cli.wasm` or pass
+`wasm_path=` to any call.
+
+## Usage
+
+Inputs are passed as `bytes` under `/work`; the tool reads/writes there and any
+new files come back as `bytes`.
+
+```python
+import geolibre_wasm as gl
+
+# Discover tools (each manifest carries a "source": "geolibre" | "whitebox")
+tools = gl.list_tools()
+manifests = gl.list_manifests()
+
+# Raster: compute slope from a DEM
+dem = open("dem.tif", "rb").read()
+res = gl.run_tool(
+    "slope",
+    args=["--input=/work/dem.tif", "--output=/work/slope.tif", "--units=degrees"],
+    input={"dem.tif": dem},
+)
+assert res.exit_code == 0
+open("slope.tif", "wb").write(res.files["slope.tif"])
+
+# Reproject (warp) to a target EPSG
+res = gl.run_tool(
+    "reproject_raster",
+    args=["--input=/work/dem.tif", "--output=/work/wgs84.tif", "--epsg=4326"],
+    input={"dem.tif": dem},
+)
+
+# Vector: GeoJSON -> GeoParquet (Hilbert-sorted, bbox covering, ZSTD by default)
+gj = open("cities.geojson", "rb").read()
+res = gl.run_tool(
+    "write_geoparquet",
+    args=["--input=/work/in.geojson", "--output=/work/out.parquet"],
+    input={"in.geojson": gj},
+)
+open("cities.parquet", "wb").write(res.files["out.parquet"])
+```
+
+Tools that write a directory tree (e.g. `raster_to_tiles`) return nested keys:
+
+```python
+res = gl.run_tool(
+    "raster_to_tiles",
+    args=["--input=/work/dem.tif", "--output_dir=/work/tiles", "--min_zoom=16", "--max_zoom=18"],
+    input={"dem.tif": dem},
+)
+for path, data in res.files.items():
+    # e.g. "tiles/16/9559/32767.png"
+    ...
+```
+
+## API
+
+- `list_tools(wasm_path=None) -> list[str]`
+- `list_manifests(wasm_path=None) -> list[dict]`
+- `run_tool(tool, args=None, input=None, wasm_path=None) -> ToolResult`
+- `ToolResult(exit_code: int, stdout: list[str], files: dict[str, bytes])`
+- `runtime_path(wasm_path=None) -> str` — resolve the runtime (explicit > `GEOLIBRE_WASM` > cached download)
+- `download_runtime(dest=None) -> str` — fetch the runtime ahead of time
+
+## License
+
+MIT

geolibre_wasm-0.4.0/pyproject.toml

@@ -0,0 +1,28 @@
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[project]
+name = "geolibre-wasm"
+version = "0.4.0"
+description = "Run the GeoLibre / whitebox_next_gen geospatial tool suite (WASM/WASI) from Python."
+readme = "README.md"
+requires-python = ">=3.9"
+license = "MIT"
+authors = [{ name = "Qiusheng Wu", email = "giswqs@gmail.com" }]
+keywords = ["geospatial", "gis", "wasm", "wasi", "raster", "vector", "geoparquet", "whitebox", "geolibre"]
+classifiers = [
+    "Programming Language :: Python :: 3",
+    "License :: OSI Approved :: MIT License",
+    "Operating System :: OS Independent",
+    "Topic :: Scientific/Engineering :: GIS",
+]
+dependencies = ["wasmtime>=20"]
+
+[project.urls]
+Homepage = "https://github.com/opengeos/geolibre-rust"
+Repository = "https://github.com/opengeos/geolibre-rust"
+Issues = "https://github.com/opengeos/geolibre-rust/issues"
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/geolibre_wasm"]

geolibre_wasm-0.4.0/src/geolibre_wasm/__init__.py

@@ -0,0 +1,40 @@
+"""GeoLibre: run the whitebox_next_gen + GeoLibre geospatial tool suite from Python.
+
+The tools are compiled to a single WebAssembly (WASI) module and executed
+in-process via wasmtime, so there is no native install, GDAL, or server. The API
+mirrors the JavaScript ``geolibre-wasm/tools`` package.
+
+Example:
+    >>> import geolibre_wasm as gl
+    >>> dem = open("dem.tif", "rb").read()
+    >>> result = gl.run_tool(
+    ...     "slope",
+    ...     args=["--input=/work/dem.tif", "--output=/work/slope.tif", "--units=degrees"],
+    ...     input={"dem.tif": dem},
+    ... )
+    >>> result.exit_code
+    0
+    >>> open("slope.tif", "wb").write(result.files["slope.tif"])
+"""
+
+from ._core import (
+    RUNTIME_VERSION,
+    ToolResult,
+    download_runtime,
+    list_manifests,
+    list_tools,
+    run_tool,
+    runtime_path,
+)
+
+__all__ = [
+    "ToolResult",
+    "RUNTIME_VERSION",
+    "list_tools",
+    "list_manifests",
+    "run_tool",
+    "runtime_path",
+    "download_runtime",
+]
+
+__version__ = "0.4.0"

geolibre_wasm-0.4.0/src/geolibre_wasm/_core.py

@@ -0,0 +1,220 @@
+"""Run the GeoLibre WASI tool runner from Python via an in-process wasmtime.
+
+The tools are the single ``geolibre-cli.wasm`` (a ``wasm32-wasip1`` module). Each
+call runs it over a private temporary directory preopened as ``/work``: input
+files are written there, the tool reads/writes via ordinary ``std::fs``, and any
+new files are returned as ``bytes``. This mirrors the JavaScript ``tools.mjs``
+API (``list_tools`` / ``list_manifests`` / ``run_tool``) so the two stay in sync.
+"""
+
+from __future__ import annotations
+
+import json
+import os
+import shutil
+import tempfile
+import urllib.request
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Dict, List, Mapping, Optional, Sequence, Union
+
+import wasmtime
+
+#: Release whose ``geolibre-cli.wasm`` asset this wrapper downloads by default.
+#: Kept in sync with the package version's ``vMAJOR.MINOR.PATCH`` tag.
+RUNTIME_VERSION = "v0.4.0"
+_ASSET = f"geolibre-cli-{RUNTIME_VERSION}.wasm"
+_RELEASE_URL = (
+    "https://github.com/opengeos/geolibre-rust/releases/download/"
+    f"{RUNTIME_VERSION}/{_ASSET}"
+)
+_STDOUT_CAPTURE = ".geolibre-stdout"
+
+PathLike = Union[str, os.PathLike]
+
+
+@dataclass
+class ToolResult:
+    """Result of running a tool.
+
+    Attributes:
+        exit_code: Process exit code (0 = success).
+        stdout: Captured stdout/stderr lines.
+        files: New files the tool wrote, keyed by path relative to ``/work``.
+            Tools that write a tree (e.g. ``raster_to_tiles``) use nested keys
+            such as ``"tiles/15/4779/16383.png"``.
+    """
+
+    exit_code: int
+    stdout: List[str]
+    files: Dict[str, bytes]
+
+
+_engine: Optional[wasmtime.Engine] = None
+_module_cache: Dict[str, "wasmtime.Module"] = {}
+
+
+def _get_engine() -> "wasmtime.Engine":
+    global _engine
+    if _engine is None:
+        _engine = wasmtime.Engine()
+    return _engine
+
+
+def _cache_path() -> Path:
+    base = os.environ.get("XDG_CACHE_HOME") or (Path.home() / ".cache")
+    return Path(base) / "geolibre" / _ASSET
+
+
+def download_runtime(dest: Optional[PathLike] = None) -> str:
+    """Download the ``geolibre-cli.wasm`` runtime from the GitHub release.
+
+    Args:
+        dest: Where to write the file. Defaults to the per-user cache
+            (``$XDG_CACHE_HOME/geolibre/`` or ``~/.cache/geolibre/``).
+
+    Returns:
+        The path to the downloaded runtime.
+    """
+    target = Path(dest) if dest is not None else _cache_path()
+    target.parent.mkdir(parents=True, exist_ok=True)
+    tmp = target.with_name(target.name + ".download")
+    urllib.request.urlretrieve(_RELEASE_URL, tmp)  # noqa: S310 (trusted release URL)
+    tmp.replace(target)
+    return str(target)
+
+
+def runtime_path(wasm_path: Optional[PathLike] = None) -> str:
+    """Resolve the runtime ``.wasm`` to use.
+
+    Resolution order: the explicit ``wasm_path`` argument, the ``GEOLIBRE_WASM``
+    environment variable, then the cached download (fetched on first use).
+
+    Args:
+        wasm_path: Explicit path to a ``geolibre-cli.wasm``; takes precedence.
+
+    Returns:
+        Filesystem path to the runtime module.
+    """
+    if wasm_path is not None:
+        return str(wasm_path)
+    env = os.environ.get("GEOLIBRE_WASM")
+    if env:
+        return env
+    cache = _cache_path()
+    if not cache.exists():
+        download_runtime(cache)
+    return str(cache)
+
+
+def _load_module(path: str) -> "wasmtime.Module":
+    module = _module_cache.get(path)
+    if module is None:
+        module = wasmtime.Module.from_file(_get_engine(), path)
+        _module_cache[path] = module
+    return module
+
+
+def _exec(
+    argv: Sequence[str],
+    inputs: Optional[Mapping[str, bytes]] = None,
+    wasm_path: Optional[PathLike] = None,
+) -> ToolResult:
+    inputs = inputs or {}
+    module = _load_module(runtime_path(wasm_path))
+    engine = _get_engine()
+    work = Path(tempfile.mkdtemp(prefix="geolibre-"))
+    try:
+        for name, data in inputs.items():
+            dest = work / name
+            dest.parent.mkdir(parents=True, exist_ok=True)
+            dest.write_bytes(bytes(data))
+
+        stdout_file = work / _STDOUT_CAPTURE
+        store = wasmtime.Store(engine)
+        wasi = wasmtime.WasiConfig()
+        wasi.argv = ["geolibre", *argv]
+        wasi.preopen_dir(str(work), "/work")
+        # Merge stdout + stderr into one capture, matching tools.mjs.
+        wasi.stdout_file = str(stdout_file)
+        wasi.stderr_file = str(stdout_file)
+        store.set_wasi(wasi)
+
+        linker = wasmtime.Linker(engine)
+        linker.define_wasi()
+        instance = linker.instantiate(store, module)
+        start = instance.exports(store)["_start"]
+
+        exit_code = 0
+        try:
+            start(store)  # WASI returns by calling proc_exit -> ExitTrap
+        except wasmtime.ExitTrap as exit_trap:
+            exit_code = exit_trap.code
+
+        stdout: List[str] = []
+        if stdout_file.exists():
+            stdout = stdout_file.read_text(errors="replace").splitlines()
+
+        input_names = set(inputs)
+        files: Dict[str, bytes] = {}
+        for path in sorted(work.rglob("*")):
+            if not path.is_file():
+                continue
+            rel = path.relative_to(work).as_posix()
+            if rel == _STDOUT_CAPTURE or rel in input_names:
+                continue
+            files[rel] = path.read_bytes()
+
+        return ToolResult(exit_code=exit_code, stdout=stdout, files=files)
+    finally:
+        shutil.rmtree(work, ignore_errors=True)
+
+
+def list_tools(wasm_path: Optional[PathLike] = None) -> List[str]:
+    """List every available tool id.
+
+    Args:
+        wasm_path: Optional explicit runtime path (see :func:`runtime_path`).
+
+    Returns:
+        The tool ids, one per registered tool.
+    """
+    result = _exec(["list"], wasm_path=wasm_path)
+    return [line.strip() for line in result.stdout if line.strip()]
+
+
+def list_manifests(wasm_path: Optional[PathLike] = None) -> List[dict]:
+    """Fetch every tool manifest (id, parameters, category, provenance, ...).
+
+    Args:
+        wasm_path: Optional explicit runtime path (see :func:`runtime_path`).
+
+    Returns:
+        A list of manifest dicts, each including a ``"source"`` field of
+        ``"geolibre"`` or ``"whitebox"``.
+    """
+    result = _exec(["manifests"], wasm_path=wasm_path)
+    return json.loads("".join(result.stdout))
+
+
+def run_tool(
+    tool: str,
+    args: Optional[Sequence[str]] = None,
+    input: Optional[Mapping[str, bytes]] = None,
+    wasm_path: Optional[PathLike] = None,
+) -> ToolResult:
+    """Run one tool over an in-memory ``/work`` filesystem.
+
+    Args:
+        tool: Tool id, e.g. ``"slope"`` (see :func:`list_tools`).
+        args: CLI args, e.g.
+            ``["--input=/work/dem.tif", "--output=/work/slope.tif", "--units=degrees"]``.
+        input: Files placed under ``/work`` before the run, keyed by filename
+            (values are ``bytes``).
+        wasm_path: Optional explicit runtime path (see :func:`runtime_path`).
+
+    Returns:
+        A :class:`ToolResult` with the exit code, captured output, and any new
+        files the tool wrote (keyed by path relative to ``/work``).
+    """
+    return _exec([tool, *(args or [])], inputs=input, wasm_path=wasm_path)

geolibre_wasm-0.4.0/tests/test_smoke.py

@@ -0,0 +1,65 @@
+"""Smoke tests for the geolibre Python wrapper.
+
+These run the real WASI tool runner. Point ``GEOLIBRE_WASM`` at a local
+``geolibre-cli.wasm`` to avoid a network download (the build stages one into
+``npm/``); otherwise the runtime is fetched from the GitHub release on first use.
+"""
+
+from __future__ import annotations
+
+import json
+
+import geolibre_wasm as geolibre
+
+
+def test_list_tools():
+    tools = geolibre.list_tools()
+    assert len(tools) > 100
+    # The GeoLibre-authored tools are registered alongside whitebox's.
+    for tool in ("slope", "reproject_raster", "write_geoparquet", "read_geoparquet"):
+        assert tool in tools
+
+
+def test_manifests_carry_provenance():
+    manifests = geolibre.list_manifests()
+    by_id = {m["id"]: m for m in manifests}
+    assert by_id["reproject_raster"]["source"] == "geolibre"
+    assert by_id["slope"]["source"] == "whitebox"
+
+
+def test_geoparquet_roundtrip():
+    geojson = json.dumps(
+        {
+            "type": "FeatureCollection",
+            "features": [
+                {
+                    "type": "Feature",
+                    "properties": {"name": "A", "val": 1},
+                    "geometry": {"type": "Point", "coordinates": [-122.3, 47.6]},
+                },
+                {
+                    "type": "Feature",
+                    "properties": {"name": "B", "val": 2},
+                    "geometry": {"type": "Point", "coordinates": [-122.4, 47.7]},
+                },
+            ],
+        }
+    ).encode()
+
+    written = geolibre.run_tool(
+        "write_geoparquet",
+        args=["--input=/work/in.geojson", "--output=/work/out.parquet"],
+        input={"in.geojson": geojson},
+    )
+    assert written.exit_code == 0
+    parquet = written.files["out.parquet"]
+    assert parquet[:4] == b"PAR1"
+
+    back = geolibre.run_tool(
+        "read_geoparquet",
+        args=["--input=/work/x.parquet", "--output=/work/back.geojson"],
+        input={"x.parquet": parquet},
+    )
+    assert back.exit_code == 0
+    fc = json.loads(back.files["back.geojson"])
+    assert len(fc["features"]) == 2