core-lens 0.1.dev136__tar.gz → 0.1.dev138__tar.gz

{core_lens-0.1.dev136 → core_lens-0.1.dev138}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: core-lens
-Version: 0.1.dev136
+Version: 0.1.dev138
 Summary: Query, analyse, and visualise CoreStack's microwatershed and Earth science data through a clean, composable Python API.
 Project-URL: Homepage, https://github.com/ApoorvaKashyap/core-lens
 Project-URL: Issues, https://github.com/ApoorvaKashyap/core-lens/issues

{core_lens-0.1.dev136 → core_lens-0.1.dev138}/docs/source/export.md

@@ -41,7 +41,13 @@ geoparquet(
 
 # Export to GeoJSON
 geojson(spatial_res, "output.json")
+
+# Export to newline-delimited GeoJSON (GeoJSONSeq)
+geojson(spatial_res, "output.ndjson", driver="GeoJSONSeq")
 ```
 
-> [!NOTE]
-> Exporting to geospatial formats automatically uses DuckDB to cast WKB geometries to proper spatial types via the `spatial` extension. CoreLens handles the installation and loading of the extension seamlessly under the hood.
+```{note}
+Exporting to **GeoParquet** uses DuckDB to cast WKB geometries to proper spatial types via the `spatial` extension. CoreLens handles the installation and loading of the extension seamlessly under the hood.
+
+Exporting to **GeoJSON** is fully natively streamed directly from Polars and Shapely without invoking GDAL or DuckDB, making it extremely fast for large datasets. Driver-specific kwargs for GeoJSON are ignored.
+```

{core_lens-0.1.dev136 → core_lens-0.1.dev138}/docs/source/intro.md

@@ -13,4 +13,6 @@ CoreLens provides a unified interface over microwatersheds, administrative bound
 - **Temporal & Seasonal Awareness**: Native support for agronomic seasons (Kharif, Rabi, Zaid) and time-range filtering.
 - **Spatial Statistics & Analysis**: Built-in methods for anomaly detection, spatial similarity, temporal correlation, and hypothesis testing.
 - **Interactive Visualisation**: Generate interactive maps using Lonboard with zero-copy GeoArrow rendering directly from Polars, and timeseries/scatter plots using Bokeh.
-- **High-Performance Exports**: Export results directly to Parquet, JSON, CSV, or spatially-enabled GeoParquet and GeoJSON using integrated DuckDB spatial extensions.
+- **Cloud & Local Data**: Native support for reading Parquet data from local filesystems or directly from cloud storage (e.g., `s3://`).
+- **High-Performance Exports**: Export results directly to Parquet, JSON, CSV, or spatially-enabled GeoParquet and GeoJSON (with a fast streaming writer).
+- **Advanced Caching**: Transparently caches boundaries and pre-computes spatial indices (via on-disk sidecars) to achieve sub-second query initialisation times.

{core_lens-0.1.dev136 → core_lens-0.1.dev138}/docs/source/queries.md

@@ -58,5 +58,6 @@ joined_view = aoi.mws.spatial_join(
 # res = joined_view.annual
 ```
 
-> [!NOTE]
-> Cross-entity spatial join execution (materialisation) is currently under development and will be added in a subsequent release. Calling materialisation methods (like `.annual`, `.static`, or `.fortnightly`) on a joined view will raise a `NotImplementedError` in the current version.
+```{note}
+Cross-entity spatial join execution (materialisation) is currently under development and will be added in a subsequent release. Calling materialisation methods (like `.annual`, `.static`, or `.fortnightly`) on a joined view will raise a `NotImplementedError` in the current version.
+```

{core_lens-0.1.dev136 → core_lens-0.1.dev138}/docs/source/quickstart.md

@@ -25,18 +25,18 @@ AoI.register(MWSEntity)
 AoI.register(TehsilEntity)
 ```
 
-Create an `AoI` (Area of Interest) by specifying the `data_root` and your boundary (either by name, bounding box, or Shapely geometry):
+Create an `AoI` (Area of Interest) by specifying the `data_root` (which can be a local path or a cloud URI like `s3://bucket/data`) and your boundary (either by name, bounding box, or Shapely geometry):
 
 ```python
-# Create an AoI scoped to a specific Tehsil
+# Create an AoI scoped to a specific Tehsil using a local path
 aoi = AoI(
     data_root="/path/to/data",
     tehsil="Pangi"
 )
 
-# Or scope it directly to a specific list of entity IDs
+# Or scope it directly to a specific list of entity IDs using an S3 bucket
 aoi_specific = AoI(
-    data_root="/path/to/data",
+    data_root="s3://my-cloud-bucket/core-lens-data",
     mws_id=["13_001", "13_002"]
 )
 

{core_lens-0.1.dev136 → core_lens-0.1.dev138}/src/core_lens/_version.py

@@ -18,7 +18,7 @@ version_tuple: tuple[int | str, ...]
 commit_id: str | None
 __commit_id__: str | None
 
-__version__ = version = '0.1.dev136'
-__version_tuple__ = version_tuple = (0, 1, 'dev136')
+__version__ = version = '0.1.dev138'
+__version_tuple__ = version_tuple = (0, 1, 'dev138')
 
 __commit_id__ = commit_id = None

{core_lens-0.1.dev136 → core_lens-0.1.dev138}/src/core_lens/schema/detection.py

@@ -178,6 +178,14 @@ def _infer_bbox_cols(schema: pl.Schema) -> tuple[str, str, str, str] | None:
     for pattern in _BBOX_PATTERNS:
         if all(col in schema for col in pattern):
             return pattern
+
+    for col_name, dtype in schema.items():
+        if isinstance(dtype, pl.Struct):
+            struct_fields = [f.name for f in dtype.fields]
+            for pattern in _BBOX_PATTERNS:
+                if all(f in struct_fields for f in pattern):
+                    return tuple(f"{col_name}.{f}" for f in pattern)  # type: ignore
+
     return None
 
 

{core_lens-0.1.dev136 → core_lens-0.1.dev138}/src/core_lens/utils/spatial.py

@@ -173,14 +173,27 @@ def build_bbox_index(
     _so = storage_options or {}
     if bbox_cols is not None:
         logger.debug("Using pre-computed bbox columns: {}", bbox_cols)
-        cols_to_read = key_cols + list(bbox_cols)
+        base_bbox_cols = list(dict.fromkeys(c.split(".")[0] for c in bbox_cols))
+        cols_to_read = key_cols + base_bbox_cols
         df = pl.read_parquet(
             static_path, columns=cols_to_read, storage_options=_so or None
         )
         minx_col, miny_col, maxx_col, maxy_col = bbox_cols
-        return df.rename(
-            {minx_col: "minx", miny_col: "miny", maxx_col: "maxx", maxy_col: "maxy"}
-        )
+
+        for alias, col_path in zip(
+            ["minx", "miny", "maxx", "maxy"],
+            [minx_col, miny_col, maxx_col, maxy_col],
+        ):
+            parts = col_path.split(".")
+            if len(parts) == 1:
+                df = df.rename({col_path: alias})
+            else:
+                expr = pl.col(parts[0])
+                for p in parts[1:]:
+                    expr = expr.struct.field(p)
+                df = df.with_columns(expr.alias(alias))
+
+        return df.select(key_cols + ["minx", "miny", "maxx", "maxy"])
 
     cols_to_read = key_cols + [geometry_col]
     if geometry_type == "latlon":