giga-spatial 0.6.2__tar.gz → 0.6.4__tar.gz

{giga_spatial-0.6.2 → giga_spatial-0.6.4}/.env_sample

@@ -12,6 +12,8 @@ export OPENCELLID_ACCESS_TOKEN=""
 export GEOREPO_API_KEY=""
 export GEOREPO_USER_EMAIL=""
 export GIGA_SCHOOL_LOCATION_API_KEY=""
+export GIGA_SCHOOL_PROFILE_API_KEY=""
+export GIGA_SCHOOL_MEASUREMENTS_API_KEY=""
 export ROOT_DATA_DIR=""
 export BRONZE_DIR=""
 export SILVER_DIR=""

{giga_spatial-0.6.2 → giga_spatial-0.6.4}/CHANGELOG.md

@@ -2,6 +2,87 @@
 
 All notable changes to this project will be documented in this file.
 
+## [v0.6.4] - 2025-06-19
+
+### Added
+
+- **GigaSchoolProfileFetcher**  
+  - New class to fetch and process school profile data from the Giga School Profile API  
+  - Supports paginated fetching, filtering by country and school ID  
+  - Includes methods to generate connectivity summary statistics by region, connection type, and source
+
+- **GigaSchoolMeasurementsFetcher**  
+  - New class to fetch and process daily real-time connectivity measurements from the Giga API  
+  - Supports filtering by date range and school  
+  - Includes performance summary generation (download/upload speeds, latency, quality flags)
+
+- **AdminBoundaries.from_geoboundaries**  
+  - New class method to download and process geoBoundaries data by country and admin level  
+  - Automatically handles HDX dataset discovery, downloading, and fallback logic
+
+- **HDXConfig.search_datasets**  
+  - Static method to search HDX datasets without full handler initialization  
+  - Supports query string, sort order, result count, HDX site selection, and custom user agent
+
+### Fixed
+
+- Typo in `MaxarImageDownloader` causing runtime error
+
+### Documentation
+
+- **Improved Configuration Guide** (`docs/user-guide/configuration.md`)  
+  - Added comprehensive table of environment variables with defaults and descriptions  
+  - Synced `.env_sample` and `config.py` with docs  
+  - Example `.env` file and guidance on path overrides using `config.set_path`  
+  - New section on `config.ensure_directories_exist` and troubleshooting tips  
+  - Clearer handling of credentials and security notes  
+  - Improved formatting and structure for clarity
+
+## [v0.6.3] - 2025-06-16
+
+### Added
+- Major refactor of `HDX` module to align with unified `BaseHandler` architecture:
+  - `HDXConfig`: fully aligned with `BaseHandlerConfig` structure.
+  - Added flexible pattern matching for resource filtering.
+  - Improved data unit resolution by country, geometry, and points.
+  - Enhanced resource filtering with exact and regex options.
+- `HDXDownloader` fully aligned with `BaseHandlerDownloader`:
+  - Simplified sequential download logic.
+  - Improved error handling, validation, and logging.
+- `HDXReader` fully aligned with `BaseHandlerReader`:
+  - Added `resolve_source_paths` and `load_all_resources` methods.
+  - Simplified source handling for single and multiple files.
+  - Cleaned up redundant and dataset-specific logic.
+- Introduced `HDXHandler` as unified orchestration layer using factory methods.
+
+- Refactor of `RelativeWealthIndex (RWI)` module:
+  - Added new `RWIHandler` class aligned with `HDXHandler` and `BaseHandler`.
+  - Simplified class names: `RWIDownloader` and `RWIReader`.
+  - Enhanced configuration with `latest_only` flag to select newest resources automatically.
+  - Simplified resource filtering and country resolution logic.
+  - Improved code maintainability, type hints, and error handling.
+
+
+- **New raster multi-band support in TifProcessor:**
+  - Added new `multi` mode for handling multi-band raster datasets.
+  - Automatic band name detection from raster metadata.
+  - Added strict mode validation (`single`, `rgb`, `rgba`, `multi`).
+  - Enhanced error handling for invalid modes and band counts.
+
+### Fixed
+- Fixed GHSL tiles loading behavior for correct coordinate system handling:
+  - Moved `TILES_URL` formatting and tile loading to `validate_configuration`.
+  - Ensures proper tile loading after CRS validation.
+
+### Documentation
+- Updated and standardized API references across documentation.
+- Standardized handler method names and usage examples.
+- Added building enrichment examples for POI processing.
+- Updated installation instructions.
+
+### Deprecated
+- Deprecated direct imports from individual handler modules.
+
 ## [v0.6.2] - 2025-06-11
 
 ### Added

{giga_spatial-0.6.2 → giga_spatial-0.6.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: giga-spatial
-Version: 0.6.2
+Version: 0.6.4
 Summary: A package for spatial data download & processing
 Home-page: https://github.com/unicef/giga-spatial
 Author: Utku Can Ozturk
@@ -51,17 +51,27 @@ Dynamic: summary
 
 # GigaSpatial
 
-# ![GigaSpatial logo](https://raw.githubusercontent.com/unicef/giga-spatial/main/docs/assets/logo.png)
-Giga is a UNICEF-ITU initiative to connect every school to the Internet and every young person to information, opportunity and choice. 
-Giga maps schools' Internet access in real time, creates models for innovative financing, and supports governments contracting for connectivity. 
+## About Giga
 
----
+[Giga](https://giga.global/) is a UNICEF-ITU initiative to connect every school to the Internet and every young person to information, opportunity and choice. 
+Giga maps schools' Internet access in real time, creates models for innovative financing, and supports governments contracting for connectivity. 
 
 ## About GigaSpatial
 
 **GigaSpatial** is a Python package developed as part of the Giga Applied Science Team to handle geospatial data efficiently. It provides tools for downloading, processing, and analyzing geospatial data, enabling users to work with datasets such as OpenStreetMap (OSM), Global Human Settlement Layer (GHSL), Microsoft Global Buildings, Google Open Buildings, and more. The package is designed to support Giga's mission by providing robust geospatial capabilities for mapping and analyzing school connectivity.
 
-### Key Features
+## Installation
+
+See the [installation docs](https://unicef.github.io/giga-spatial/getting-started/installation/) for all details. GigaSpatial requires Python 3.10 or above and depends on the following key packages:
+
+- geopandas
+- pandas
+- shapely
+- rasterio
+
+We recommend using a virtual environment for installation. See the [installation docs](https://unicef.github.io/giga-spatial/getting-started/installation/) for more details.
+
+## Key Features
 - **Data Downloading**: Download geospatial data from various sources including GHSL, Microsoft Global Buildings, Google Open Buildings, OpenCellID, and HDX datasets.
 - **Data Processing**: Process and transform geospatial data, such as GeoTIFF files and vector data, with support for compression and efficient handling.
 - **View Generators**: 
@@ -74,7 +84,7 @@ Giga maps schools' Internet access in real time, creates models for innovative f
   - Centralized configuration via environment variables or `.env` file
   - Easy setup of API keys and paths
 
-### Supported Datasets
+## Supported Datasets
 
 The `gigaspatial` package supports data from the following providers:
 
@@ -84,7 +94,7 @@ The `gigaspatial` package supports data from the following providers:
 
 ---
 
-### View Generators
+## View Generators
 
 The **view generators** in GigaSpatial are designed to enrich the spatial context of school locations and map data into grid or POI locations. This enables users to analyze and visualize geospatial data in meaningful ways.
 

{giga_spatial-0.6.2 → giga_spatial-0.6.4}/README.md

@@ -5,17 +5,27 @@
 
 # GigaSpatial
 
-# ![GigaSpatial logo](https://raw.githubusercontent.com/unicef/giga-spatial/main/docs/assets/logo.png)
-Giga is a UNICEF-ITU initiative to connect every school to the Internet and every young person to information, opportunity and choice. 
-Giga maps schools' Internet access in real time, creates models for innovative financing, and supports governments contracting for connectivity. 
+## About Giga
 
----
+[Giga](https://giga.global/) is a UNICEF-ITU initiative to connect every school to the Internet and every young person to information, opportunity and choice. 
+Giga maps schools' Internet access in real time, creates models for innovative financing, and supports governments contracting for connectivity. 
 
 ## About GigaSpatial
 
 **GigaSpatial** is a Python package developed as part of the Giga Applied Science Team to handle geospatial data efficiently. It provides tools for downloading, processing, and analyzing geospatial data, enabling users to work with datasets such as OpenStreetMap (OSM), Global Human Settlement Layer (GHSL), Microsoft Global Buildings, Google Open Buildings, and more. The package is designed to support Giga's mission by providing robust geospatial capabilities for mapping and analyzing school connectivity.
 
-### Key Features
+## Installation
+
+See the [installation docs](https://unicef.github.io/giga-spatial/getting-started/installation/) for all details. GigaSpatial requires Python 3.10 or above and depends on the following key packages:
+
+- geopandas
+- pandas
+- shapely
+- rasterio
+
+We recommend using a virtual environment for installation. See the [installation docs](https://unicef.github.io/giga-spatial/getting-started/installation/) for more details.
+
+## Key Features
 - **Data Downloading**: Download geospatial data from various sources including GHSL, Microsoft Global Buildings, Google Open Buildings, OpenCellID, and HDX datasets.
 - **Data Processing**: Process and transform geospatial data, such as GeoTIFF files and vector data, with support for compression and efficient handling.
 - **View Generators**: 
@@ -28,7 +38,7 @@ Giga maps schools' Internet access in real time, creates models for innovative f
   - Centralized configuration via environment variables or `.env` file
   - Easy setup of API keys and paths
 
-### Supported Datasets
+## Supported Datasets
 
 The `gigaspatial` package supports data from the following providers:
 
@@ -38,7 +48,7 @@ The `gigaspatial` package supports data from the following providers:
 
 ---
 
-### View Generators
+## View Generators
 
 The **view generators** in GigaSpatial are designed to enrich the spatial context of school locations and map data into grid or POI locations. This enables users to analyze and visualize geospatial data in meaningful ways.
 

giga_spatial-0.6.4/docs/examples/basic.md

@@ -0,0 +1,185 @@
+# Data Handler Examples
+
+This guide provides examples of how to use various data handlers in GigaSpatial to access and process different types of spatial data.
+
+## Population Data (WorldPop)
+
+```python
+from gigaspatial.handlers import WorldPopHandler
+
+# Get population data for a specific country and year
+config = {
+    "country_code": "KEN",
+    "year": 2020,
+}
+
+# Initialize the WorldPop handler
+worldpop = WorldPopDownloader(config = config)
+path_to_data = worldpop.download_dataset()
+```
+
+## Building Footprints
+
+### Google Open Buildings
+
+```python
+from gigaspatial.handlers import GoogleOpenBuildingsHandler
+from gigaspatial.generators import PoiViewGenerator
+from gigaspatial.core.io import LocalDataStore
+import geopandas as gpd
+
+# Initialize data store and handlers
+data_store = LocalDataStore()
+gob_handler = GoogleOpenBuildingsHandler(data_store=data_store)
+
+# Example 1: Load building data for a country
+country_code = "KEN"  # Kenya
+polygons_gdf = gob_handler.load_polygons(country_code, ensure_available=True)
+print(f"Loaded {len(polygons_gdf)} building polygons")
+
+# Example 2: Load building data for specific points
+points = [(36.8219, -1.2921), (36.8172, -1.2867)]  # Nairobi coordinates
+points_gdf = gob_handler.load_polygons(points, ensure_available=True)
+print(f"Loaded {len(points_gdf)} building polygons for points")
+
+# Example 3: Map nearest buildings to points using PoiViewGenerator
+# Initialize POI view generator with points
+poi_generator = PoiViewGenerator(points=points)
+
+# Map nearest Google buildings to POIs
+result_gdf = poi_generator.map_google_buildings(gob_handler)
+print("\nPOIs with nearest building information:")
+print(result_gdf[["poi_id", "nearest_google_building_id", "nearest_google_building_distance"]].head())
+
+# Example 4: Save the enriched POI view
+output_path = poi_generator.save_view("nairobi_buildings", output_format="geojson")
+print(f"\nSaved enriched POI view to: {output_path}")
+```
+
+This example demonstrates:
+1. Loading building data for a country or specific points
+2. Using PoiViewGenerator to map nearest buildings to points of interest
+3. Saving the enriched POI view with building information
+
+The resulting GeoDataFrame includes:
+- Original POI information
+- Nearest building ID
+- Distance to the nearest building
+
+### Microsoft Global Buildings
+
+```python
+from gigaspatial.handlers import MSBuildingsDownloader
+
+# Initialize the handler
+mgb = MSBuildingsDownloader()
+
+points = [(1.25214, 5.5124), (3.45234, 12.51232)]
+
+# Get building footprints
+list_of_paths = mgb.download(
+    points=points
+)
+```
+
+## Satellite Imagery
+
+### Maxar Imagery
+
+```python
+from gigaspatial.handlers import MaxarImageHandler
+
+# Initialize woith default config which reads credentials config from your environment
+maxar = MaxarImageDownloader()
+
+# Download imagery
+maxar.download_images_by_coordinates(
+    data=coordinates,
+    res_meters_pixel=0.6,
+    output_dir="bronze/maxar",
+    bbox_size = 300.0,
+    image_prefix = "maxar_"
+)
+
+```
+
+### Mapbox Imagery
+
+```python
+from gigaspatial.handlers import MapboxImageDownloader
+
+# Initialize with your access token or config will be read from your environment
+mapbox = MapboxImageDownloader(access_token="your_access_token", style_id="mapbox/satellite-v9")
+
+# Get satellite imagery
+mapbox.download_images_by_coordinates(
+    data=coordinates,
+    res_meters_pixel=300.0,
+    output_dir="bronze/mapbox",
+    image_size=(256,256),
+    image_prefix="mapbox_"
+)
+```
+
+## Internet Speed Data (Ookla)
+
+```python
+from gigaspatial.core.io import LocalDataStore
+from gigaspatial.handlers import (
+    OoklaSpeedtestTileConfig, CountryOoklaTiles
+)
+
+# Initialize OoklaSpeedtestTileConfig for a specific quarter and year
+ookla_config = OoklaSpeedtestTileConfig(
+    service_type="fixed", year=2023, quarter=3, data_store=LocalDataStore())
+
+# Download and read the Ookla tile data
+df = ookla_config.read_tile()
+print(df.head())  # Display the first few rows of the dataset
+
+# Generate country-specific Ookla tiles
+country_ookla_tiles = CountryOoklaTiles.from_country("KEN", ookla_config)
+
+# Convert to DataFrame and display
+country_df = country_ookla_tiles.to_dataframe()
+print(country_df.head())
+
+# Convert to GeoDataFrame and display
+country_gdf = country_ookla_tiles.to_geodataframe()
+print(country_gdf.head())
+
+```
+
+## Administrative Boundaries
+
+```python
+from gigaspatial.handlers import AdminBoundaries
+
+# Load level-1 administrative boundaries for Kenya
+admin_boundaries = AdminBoundaries.create(country_code="KEN", admin_level=1)
+
+# Convert to a GeoDataFrame
+gdf = admin_boundaries.to_geodataframe()
+```
+
+
+## OpenStreetMap Data
+
+```python
+from gigaspatial.handlers.osm import OSMAmenityFetcher
+
+# Example 1: Fetching school amenities in Kenya
+fetcher = OSMAmenityFetcher(country_iso2="KE", amenity_types=["school"])
+schools_df = fetcher.get_locations()
+print(schools_df.head())
+
+# Example 2: Fetching hospital and clinic amenities in Tanzania
+fetcher = OSMAmenityFetcher(country_iso2="TZ", amenity_types=["hospital", "clinic"])
+healthcare_df = fetcher.get_locations()
+print(healthcare_df.head())
+
+# Example 3: Fetching restaurant amenities in Ghana since 2020
+fetcher = OSMAmenityFetcher(country_iso2="GH", amenity_types=["restaurant"])
+restaurants_df = fetcher.get_locations(since_year=2020)
+print(restaurants_df.head())
+```

giga_spatial-0.6.4/docs/examples/downloading/ghsl.md

@@ -0,0 +1,61 @@
+# Downloading and Processing GHSL Data
+
+This example demonstrates how to download and process data from the Global Human Settlement Layer (GHSL) using the `GHSLDataHandler` class.
+
+## Prerequisites
+
+Ensure you have installed the `gigaspatial` package and set up the necessary configuration. Follow the [Installation Guide](../getting-started/installation.md) if you haven't already.
+
+## Example Code
+
+```python
+from gigaspatial.handlers import GHSLDataHandler
+
+# Initialize the handler with desired product and parameters
+ghsl_handler = GHSLDataHandler(
+    product="GHS_BUILT_S",  # Built-up surface
+    year=2020,
+    resolution=100,  # 100m resolution
+)
+
+# Download and load data for a specific country
+country_code = "TUR"
+downloaded_files = ghsl_handler.load_data(country_code, ensure_available=True)
+
+# Load the data into a DataFrame
+df = ghsl_handler.load_into_dataframe(country_code, ensure_available=True)
+print(df.head())
+
+# You can also load data for specific points
+points = [(38.404581, 27.4816677), (39.8915702, 32.7809618)]  # Example coordinates
+df_points = ghsl_handler.load_into_dataframe(points, ensure_available=True)
+```
+
+## Explanation
+
+- **GHSLDataHandler**: This class provides a unified interface for downloading and processing GHSL data.
+- **Available Products**:
+  - `GHS_BUILT_S`: Built-up surface
+  - `GHS_BUILT_H_AGBH`: Average building height
+  - `GHS_BUILT_H_ANBH`: Average number of building heights
+  - `GHS_BUILT_V`: Building volume
+  - `GHS_POP`: Population
+  - `GHS_SMOD`: Settlement model
+- **Parameters**:
+  - `product`: The GHSL product to use
+  - `year`: The year of the data (default: 2020)
+  - `resolution`: The resolution in meters (default: 100)
+- **Methods**:
+  - `load_data()`: Downloads and loads the data
+  - `load_into_dataframe()`: Loads the data into a pandas DataFrame
+
+## Next Steps
+
+Once the data is downloaded and processed, you can:
+1. Store the data using the `DataStore` class
+2. Visualize the data using `geopandas` and `matplotlib`
+3. Process the data further using the [Processing Examples](../processing/tif.md)
+
+---
+
+[Back to Examples](../index.md)

{giga_spatial-0.6.2 → giga_spatial-0.6.4}/docs/getting-started/installation.md

@@ -1,6 +1,6 @@
 # Installation Guide
 
-This guide will walk you through the steps to install the `gigaspatial` package on your system. The package is compatible with Python 3.7 and above.
+This guide will walk you through the steps to install the `gigaspatial` package on your system. The package is compatible with Python 3.10 and above.
 
 ## Prerequisites
 
@@ -12,27 +12,34 @@ python --version
 
 If Python is not installed, you can download it from the [official Python website](https://www.python.org/downloads/).
 
-## Installing via pip
+## Installing from PyPI
 
-You can install the `gigaspatial` package directly from the source using `pip`. Follow these steps:
+The easiest way to install `gigaspatial` is directly from PyPI using pip:
 
-1. **Clone the Repository** (if you haven't already):
+```bash
+pip install giga-spatial
+```
+
+This will install the latest stable version of the package along with all its dependencies.
+
+## Installing from Source
+
+If you need to install a specific version or want to contribute to the development, you can install from the source:
+
+1. **Clone the Repository**:
    ```bash
    git clone https://github.com/unicef/giga-spatial.git
    cd giga-spatial
    ```
 
 2. **Install the Package**:
-   Run the following command in your terminal to install the package:
    ```bash
    pip install .
    ```
 
-   This command will install `gigaspatial` along with its dependencies.
-
-## Installing in Development Mode
+### Installing in Development Mode
 
-If you plan to contribute to the package or modify the source code, you can install it in development mode. This allows you to make changes to the code without reinstalling the package. To install in development mode, run:
+If you plan to contribute to the package or modify the source code, you can install it in development mode. This allows you to make changes to the code without reinstalling the package:
 
 ```bash
 pip install -e .
@@ -71,7 +78,7 @@ If you encounter any issues during installation, consider the following:
   ```bash
   python -m venv venv
   source venv/bin/activate  # On Windows, use `venv\Scripts\activate`
-  pip install .
+  pip install giga-spatial  # or pip install . if installing from source
   ```
 
 ---

{giga_spatial-0.6.2 → giga_spatial-0.6.4}/docs/getting-started/quickstart.md

@@ -54,39 +54,57 @@ config.set_path("views", "/path/to/your/views_data")
 
 API keys and tokens should be set through environment variables.
 
-## Downloading Geospatial Data
+## Downloading and Processing Geospatial Data
 
-To download geospatial data, you can use the `GHSLDataDownloader` class from the `handlers` module. Here's an example of downloading data for a specific country:
+The `gigaspatial` package provides several handlers for different types of geospatial data. Here are examples for two commonly used handlers:
+
+### GHSL (Global Human Settlement Layer) Data
+
+The `GHSLDataHandler` provides access to various GHSL products including built-up surface, building height, population, and settlement model data:
 
 ```python
-from gigaspatial.handlers.ghsl import GHSLDataDownloader
+from gigaspatial.handlers import GHSLDataHandler
 
-# Initialize the downloader
-downloader = GHSLDataDownloader({
-    "product": "GHS_BUILT_S",
-    "year": 2020,
-    "resolution": 100,
-    "coord_system": "WGS84"
-})
+# Initialize the handler with desired product and parameters
+ghsl_handler = GHSLDataHandler(
+    product="GHS_BUILT_S",  # Built-up surface
+    year=2020,
+    resolution=100,  # 100m resolution
+)
 
 # Download data for a specific country
-country_code = "TUR" 
-downloader.download_by_country(country_code)
+country_code = "TUR"
+downloaded_files = ghsl_handler.load_data(country_code, ensure_available = True)
+
+# Load the data into a DataFrame
+df = ghsl_handler.load_into_dataframe(country_code, ensure_available = True)
+print(df.head())
+
+# You can also load data for specific points or geometries
+points = [(38.404581,27.4816677), (39.8915702, 32.7809618)]
+df_points = ghsl_handler.load_into_dataframe(points, ensure_available = True)
 ```
 
-## Processing Geospatial Data
+### Google Open Buildings Data
 
-Once the data is downloaded, you can process it using the `TifProcessor` class from the `processing` module. Here's an example of processing GHSL data:
+The `GoogleOpenBuildingsHandler` provides access to Google's Open Buildings dataset, which includes building footprints and points:
 
 ```python
-from gigaspatial.processing.tif_processor import TifProcessor
+from gigaspatial.handlers import GoogleOpenBuildingsHandler
 
-# Initialize the processor
-processor = TifProcessor(dataset_path="/path/to/ghsl/data/ghsl_data.tif", data_store=None, mode="single")
+# Initialize the handler
+gob_handler = GoogleOpenBuildingsHandler()
 
-# Process the GHSL data
-processed_data = processor.to_dataframe()
-print(processed_data.head())
+# Download and load building polygons for a country
+country_code = "TUR"
+polygons_gdf = gob_handler.load_polygons(country_code, ensure_available = True)
+
+# Download and load building points for a country
+points_gdf = gob_handler.load_points(country_code, ensure_available = True)
+
+# You can also load data for specific points or geometries
+points = [(38.404581, 27.4816677), (39.8915702, 32.7809618)]
+polygons_gdf = gob_handler.load_polygons(points, ensure_available = True)
 ```
 
 ## Storing Geospatial Data
@@ -94,7 +112,7 @@ print(processed_data.head())
 You can store the processed data in various formats using the `DataStore` class from the `core.io` module. Here's an example of saving data to a parquet file:
 
 ```python
-from gigaspatial.core.io.local_data_store import LocalDataStore
+from gigaspatial.core.io import LocalDataStore
 
 # Initialize the data store
 data_store = LocalDataStore()
@@ -113,7 +131,6 @@ from gigaspatial.core.io.writers import write_dataset
 write_dataset(data=processed_data, data_store=data_store, path="/path/to/your/output/processed_data.geojson")
 ```
 
-
 ## Visualizing Geospatial Data
 
 To visualize the geospatial data, you can use libraries like `geopandas` and `matplotlib`. Here's an example of plotting the processed data on a map: