giga-spatial 0.6.1__tar.gz → 0.6.3__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (93) hide show
  1. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/.env_sample +1 -0
  2. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/CHANGELOG.md +64 -0
  3. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/PKG-INFO +45 -24
  4. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/README.md +44 -23
  5. giga_spatial-0.6.3/docs/assets/datasets.png +0 -0
  6. giga_spatial-0.6.3/docs/changelog.md +21 -0
  7. giga_spatial-0.6.3/docs/examples/basic.md +185 -0
  8. giga_spatial-0.6.3/docs/examples/downloading/ghsl.md +61 -0
  9. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/docs/getting-started/installation.md +17 -10
  10. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/docs/getting-started/quickstart.md +39 -22
  11. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/giga_spatial.egg-info/PKG-INFO +45 -24
  12. giga_spatial-0.6.3/gigaspatial/__init__.py +1 -0
  13. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/config.py +9 -3
  14. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/handlers/__init__.py +2 -2
  15. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/handlers/ghsl.py +10 -10
  16. giga_spatial-0.6.3/gigaspatial/handlers/hdx.py +510 -0
  17. giga_spatial-0.6.3/gigaspatial/handlers/rwi.py +155 -0
  18. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/processing/tif_processor.py +88 -2
  19. giga_spatial-0.6.1/docs/assets/datasets.png +0 -0
  20. giga_spatial-0.6.1/docs/changelog.md +0 -1
  21. giga_spatial-0.6.1/docs/examples/basic.md +0 -249
  22. giga_spatial-0.6.1/docs/examples/downloading/ghsl.md +0 -38
  23. giga_spatial-0.6.1/gigaspatial/__init__.py +0 -1
  24. giga_spatial-0.6.1/gigaspatial/handlers/hdx.py +0 -241
  25. giga_spatial-0.6.1/gigaspatial/handlers/rwi.py +0 -157
  26. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/CODE_OF_CONDUCT.md +0 -0
  27. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/CONTRIBUTING.md +0 -0
  28. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/LICENSE +0 -0
  29. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/MANIFEST.in +0 -0
  30. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/PULL_REQUEST_TEMPLATE.md +0 -0
  31. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/docs/.DS_Store +0 -0
  32. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/docs/api/core.md +0 -0
  33. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/docs/api/generators.md +0 -0
  34. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/docs/api/grid.md +0 -0
  35. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/docs/api/handlers.md +0 -0
  36. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/docs/api/index.md +0 -0
  37. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/docs/api/processing.md +0 -0
  38. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/docs/assets/GIGA_horizontal_notext_white.webp +0 -0
  39. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/docs/assets/logo.png +0 -0
  40. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/docs/contributing.md +0 -0
  41. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/docs/examples/advanced.md +0 -0
  42. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/docs/examples/downloading/osm.md +0 -0
  43. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/docs/examples/index.md +0 -0
  44. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/docs/examples/processing/tif.md +0 -0
  45. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/docs/examples/use-cases.md +0 -0
  46. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/docs/index.md +0 -0
  47. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/docs/license.md +0 -0
  48. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/docs/stylesheets/extra.css +0 -0
  49. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/docs/user-guide/configuration.md +0 -0
  50. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/docs/user-guide/index.md +0 -0
  51. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/giga_spatial.egg-info/SOURCES.txt +0 -0
  52. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/giga_spatial.egg-info/dependency_links.txt +0 -0
  53. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/giga_spatial.egg-info/requires.txt +0 -0
  54. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/giga_spatial.egg-info/top_level.txt +0 -0
  55. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/core/__init__.py +0 -0
  56. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/core/io/__init__.py +0 -0
  57. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/core/io/adls_data_store.py +0 -0
  58. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/core/io/data_api.py +0 -0
  59. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/core/io/data_store.py +0 -0
  60. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/core/io/local_data_store.py +0 -0
  61. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/core/io/readers.py +0 -0
  62. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/core/io/writers.py +0 -0
  63. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/core/schemas/__init__.py +0 -0
  64. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/core/schemas/entity.py +0 -0
  65. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/generators/__init__.py +0 -0
  66. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/generators/poi.py +0 -0
  67. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/generators/zonal/__init__.py +0 -0
  68. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/generators/zonal/base.py +0 -0
  69. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/generators/zonal/geometry.py +0 -0
  70. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/generators/zonal/mercator.py +0 -0
  71. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/grid/__init__.py +0 -0
  72. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/grid/mercator_tiles.py +0 -0
  73. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/handlers/base.py +0 -0
  74. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/handlers/boundaries.py +0 -0
  75. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/handlers/giga.py +0 -0
  76. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/handlers/google_open_buildings.py +0 -0
  77. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/handlers/mapbox_image.py +0 -0
  78. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/handlers/maxar_image.py +0 -0
  79. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/handlers/microsoft_global_buildings.py +0 -0
  80. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/handlers/ookla_speedtest.py +0 -0
  81. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/handlers/opencellid.py +0 -0
  82. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/handlers/osm.py +0 -0
  83. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/handlers/overture.py +0 -0
  84. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/handlers/unicef_georepo.py +0 -0
  85. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/handlers/worldpop.py +0 -0
  86. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/processing/__init__.py +0 -0
  87. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/processing/geo.py +0 -0
  88. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/processing/sat_images.py +0 -0
  89. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/gigaspatial/processing/utils.py +0 -0
  90. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/pyproject.toml +0 -0
  91. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/requirements.txt +0 -0
  92. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/setup.cfg +0 -0
  93. {giga_spatial-0.6.1 → giga_spatial-0.6.3}/setup.py +0 -0
@@ -12,6 +12,7 @@ export OPENCELLID_ACCESS_TOKEN=""
12
12
  export GEOREPO_API_KEY=""
13
13
  export GEOREPO_USER_EMAIL=""
14
14
  export GIGA_SCHOOL_LOCATION_API_KEY=""
15
+ export ROOT_DATA_DIR=""
15
16
  export BRONZE_DIR=""
16
17
  export SILVER_DIR=""
17
18
  export GOLD_DIR=""
@@ -2,6 +2,70 @@
2
2
 
3
3
  All notable changes to this project will be documented in this file.
4
4
 
5
+ ## [v0.6.3] - 2025-06-16
6
+
7
+ ### Added
8
+ - Major refactor of `HDX` module to align with unified `BaseHandler` architecture:
9
+ - `HDXConfig`: fully aligned with `BaseHandlerConfig` structure.
10
+ - Added flexible pattern matching for resource filtering.
11
+ - Improved data unit resolution by country, geometry, and points.
12
+ - Enhanced resource filtering with exact and regex options.
13
+ - `HDXDownloader` fully aligned with `BaseHandlerDownloader`:
14
+ - Simplified sequential download logic.
15
+ - Improved error handling, validation, and logging.
16
+ - `HDXReader` fully aligned with `BaseHandlerReader`:
17
+ - Added `resolve_source_paths` and `load_all_resources` methods.
18
+ - Simplified source handling for single and multiple files.
19
+ - Cleaned up redundant and dataset-specific logic.
20
+ - Introduced `HDXHandler` as unified orchestration layer using factory methods.
21
+
22
+ - Refactor of `RelativeWealthIndex (RWI)` module:
23
+ - Added new `RWIHandler` class aligned with `HDXHandler` and `BaseHandler`.
24
+ - Simplified class names: `RWIDownloader` and `RWIReader`.
25
+ - Enhanced configuration with `latest_only` flag to select newest resources automatically.
26
+ - Simplified resource filtering and country resolution logic.
27
+ - Improved code maintainability, type hints, and error handling.
28
+
29
+
30
+ - **New raster multi-band support in TifProcessor:**
31
+ - Added new `multi` mode for handling multi-band raster datasets.
32
+ - Automatic band name detection from raster metadata.
33
+ - Added strict mode validation (`single`, `rgb`, `rgba`, `multi`).
34
+ - Enhanced error handling for invalid modes and band counts.
35
+
36
+ ### Fixed
37
+ - Fixed GHSL tiles loading behavior for correct coordinate system handling:
38
+ - Moved `TILES_URL` formatting and tile loading to `validate_configuration`.
39
+ - Ensures proper tile loading after CRS validation.
40
+
41
+ ### Documentation
42
+ - Updated and standardized API references across documentation.
43
+ - Standardized handler method names and usage examples.
44
+ - Added building enrichment examples for POI processing.
45
+ - Updated installation instructions.
46
+
47
+ ### Deprecated
48
+ - Deprecated direct imports from individual handler modules.
49
+
50
+ ## [v0.6.2] - 2025-06-11
51
+
52
+ ### Added
53
+ - New `ROOT_DATA_DIR` configuration option to set a base directory for all data tiers
54
+ - Can be configured via environment variable `ROOT_DATA_DIR` or `.env` file
55
+ - Defaults to current directory (`.`) if not specified
56
+ - All tier data paths (bronze, silver, gold, views) are now constructed relative to this root directory
57
+ - Example: Setting `ROOT_DATA_DIR=/data/gigaspatial` will store all data under `/data/gigaspatial/bronze`, `/data/gigaspatial/silver`, etc.
58
+
59
+ ### Fixed
60
+ - Fixed URL formatting in GHSL tiles by using Enum value instead of Enum member
61
+ - Ensures consistent URL formatting with numeric values (4326) instead of Enum names (WGS84)
62
+ - Fixes URL formatting issue across different Python environments
63
+
64
+ - Refactored GHSL downloader to follow DataStore abstraction
65
+ - Directory creation is now handled by DataStore implementation
66
+ - Removed redundant directory creation logic from download_data_unit method
67
+ - Improves separation of concerns and makes the code more maintainable
68
+
5
69
  ## [v0.6.1] - 2025-06-09
6
70
 
7
71
  ### Fixed
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: giga-spatial
3
- Version: 0.6.1
3
+ Version: 0.6.3
4
4
  Summary: A package for spatial data download & processing
5
5
  Home-page: https://github.com/unicef/giga-spatial
6
6
  Author: Utku Can Ozturk
@@ -51,56 +51,77 @@ Dynamic: summary
51
51
 
52
52
  # GigaSpatial
53
53
 
54
- # ![GigaSpatial logo](https://github.com/unicef/giga-spatial/blob/main/docs/assets/logo.png)
55
- Giga is a UNICEF-ITU initiative to connect every school to the Internet and every young person to information, opportunity and choice.
56
- Giga maps schools’ Internet access in real time, creates models for innovative financing, and supports governments contracting for connectivity.
54
+ ## About Giga
57
55
 
58
- ---
56
+ [Giga](https://giga.global/) is a UNICEF-ITU initiative to connect every school to the Internet and every young person to information, opportunity and choice.
57
+ Giga maps schools' Internet access in real time, creates models for innovative financing, and supports governments contracting for connectivity.
59
58
 
60
59
  ## About GigaSpatial
61
60
 
62
- **GigaSpatial** is a Python package developed as part of the Giga Applied Science Team to handle geospatial data efficiently. It provides tools for downloading, processing, and analyzing geospatial data, enabling users to work with datasets such as OpenStreetMap (OSM) and the Global Human Settlement Layer (GHSL). The package is designed to support Giga's mission by providing robust geospatial capabilities for mapping and analyzing school connectivity.
61
+ **GigaSpatial** is a Python package developed as part of the Giga Applied Science Team to handle geospatial data efficiently. It provides tools for downloading, processing, and analyzing geospatial data, enabling users to work with datasets such as OpenStreetMap (OSM), Global Human Settlement Layer (GHSL), Microsoft Global Buildings, Google Open Buildings, and more. The package is designed to support Giga's mission by providing robust geospatial capabilities for mapping and analyzing school connectivity.
63
62
 
64
- ### Key Features
65
- - **Data Downloading**: Download geospatial data from various sources.
66
- - **Data Processing**: Process and transform geospatial data, such as GeoTIFF files and vector data.
67
- - **View Generators**: Enrich the spatial context and map data into grid or POI (Point of Interest) locations.
68
- - **Grid System**: Create and manipulate grid-based geospatial data for analysis and modeling.
69
- - **Data Storage**: Store and retrieve geospatial data in various formats.
70
- - **Configuration Management**: Easily configure paths, API keys, and other settings using environment variables or manual setup.
63
+ ## Installation
71
64
 
72
- ---
65
+ See the [installation docs](https://unicef.github.io/giga-spatial/getting-started/installation/) for all details. GigaSpatial requires Python 3.10 or above and depends on the following key packages:
66
+
67
+ - geopandas
68
+ - pandas
69
+ - shapely
70
+ - rasterio
71
+
72
+ We recommend using a virtual environment for installation. See the [installation docs](https://unicef.github.io/giga-spatial/getting-started/installation/) for more details.
73
+
74
+ ## Key Features
75
+ - **Data Downloading**: Download geospatial data from various sources including GHSL, Microsoft Global Buildings, Google Open Buildings, OpenCellID, and HDX datasets.
76
+ - **Data Processing**: Process and transform geospatial data, such as GeoTIFF files and vector data, with support for compression and efficient handling.
77
+ - **View Generators**:
78
+ - Enrich spatial context with POI (Point of Interest) data
79
+ - Support for raster point sampling and zonal statistics
80
+ - Area-weighted aggregation for polygon-based statistics
81
+ - **Grid System**: Create and manipulate grid-based geospatial data for analysis and modeling.
82
+ - **Data Storage**: Flexible storage options with both local and cloud (ADLS) support.
83
+ - **Configuration Management**:
84
+ - Centralized configuration via environment variables or `.env` file
85
+ - Easy setup of API keys and paths
73
86
 
74
87
  ## Supported Datasets
75
88
 
76
89
  The `gigaspatial` package supports data from the following providers:
77
90
 
78
91
  <div align="center">
79
- <img src="https://github.com/unicef/giga-spatial/blob/main/docs/assets/datasets.png" alt="Dataset Providers" style="width: 70%; height: auto;"/>
92
+ <img src="https://raw.githubusercontent.com/unicef/giga-spatial/main/docs/assets/datasets.png" alt="Dataset Providers" style="width: 75%; height: auto;"/>
80
93
  </div>
81
94
 
82
95
  ---
83
96
 
84
- ### View Generators
97
+ ## View Generators
85
98
 
86
99
  The **view generators** in GigaSpatial are designed to enrich the spatial context of school locations and map data into grid or POI locations. This enables users to analyze and visualize geospatial data in meaningful ways.
87
100
 
88
101
  ### Key Capabilities
89
102
  1. **Spatial Context Enrichment**:
90
- - Automatic attribution of geospatial variables to school locations.
91
- - Contextual layers for environmental, infrastructural, and socioeconomic factors.
92
- - Multi-resolution data availability for different analytical needs.
103
+ - Automatic attribution of geospatial variables to school locations
104
+ - Contextual layers for environmental, infrastructural, and socioeconomic factors
105
+ - Multi-resolution data availability for different analytical needs
106
+ - Support for both point and polygon-based enrichment
93
107
 
94
108
  2. **Mapping to Grid or POI Locations**:
95
- - Map geospatial data to grid cells for scalable analysis.
96
- - Map data to POI locations for detailed, location-specific insights.
109
+ - Map geospatial data to grid cells for scalable analysis
110
+ - Map data to POI locations for detailed, location-specific insights
111
+ - Support for chained enrichment using multiple datasets
112
+ - Built-in support for administrative boundary annotations
97
113
 
98
114
  ---
99
115
 
100
116
  ## Why Use GigaSpatial?
101
- - **Efficient Geospatial Handling**: Streamline the process of downloading, processing, and analyzing geospatial data.
102
- - **Scalable Analysis**: Map data to grid cells or POI locations for both scalable and detailed insights.
103
- - **Open Source**: Contribute to and benefit from a collaborative, transparent, and innovative geospatial toolset.
117
+ - **Efficient Geospatial Handling**: Streamline the process of downloading, processing, and analyzing geospatial data
118
+ - **Scalable Analysis**: Map data to grid cells or POI locations for both scalable and detailed insights
119
+ - **Open Source**: Contribute to and benefit from a collaborative, transparent, and innovative geospatial toolset
120
+ - **Modern Architecture**: Built with maintainability and extensibility in mind, featuring:
121
+ - Base handler orchestration for unified lifecycle management
122
+ - Dedicated reader classes for major datasets
123
+ - Modular source resolution for flexible data access
124
+ - Comprehensive error handling and logging
104
125
 
105
126
  ## Why Open Source?
106
127
 
@@ -5,56 +5,77 @@
5
5
 
6
6
  # GigaSpatial
7
7
 
8
- # ![GigaSpatial logo](https://github.com/unicef/giga-spatial/blob/main/docs/assets/logo.png)
9
- Giga is a UNICEF-ITU initiative to connect every school to the Internet and every young person to information, opportunity and choice.
10
- Giga maps schools’ Internet access in real time, creates models for innovative financing, and supports governments contracting for connectivity.
8
+ ## About Giga
11
9
 
12
- ---
10
+ [Giga](https://giga.global/) is a UNICEF-ITU initiative to connect every school to the Internet and every young person to information, opportunity and choice.
11
+ Giga maps schools' Internet access in real time, creates models for innovative financing, and supports governments contracting for connectivity.
13
12
 
14
13
  ## About GigaSpatial
15
14
 
16
- **GigaSpatial** is a Python package developed as part of the Giga Applied Science Team to handle geospatial data efficiently. It provides tools for downloading, processing, and analyzing geospatial data, enabling users to work with datasets such as OpenStreetMap (OSM) and the Global Human Settlement Layer (GHSL). The package is designed to support Giga's mission by providing robust geospatial capabilities for mapping and analyzing school connectivity.
15
+ **GigaSpatial** is a Python package developed as part of the Giga Applied Science Team to handle geospatial data efficiently. It provides tools for downloading, processing, and analyzing geospatial data, enabling users to work with datasets such as OpenStreetMap (OSM), Global Human Settlement Layer (GHSL), Microsoft Global Buildings, Google Open Buildings, and more. The package is designed to support Giga's mission by providing robust geospatial capabilities for mapping and analyzing school connectivity.
17
16
 
18
- ### Key Features
19
- - **Data Downloading**: Download geospatial data from various sources.
20
- - **Data Processing**: Process and transform geospatial data, such as GeoTIFF files and vector data.
21
- - **View Generators**: Enrich the spatial context and map data into grid or POI (Point of Interest) locations.
22
- - **Grid System**: Create and manipulate grid-based geospatial data for analysis and modeling.
23
- - **Data Storage**: Store and retrieve geospatial data in various formats.
24
- - **Configuration Management**: Easily configure paths, API keys, and other settings using environment variables or manual setup.
17
+ ## Installation
25
18
 
26
- ---
19
+ See the [installation docs](https://unicef.github.io/giga-spatial/getting-started/installation/) for all details. GigaSpatial requires Python 3.10 or above and depends on the following key packages:
20
+
21
+ - geopandas
22
+ - pandas
23
+ - shapely
24
+ - rasterio
25
+
26
+ We recommend using a virtual environment for installation. See the [installation docs](https://unicef.github.io/giga-spatial/getting-started/installation/) for more details.
27
+
28
+ ## Key Features
29
+ - **Data Downloading**: Download geospatial data from various sources including GHSL, Microsoft Global Buildings, Google Open Buildings, OpenCellID, and HDX datasets.
30
+ - **Data Processing**: Process and transform geospatial data, such as GeoTIFF files and vector data, with support for compression and efficient handling.
31
+ - **View Generators**:
32
+ - Enrich spatial context with POI (Point of Interest) data
33
+ - Support for raster point sampling and zonal statistics
34
+ - Area-weighted aggregation for polygon-based statistics
35
+ - **Grid System**: Create and manipulate grid-based geospatial data for analysis and modeling.
36
+ - **Data Storage**: Flexible storage options with both local and cloud (ADLS) support.
37
+ - **Configuration Management**:
38
+ - Centralized configuration via environment variables or `.env` file
39
+ - Easy setup of API keys and paths
27
40
 
28
41
  ## Supported Datasets
29
42
 
30
43
  The `gigaspatial` package supports data from the following providers:
31
44
 
32
45
  <div align="center">
33
- <img src="https://github.com/unicef/giga-spatial/blob/main/docs/assets/datasets.png" alt="Dataset Providers" style="width: 70%; height: auto;"/>
46
+ <img src="https://raw.githubusercontent.com/unicef/giga-spatial/main/docs/assets/datasets.png" alt="Dataset Providers" style="width: 75%; height: auto;"/>
34
47
  </div>
35
48
 
36
49
  ---
37
50
 
38
- ### View Generators
51
+ ## View Generators
39
52
 
40
53
  The **view generators** in GigaSpatial are designed to enrich the spatial context of school locations and map data into grid or POI locations. This enables users to analyze and visualize geospatial data in meaningful ways.
41
54
 
42
55
  ### Key Capabilities
43
56
  1. **Spatial Context Enrichment**:
44
- - Automatic attribution of geospatial variables to school locations.
45
- - Contextual layers for environmental, infrastructural, and socioeconomic factors.
46
- - Multi-resolution data availability for different analytical needs.
57
+ - Automatic attribution of geospatial variables to school locations
58
+ - Contextual layers for environmental, infrastructural, and socioeconomic factors
59
+ - Multi-resolution data availability for different analytical needs
60
+ - Support for both point and polygon-based enrichment
47
61
 
48
62
  2. **Mapping to Grid or POI Locations**:
49
- - Map geospatial data to grid cells for scalable analysis.
50
- - Map data to POI locations for detailed, location-specific insights.
63
+ - Map geospatial data to grid cells for scalable analysis
64
+ - Map data to POI locations for detailed, location-specific insights
65
+ - Support for chained enrichment using multiple datasets
66
+ - Built-in support for administrative boundary annotations
51
67
 
52
68
  ---
53
69
 
54
70
  ## Why Use GigaSpatial?
55
- - **Efficient Geospatial Handling**: Streamline the process of downloading, processing, and analyzing geospatial data.
56
- - **Scalable Analysis**: Map data to grid cells or POI locations for both scalable and detailed insights.
57
- - **Open Source**: Contribute to and benefit from a collaborative, transparent, and innovative geospatial toolset.
71
+ - **Efficient Geospatial Handling**: Streamline the process of downloading, processing, and analyzing geospatial data
72
+ - **Scalable Analysis**: Map data to grid cells or POI locations for both scalable and detailed insights
73
+ - **Open Source**: Contribute to and benefit from a collaborative, transparent, and innovative geospatial toolset
74
+ - **Modern Architecture**: Built with maintainability and extensibility in mind, featuring:
75
+ - Base handler orchestration for unified lifecycle management
76
+ - Dedicated reader classes for major datasets
77
+ - Modular source resolution for flexible data access
78
+ - Comprehensive error handling and logging
58
79
 
59
80
  ## Why Open Source?
60
81
 
@@ -0,0 +1,21 @@
1
+ # Changelog
2
+
3
+ ## [Unreleased]
4
+
5
+ ### Changed
6
+ - Simplified import structure: all handlers now available directly from `gigaspatial.handlers`
7
+ - Updated `LocalDataStore` import path to `gigaspatial.core.io`
8
+ - Standardized handler method names across the codebase
9
+ - Updated documentation and examples to reflect current API
10
+
11
+ ### Added
12
+ - New `PoiViewGenerator` for mapping nearest buildings to points
13
+ - Enhanced building data handling with improved point-to-building mapping
14
+
15
+ ### Fixed
16
+ - Import path inconsistencies
17
+ - Documentation and installation instructions
18
+
19
+ ### Deprecated
20
+ - Direct imports from individual handler modules
21
+ - Old downloader classes in favor of unified handlers
@@ -0,0 +1,185 @@
1
+ # Data Handler Examples
2
+
3
+ This guide provides examples of how to use various data handlers in GigaSpatial to access and process different types of spatial data.
4
+
5
+ ## Population Data (WorldPop)
6
+
7
+ ```python
8
+ from gigaspatial.handlers import WorldPopHandler
9
+
10
+ # Get population data for a specific country and year
11
+ config = {
12
+ "country_code": "KEN",
13
+ "year": 2020,
14
+ }
15
+
16
+ # Initialize the WorldPop handler
17
+ worldpop = WorldPopDownloader(config = config)
18
+ path_to_data = worldpop.download_dataset()
19
+ ```
20
+
21
+ ## Building Footprints
22
+
23
+ ### Google Open Buildings
24
+
25
+ ```python
26
+ from gigaspatial.handlers import GoogleOpenBuildingsHandler
27
+ from gigaspatial.generators import PoiViewGenerator
28
+ from gigaspatial.core.io import LocalDataStore
29
+ import geopandas as gpd
30
+
31
+ # Initialize data store and handlers
32
+ data_store = LocalDataStore()
33
+ gob_handler = GoogleOpenBuildingsHandler(data_store=data_store)
34
+
35
+ # Example 1: Load building data for a country
36
+ country_code = "KEN" # Kenya
37
+ polygons_gdf = gob_handler.load_polygons(country_code, ensure_available=True)
38
+ print(f"Loaded {len(polygons_gdf)} building polygons")
39
+
40
+ # Example 2: Load building data for specific points
41
+ points = [(36.8219, -1.2921), (36.8172, -1.2867)] # Nairobi coordinates
42
+ points_gdf = gob_handler.load_polygons(points, ensure_available=True)
43
+ print(f"Loaded {len(points_gdf)} building polygons for points")
44
+
45
+ # Example 3: Map nearest buildings to points using PoiViewGenerator
46
+ # Initialize POI view generator with points
47
+ poi_generator = PoiViewGenerator(points=points)
48
+
49
+ # Map nearest Google buildings to POIs
50
+ result_gdf = poi_generator.map_google_buildings(gob_handler)
51
+ print("\nPOIs with nearest building information:")
52
+ print(result_gdf[["poi_id", "nearest_google_building_id", "nearest_google_building_distance"]].head())
53
+
54
+ # Example 4: Save the enriched POI view
55
+ output_path = poi_generator.save_view("nairobi_buildings", output_format="geojson")
56
+ print(f"\nSaved enriched POI view to: {output_path}")
57
+ ```
58
+
59
+ This example demonstrates:
60
+ 1. Loading building data for a country or specific points
61
+ 2. Using PoiViewGenerator to map nearest buildings to points of interest
62
+ 3. Saving the enriched POI view with building information
63
+
64
+ The resulting GeoDataFrame includes:
65
+ - Original POI information
66
+ - Nearest building ID
67
+ - Distance to the nearest building
68
+
69
+ ### Microsoft Global Buildings
70
+
71
+ ```python
72
+ from gigaspatial.handlers import MSBuildingsDownloader
73
+
74
+ # Initialize the handler
75
+ mgb = MSBuildingsDownloader()
76
+
77
+ points = [(1.25214, 5.5124), (3.45234, 12.51232)]
78
+
79
+ # Get building footprints
80
+ list_of_paths = mgb.download(
81
+ points=points
82
+ )
83
+ ```
84
+
85
+ ## Satellite Imagery
86
+
87
+ ### Maxar Imagery
88
+
89
+ ```python
90
+ from gigaspatial.handlers import MaxarImageHandler
91
+
92
+ # Initialize woith default config which reads credentials config from your environment
93
+ maxar = MaxarImageDownloader()
94
+
95
+ # Download imagery
96
+ maxar.download_images_by_coordinates(
97
+ data=coordinates,
98
+ res_meters_pixel=0.6,
99
+ output_dir="bronze/maxar",
100
+ bbox_size = 300.0,
101
+ image_prefix = "maxar_"
102
+ )
103
+
104
+ ```
105
+
106
+ ### Mapbox Imagery
107
+
108
+ ```python
109
+ from gigaspatial.handlers import MapboxImageDownloader
110
+
111
+ # Initialize with your access token or config will be read from your environment
112
+ mapbox = MapboxImageDownloader(access_token="your_access_token", style_id="mapbox/satellite-v9")
113
+
114
+ # Get satellite imagery
115
+ mapbox.download_images_by_coordinates(
116
+ data=coordinates,
117
+ res_meters_pixel=300.0,
118
+ output_dir="bronze/mapbox",
119
+ image_size=(256,256),
120
+ image_prefix="mapbox_"
121
+ )
122
+ ```
123
+
124
+ ## Internet Speed Data (Ookla)
125
+
126
+ ```python
127
+ from gigaspatial.core.io import LocalDataStore
128
+ from gigaspatial.handlers import (
129
+ OoklaSpeedtestTileConfig, CountryOoklaTiles
130
+ )
131
+
132
+ # Initialize OoklaSpeedtestTileConfig for a specific quarter and year
133
+ ookla_config = OoklaSpeedtestTileConfig(
134
+ service_type="fixed", year=2023, quarter=3, data_store=LocalDataStore())
135
+
136
+ # Download and read the Ookla tile data
137
+ df = ookla_config.read_tile()
138
+ print(df.head()) # Display the first few rows of the dataset
139
+
140
+ # Generate country-specific Ookla tiles
141
+ country_ookla_tiles = CountryOoklaTiles.from_country("KEN", ookla_config)
142
+
143
+ # Convert to DataFrame and display
144
+ country_df = country_ookla_tiles.to_dataframe()
145
+ print(country_df.head())
146
+
147
+ # Convert to GeoDataFrame and display
148
+ country_gdf = country_ookla_tiles.to_geodataframe()
149
+ print(country_gdf.head())
150
+
151
+ ```
152
+
153
+ ## Administrative Boundaries
154
+
155
+ ```python
156
+ from gigaspatial.handlers import AdminBoundaries
157
+
158
+ # Load level-1 administrative boundaries for Kenya
159
+ admin_boundaries = AdminBoundaries.create(country_code="KEN", admin_level=1)
160
+
161
+ # Convert to a GeoDataFrame
162
+ gdf = admin_boundaries.to_geodataframe()
163
+ ```
164
+
165
+
166
+ ## OpenStreetMap Data
167
+
168
+ ```python
169
+ from gigaspatial.handlers.osm import OSMAmenityFetcher
170
+
171
+ # Example 1: Fetching school amenities in Kenya
172
+ fetcher = OSMAmenityFetcher(country_iso2="KE", amenity_types=["school"])
173
+ schools_df = fetcher.get_locations()
174
+ print(schools_df.head())
175
+
176
+ # Example 2: Fetching hospital and clinic amenities in Tanzania
177
+ fetcher = OSMAmenityFetcher(country_iso2="TZ", amenity_types=["hospital", "clinic"])
178
+ healthcare_df = fetcher.get_locations()
179
+ print(healthcare_df.head())
180
+
181
+ # Example 3: Fetching restaurant amenities in Ghana since 2020
182
+ fetcher = OSMAmenityFetcher(country_iso2="GH", amenity_types=["restaurant"])
183
+ restaurants_df = fetcher.get_locations(since_year=2020)
184
+ print(restaurants_df.head())
185
+ ```
@@ -0,0 +1,61 @@
1
+ # Downloading and Processing GHSL Data
2
+
3
+ This example demonstrates how to download and process data from the Global Human Settlement Layer (GHSL) using the `GHSLDataHandler` class.
4
+
5
+ ## Prerequisites
6
+
7
+ Ensure you have installed the `gigaspatial` package and set up the necessary configuration. Follow the [Installation Guide](../getting-started/installation.md) if you haven't already.
8
+
9
+ ## Example Code
10
+
11
+ ```python
12
+ from gigaspatial.handlers import GHSLDataHandler
13
+
14
+ # Initialize the handler with desired product and parameters
15
+ ghsl_handler = GHSLDataHandler(
16
+ product="GHS_BUILT_S", # Built-up surface
17
+ year=2020,
18
+ resolution=100, # 100m resolution
19
+ )
20
+
21
+ # Download and load data for a specific country
22
+ country_code = "TUR"
23
+ downloaded_files = ghsl_handler.load_data(country_code, ensure_available=True)
24
+
25
+ # Load the data into a DataFrame
26
+ df = ghsl_handler.load_into_dataframe(country_code, ensure_available=True)
27
+ print(df.head())
28
+
29
+ # You can also load data for specific points
30
+ points = [(38.404581, 27.4816677), (39.8915702, 32.7809618)] # Example coordinates
31
+ df_points = ghsl_handler.load_into_dataframe(points, ensure_available=True)
32
+ ```
33
+
34
+ ## Explanation
35
+
36
+ - **GHSLDataHandler**: This class provides a unified interface for downloading and processing GHSL data.
37
+ - **Available Products**:
38
+ - `GHS_BUILT_S`: Built-up surface
39
+ - `GHS_BUILT_H_AGBH`: Average building height
40
+ - `GHS_BUILT_H_ANBH`: Average number of building heights
41
+ - `GHS_BUILT_V`: Building volume
42
+ - `GHS_POP`: Population
43
+ - `GHS_SMOD`: Settlement model
44
+ - **Parameters**:
45
+ - `product`: The GHSL product to use
46
+ - `year`: The year of the data (default: 2020)
47
+ - `resolution`: The resolution in meters (default: 100)
48
+ - **Methods**:
49
+ - `load_data()`: Downloads and loads the data
50
+ - `load_into_dataframe()`: Loads the data into a pandas DataFrame
51
+
52
+ ## Next Steps
53
+
54
+ Once the data is downloaded and processed, you can:
55
+ 1. Store the data using the `DataStore` class
56
+ 2. Visualize the data using `geopandas` and `matplotlib`
57
+ 3. Process the data further using the [Processing Examples](../processing/tif.md)
58
+
59
+ ---
60
+
61
+ [Back to Examples](../index.md)
@@ -1,6 +1,6 @@
1
1
  # Installation Guide
2
2
 
3
- This guide will walk you through the steps to install the `gigaspatial` package on your system. The package is compatible with Python 3.7 and above.
3
+ This guide will walk you through the steps to install the `gigaspatial` package on your system. The package is compatible with Python 3.10 and above.
4
4
 
5
5
  ## Prerequisites
6
6
 
@@ -12,27 +12,34 @@ python --version
12
12
 
13
13
  If Python is not installed, you can download it from the [official Python website](https://www.python.org/downloads/).
14
14
 
15
- ## Installing via pip
15
+ ## Installing from PyPI
16
16
 
17
- You can install the `gigaspatial` package directly from the source using `pip`. Follow these steps:
17
+ The easiest way to install `gigaspatial` is directly from PyPI using pip:
18
18
 
19
- 1. **Clone the Repository** (if you haven't already):
19
+ ```bash
20
+ pip install giga-spatial
21
+ ```
22
+
23
+ This will install the latest stable version of the package along with all its dependencies.
24
+
25
+ ## Installing from Source
26
+
27
+ If you need to install a specific version or want to contribute to the development, you can install from the source:
28
+
29
+ 1. **Clone the Repository**:
20
30
  ```bash
21
31
  git clone https://github.com/unicef/giga-spatial.git
22
32
  cd giga-spatial
23
33
  ```
24
34
 
25
35
  2. **Install the Package**:
26
- Run the following command in your terminal to install the package:
27
36
  ```bash
28
37
  pip install .
29
38
  ```
30
39
 
31
- This command will install `gigaspatial` along with its dependencies.
32
-
33
- ## Installing in Development Mode
40
+ ### Installing in Development Mode
34
41
 
35
- If you plan to contribute to the package or modify the source code, you can install it in development mode. This allows you to make changes to the code without reinstalling the package. To install in development mode, run:
42
+ If you plan to contribute to the package or modify the source code, you can install it in development mode. This allows you to make changes to the code without reinstalling the package:
36
43
 
37
44
  ```bash
38
45
  pip install -e .
@@ -71,7 +78,7 @@ If you encounter any issues during installation, consider the following:
71
78
  ```bash
72
79
  python -m venv venv
73
80
  source venv/bin/activate # On Windows, use `venv\Scripts\activate`
74
- pip install .
81
+ pip install giga-spatial # or pip install . if installing from source
75
82
  ```
76
83
 
77
84
  ---