topo_map_processor 0.1.5__tar.gz → 0.1.7__tar.gz

{topo_map_processor-0.1.5 → topo_map_processor-0.1.7}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: topo_map_processor
-Version: 0.1.5
+Version: 0.1.7
 Summary: Library for processing topographic maps
 Project-URL: homepage, https://github.com/ramSeraph/topo_map_processor
 Project-URL: repository, https://github.com/ramSeraph/topo_map_processor.git
@@ -31,17 +31,19 @@ License: This is free and unencumbered software released into the public domain.
         For more information, please refer to <https://unlicense.org>
 License-File: LICENSE
 Requires-Python: >=3.12
-Requires-Dist: easyocr>=1.7.2
-Requires-Dist: imgcat>=0.6.0
 Requires-Dist: mercantile>=1.2.1
-Requires-Dist: numpy>=2.3.1
-Requires-Dist: opencv-python-headless>=4.11.0.86
-Requires-Dist: pillow>=11.3.0
 Requires-Dist: pmtiles>=3.4.1
-Requires-Dist: pyproj>=3.7.1
-Requires-Dist: pytesseract>=0.3.13
-Requires-Dist: rasterio>=1.4.3
 Requires-Dist: shapely>=2.1.1
+Provides-Extra: easyocr
+Requires-Dist: easyocr>=1.7.2; extra == 'easyocr'
+Provides-Extra: parse
+Requires-Dist: imgcat>=0.6.0; extra == 'parse'
+Requires-Dist: numpy>=2.3.1; extra == 'parse'
+Requires-Dist: opencv-python-headless>=4.11.0.86; extra == 'parse'
+Requires-Dist: pillow>=11.3.0; extra == 'parse'
+Requires-Dist: pyproj>=3.7.1; extra == 'parse'
+Requires-Dist: pytesseract>=0.3.13; extra == 'parse'
+Requires-Dist: rasterio>=1.4.3; extra == 'parse'
 Description-Content-Type: text/markdown
 
 # Topo Map Processor [![PyPI - Latest Version](https://img.shields.io/pypi/v/topo_map_processor)](https://pypi.org/project/topo_map_processor/) [![GitHub Tag](https://img.shields.io/github/v/tag/ramSeraph/topo_map_processor?filter=v*)](https://github.com/ramSeraph/topo_map_processor/releases/latest)
@@ -51,7 +53,7 @@ A Python utility for processing topographic maps.
 
 ## Description
 
-This project provides a collection of command-line tools and a Python library to process, analyze, and manipulate topographic map data, specifically for creating web-mappable tiles from GeoTIFF files. The tools allow you to create tiles, partition large tile sets, and update existing tile sets efficiently.
+This project provides a collection of command-line tools and a Python library to process, georeference, and export topographic map data in raster form, specifically for creating web-mappable tiles from GeoTIFF files. The tools allow you to create tiles, partition large tile sets, and update existing tile sets efficiently.
 
 ## Installation
 
@@ -106,7 +108,7 @@ The main steps, executed by the `processor.process()` method, are as follows:
 1.  **Image Preparation (`rotate`):** To minimize computation, the processor first creates a smaller, shrunken version of the full-resolution map. On this small image, it locates the main contour of the map's frame (the border area). By calculating the angle of this contour's minimum bounding box, it determines the skew of the map. The full-resolution image is then rotated using this angle to make the frame perfectly horizontal, correcting for any tilt introduced during scanning.
 2.  **Corner Detection (`get_corners`):** Once the image is straightened, the processor uses the same map frame contour to identify the four corner regions. By narrowing the search to these smaller areas, it can apply more precise logic to find the exact pixel coordinates of the map's neatline corners. This is a critical step that often requires custom implementation in a subclass (`get_intersection_point`) to handle the specific visual features of the map's corners (e.g., crosses, specific colors, patterns).
 3.  **Grid Line Removal (`remove_grid_lines`):** If configured, the processor can detect and remove grid lines (graticules) from the map image. This is achieved by implementing the `locate_grid_lines` method. The removal helps create a cleaner final map by inpainting the areas where the lines were.
-4.  **Georeferencing (`georeference`):** Using the detected corner coordinates and the corresponding real-world geographic coordinates (provided via the `index_map`), the tool creates Ground Control Points (GCPs). It then uses `gdal_translate` to create an initial georeferenced GeoTIFF.
+4.  **Georeferencing (`georeference`):** Using the detected corner coordinates and the corresponding real-world geographic coordinates (provided via the `index_box`), the tool creates Ground Control Points (GCPs). It then uses `gdal_translate` to create an initial georeferenced GeoTIFF.
 5.  **Warping and Cutting (`warp`):** The georeferenced image is then warped into the standard `EPSG:3857` projection used by most web maps. During this step, it's also precisely cropped to the map's boundary (neatline) using a cutline derived from the corner points. The output is a clean, map-only image, correctly projected.
 6.  **Exporting (`export`):** The final warped image is converted into a Cloud-Optimized GeoTIFF (COG), which is highly efficient for web-based tiling. A corresponding bounds file (`.geojsonl`) is also created, defining the geographic extent of the map sheet.
 
@@ -116,15 +118,15 @@ The output of this library workflow—a directory of COG `.tif` files and a dire
 
 The `TopoMapProcessor` class provides a framework for processing individual map sheets. To use it, you need to create a subclass and implement the required methods.
 
-Here's an example of how to use the `TopoMapProcessor` class, based on the provided `parse_jica.py` script:
+Here's an example of how to use the `TopoMapProcessor` class.
 
 ```python
-from topo_map_processor import TopoMapProcessor, LineRemovalParams
+from topo_map_processor.processor import TopoMapProcessor, LineRemovalParams
 
 class SampleProcessor(TopoMapProcessor):
 
-    def __init__(self, filepath, extra, index_map):
-        super().__init__(filepath, extra, index_map)
+    def __init__(self, filepath, extra, index_box, index_props):
+        super().__init__(filepath, extra, index_box, index_props)
         # ... (additional initialization)
 
     def get_inter_dir(self):
@@ -155,14 +157,16 @@ def process_files():
     # ... (load image files and special cases)
     for filepath in image_files:
         extra = special_cases.get(filepath.name, {})
-        index_map = get_index_map_jica()
-        processor = SampleProcessor(filepath, extra, index_map)
+        index_box, index_props = get_index_data(filepath.name)  # This should return the corresponding index box and properties for the file
+        processor = SampleProcessor(filepath, extra, index_box, index_props)
         processor.process()
 ```
 
 The library is configurable and most parameters can be overridden at a per sheet level using the `extra` dictionary.
 
-The `index_map` is used to map specific file names to their corresponding metadata, which can be useful for processing specific datasets. In particular the index_map is expected to contain the geojson geometry of the map sheet in the `geometry` key, which is used to create the bounds file and georeference the sheet. The geometry is expected to be in counter clockwise order starting at the top left corner of the map sheet.
+The `index_box` is used to provide the coordinates of the corners of the sheet. This information is used to create the bounds file and georeference the sheet. This is expected to be a list of coordinates in counter clockwise order starting at the top left corner of the map sheet.
+
+The optional `index_properties` dictionary can contain additional metadata about the sheet, such as its name, description, and other properties which end up in the generated bounds files.
 
 The end result is a Cloud optimized GeoTIFF file, a bounds file in GeoJSON format from which the tiles can be generated, using the `tile` command-line tool, and a set of tiles in the specified directory.
 
@@ -203,6 +207,14 @@ Similar to corner detection, the `locate_grid_lines` method can be implemented u
 
 -   **`get_grid_line_corrections`**: This method serves as a refinement step to improve accuracy. While the transformer provides a very close estimate, this function can correct for minor printing or scanning inaccuracies. For each projected grid intersection, it searches a small area in the actual image for the *true* intersection of printed lines, adjusting the point to match the visual data. This ensures that the lines targeted for removal are precisely the ones printed on the map.
 
+### Tile Sources
+
+The `topo_map_processor.tools.tile_sources` module provides classes for reading tiles from different sources.
+
+-   `DiskSource`: Reads tiles from a directory on the local disk.
+-   `PartitionedPMTilesSource`: Reads tiles from a set of partitioned PMTiles files.
+-   `DiskAndPartitionedPMTilesSource`: Reads tiles from both a local directory and a set of partitioned PMTiles files, useful for retiling.
+
 ### Command-Line Tools
 
 This project provides several command-line tools to work with topographic maps. These tools are dynamically loaded, and you can get the most up-to-date usage information by running the tool with the `--help` flag.
@@ -247,7 +259,7 @@ The tool works in two stages. First, it calculates the full set of sheets that n
 In the second stage, it retiles the affected area. It uses the pull list to create a temporary virtual raster (`.vrt`), generates new base tiles for the affected region, and then reconstructs the upper-level tiles by pulling existing, unaffected tiles from the PMTiles source.
 
 ```bash
-retile --retile-list-file <file> --bounds-file <file> [--max-zoom <zoom>] [--min-zoom <zoom>] [--tiffs-dir <dir>] [--tiles-dir <dir>] [--from-pmtiles-prefix <prefix>] [--sheets-to-pull-list-outfile <file>]
+retile --retile-list-file <file> --bounds-file <file> [--max-zoom <zoom>] [--min-zoom <zoom>] [--tiffs-dir <dir>] [--tiles-dir <dir>] [--from-pmtiles-prefix <prefix>] [--sheets-to-pull-list-outfile <file>] [--num-parallel <processes>]
 ```
 
 -   `--retile-list-file`: File containing the list of sheets to retile. (Required)
@@ -258,19 +270,21 @@ retile --retile-list-file <file> --bounds-file <file> [--max-zoom <zoom>] [--min
 -   `--tiles-dir`: Directory where the tiles will be created. (Required if not in pull-list generation mode)
 -   `--from-pmtiles-prefix`: Prefix for the PMTiles source from which to pull existing tiles. (Required if not in pull-list generation mode)
 -   `--sheets-to-pull-list-outfile`: If provided, the script will only calculate the full list of sheets that need to be processed (including adjacent ones) and write it to this file, then exit.
+-   `--num-parallel`: Number of parallel processes to use for tiling (default: number of CPU cores).
 
 #### `tile`
 
 Creates web map tiles from a directory of GeoTIFF files. It combines them into a virtual raster (`.vrt`) for efficient processing.
 
 ```bash
-tile <tiles-dir> <tiffs-dir> <max-zoom> [<min-zoom>]
+tile --tiles-dir <dir> --tiffs-dir <dir> --max-zoom <zoom> [--min-zoom <zoom>] [--num-parallel <processes>]
 ```
 
 -   `<tiles-dir>`: Directory to store the output tiles. (Required)
 -   `<tiffs-dir>`: Directory containing the input GeoTIFF files. (Required)
 -   `<max-zoom>`: Maximum zoom level for tiling. (Required)
 -   `<min-zoom>`: Minimum zoom level for tiling (default: 0).
+-   `--num-parallel`: Number of parallel processes to use for tiling (default: number of CPU cores).
 
 ### Running Tools Directly with `uvx`
 

{topo_map_processor-0.1.5 → topo_map_processor-0.1.7}/README.md

@@ -5,7 +5,7 @@ A Python utility for processing topographic maps.
 
 ## Description
 
-This project provides a collection of command-line tools and a Python library to process, analyze, and manipulate topographic map data, specifically for creating web-mappable tiles from GeoTIFF files. The tools allow you to create tiles, partition large tile sets, and update existing tile sets efficiently.
+This project provides a collection of command-line tools and a Python library to process, georeference, and export topographic map data in raster form, specifically for creating web-mappable tiles from GeoTIFF files. The tools allow you to create tiles, partition large tile sets, and update existing tile sets efficiently.
 
 ## Installation
 
@@ -60,7 +60,7 @@ The main steps, executed by the `processor.process()` method, are as follows:
 1.  **Image Preparation (`rotate`):** To minimize computation, the processor first creates a smaller, shrunken version of the full-resolution map. On this small image, it locates the main contour of the map's frame (the border area). By calculating the angle of this contour's minimum bounding box, it determines the skew of the map. The full-resolution image is then rotated using this angle to make the frame perfectly horizontal, correcting for any tilt introduced during scanning.
 2.  **Corner Detection (`get_corners`):** Once the image is straightened, the processor uses the same map frame contour to identify the four corner regions. By narrowing the search to these smaller areas, it can apply more precise logic to find the exact pixel coordinates of the map's neatline corners. This is a critical step that often requires custom implementation in a subclass (`get_intersection_point`) to handle the specific visual features of the map's corners (e.g., crosses, specific colors, patterns).
 3.  **Grid Line Removal (`remove_grid_lines`):** If configured, the processor can detect and remove grid lines (graticules) from the map image. This is achieved by implementing the `locate_grid_lines` method. The removal helps create a cleaner final map by inpainting the areas where the lines were.
-4.  **Georeferencing (`georeference`):** Using the detected corner coordinates and the corresponding real-world geographic coordinates (provided via the `index_map`), the tool creates Ground Control Points (GCPs). It then uses `gdal_translate` to create an initial georeferenced GeoTIFF.
+4.  **Georeferencing (`georeference`):** Using the detected corner coordinates and the corresponding real-world geographic coordinates (provided via the `index_box`), the tool creates Ground Control Points (GCPs). It then uses `gdal_translate` to create an initial georeferenced GeoTIFF.
 5.  **Warping and Cutting (`warp`):** The georeferenced image is then warped into the standard `EPSG:3857` projection used by most web maps. During this step, it's also precisely cropped to the map's boundary (neatline) using a cutline derived from the corner points. The output is a clean, map-only image, correctly projected.
 6.  **Exporting (`export`):** The final warped image is converted into a Cloud-Optimized GeoTIFF (COG), which is highly efficient for web-based tiling. A corresponding bounds file (`.geojsonl`) is also created, defining the geographic extent of the map sheet.
 
@@ -70,15 +70,15 @@ The output of this library workflow—a directory of COG `.tif` files and a dire
 
 The `TopoMapProcessor` class provides a framework for processing individual map sheets. To use it, you need to create a subclass and implement the required methods.
 
-Here's an example of how to use the `TopoMapProcessor` class, based on the provided `parse_jica.py` script:
+Here's an example of how to use the `TopoMapProcessor` class.
 
 ```python
-from topo_map_processor import TopoMapProcessor, LineRemovalParams
+from topo_map_processor.processor import TopoMapProcessor, LineRemovalParams
 
 class SampleProcessor(TopoMapProcessor):
 
-    def __init__(self, filepath, extra, index_map):
-        super().__init__(filepath, extra, index_map)
+    def __init__(self, filepath, extra, index_box, index_props):
+        super().__init__(filepath, extra, index_box, index_props)
         # ... (additional initialization)
 
     def get_inter_dir(self):
@@ -109,14 +109,16 @@ def process_files():
     # ... (load image files and special cases)
     for filepath in image_files:
         extra = special_cases.get(filepath.name, {})
-        index_map = get_index_map_jica()
-        processor = SampleProcessor(filepath, extra, index_map)
+        index_box, index_props = get_index_data(filepath.name)  # This should return the corresponding index box and properties for the file
+        processor = SampleProcessor(filepath, extra, index_box, index_props)
         processor.process()
 ```
 
 The library is configurable and most parameters can be overridden at a per sheet level using the `extra` dictionary.
 
-The `index_map` is used to map specific file names to their corresponding metadata, which can be useful for processing specific datasets. In particular the index_map is expected to contain the geojson geometry of the map sheet in the `geometry` key, which is used to create the bounds file and georeference the sheet. The geometry is expected to be in counter clockwise order starting at the top left corner of the map sheet.
+The `index_box` is used to provide the coordinates of the corners of the sheet. This information is used to create the bounds file and georeference the sheet. This is expected to be a list of coordinates in counter clockwise order starting at the top left corner of the map sheet.
+
+The optional `index_properties` dictionary can contain additional metadata about the sheet, such as its name, description, and other properties which end up in the generated bounds files.
 
 The end result is a Cloud optimized GeoTIFF file, a bounds file in GeoJSON format from which the tiles can be generated, using the `tile` command-line tool, and a set of tiles in the specified directory.
 
@@ -157,6 +159,14 @@ Similar to corner detection, the `locate_grid_lines` method can be implemented u
 
 -   **`get_grid_line_corrections`**: This method serves as a refinement step to improve accuracy. While the transformer provides a very close estimate, this function can correct for minor printing or scanning inaccuracies. For each projected grid intersection, it searches a small area in the actual image for the *true* intersection of printed lines, adjusting the point to match the visual data. This ensures that the lines targeted for removal are precisely the ones printed on the map.
 
+### Tile Sources
+
+The `topo_map_processor.tools.tile_sources` module provides classes for reading tiles from different sources.
+
+-   `DiskSource`: Reads tiles from a directory on the local disk.
+-   `PartitionedPMTilesSource`: Reads tiles from a set of partitioned PMTiles files.
+-   `DiskAndPartitionedPMTilesSource`: Reads tiles from both a local directory and a set of partitioned PMTiles files, useful for retiling.
+
 ### Command-Line Tools
 
 This project provides several command-line tools to work with topographic maps. These tools are dynamically loaded, and you can get the most up-to-date usage information by running the tool with the `--help` flag.
@@ -201,7 +211,7 @@ The tool works in two stages. First, it calculates the full set of sheets that n
 In the second stage, it retiles the affected area. It uses the pull list to create a temporary virtual raster (`.vrt`), generates new base tiles for the affected region, and then reconstructs the upper-level tiles by pulling existing, unaffected tiles from the PMTiles source.
 
 ```bash
-retile --retile-list-file <file> --bounds-file <file> [--max-zoom <zoom>] [--min-zoom <zoom>] [--tiffs-dir <dir>] [--tiles-dir <dir>] [--from-pmtiles-prefix <prefix>] [--sheets-to-pull-list-outfile <file>]
+retile --retile-list-file <file> --bounds-file <file> [--max-zoom <zoom>] [--min-zoom <zoom>] [--tiffs-dir <dir>] [--tiles-dir <dir>] [--from-pmtiles-prefix <prefix>] [--sheets-to-pull-list-outfile <file>] [--num-parallel <processes>]
 ```
 
 -   `--retile-list-file`: File containing the list of sheets to retile. (Required)
@@ -212,19 +222,21 @@ retile --retile-list-file <file> --bounds-file <file> [--max-zoom <zoom>] [--min
 -   `--tiles-dir`: Directory where the tiles will be created. (Required if not in pull-list generation mode)
 -   `--from-pmtiles-prefix`: Prefix for the PMTiles source from which to pull existing tiles. (Required if not in pull-list generation mode)
 -   `--sheets-to-pull-list-outfile`: If provided, the script will only calculate the full list of sheets that need to be processed (including adjacent ones) and write it to this file, then exit.
+-   `--num-parallel`: Number of parallel processes to use for tiling (default: number of CPU cores).
 
 #### `tile`
 
 Creates web map tiles from a directory of GeoTIFF files. It combines them into a virtual raster (`.vrt`) for efficient processing.
 
 ```bash
-tile <tiles-dir> <tiffs-dir> <max-zoom> [<min-zoom>]
+tile --tiles-dir <dir> --tiffs-dir <dir> --max-zoom <zoom> [--min-zoom <zoom>] [--num-parallel <processes>]
 ```
 
 -   `<tiles-dir>`: Directory to store the output tiles. (Required)
 -   `<tiffs-dir>`: Directory containing the input GeoTIFF files. (Required)
 -   `<max-zoom>`: Maximum zoom level for tiling. (Required)
 -   `<min-zoom>`: Minimum zoom level for tiling (default: 0).
+-   `--num-parallel`: Number of parallel processes to use for tiling (default: number of CPU cores).
 
 ### Running Tools Directly with `uvx`
 

{topo_map_processor-0.1.5 → topo_map_processor-0.1.7}/pyproject.toml

@@ -8,22 +8,28 @@ authors = [
     { name = "Sreeram Kandimalla", email = "kandimalla.sreeram@gmail.com" },
 ]
 dependencies = [
-    "easyocr>=1.7.2",
-    "imgcat>=0.6.0",
     "mercantile>=1.2.1",
-    "numpy>=2.3.1",
-    "opencv-python-headless>=4.11.0.86",
-    "pillow>=11.3.0",
     "pmtiles>=3.4.1",
-    "pyproj>=3.7.1",
-    "pytesseract>=0.3.13",
-    "rasterio>=1.4.3",
     "shapely>=2.1.1",
 ]
 dynamic = [
     "version"
 ]
 
+[project.optional-dependencies]
+easyocr = [
+    "easyocr>=1.7.2",
+]
+parse = [
+    "numpy>=2.3.1",
+    "pillow>=11.3.0",
+    "imgcat>=0.6.0",
+    "opencv-python-headless>=4.11.0.86",
+    "pyproj>=3.7.1",
+    "pytesseract>=0.3.13",
+    "rasterio>=1.4.3",
+]
+
 [project.urls]
 homepage = "https://github.com/ramSeraph/topo_map_processor"
 repository = "https://github.com/ramSeraph/topo_map_processor.git"
@@ -34,6 +40,8 @@ collect-bounds = "topo_map_processor.tools.collect_bounds:cli"
 partition = "topo_map_processor.tools.partition:cli"
 retile = "topo_map_processor.tools.retile:cli"
 tile = "topo_map_processor.tools.tile:cli"
+generate-lists = "topo_map_processor.tools.execute_bash:generate_lists"
+upload-to-release = "topo_map_processor.tools.execute_bash:upload_to_release"
 
 
 [build-system]
@@ -49,6 +57,15 @@ source = "vcs"
 [tool.hatch.build.hooks.vcs]
 version-file = "topo_map_processor/_version.py"
 
-tool.hatch.build.targets.sdist.include = [
-    "topo_map_processor"
+[tool.hatch.build.targets.sdist]
+include = [
+    "topo_map_processor",
+    "topo_map_processor/bash_scripts/**",
 ]
+
+[tool.hatch.build.targets.wheel]
+include = [
+    "topo_map_processor",
+    "topo_map_processor/bash_scripts/**",
+]
+

{topo_map_processor-0.1.5 → topo_map_processor-0.1.7}/topo_map_processor/_version.py

@@ -17,5 +17,5 @@ __version__: str
 __version_tuple__: VERSION_TUPLE
 version_tuple: VERSION_TUPLE
 
-__version__ = version = '0.1.5'
-__version_tuple__ = version_tuple = (0, 1, 5)
+__version__ = version = '0.1.7'
+__version_tuple__ = version_tuple = (0, 1, 7)

topo_map_processor-0.1.7/topo_map_processor/bash_scripts/generate_lists.sh

@@ -0,0 +1,32 @@
+#!/bin/bash
+
+release_base=$1
+ext=$2
+
+if [ -z "$release_base" ] || [ -z "$ext" ]; then
+  echo "Usage: $0 <release_base> <extension>"
+  exit 1
+fi
+
+#owner=$(gh repo view --json owner -q .owner.login)
+#name=$(gh repo view --json name -q .name)
+#repo="$owner/$name"
+
+echo "getting file list"
+echo "name,size,url" > listing_files.csv
+
+# Get all releases, then filter for the base release and any -extra releases.
+releases_to_process=$(gh release list --repo "$repo" --json tagName -q '.[].tagName' | grep -E "^${release_base}(-extra[0-9]+)?$")
+releases_to_process=$(echo "$releases_to_process" | sort -u)
+
+echo "Will process releases: $releases_to_process"
+
+for release in $releases_to_process; do
+  echo "Processing release: $release"
+  gh release view "$release" --json assets -q ".assets[] | select(.name | endswith(\"$ext\")) | \"\\(.name),\\(.size),\\(.url)\"" >> listing_files.csv
+done
+
+gh release upload "$release_base" listing_files.csv --clobber
+
+rm listing_files.csv
+

topo_map_processor-0.1.7/topo_map_processor/bash_scripts/upload_to_release.sh

@@ -0,0 +1,148 @@
+#!/bin/bash
+
+# A script to upload files from a folder to a GitHub release,
+# skipping files that already exist in the release.
+# If the fourth argument is 'yes', it will overwrite existing assets.
+
+set -e
+set -o pipefail
+
+# --- Configuration ---
+# TAG: The git tag of the release to upload to.
+# FOLDER: The local folder containing the files to upload.
+# OVERWRITE: Optional. Set to 'yes' to allow overwriting existing assets.
+
+if [ "$#" -lt 3 ] || [ "$#" -gt 4 ]; then
+    echo "Usage: $0 <tag> <folder> <extension_without_the_leading_dot> [yes_to_overwrite]"
+    exit 1
+fi
+
+TAG="$1"
+FOLDER="$2"
+EXT="$3"
+OVERWRITE_FLAG=${4:-no}
+
+# --- Pre-flight Checks ---
+
+# Check if gh is installed
+if ! command -v gh &> /dev/null; then
+    echo "Error: gh command-line tool is not installed. Please install it to continue."
+    exit 1
+fi
+
+if [ -z "$TAG" ]; then
+    echo "Error: Release tag is not specified."
+    exit 1
+fi
+
+if [ ! -d "$FOLDER" ]; then
+    echo "Error: Folder '$FOLDER' not found."
+    exit 1
+fi
+
+if [ -z "$EXT" ]; then
+    echo "Error: Extension '$EXT' is not specified."
+    exit 1
+fi
+
+ALLOW_OVERWRITE=0
+if [ "$OVERWRITE_FLAG" == "yes" ]; then
+    ALLOW_OVERWRITE=1
+    echo "Overwrite flag is set. Existing assets will be overwritten."
+fi
+
+# --- Main Logic ---
+
+echo "Fetching existing assets for releases matching pattern '${TAG}(-extra[0-9]+)?'..."
+
+# Get all releases, then filter for the base release and any -extra releases, and sort them
+RELEASES_TO_PROCESS=$(gh release list --json tagName -q '.[].tagName' | grep -E "^${TAG}(-extra[0-9]+)?$" | sort -V)
+
+if [ -z "$RELEASES_TO_PROCESS" ]; then
+    echo "Error: No releases found matching pattern '${TAG}(-extra[0-9]+)?'." >&2
+    exit 1
+fi
+
+echo "Found releases: $RELEASES_TO_PROCESS"
+
+# Store assets with their release tags. Format: "asset_name release_tag"
+EXISTING_ASSETS_WITH_RELEASES=""
+for release in $RELEASES_TO_PROCESS; do
+  echo "Fetching assets from release: $release"
+  assets=$(gh release view "$release" --json assets -q '.assets[].name' 2>/dev/null || echo "")
+  if [ -n "$assets" ]; then
+    while IFS= read -r asset_name; do
+      if [ -n "$asset_name" ]; then
+        EXISTING_ASSETS_WITH_RELEASES="${EXISTING_ASSETS_WITH_RELEASES}${asset_name} ${release}"$'
+'
+      fi
+    done <<< "$assets"
+  fi
+done
+
+if [ -z "$EXISTING_ASSETS_WITH_RELEASES" ]; then
+    echo "Could not fetch any assets. This could mean the releases do not have assets, or you lack permissions."
+fi
+
+# Determine available releases and their current asset counts
+AVAILABLE_RELEASES=()
+AVAILABLE_ASSET_COUNTS=()
+for release in $RELEASES_TO_PROCESS; do
+    count=$(gh release view "$release" --json assets -q '.assets | length' 2>/dev/null || echo "0")
+    
+    MAX_ASSETS=998
+    if [ "$release" == "$TAG" ]; then
+        MAX_ASSETS=988
+    fi
+
+    if [ "$count" -lt "$MAX_ASSETS" ]; then
+        AVAILABLE_RELEASES+=("$release")
+        AVAILABLE_ASSET_COUNTS+=("$count")
+    fi
+done
+
+echo "Starting upload process from folder '$FOLDER'..."
+
+# Find all files in the folder that are not yet in any release and loop through them.
+find "${FOLDER}" -type f | grep "^.*\.${EXT}$" | while read -r FILE_PATH; do
+    FILENAME=$(basename "$FILE_PATH")
+
+    RELEASE_FOR_ASSET=$(echo -n "$EXISTING_ASSETS_WITH_RELEASES" | grep "^${FILENAME} " | head -n 1 | awk '{print $2}')
+
+    if [ -n "$RELEASE_FOR_ASSET" ]; then
+        # Asset exists
+        if [ "$ALLOW_OVERWRITE" -eq 1 ]; then
+            echo "  -> Overwriting '$FILENAME' in release '$RELEASE_FOR_ASSET'..."
+            gh release upload "$RELEASE_FOR_ASSET" "$FILE_PATH" --clobber
+        else
+            echo "  -> Skipping '$FILENAME', it already exists in release '$RELEASE_FOR_ASSET'."
+        fi
+        continue
+    fi
+
+    # Find a release to upload to for a new asset
+    if [ ${#AVAILABLE_RELEASES[@]} -eq 0 ]; then
+        echo "Error: All existing releases are full. No space to upload '$FILENAME'." >&2
+        exit 1
+    fi
+
+    UPLOAD_TARGET=${AVAILABLE_RELEASES[0]}
+
+    echo "  -> Uploading '$FILENAME' to '$UPLOAD_TARGET'..."
+    gh release upload "$UPLOAD_TARGET" "$FILE_PATH"
+
+    # Update asset count and remove release from available list if full
+    AVAILABLE_ASSET_COUNTS[0]=$((${AVAILABLE_ASSET_COUNTS[0]} + 1))
+    
+    MAX_ASSETS=998
+    if [ "$UPLOAD_TARGET" == "$TAG" ]; then
+        MAX_ASSETS=988
+    fi
+
+    if [ "${AVAILABLE_ASSET_COUNTS[0]}" -ge "$MAX_ASSETS" ]; then
+        AVAILABLE_RELEASES=("${AVAILABLE_RELEASES[@]:1}")
+        AVAILABLE_ASSET_COUNTS=("${AVAILABLE_ASSET_COUNTS[@]:1}")
+    fi
+done
+
+echo "Upload process complete."

topo_map_processor-0.1.5/topo_map_processor/__init__.py → topo_map_processor-0.1.7/topo_map_processor/processor.py

@@ -14,7 +14,6 @@ import cv2
 import numpy as np
 
 import pytesseract
-import easyocr
 from PIL import Image
 from imgcat import imgcat
 
@@ -70,9 +69,10 @@ class GCPBasedTransformer:
 
 
 class TopoMapProcessor:
-    def __init__(self, filepath, extra, index_map):
+    def __init__(self, filepath, extra, index_box, index_properties={}):
         self.filepath = filepath
-        self.index_map = index_map
+        self.index_box = index_box
+        self.index_properties = index_properties
 
         self.SHOW_IMG = os.getenv('SHOW_IMG', '0') == '1'
         self.INSPECT = os.getenv('INSPECT', '0') == '1'
@@ -141,11 +141,8 @@ class TopoMapProcessor:
         ext = self.filepath.suffix
         return self.filepath.name.replace(ext, '')
 
-    def get_sheet_ibox(self, sheet_id=None):
-        if sheet_id is None:
-            sheet_id = self.get_id()
-
-        return self.index_map[sheet_id]
+    def get_sheet_ibox(self):
+        return self.index_box
 
     def get_data_dir(self):
         return Path('data')
@@ -553,6 +550,7 @@ class TopoMapProcessor:
         global easy_ocr_reader
     
         if easy_ocr_reader is None:
+            import easyocr
             easy_ocr_reader = easyocr.Reader(['en'], gpu=False)
     
         mask = np.zeros(img.shape[:2], dtype=np.uint8)
@@ -1607,12 +1605,15 @@ class TopoMapProcessor:
     def get_cutline_props(self):
         crs_proj = self.get_crs_proj()
 
-        props = {
+        props = {}
+        props.update(self.index_properties)
+
+        props.update({
             'id': self.get_id(),
             'crs': crs_proj,
             'gcps': self.get_gcps(pre_rotated=True),
             'pixel_cutline': self.get_full_pixel_cutline(pre_rotated=True),
-        }
+        })
         return props
 
     def create_cutline(self, ibox, file):

topo_map_processor-0.1.7/topo_map_processor/tools/execute_bash.py

@@ -0,0 +1,54 @@
+import subprocess
+import sys
+import shutil
+import importlib.resources as resources
+
+def execute_bash_script(script_name):
+    """
+    Executes a bash script from the bash_scripts directory, locating it
+    relative to the project structure.
+    """
+    if not shutil.which("bash"):
+        raise FileNotFoundError("bash executable not found in PATH")
+
+    # Use importlib.resources to access the script as package data
+    # The script is located in topo_map_processor/bash_scripts
+    try:
+        script_path = resources.files('topo_map_processor.bash_scripts').joinpath(script_name)
+    except FileNotFoundError:
+        raise FileNotFoundError(f"The script {script_name} does not exist or is not a file. Searched in package data.")
+
+    # Execute the bash script and stream output, merging stderr into stdout
+    process = subprocess.Popen(
+        [shutil.which("bash"), str(script_path)] + sys.argv[1:],
+        stdout=subprocess.PIPE,
+        stderr=subprocess.STDOUT,  # Redirect stderr to stdout
+        text=True,
+        bufsize=1  # Line-buffered
+    )
+
+    # Stream combined stdout and stderr
+    for line in process.stdout:
+        sys.stdout.write(line)
+        sys.stdout.flush()
+
+    process.wait()
+
+    if process.returncode != 0:
+        raise Exception(f"Bash script '{script_name}' failed with exit code {process.returncode}")
+
+def generate_lists():
+    """
+    Executes the generate_lists.sh bash script.
+    """
+    execute_bash_script("generate_lists.sh")
+
+def upload_to_release():
+    """
+    Executes the upload_to_release.sh bash script.
+    """
+    execute_bash_script("upload_to_release.sh")
+
+
+if __name__ == "__main__":
+    generate_lists()

{topo_map_processor-0.1.5 → topo_map_processor-0.1.7}/topo_map_processor/tools/partition.py

@@ -16,7 +16,7 @@ from pprint import pprint
 from pmtiles.tile import zxy_to_tileid, TileType, Compression
 from pmtiles.writer import Writer as PMTilesWriter
 
-from tile_sources import (
+from .tile_sources import (
     DiskSource,
     DiskAndPartitionedPMTilesSource,
     MissingTileError