topo_map_processor 0.1.3__tar.gz → 0.1.4__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (20) hide show
  1. topo_map_processor-0.1.4/PKG-INFO +301 -0
  2. topo_map_processor-0.1.4/README.md +255 -0
  3. {topo_map_processor-0.1.3 → topo_map_processor-0.1.4}/topo_map_processor/_version.py +2 -2
  4. topo_map_processor-0.1.4/topo_map_processor/tools/collect_bounds.py +25 -0
  5. {topo_map_processor-0.1.3 → topo_map_processor-0.1.4}/topo_map_processor/tools/partition.py +10 -10
  6. {topo_map_processor-0.1.3 → topo_map_processor-0.1.4}/topo_map_processor/tools/retile.py +16 -8
  7. {topo_map_processor-0.1.3 → topo_map_processor-0.1.4}/topo_map_processor/tools/update_bounds.py +4 -3
  8. topo_map_processor-0.1.3/PKG-INFO +0 -187
  9. topo_map_processor-0.1.3/README.md +0 -141
  10. topo_map_processor-0.1.3/topo_map_processor/tools/collect_bounds.py +0 -23
  11. {topo_map_processor-0.1.3 → topo_map_processor-0.1.4}/.github/workflows/publish.yml +0 -0
  12. {topo_map_processor-0.1.3 → topo_map_processor-0.1.4}/.gitignore +0 -0
  13. {topo_map_processor-0.1.3 → topo_map_processor-0.1.4}/.python-version +0 -0
  14. {topo_map_processor-0.1.3 → topo_map_processor-0.1.4}/LICENSE +0 -0
  15. {topo_map_processor-0.1.3 → topo_map_processor-0.1.4}/pyproject.toml +0 -0
  16. {topo_map_processor-0.1.3 → topo_map_processor-0.1.4}/topo_map_processor/__init__.py +0 -0
  17. {topo_map_processor-0.1.3 → topo_map_processor-0.1.4}/topo_map_processor/tools/__init__.py +0 -0
  18. {topo_map_processor-0.1.3 → topo_map_processor-0.1.4}/topo_map_processor/tools/tile.py +0 -0
  19. {topo_map_processor-0.1.3 → topo_map_processor-0.1.4}/topo_map_processor/tools/tile_sources.py +0 -0
  20. {topo_map_processor-0.1.3 → topo_map_processor-0.1.4}/uv.lock +0 -0
@@ -0,0 +1,301 @@
1
+ Metadata-Version: 2.4
2
+ Name: topo_map_processor
3
+ Version: 0.1.4
4
+ Summary: Library for processing topographic maps
5
+ Project-URL: homepage, https://github.com/ramSeraph/topo_map_processor
6
+ Project-URL: repository, https://github.com/ramSeraph/topo_map_processor.git
7
+ Author-email: Sreeram Kandimalla <kandimalla.sreeram@gmail.com>
8
+ License: This is free and unencumbered software released into the public domain.
9
+
10
+ Anyone is free to copy, modify, publish, use, compile, sell, or
11
+ distribute this software, either in source code form or as a compiled
12
+ binary, for any purpose, commercial or non-commercial, and by any
13
+ means.
14
+
15
+ In jurisdictions that recognize copyright laws, the author or authors
16
+ of this software dedicate any and all copyright interest in the
17
+ software to the public domain. We make this dedication for the benefit
18
+ of the public at large and to the detriment of our heirs and
19
+ successors. We intend this dedication to be an overt act of
20
+ relinquishment in perpetuity of all present and future rights to this
21
+ software under copyright law.
22
+
23
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
24
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
25
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
26
+ IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
27
+ OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
28
+ ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
29
+ OTHER DEALINGS IN THE SOFTWARE.
30
+
31
+ For more information, please refer to <https://unlicense.org>
32
+ License-File: LICENSE
33
+ Requires-Python: >=3.12
34
+ Requires-Dist: easyocr>=1.7.2
35
+ Requires-Dist: imgcat>=0.6.0
36
+ Requires-Dist: mercantile>=1.2.1
37
+ Requires-Dist: numpy>=2.3.1
38
+ Requires-Dist: opencv-python-headless>=4.11.0.86
39
+ Requires-Dist: pillow>=11.3.0
40
+ Requires-Dist: pmtiles>=3.4.1
41
+ Requires-Dist: pyproj>=3.7.1
42
+ Requires-Dist: pytesseract>=0.3.13
43
+ Requires-Dist: rasterio>=1.4.3
44
+ Requires-Dist: shapely>=2.1.1
45
+ Description-Content-Type: text/markdown
46
+
47
+ # Topo Map Processor [![PyPI - Latest Version](https://img.shields.io/pypi/v/topo_map_processor)](https://pypi.org/project/topo_map_processor/) [![GitHub Tag](https://img.shields.io/github/v/tag/ramSeraph/topo_map_processor?filter=v*)](https://github.com/ramSeraph/topo_map_processor/releases/latest)
48
+
49
+
50
+ A Python utility for processing topographic maps.
51
+
52
+ ## Description
53
+
54
+ This project provides a collection of command-line tools and a Python library to process, analyze, and manipulate topographic map data, specifically for creating web-mappable tiles from GeoTIFF files. The tools allow you to create tiles, partition large tile sets, and update existing tile sets efficiently.
55
+
56
+ ## Installation
57
+
58
+ It is recommended to use a virtual environment. [uv](https://github.com/astral-sh/uv) is a fast, modern Python package installer and is the recommended way to manage this project.
59
+
60
+ ### For Library or Development
61
+
62
+ If you want to use `topo_map_processor` as a library or contribute to its development, install it from PyPI into a virtual environment.
63
+
64
+ **Using `uv` (Recommended):**
65
+ ```bash
66
+ # Install uv if you don't have it
67
+ pip install uv
68
+
69
+ # Create and activate a virtual environment
70
+ uv venv
71
+ source .venv/bin/activate
72
+
73
+ # Install the package from PyPI
74
+ uv pip install topo_map_processor
75
+ ```
76
+
77
+ **Using `pip`:**
78
+ ```bash
79
+ # Create and activate a virtual environment
80
+ python -m venv .venv
81
+ source .venv/bin/activate
82
+
83
+ # Install the package from PyPI
84
+ pip install topo_map_processor
85
+ ```
86
+
87
+ To contribute, clone the repository and install in editable mode:
88
+ ```bash
89
+ git clone https://github.com/ramSeraph/topo_map_processor.git
90
+ cd topo_map_processor
91
+ uv venv
92
+ source .venv/bin/activate
93
+ uv pip install -e .
94
+ ```
95
+
96
+ ## Usage
97
+
98
+ This project can be used as a library but also provides a set of command-line tools to help with managing the datasets, tiling and creating `pmtiles` files.
99
+
100
+ ### Library Workflow
101
+
102
+ The core of this project is the `TopoMapProcessor` class, which provides a structured workflow for converting a raw scanned topographic map into a Cloud-Optimized GeoTIFF (COG) ready for tiling. The process is designed to be adaptable to different map series by subclassing and implementing key methods.
103
+
104
+ The main steps, executed by the `processor.process()` method, are as follows:
105
+
106
+ 1. **Image Preparation (`rotate`):** To minimize computation, the processor first creates a smaller, shrunken version of the full-resolution map. On this small image, it locates the main contour of the map's frame (the border area). By calculating the angle of this contour's minimum bounding box, it determines the skew of the map. The full-resolution image is then rotated using this angle to make the frame perfectly horizontal, correcting for any tilt introduced during scanning.
107
+ 2. **Corner Detection (`get_corners`):** Once the image is straightened, the processor uses the same map frame contour to identify the four corner regions. By narrowing the search to these smaller areas, it can apply more precise logic to find the exact pixel coordinates of the map's neatline corners. This is a critical step that often requires custom implementation in a subclass (`get_intersection_point`) to handle the specific visual features of the map's corners (e.g., crosses, specific colors, patterns).
108
+ 3. **Grid Line Removal (`remove_grid_lines`):** If configured, the processor can detect and remove grid lines (graticules) from the map image. This is achieved by implementing the `locate_grid_lines` method. The removal helps create a cleaner final map by inpainting the areas where the lines were.
109
+ 4. **Georeferencing (`georeference`):** Using the detected corner coordinates and the corresponding real-world geographic coordinates (provided via the `index_map`), the tool creates Ground Control Points (GCPs). It then uses `gdal_translate` to create an initial georeferenced GeoTIFF.
110
+ 5. **Warping and Cutting (`warp`):** The georeferenced image is then warped into the standard `EPSG:3857` projection used by most web maps. During this step, it's also precisely cropped to the map's boundary (neatline) using a cutline derived from the corner points. The output is a clean, map-only image, correctly projected.
111
+ 6. **Exporting (`export`):** The final warped image is converted into a Cloud-Optimized GeoTIFF (COG), which is highly efficient for web-based tiling. A corresponding bounds file (`.geojsonl`) is also created, defining the geographic extent of the map sheet.
112
+
113
+ The output of this library workflow—a directory of COG `.tif` files and a directory of `.geojsonl` bounds files—serves as the direct input for the command-line tools like `tile`, `collect-bounds`, and `partition`, which handle the final steps of creating a complete, partitioned web map tile set.
114
+
115
+ ### Library Usage
116
+
117
+ The `TopoMapProcessor` class provides a framework for processing individual map sheets. To use it, you need to create a subclass and implement the required methods.
118
+
119
+ Here's an example of how to use the `TopoMapProcessor` class, based on the provided `parse_jica.py` script:
120
+
121
+ ```python
122
+ from topo_map_processor import TopoMapProcessor, LineRemovalParams
123
+
124
+ class SampleProcessor(TopoMapProcessor):
125
+
126
+ def __init__(self, filepath, extra, index_map):
127
+ super().__init__(filepath, extra, index_map)
128
+ # ... (additional initialization)
129
+
130
+ def get_inter_dir(self):
131
+ return Path('data/inter')
132
+
133
+ def get_gtiff_dir(self):
134
+ return Path('export/gtiffs')
135
+
136
+ def get_bounds_dir(self):
137
+ return Path('export/bounds')
138
+
139
+ def get_crs_proj(self):
140
+ return '+proj=tmerc +lat_0=0 +lon_0=84 +k=0.9999 +x_0=500000 +y_0=0 +units=m +ellps=evrst30 +towgs84=293.17,726.18,245.36,0,0,0,0 +no_defs'
141
+
142
+ def get_scale(self):
143
+ return 25000
144
+
145
+ def get_intersection_point(self, img, direction, anchor_angle):
146
+ # ... (implementation for finding intersection points which are the corners of the mapframe)
147
+ # you are expected to pick from the various locate intersection points implementations in the library or write your own
148
+
149
+ def locate_grid_lines(self):
150
+ # ... (implementation for locating grid lines so as to remove them)
151
+ # you are expected to pick from the various locate grid lines implementations in the library or write your own
152
+
153
+ # Example of how to run the processor
154
+ def process_files():
155
+ # ... (load image files and special cases)
156
+ for filepath in image_files:
157
+ extra = special_cases.get(filepath.name, {})
158
+ index_map = get_index_map_jica()
159
+ processor = SampleProcessor(filepath, extra, index_map)
160
+ processor.process()
161
+ ```
162
+
163
+ The library is configurable and most parameters can be overridden at a per sheet level using the `extra` dictionary.
164
+
165
+ The `index_map` is used to map specific file names to their corresponding metadata, which can be useful for processing specific datasets. In particular the index_map is expected to contain the geojson geometry of the map sheet in the `geometry` key, which is used to create the bounds file and georeference the sheet. The geometry is expected to be in counter clockwise order starting at the top left corner of the map sheet.
166
+
167
+ The end result is a Cloud optimized GeoTIFF file, a bounds file in GeoJSON format from which the tiles can be generated, using the `tile` command-line tool, and a set of tiles in the specified directory.
168
+
169
+ I would recommend looking at the [nepal_survey_maps](https://github.com/ramSeraph/nepal_survey_maps) repository for a sample implementation
170
+
171
+ ### Corner Detection Strategies
172
+
173
+ The `get_intersection_point` method is the most critical part to customize for a new map series. The library provides several helper methods that can be used to implement this logic, each suited for different types of corner features:
174
+
175
+ - **`get_nearest_intersection_point`**: This method is useful when the map corners are defined by the intersection of two perpendicular lines (like a simple crosshair). It works by:
176
+ 1. Detecting all horizontal and vertical lines in the corner region.
177
+ 2. Finding all intersection points between these lines.
178
+ 3. Identifying the point closest to the expected corner position, while also matching an expected angle and distance from an anchor point.
179
+
180
+ - **`get_4way_intersection_point`**: This is a more robust version for crosshair-style corners. It specifically looks for intersections where four lines meet, making it less prone to errors from stray lines in the map collar.
181
+
182
+ - **`get_biggest_contour_corner`**: This strategy is designed for corners that are marked by a solid shape or a filled area (e.g., a colored square or circle). It works by:
183
+ 1. Isolating the specific color of the corner feature.
184
+ 2. Finding the largest contour (shape) of that color in the corner region.
185
+ 3. Returning the outermost point of that contour as the corner.
186
+
187
+ - **`get_nearest_intersection_point_from_biggest_corner_contour`**: This is a hybrid approach. It first uses the "biggest contour" method to locate a general corner area (e.g., a colored box) and then searches for a precise line intersection *within* that area. This is effective for maps that have both a colored corner block and a fine crosshair inside it.
188
+
189
+ By combining these methods in your subclass's implementation of `get_intersection_point`, you can build a reliable corner detection system for your specific map type.
190
+
191
+ ### Grid Line Detection Strategies
192
+
193
+ Similar to corner detection, the `locate_grid_lines` method can be implemented using a few key helper methods provided by the library. The primary goal is to identify the precise pixel paths of the map's grid lines (graticules) so they can be removed.
194
+
195
+ - **`locate_grid_lines_using_transformer`**: This is the main strategy for identifying grid lines. It leverages the georeferencing information (the `GCPTransformer`) established from the map corners. The process is as follows:
196
+ 1. It defines a regular grid in the map's real-world coordinate system (e.g., a line every 1000 meters).
197
+ 2. It uses the transformer to project this theoretical grid back onto the pixel space of the image.
198
+ 3. This results in a highly accurate set of lines that represent where the grid lines should be, even accounting for distortions in the map.
199
+
200
+ - **`get_grid_line_corrections`**: This method serves as a refinement step to improve accuracy. While the transformer provides a very close estimate, this function can correct for minor printing or scanning inaccuracies. For each projected grid intersection, it searches a small area in the actual image for the *true* intersection of printed lines, adjusting the point to match the visual data. This ensures that the lines targeted for removal are precisely the ones printed on the map.
201
+
202
+ ### Command-Line Tools
203
+
204
+ This project provides several command-line tools to work with topographic maps. These tools are dynamically loaded, and you can get the most up-to-date usage information by running the tool with the `--help` flag.
205
+
206
+ #### `collect-bounds`
207
+
208
+ Collects individual GeoJSONL bound files into a single GeoJSON FeatureCollection.
209
+
210
+ ```bash
211
+ collect-bounds --bounds-dir <directory> --output-file <file>
212
+ ```
213
+
214
+ - `--bounds-dir`: Directory containing GeoJSONL files. (Required)
215
+ - `--output-file`: Output GeoJSON file path. (Required)
216
+
217
+ #### `partition`
218
+
219
+ Partitions a large set of tiles into smaller PMTiles files. This is useful for managing large datasets, for example, to stay within file size limits for services like GitHub Releases.
220
+
221
+ ```bash
222
+ partition --to-pmtiles-prefix <prefix> --from-tiles-dir <dir> --name <name> --description <desc> --max-zoom <zoom> [--min-zoom <zoom>] [--from-pmtiles-prefix <prefix>] [--only-disk] (--attribution <text> | --attribution-file <file>)
223
+ ```
224
+
225
+ - `--to-pmtiles-prefix`: Prefix for the output PMTiles files. (Required)
226
+ - `--from-tiles-dir`: Directory containing the input tiles. (Required)
227
+ - `--name`: Name of the mosaic. (Required)
228
+ - `--description`: Description of the mosaic. (Required)
229
+ - `--max-zoom`: Maximum zoom level for the mosaic. (Required)
230
+ - `--min-zoom`: Minimum zoom level for the mosaic (default: 0).
231
+ - `--from-pmtiles-prefix`: Prefix for input PMTiles files (required if not using `--only-disk`).
232
+ - `--only-disk`: Only read tiles from the local disk.
233
+ - `--attribution`: Attribution text for the mosaic. (Required, unless `--attribution-file` is used)
234
+ - `--attribution-file`: File containing attribution text. (Required, unless `--attribution` is used)
235
+
236
+ #### `retile`
237
+
238
+ Retiles a specific list of sheets (GeoTIFFs) and updates the corresponding tiles in an existing tile set. It can determine which other sheets are affected and need to be pulled in to correctly retile the area.
239
+
240
+ ```bash
241
+ retile --retile-list-file <file> --bounds-file <file> --max-zoom <zoom> [--min-zoom <zoom>] [--tiffs-dir <dir>] [--tiles-dir <dir>] [--from-pmtiles-prefix <prefix>] [--sheets-to-pull-list-outfile <file>]
242
+ ```
243
+
244
+ - `--retile-list-file`: File containing the list of sheets to retile. (Required)
245
+ - `--bounds-file`: GeoJSON file containing the list of available sheets and their geographic bounds. (Required)
246
+ - `--max-zoom`: Maximum zoom level to create tiles for. (Required)
247
+ - `--min-zoom`: Minimum zoom level to create tiles for (default: 0).
248
+ - `--tiffs-dir`: Directory where the GeoTIFFs are located.
249
+ - `--tiles-dir`: Directory where the tiles will be created.
250
+ - `--from-pmtiles-prefix`: Prefix for the PMTiles source from which to pull existing tiles.
251
+ - `--sheets-to-pull-list-outfile`: If provided, the script will only calculate the full list of sheets that need to be processed (including adjacent ones) and write it to this file, then exit.
252
+
253
+ #### `tile`
254
+
255
+ Creates web map tiles from a directory of GeoTIFF files. It combines them into a virtual raster (`.vrt`) for efficient processing.
256
+
257
+ ```bash
258
+ tile <tiles-dir> <tiffs-dir> <max-zoom> [<min-zoom>]
259
+ ```
260
+
261
+ - `<tiles-dir>`: Directory to store the output tiles. (Required)
262
+ - `<tiffs-dir>`: Directory containing the input GeoTIFF files. (Required)
263
+ - `<max-zoom>`: Maximum zoom level for tiling. (Required)
264
+ - `<min-zoom>`: Minimum zoom level for tiling (default: 0).
265
+
266
+ #### `update-bounds`
267
+
268
+ Updates a main GeoJSON bounds file with new or modified bounds from individual `.geojsonl` files.
269
+
270
+ ```bash
271
+ update-bounds --bounds-file <file> --bounds-dir <dir> --retile-list-file <file>
272
+ ```
273
+
274
+ - `--bounds-file`: The main GeoJSON bounds file to update. (Required)
275
+ - `--bounds-dir`: Directory containing the new or updated individual `.geojsonl` files. (Required)
276
+ - `--retile-list-file`: A file containing the list of sheet names that have been updated (corresponding to the files in `--bounds-dir`). (Required)
277
+
278
+ ### Running Tools Directly with `uvx`
279
+
280
+ You can run the command-line tools directly from PyPI without a persistent installation using `uvx`. This is convenient for one-off tasks.
281
+
282
+ ```bash
283
+
284
+ # Example: Run the 'collect-bounds' tool
285
+ uvx --from topo_map_processor collect-bounds --bounds-dir /path/to/bounds --output-file bounds.geojson
286
+
287
+ # tile and retile have a dependency gdal python library,
288
+ # and some special handling is required so as to install a version of gdal python bindings which matches the version of gdal installed in your system
289
+ # Also numpy and pillow dependencies are required so that the gdal python bindings have some of the resampling features enabled
290
+ # Example: Get help for the 'tile' command
291
+ GDAL_VERSION=$(gdalinfo --version | cut -d"," -f1 | cut -d" " -f2)
292
+ uvx --with numpy --with pillow --with gdal==$GDAL_VERSION --from topo_map_processor tile --help
293
+ ```
294
+
295
+ ## Contributing
296
+
297
+ Contributions are welcome! Please feel free to submit a pull request.
298
+
299
+ ## License
300
+
301
+ This project is under the UNLICENSE
@@ -0,0 +1,255 @@
1
+ # Topo Map Processor [![PyPI - Latest Version](https://img.shields.io/pypi/v/topo_map_processor)](https://pypi.org/project/topo_map_processor/) [![GitHub Tag](https://img.shields.io/github/v/tag/ramSeraph/topo_map_processor?filter=v*)](https://github.com/ramSeraph/topo_map_processor/releases/latest)
2
+
3
+
4
+ A Python utility for processing topographic maps.
5
+
6
+ ## Description
7
+
8
+ This project provides a collection of command-line tools and a Python library to process, analyze, and manipulate topographic map data, specifically for creating web-mappable tiles from GeoTIFF files. The tools allow you to create tiles, partition large tile sets, and update existing tile sets efficiently.
9
+
10
+ ## Installation
11
+
12
+ It is recommended to use a virtual environment. [uv](https://github.com/astral-sh/uv) is a fast, modern Python package installer and is the recommended way to manage this project.
13
+
14
+ ### For Library or Development
15
+
16
+ If you want to use `topo_map_processor` as a library or contribute to its development, install it from PyPI into a virtual environment.
17
+
18
+ **Using `uv` (Recommended):**
19
+ ```bash
20
+ # Install uv if you don't have it
21
+ pip install uv
22
+
23
+ # Create and activate a virtual environment
24
+ uv venv
25
+ source .venv/bin/activate
26
+
27
+ # Install the package from PyPI
28
+ uv pip install topo_map_processor
29
+ ```
30
+
31
+ **Using `pip`:**
32
+ ```bash
33
+ # Create and activate a virtual environment
34
+ python -m venv .venv
35
+ source .venv/bin/activate
36
+
37
+ # Install the package from PyPI
38
+ pip install topo_map_processor
39
+ ```
40
+
41
+ To contribute, clone the repository and install in editable mode:
42
+ ```bash
43
+ git clone https://github.com/ramSeraph/topo_map_processor.git
44
+ cd topo_map_processor
45
+ uv venv
46
+ source .venv/bin/activate
47
+ uv pip install -e .
48
+ ```
49
+
50
+ ## Usage
51
+
52
+ This project can be used as a library but also provides a set of command-line tools to help with managing the datasets, tiling and creating `pmtiles` files.
53
+
54
+ ### Library Workflow
55
+
56
+ The core of this project is the `TopoMapProcessor` class, which provides a structured workflow for converting a raw scanned topographic map into a Cloud-Optimized GeoTIFF (COG) ready for tiling. The process is designed to be adaptable to different map series by subclassing and implementing key methods.
57
+
58
+ The main steps, executed by the `processor.process()` method, are as follows:
59
+
60
+ 1. **Image Preparation (`rotate`):** To minimize computation, the processor first creates a smaller, shrunken version of the full-resolution map. On this small image, it locates the main contour of the map's frame (the border area). By calculating the angle of this contour's minimum bounding box, it determines the skew of the map. The full-resolution image is then rotated using this angle to make the frame perfectly horizontal, correcting for any tilt introduced during scanning.
61
+ 2. **Corner Detection (`get_corners`):** Once the image is straightened, the processor uses the same map frame contour to identify the four corner regions. By narrowing the search to these smaller areas, it can apply more precise logic to find the exact pixel coordinates of the map's neatline corners. This is a critical step that often requires custom implementation in a subclass (`get_intersection_point`) to handle the specific visual features of the map's corners (e.g., crosses, specific colors, patterns).
62
+ 3. **Grid Line Removal (`remove_grid_lines`):** If configured, the processor can detect and remove grid lines (graticules) from the map image. This is achieved by implementing the `locate_grid_lines` method. The removal helps create a cleaner final map by inpainting the areas where the lines were.
63
+ 4. **Georeferencing (`georeference`):** Using the detected corner coordinates and the corresponding real-world geographic coordinates (provided via the `index_map`), the tool creates Ground Control Points (GCPs). It then uses `gdal_translate` to create an initial georeferenced GeoTIFF.
64
+ 5. **Warping and Cutting (`warp`):** The georeferenced image is then warped into the standard `EPSG:3857` projection used by most web maps. During this step, it's also precisely cropped to the map's boundary (neatline) using a cutline derived from the corner points. The output is a clean, map-only image, correctly projected.
65
+ 6. **Exporting (`export`):** The final warped image is converted into a Cloud-Optimized GeoTIFF (COG), which is highly efficient for web-based tiling. A corresponding bounds file (`.geojsonl`) is also created, defining the geographic extent of the map sheet.
66
+
67
+ The output of this library workflow—a directory of COG `.tif` files and a directory of `.geojsonl` bounds files—serves as the direct input for the command-line tools like `tile`, `collect-bounds`, and `partition`, which handle the final steps of creating a complete, partitioned web map tile set.
68
+
69
+ ### Library Usage
70
+
71
+ The `TopoMapProcessor` class provides a framework for processing individual map sheets. To use it, you need to create a subclass and implement the required methods.
72
+
73
+ Here's an example of how to use the `TopoMapProcessor` class, based on the provided `parse_jica.py` script:
74
+
75
+ ```python
76
+ from topo_map_processor import TopoMapProcessor, LineRemovalParams
77
+
78
+ class SampleProcessor(TopoMapProcessor):
79
+
80
+ def __init__(self, filepath, extra, index_map):
81
+ super().__init__(filepath, extra, index_map)
82
+ # ... (additional initialization)
83
+
84
+ def get_inter_dir(self):
85
+ return Path('data/inter')
86
+
87
+ def get_gtiff_dir(self):
88
+ return Path('export/gtiffs')
89
+
90
+ def get_bounds_dir(self):
91
+ return Path('export/bounds')
92
+
93
+ def get_crs_proj(self):
94
+ return '+proj=tmerc +lat_0=0 +lon_0=84 +k=0.9999 +x_0=500000 +y_0=0 +units=m +ellps=evrst30 +towgs84=293.17,726.18,245.36,0,0,0,0 +no_defs'
95
+
96
+ def get_scale(self):
97
+ return 25000
98
+
99
+ def get_intersection_point(self, img, direction, anchor_angle):
100
+ # ... (implementation for finding intersection points which are the corners of the mapframe)
101
+ # you are expected to pick from the various locate intersection points implementations in the library or write your own
102
+
103
+ def locate_grid_lines(self):
104
+ # ... (implementation for locating grid lines so as to remove them)
105
+ # you are expected to pick from the various locate grid lines implementations in the library or write your own
106
+
107
+ # Example of how to run the processor
108
+ def process_files():
109
+ # ... (load image files and special cases)
110
+ for filepath in image_files:
111
+ extra = special_cases.get(filepath.name, {})
112
+ index_map = get_index_map_jica()
113
+ processor = SampleProcessor(filepath, extra, index_map)
114
+ processor.process()
115
+ ```
116
+
117
+ The library is configurable and most parameters can be overridden at a per sheet level using the `extra` dictionary.
118
+
119
+ The `index_map` is used to map specific file names to their corresponding metadata, which can be useful for processing specific datasets. In particular the index_map is expected to contain the geojson geometry of the map sheet in the `geometry` key, which is used to create the bounds file and georeference the sheet. The geometry is expected to be in counter clockwise order starting at the top left corner of the map sheet.
120
+
121
+ The end result is a Cloud optimized GeoTIFF file, a bounds file in GeoJSON format from which the tiles can be generated, using the `tile` command-line tool, and a set of tiles in the specified directory.
122
+
123
+ I would recommend looking at the [nepal_survey_maps](https://github.com/ramSeraph/nepal_survey_maps) repository for a sample implementation
124
+
125
+ ### Corner Detection Strategies
126
+
127
+ The `get_intersection_point` method is the most critical part to customize for a new map series. The library provides several helper methods that can be used to implement this logic, each suited for different types of corner features:
128
+
129
+ - **`get_nearest_intersection_point`**: This method is useful when the map corners are defined by the intersection of two perpendicular lines (like a simple crosshair). It works by:
130
+ 1. Detecting all horizontal and vertical lines in the corner region.
131
+ 2. Finding all intersection points between these lines.
132
+ 3. Identifying the point closest to the expected corner position, while also matching an expected angle and distance from an anchor point.
133
+
134
+ - **`get_4way_intersection_point`**: This is a more robust version for crosshair-style corners. It specifically looks for intersections where four lines meet, making it less prone to errors from stray lines in the map collar.
135
+
136
+ - **`get_biggest_contour_corner`**: This strategy is designed for corners that are marked by a solid shape or a filled area (e.g., a colored square or circle). It works by:
137
+ 1. Isolating the specific color of the corner feature.
138
+ 2. Finding the largest contour (shape) of that color in the corner region.
139
+ 3. Returning the outermost point of that contour as the corner.
140
+
141
+ - **`get_nearest_intersection_point_from_biggest_corner_contour`**: This is a hybrid approach. It first uses the "biggest contour" method to locate a general corner area (e.g., a colored box) and then searches for a precise line intersection *within* that area. This is effective for maps that have both a colored corner block and a fine crosshair inside it.
142
+
143
+ By combining these methods in your subclass's implementation of `get_intersection_point`, you can build a reliable corner detection system for your specific map type.
144
+
145
+ ### Grid Line Detection Strategies
146
+
147
+ Similar to corner detection, the `locate_grid_lines` method can be implemented using a few key helper methods provided by the library. The primary goal is to identify the precise pixel paths of the map's grid lines (graticules) so they can be removed.
148
+
149
+ - **`locate_grid_lines_using_transformer`**: This is the main strategy for identifying grid lines. It leverages the georeferencing information (the `GCPTransformer`) established from the map corners. The process is as follows:
150
+ 1. It defines a regular grid in the map's real-world coordinate system (e.g., a line every 1000 meters).
151
+ 2. It uses the transformer to project this theoretical grid back onto the pixel space of the image.
152
+ 3. This results in a highly accurate set of lines that represent where the grid lines should be, even accounting for distortions in the map.
153
+
154
+ - **`get_grid_line_corrections`**: This method serves as a refinement step to improve accuracy. While the transformer provides a very close estimate, this function can correct for minor printing or scanning inaccuracies. For each projected grid intersection, it searches a small area in the actual image for the *true* intersection of printed lines, adjusting the point to match the visual data. This ensures that the lines targeted for removal are precisely the ones printed on the map.
155
+
156
+ ### Command-Line Tools
157
+
158
+ This project provides several command-line tools to work with topographic maps. These tools are dynamically loaded, and you can get the most up-to-date usage information by running the tool with the `--help` flag.
159
+
160
+ #### `collect-bounds`
161
+
162
+ Collects individual GeoJSONL bound files into a single GeoJSON FeatureCollection.
163
+
164
+ ```bash
165
+ collect-bounds --bounds-dir <directory> --output-file <file>
166
+ ```
167
+
168
+ - `--bounds-dir`: Directory containing GeoJSONL files. (Required)
169
+ - `--output-file`: Output GeoJSON file path. (Required)
170
+
171
+ #### `partition`
172
+
173
+ Partitions a large set of tiles into smaller PMTiles files. This is useful for managing large datasets, for example, to stay within file size limits for services like GitHub Releases.
174
+
175
+ ```bash
176
+ partition --to-pmtiles-prefix <prefix> --from-tiles-dir <dir> --name <name> --description <desc> --max-zoom <zoom> [--min-zoom <zoom>] [--from-pmtiles-prefix <prefix>] [--only-disk] (--attribution <text> | --attribution-file <file>)
177
+ ```
178
+
179
+ - `--to-pmtiles-prefix`: Prefix for the output PMTiles files. (Required)
180
+ - `--from-tiles-dir`: Directory containing the input tiles. (Required)
181
+ - `--name`: Name of the mosaic. (Required)
182
+ - `--description`: Description of the mosaic. (Required)
183
+ - `--max-zoom`: Maximum zoom level for the mosaic. (Required)
184
+ - `--min-zoom`: Minimum zoom level for the mosaic (default: 0).
185
+ - `--from-pmtiles-prefix`: Prefix for input PMTiles files (required if not using `--only-disk`).
186
+ - `--only-disk`: Only read tiles from the local disk.
187
+ - `--attribution`: Attribution text for the mosaic. (Required, unless `--attribution-file` is used)
188
+ - `--attribution-file`: File containing attribution text. (Required, unless `--attribution` is used)
189
+
190
+ #### `retile`
191
+
192
+ Retiles a specific list of sheets (GeoTIFFs) and updates the corresponding tiles in an existing tile set. It can determine which other sheets are affected and need to be pulled in to correctly retile the area.
193
+
194
+ ```bash
195
+ retile --retile-list-file <file> --bounds-file <file> --max-zoom <zoom> [--min-zoom <zoom>] [--tiffs-dir <dir>] [--tiles-dir <dir>] [--from-pmtiles-prefix <prefix>] [--sheets-to-pull-list-outfile <file>]
196
+ ```
197
+
198
+ - `--retile-list-file`: File containing the list of sheets to retile. (Required)
199
+ - `--bounds-file`: GeoJSON file containing the list of available sheets and their geographic bounds. (Required)
200
+ - `--max-zoom`: Maximum zoom level to create tiles for. (Required)
201
+ - `--min-zoom`: Minimum zoom level to create tiles for (default: 0).
202
+ - `--tiffs-dir`: Directory where the GeoTIFFs are located.
203
+ - `--tiles-dir`: Directory where the tiles will be created.
204
+ - `--from-pmtiles-prefix`: Prefix for the PMTiles source from which to pull existing tiles.
205
+ - `--sheets-to-pull-list-outfile`: If provided, the script will only calculate the full list of sheets that need to be processed (including adjacent ones) and write it to this file, then exit.
206
+
207
+ #### `tile`
208
+
209
+ Creates web map tiles from a directory of GeoTIFF files. It combines them into a virtual raster (`.vrt`) for efficient processing.
210
+
211
+ ```bash
212
+ tile <tiles-dir> <tiffs-dir> <max-zoom> [<min-zoom>]
213
+ ```
214
+
215
+ - `<tiles-dir>`: Directory to store the output tiles. (Required)
216
+ - `<tiffs-dir>`: Directory containing the input GeoTIFF files. (Required)
217
+ - `<max-zoom>`: Maximum zoom level for tiling. (Required)
218
+ - `<min-zoom>`: Minimum zoom level for tiling (default: 0).
219
+
220
+ #### `update-bounds`
221
+
222
+ Updates a main GeoJSON bounds file with new or modified bounds from individual `.geojsonl` files.
223
+
224
+ ```bash
225
+ update-bounds --bounds-file <file> --bounds-dir <dir> --retile-list-file <file>
226
+ ```
227
+
228
+ - `--bounds-file`: The main GeoJSON bounds file to update. (Required)
229
+ - `--bounds-dir`: Directory containing the new or updated individual `.geojsonl` files. (Required)
230
+ - `--retile-list-file`: A file containing the list of sheet names that have been updated (corresponding to the files in `--bounds-dir`). (Required)
231
+
232
+ ### Running Tools Directly with `uvx`
233
+
234
+ You can run the command-line tools directly from PyPI without a persistent installation using `uvx`. This is convenient for one-off tasks.
235
+
236
+ ```bash
237
+
238
+ # Example: Run the 'collect-bounds' tool
239
+ uvx --from topo_map_processor collect-bounds --bounds-dir /path/to/bounds --output-file bounds.geojson
240
+
241
+ # tile and retile have a dependency gdal python library,
242
+ # and some special handling is required so as to install a version of gdal python bindings which matches the version of gdal installed in your system
243
+ # Also numpy and pillow dependencies are required so that the gdal python bindings have some of the resampling features enabled
244
+ # Example: Get help for the 'tile' command
245
+ GDAL_VERSION=$(gdalinfo --version | cut -d"," -f1 | cut -d" " -f2)
246
+ uvx --with numpy --with pillow --with gdal==$GDAL_VERSION --from topo_map_processor tile --help
247
+ ```
248
+
249
+ ## Contributing
250
+
251
+ Contributions are welcome! Please feel free to submit a pull request.
252
+
253
+ ## License
254
+
255
+ This project is under the UNLICENSE
@@ -17,5 +17,5 @@ __version__: str
17
17
  __version_tuple__: VERSION_TUPLE
18
18
  version_tuple: VERSION_TUPLE
19
19
 
20
- __version__ = version = '0.1.3'
21
- __version_tuple__ = version_tuple = (0, 1, 3)
20
+ __version__ = version = '0.1.4'
21
+ __version_tuple__ = version_tuple = (0, 1, 4)
@@ -0,0 +1,25 @@
1
+
2
+ import argparse
3
+
4
+ def cli():
5
+ parser = argparse.ArgumentParser(description='Collect GeoJSONL files into a single GeoJSON file.')
6
+ parser.add_argument('--bounds-dir', required=True, help='Directory containing GeoJSONL files')
7
+ parser.add_argument('--output-file', required=True, help='Output GeoJSON file path')
8
+
9
+ args = parser.parse_args()
10
+
11
+ paths = list(args.bounds_dir.glob('*.geojsonl'))
12
+ with open(args.output_file, 'w') as f:
13
+ f.write('{ "type": "FeatureCollection", "features": [\n')
14
+ for i,p in enumerate(paths):
15
+ line = p.read_text().strip('\n')
16
+
17
+ if i != len(paths) - 1:
18
+ line += ','
19
+ line += '\n'
20
+ f.write(line)
21
+ f.write(']}\n')
22
+
23
+ if __name__ == "__main__":
24
+ cli()
25
+
@@ -315,17 +315,17 @@ def create_pmtiles(partition_info, reader, to_pmtiles_prefix, name, description,
315
315
 
316
316
  def cli():
317
317
  parser = argparse.ArgumentParser()
318
- parser.add_argument('--only-disk', action='store_true')
319
- parser.add_argument('--to-pmtiles-prefix', required=True)
320
- parser.add_argument('--from-pmtiles-prefix')
321
- parser.add_argument('--from-tiles-dir', required=True)
322
- parser.add_argument('--name', required=True)
323
- parser.add_argument('--description', required=True)
324
- parser.add_argument('--min-zoom', default=0, type=int)
325
- parser.add_argument('--max-zoom', required=True, type=int)
318
+ parser.add_argument('--only-disk', action='store_true', help='Only read tiles from disk.')
319
+ parser.add_argument('--to-pmtiles-prefix', required=True, help='Prefix for the output PMTiles files.')
320
+ parser.add_argument('--from-pmtiles-prefix', help='Prefix for the input PMTiles files. Required if not using --only-disk.')
321
+ parser.add_argument('--from-tiles-dir', required=True, help='Directory containing the input tiles')
322
+ parser.add_argument('--name', required=True, help='Name of the mosaic.')
323
+ parser.add_argument('--description', required=True, help='Description of the mosaic.')
324
+ parser.add_argument('--min-zoom', default=0, type=int, help='Minimum zoom level for the mosaic.')
325
+ parser.add_argument('--max-zoom', required=True, type=int, help='Maximum zoom level for the mosaic.')
326
326
  group = parser.add_mutually_exclusive_group(required=True)
327
- group.add_argument('--attribution')
328
- group.add_argument('--attribution-file')
327
+ group.add_argument('--attribution', help='Attribution text for the mosaic.')
328
+ group.add_argument('--attribution-file', help='File containing attribution text for the mosaic.')
329
329
  args = parser.parse_args()
330
330
 
331
331
  if args.attribution_file:
@@ -217,19 +217,23 @@ def cli():
217
217
  parser = argparse.ArgumentParser()
218
218
  parser.add_argument('--retile-list-file', required=True, help='File containing list of sheets to retile')
219
219
  parser.add_argument('--bounds-file', required=True, help='Geojson file containing list of available sheets and their georaphic bounds')
220
- parser.add_argument('--min-zoom', default=0, type=int)
221
- parser.add_argument('--max-zoom', required=True, type=int)
220
+ parser.add_argument('--min-zoom', default=0, type=int, help='Minimum zoom level to create tiles for')
221
+ parser.add_argument('--max-zoom', required=True, type=int, help='Maximum zoom level to create tiles for')
222
222
  parser.add_argument('--sheets-to-pull-list-outfile', default=None,
223
- help='Output into which we write the list of sheet that need to be pulled, if set the script ends after created the list file')
224
- parser.add_argument('--from-pmtiles-prefix')
225
- parser.add_argument('--tiles-dir', required=True)
226
- parser.add_argument('--tiffs-dir', required=True)
223
+ help='Output into which we write the list of sheet that need to be pulled, if set, the script ends after it created the list file')
224
+ parser.add_argument('--from-pmtiles-prefix', help='Prefix to the PMTiles source from which we pull tiles, if needs to be set when --sheets-to-pull-list-outfile is not set')
225
+ parser.add_argument('--tiles-dir', help='Directory where the tiles will be created')
226
+ parser.add_argument('--tiffs-dir', help='Directory where the tiffs are present')
227
227
 
228
228
  args = parser.parse_args()
229
229
  if not args.sheets_to_pull_list_outfile:
230
230
  missing = []
231
231
  if not args.from_pmtiles_prefix:
232
232
  missing.append('--from-pmtiles-prefix')
233
+ if not args.tiles_dir:
234
+ missing.append('--tiles-dir')
235
+ if not args.tiffs_dir:
236
+ missing.append('--tiffs-dir')
233
237
 
234
238
  if missing:
235
239
  parser.error(f"The following arguments are required when --sheets-to-pull-list-outfile is not provided: {', '.join(missing)}")
@@ -255,20 +259,24 @@ def cli():
255
259
  for sheet in to_add:
256
260
  sheets_to_pull.add(sheet)
257
261
 
258
- Path(args.tiffs_dir).mkdir(exist_ok=True, parents=True)
259
- Path(args.tiles_dir).mkdir(exist_ok=True, parents=True)
260
262
  print(f'{sheets_to_pull=}')
263
+
261
264
  if args.sheets_to_pull_list_outfile is not None:
265
+ if args.tiffs_dir is not None:
266
+ Path(args.tiffs_dir).mkdir(exist_ok=True, parents=True)
262
267
  Path(args.sheets_to_pull_list_outfile).write_text('\n'.join(sheets_to_pull) + '\n')
263
268
  exit(0)
264
269
 
265
270
 
271
+
266
272
  print('check the sheets availability')
267
273
  check_sheets(sheets_to_pull, args.tiffs_dir)
268
274
 
269
275
  print('creating vrt file from sheets involved')
270
276
  vrt_file = create_vrt_file(sheets_to_pull, args.tiffs_dir)
271
277
 
278
+ Path(args.tiles_dir).mkdir(exist_ok=True, parents=True)
279
+
272
280
  print('creating tiles for base zoom with a vrt')
273
281
  create_base_tiles(f'{vrt_file}', str(args.tiles_dir), f'{args.max_zoom}')
274
282
 
@@ -5,9 +5,9 @@ from pathlib import Path
5
5
 
6
6
  def cli():
7
7
  parser = argparse.ArgumentParser(description="Update bounds in a GeoJSON file.")
8
- parser.add_argument("bounds-file", help="Input GeoJSON file with bounds.")
9
- parser.add_argument("bounds-dir", help="Directory containing the individual bounds files")
10
- parser.add_argument("retile-list-file", help="File containing the list of sheets to update.")
8
+ parser.add_argument("--bounds-file", required=True, help="Input GeoJSON file with bounds.")
9
+ parser.add_argument("--bounds-dir", required=True, help="Directory containing the individual bounds files")
10
+ parser.add_argument("--retile-list-file", required=True, help="File containing the list of sheets to update.. .tif extension is expected")
11
11
 
12
12
  args = parser.parse_args()
13
13
 
@@ -29,6 +29,7 @@ def cli():
29
29
  sheets = retile_list_file.read_text().strip().splitlines()
30
30
  sheets = [s.strip() for s in sheets if s.strip()]
31
31
  for s in sheets:
32
+ s = s.replace('.tif', '')
32
33
  bfile = bounds_dir / f'{s}.geojsonl'
33
34
  if not bfile.exists():
34
35
  raise FileNotFoundError(f"Bounds file for sheet '{s}' does not exist: {bfile}")
@@ -1,187 +0,0 @@
1
- Metadata-Version: 2.4
2
- Name: topo_map_processor
3
- Version: 0.1.3
4
- Summary: Library for processing topographic maps
5
- Project-URL: homepage, https://github.com/ramSeraph/topo_map_processor
6
- Project-URL: repository, https://github.com/ramSeraph/topo_map_processor.git
7
- Author-email: Sreeram Kandimalla <kandimalla.sreeram@gmail.com>
8
- License: This is free and unencumbered software released into the public domain.
9
-
10
- Anyone is free to copy, modify, publish, use, compile, sell, or
11
- distribute this software, either in source code form or as a compiled
12
- binary, for any purpose, commercial or non-commercial, and by any
13
- means.
14
-
15
- In jurisdictions that recognize copyright laws, the author or authors
16
- of this software dedicate any and all copyright interest in the
17
- software to the public domain. We make this dedication for the benefit
18
- of the public at large and to the detriment of our heirs and
19
- successors. We intend this dedication to be an overt act of
20
- relinquishment in perpetuity of all present and future rights to this
21
- software under copyright law.
22
-
23
- THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
24
- EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
25
- MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
26
- IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
27
- OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
28
- ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
29
- OTHER DEALINGS IN THE SOFTWARE.
30
-
31
- For more information, please refer to <https://unlicense.org>
32
- License-File: LICENSE
33
- Requires-Python: >=3.12
34
- Requires-Dist: easyocr>=1.7.2
35
- Requires-Dist: imgcat>=0.6.0
36
- Requires-Dist: mercantile>=1.2.1
37
- Requires-Dist: numpy>=2.3.1
38
- Requires-Dist: opencv-python-headless>=4.11.0.86
39
- Requires-Dist: pillow>=11.3.0
40
- Requires-Dist: pmtiles>=3.4.1
41
- Requires-Dist: pyproj>=3.7.1
42
- Requires-Dist: pytesseract>=0.3.13
43
- Requires-Dist: rasterio>=1.4.3
44
- Requires-Dist: shapely>=2.1.1
45
- Description-Content-Type: text/markdown
46
-
47
- # Topo Map Processor
48
-
49
- A Python utility for processing topographic maps.
50
-
51
- ## Description
52
-
53
- This project provides a collection of command-line tools and a Python library to process, analyze, and manipulate topographic map data, specifically for creating web-mappable tiles from GeoTIFF files. The tools allow you to create tiles, partition large tile sets, and update existing tile sets efficiently.
54
-
55
- ## Installation
56
-
57
- To install the necessary dependencies, it is recommended to use a virtual environment.
58
-
59
- ```bash
60
- python -m venv .venv
61
- source .venv/bin/activate
62
- pip install -e .
63
- ```
64
-
65
- ## Usage
66
-
67
- This project can be used as a library but also provides as a set of command-line tools to help with managing the datasets, tiling and creating `pmtiles` files.
68
-
69
- ### Library Usage
70
-
71
- The `TopoMapProcessor` class provides a framework for processing individual map sheets. To use it, you need to create a subclass and implement the required methods.
72
-
73
- Here's an example of how to use the `TopoMapProcessor` class, based on the provided `parse_jica.py` script:
74
-
75
- ```python
76
- from topo_map_processor import TopoMapProcessor, LineRemovalParams
77
-
78
- class SampleProcessor(TopoMapProcessor):
79
-
80
- def __init__(self, filepath, extra, index_map):
81
- super().__init__(filepath, extra, index_map)
82
- # ... (additional initialization)
83
-
84
- def get_inter_dir(self):
85
- return Path('data/inter')
86
-
87
- def get_gtiff_dir(self):
88
- return Path('export/gtiffs')
89
-
90
- def get_bounds_dir(self):
91
- return Path('export/bounds')
92
-
93
- def get_crs_proj(self):
94
- return '+proj=tmerc +lat_0=0 +lon_0=84 +k=0.9999 +x_0=500000 +y_0=0 +units=m +ellps=evrst30 +towgs84=293.17,726.18,245.36,0,0,0,0 +no_defs'
95
-
96
- def get_scale(self):
97
- return 25000
98
-
99
- def get_intersection_point(self, img, direction, anchor_angle):
100
- # ... (implementation for finding intersection points which are the corners of the mapframe)
101
- # you are expected to pick from the various locate intersection points implementations in the library or write your own
102
-
103
- def locate_grid_lines(self):
104
- # ... (implementation for locating grid lines so as to remove them)
105
- # you are expected to pick from the various locate grid lines implementations in the library or write your own
106
-
107
- # Example of how to run the processor
108
- def process_files():
109
- # ... (load image files and special cases)
110
- for filepath in image_files:
111
- extra = special_cases.get(filepath.name, {})
112
- index_map = get_index_map_jica()
113
- processor = SampleProcessor(filepath, extra, index_map)
114
- processor.process()
115
- ```
116
-
117
- The library is configurable and most parameters can be overriden at a per sheet level using the `extra` dictionary.
118
-
119
- The `index_map` is used to map specific file names to their corresponding metadata, which can be useful for processing specific datasets. In particular the index_map is expected to contain the geojson geometry of the map sheet in the `geometry` key, which is used to create the bounds file and georeference the sheet. The geometry is expected to be in counter clockwise order startiung at the top left corner of the map sheet.
120
-
121
- The end result is a Cloud optimized GeoTIFF file, a bounds file in GeoJSON format from which the tiles can be generated, using the `tile` command-line tool, and a set of tiles in the specified directory.
122
-
123
- I would recommend looking at the [nepal_survey_maps](https://github.com/ramSeraph/nepal_survey_maps) repository for a sample implementation
124
-
125
- ### Command-Line Tools
126
-
127
- This project provides several command-line tools to work with topographic maps.
128
-
129
- #### `tile`
130
-
131
- Creates tiles from a directory of GeoTIFF files.
132
-
133
- ```bash
134
- tile --tiles-dir <tiles-dir> --tiffs-dir <tiffs-dir> --max-zoom <max-zoom> [--min-zoom <min-zoom>]
135
- ```
136
-
137
- - `<tiles-dir>`: Directory to store the output tiles.
138
- - `<tiffs-dir>`: Directory containing the input GeoTIFF files.
139
- - `<max-zoom>`: Maximum zoom level for tiling.
140
- - `<min-zoom>`: Minimum zoom level for tiling (default: 0).
141
-
142
- #### `partition`
143
-
144
- Partitions a large set of tiles into smaller PMTiles files, which is useful for managing large datasets.
145
-
146
- ```bash
147
- partition --to-pmtiles-prefix <prefix> --from-tiles-dir <dir> --name <name> --description <desc> --max-zoom <max-zoom> [--from-pmtiles-prefix <prefix>] [--only-disk] [--min-zoom <min-zoom>] (--attribution <attr> | --attribution-file <file>)
148
- ```
149
-
150
- #### `retile`
151
-
152
- Retiles a specific list of sheets (GeoTIFFs) and updates the corresponding tiles.
153
-
154
- ```bash
155
- retile --retile-list-file <file> --bounds-file <file> --max-zoom <max-zoom> --tiles-dir <dir> --tiffs-dir <dir> [--min-zoom <min-zoom>] [--from-pmtiles-prefix <prefix>]
156
- ```
157
-
158
- #### `collect-bounds`
159
-
160
- Collects individual GeoJSONL bound files into a single GeoJSON FeatureCollection.
161
-
162
- ```bash
163
- collect-bounds <bounds-dir> <output-file>
164
- ```
165
-
166
- - `<bounds-dir>`: Directory containing `.geojsonl` files.
167
- - `<output-file>`: The output GeoJSON file.
168
-
169
- #### `update-bounds`
170
-
171
- Updates a GeoJSON bounds file with new information from a directory of individual bounds files.
172
-
173
- ```bash
174
- update-bounds <bounds-file> <bounds-dir> <retile-list-file>
175
- ```
176
-
177
- - `<bounds-file>`: The GeoJSON file to update.
178
- - `<bounds-dir>`: Directory with the new `.geojsonl` files.
179
- - `<retile-list-file>`: A file containing the list of sheets that have been updated.
180
-
181
- ## Contributing
182
-
183
- Contributions are welcome! Please feel free to submit a pull request.
184
-
185
- ## License
186
-
187
- This project is under UNLICENSE
@@ -1,141 +0,0 @@
1
- # Topo Map Processor
2
-
3
- A Python utility for processing topographic maps.
4
-
5
- ## Description
6
-
7
- This project provides a collection of command-line tools and a Python library to process, analyze, and manipulate topographic map data, specifically for creating web-mappable tiles from GeoTIFF files. The tools allow you to create tiles, partition large tile sets, and update existing tile sets efficiently.
8
-
9
- ## Installation
10
-
11
- To install the necessary dependencies, it is recommended to use a virtual environment.
12
-
13
- ```bash
14
- python -m venv .venv
15
- source .venv/bin/activate
16
- pip install -e .
17
- ```
18
-
19
- ## Usage
20
-
21
- This project can be used as a library but also provides as a set of command-line tools to help with managing the datasets, tiling and creating `pmtiles` files.
22
-
23
- ### Library Usage
24
-
25
- The `TopoMapProcessor` class provides a framework for processing individual map sheets. To use it, you need to create a subclass and implement the required methods.
26
-
27
- Here's an example of how to use the `TopoMapProcessor` class, based on the provided `parse_jica.py` script:
28
-
29
- ```python
30
- from topo_map_processor import TopoMapProcessor, LineRemovalParams
31
-
32
- class SampleProcessor(TopoMapProcessor):
33
-
34
- def __init__(self, filepath, extra, index_map):
35
- super().__init__(filepath, extra, index_map)
36
- # ... (additional initialization)
37
-
38
- def get_inter_dir(self):
39
- return Path('data/inter')
40
-
41
- def get_gtiff_dir(self):
42
- return Path('export/gtiffs')
43
-
44
- def get_bounds_dir(self):
45
- return Path('export/bounds')
46
-
47
- def get_crs_proj(self):
48
- return '+proj=tmerc +lat_0=0 +lon_0=84 +k=0.9999 +x_0=500000 +y_0=0 +units=m +ellps=evrst30 +towgs84=293.17,726.18,245.36,0,0,0,0 +no_defs'
49
-
50
- def get_scale(self):
51
- return 25000
52
-
53
- def get_intersection_point(self, img, direction, anchor_angle):
54
- # ... (implementation for finding intersection points which are the corners of the mapframe)
55
- # you are expected to pick from the various locate intersection points implementations in the library or write your own
56
-
57
- def locate_grid_lines(self):
58
- # ... (implementation for locating grid lines so as to remove them)
59
- # you are expected to pick from the various locate grid lines implementations in the library or write your own
60
-
61
- # Example of how to run the processor
62
- def process_files():
63
- # ... (load image files and special cases)
64
- for filepath in image_files:
65
- extra = special_cases.get(filepath.name, {})
66
- index_map = get_index_map_jica()
67
- processor = SampleProcessor(filepath, extra, index_map)
68
- processor.process()
69
- ```
70
-
71
- The library is configurable and most parameters can be overriden at a per sheet level using the `extra` dictionary.
72
-
73
- The `index_map` is used to map specific file names to their corresponding metadata, which can be useful for processing specific datasets. In particular the index_map is expected to contain the geojson geometry of the map sheet in the `geometry` key, which is used to create the bounds file and georeference the sheet. The geometry is expected to be in counter clockwise order startiung at the top left corner of the map sheet.
74
-
75
- The end result is a Cloud optimized GeoTIFF file, a bounds file in GeoJSON format from which the tiles can be generated, using the `tile` command-line tool, and a set of tiles in the specified directory.
76
-
77
- I would recommend looking at the [nepal_survey_maps](https://github.com/ramSeraph/nepal_survey_maps) repository for a sample implementation
78
-
79
- ### Command-Line Tools
80
-
81
- This project provides several command-line tools to work with topographic maps.
82
-
83
- #### `tile`
84
-
85
- Creates tiles from a directory of GeoTIFF files.
86
-
87
- ```bash
88
- tile --tiles-dir <tiles-dir> --tiffs-dir <tiffs-dir> --max-zoom <max-zoom> [--min-zoom <min-zoom>]
89
- ```
90
-
91
- - `<tiles-dir>`: Directory to store the output tiles.
92
- - `<tiffs-dir>`: Directory containing the input GeoTIFF files.
93
- - `<max-zoom>`: Maximum zoom level for tiling.
94
- - `<min-zoom>`: Minimum zoom level for tiling (default: 0).
95
-
96
- #### `partition`
97
-
98
- Partitions a large set of tiles into smaller PMTiles files, which is useful for managing large datasets.
99
-
100
- ```bash
101
- partition --to-pmtiles-prefix <prefix> --from-tiles-dir <dir> --name <name> --description <desc> --max-zoom <max-zoom> [--from-pmtiles-prefix <prefix>] [--only-disk] [--min-zoom <min-zoom>] (--attribution <attr> | --attribution-file <file>)
102
- ```
103
-
104
- #### `retile`
105
-
106
- Retiles a specific list of sheets (GeoTIFFs) and updates the corresponding tiles.
107
-
108
- ```bash
109
- retile --retile-list-file <file> --bounds-file <file> --max-zoom <max-zoom> --tiles-dir <dir> --tiffs-dir <dir> [--min-zoom <min-zoom>] [--from-pmtiles-prefix <prefix>]
110
- ```
111
-
112
- #### `collect-bounds`
113
-
114
- Collects individual GeoJSONL bound files into a single GeoJSON FeatureCollection.
115
-
116
- ```bash
117
- collect-bounds <bounds-dir> <output-file>
118
- ```
119
-
120
- - `<bounds-dir>`: Directory containing `.geojsonl` files.
121
- - `<output-file>`: The output GeoJSON file.
122
-
123
- #### `update-bounds`
124
-
125
- Updates a GeoJSON bounds file with new information from a directory of individual bounds files.
126
-
127
- ```bash
128
- update-bounds <bounds-file> <bounds-dir> <retile-list-file>
129
- ```
130
-
131
- - `<bounds-file>`: The GeoJSON file to update.
132
- - `<bounds-dir>`: Directory with the new `.geojsonl` files.
133
- - `<retile-list-file>`: A file containing the list of sheets that have been updated.
134
-
135
- ## Contributing
136
-
137
- Contributions are welcome! Please feel free to submit a pull request.
138
-
139
- ## License
140
-
141
- This project is under UNLICENSE
@@ -1,23 +0,0 @@
1
-
2
- import sys
3
- from pathlib import Path
4
-
5
- def cli():
6
- bounds_dir = Path(sys.argv[1])
7
- output_file = Path(sys.argv[2])
8
-
9
- paths = list(bounds_dir.glob('*.geojsonl'))
10
- with open(output_file, 'w') as f:
11
- f.write('{ "type": "FeatureCollection", "features": [\n')
12
- for i,p in enumerate(paths):
13
- line = p.read_text().strip('\n')
14
-
15
- if i != len(paths) - 1:
16
- line += ','
17
- line += '\n'
18
- f.write(line)
19
- f.write(']}\n')
20
-
21
- if __name__ == "__main__":
22
- cli()
23
-