hybig-py 2.0.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,51 @@
1
+ # Temporary output directories
2
+ tmp*
3
+ .DS_Store
4
+ tmp
5
+ config-*.json
6
+ config.json
7
+
8
+ # Byte-compiled / optimized / DLL files
9
+ __pycache__/
10
+ *.py[cod]
11
+ *$py.class
12
+
13
+ # Unit test / coverage reports
14
+ htmlcov/
15
+ .tox/
16
+ .nox/
17
+ .coverage
18
+ .coverage.*
19
+ .cache
20
+ nosetests.xml
21
+ coverage.xml
22
+ *.cover
23
+ .hypothesis/
24
+ .pytest_cache/
25
+ coverage
26
+ test-reports
27
+ tests/reports
28
+
29
+ # Jupyter Notebook
30
+ .ipynb_checkpoints
31
+
32
+ # IPython
33
+ profile_default/
34
+ ipython_config.py
35
+
36
+ # pyenv
37
+ .python-version
38
+
39
+ # Environments
40
+ .env
41
+ .venv
42
+ env/
43
+ venv/
44
+ ENV/
45
+ env.bak/
46
+ venv.bak/
47
+
48
+ # mypy
49
+ .mypy_cache/
50
+ .dmypy.json
51
+ dmypy.json
hybig_py-2.0.0/LICENSE ADDED
@@ -0,0 +1,60 @@
1
+ Copyright © 2022 United States Government as represented by the Administrator of the National Aeronautics and Space Administration. All Rights Reserved.
2
+
3
+ Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License.
4
+ You may obtain a copy of the License at
5
+
6
+ http://www.apache.org/licenses/LICENSE-2.0
7
+
8
+ Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS,
9
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
10
+
11
+ ---
12
+
13
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
14
+
15
+ 1. Definitions.
16
+
17
+ "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document.
18
+
19
+ "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License.
20
+
21
+ "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity.
22
+
23
+ "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License.
24
+
25
+ "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files.
26
+
27
+ "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types.
28
+
29
+ "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below).
30
+
31
+ "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof.
32
+
33
+ "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution."
34
+
35
+ "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work.
36
+
37
+ 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form.
38
+
39
+ 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed.
40
+
41
+ 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions:
42
+
43
+ You must give any other recipients of the Work or Derivative Works a copy of this License; and
44
+ You must cause any modified files to carry prominent notices stating that You changed the files; and
45
+ You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and
46
+ If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License.
47
+
48
+ You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License.
49
+
50
+ 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions.
51
+
52
+ 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file.
53
+
54
+ 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License.
55
+
56
+ 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages.
57
+
58
+ 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability.
59
+
60
+ END OF TERMS AND CONDITIONS
@@ -0,0 +1,507 @@
1
+ Metadata-Version: 2.3
2
+ Name: hybig-py
3
+ Version: 2.0.0
4
+ Summary: Python package designed to produce browse imagery compatible with NASA's Global Image Browse Services (GIBS).
5
+ Project-URL: Homepage, https://github.com/nasa/harmony-browse-image-generator
6
+ Project-URL: Issues, https://github.com/nasa/harmony-browse-image-generator/issues
7
+ Author-email: Matt Savoie <savoie@colorado.edu>, Owen Littlejohns <owen.m.littlejohns@nasa.gov>
8
+ Maintainer-email: Matt Savoie <savoie@colorado.edu>, Owen Littlejohns <owen.m.littlejohns@nasa.gov>
9
+ License-File: LICENSE
10
+ Classifier: License :: OSI Approved :: Apache Software License
11
+ Classifier: Operating System :: OS Independent
12
+ Classifier: Programming Language :: Python :: 3
13
+ Requires-Python: >=3.10
14
+ Requires-Dist: gdal==3.6.2
15
+ Requires-Dist: harmony-service-lib~=1.0.27
16
+ Requires-Dist: matplotlib==3.9.0
17
+ Requires-Dist: numpy==1.26.4
18
+ Requires-Dist: pillow==10.3.0
19
+ Requires-Dist: pyproj==3.6.1
20
+ Requires-Dist: pystac~=0.5.6
21
+ Requires-Dist: rasterio==1.3.10
22
+ Requires-Dist: rioxarray==0.15.5
23
+ Description-Content-Type: text/markdown
24
+
25
+ # Harmony Browse Image Generator (HyBIG).
26
+
27
+ This repository contains code designed to produce browse imagery. Its default behaviour
28
+ produces images compatible with the NASA Global Image Browse
29
+ Services ([GIBS](https://www.earthdata.nasa.gov/eosdis/science-system-description/eosdis-components/gibs)).
30
+
31
+ This means that default parameters for images are selected to match the
32
+ visualization generation requirements and recommendations put forth in the GIBS
33
+ Interface Control Document (ICD), which can be found on [Earthdata
34
+ Wiki](https://wiki.earthdata.nasa.gov/display/GITC/Ingest+Delivery+Methods)
35
+ along with [additional GIBS
36
+ documentation](https://nasa-gibs.github.io/gibs-api-docs/).
37
+
38
+ HyBIG creates paletted PNG images and associated metadata from GeoTIFF input
39
+ images. Scientific parameter raster data as well as RGB[A] raster images can
40
+ be converted to browse PNGs. These browse images undergo transformation by
41
+ reprojection, tiling and coloring to seamlessly integrate with GIBS.
42
+
43
+ The repository contains code and infrastructure to support both the HyBIG
44
+ Service as well as `hybig-py`. The HyBIG Service is packaged as a Docker
45
+ container that is deployed to [NASA's
46
+ Harmony](https://harmony.earthdata.nasa.gov/) system. The business logic is
47
+ contained in the [`hybig-py` library](https://pypi.org/project/hybig-py/) which
48
+ exposes functions to generate browse images in python scripts.
49
+
50
+ ### hybig-py
51
+
52
+ The browse image generation logic is packaged in the hybig-py
53
+ library. Currently, a single function, `create_browse` is exposed to the user.
54
+
55
+ ```python
56
+ def create_browse(
57
+ source_tiff: str,
58
+ params: dict = None,
59
+ palette: str | ColorPalette | None = None,
60
+ logger: Logger = None,
61
+ ) -> list[tuple[Path, Path, Path]]:
62
+ """Create browse imagery from an input geotiff.
63
+
64
+ This is the exposed library function to allow users to create browse images
65
+ from the hybig-py library. It parses the input params and constructs the
66
+ correct Harmony input structure [Message.Format] to call the service's
67
+ entry point create_browse_imagery.
68
+
69
+ Output images are created and deposited into the input GeoTIFF's directory.
70
+
71
+ Args:
72
+ source_tiff: str, location of the input geotiff to process.
73
+
74
+ params: [dict | None], A dictionary with the following keys:
75
+
76
+ mime: [str], MIME type of the output image (default: 'image/png').
77
+ any string that contains 'jpeg' will return a jpeg image,
78
+ otherwise create a png.
79
+
80
+ crs: [dict | None], Target image's Coordinate Reference System.
81
+ A dictionary with 'epsg', 'proj4' or 'wkt' key.
82
+
83
+ scale_extent: [dict | None], Scale Extents for the image. This dictionary
84
+ contains "x" and "y" keys each whose value which is a dictionary
85
+ of "min", "max" values in the same units as the crs.
86
+ e.g.: { "x": { "min": 0.5, "max": 125 },
87
+ "y": { "min": 52, "max": 75.22 } }
88
+
89
+ scale_size: [dict | None], Scale sizes for the image. The dictionary
90
+ contains "x" and "y" keys with the horizontal and veritcal
91
+ resolution in the same units as the crs.
92
+ e.g.: { "x": 10, "y": 10 }
93
+
94
+ height: [int | None], height of the output image in gridcells.
95
+
96
+ width: [int | none], width of the output image in gridcells.
97
+
98
+ palette: [str | ColorPalette | none], either a URL to a remote color palette
99
+ that is fetched and loaded or a ColorPalette object used to color
100
+ the output browse image. If not provided, a grayscale image is
101
+ generated.
102
+
103
+ logger: [Logger | None], a configured Logger object. If None a default
104
+ logger will be used.
105
+
106
+ Note:
107
+ if supplied, scale_size, scale_extent, height and width must be
108
+ internally consistent. To define a valid output grid:
109
+ * Specify scale_extent and 1 of:
110
+ * height and width
111
+ * scale_sizes (in the x and y horizontal spatial dimensions)
112
+ * Specify all three of the above, but ensure values are consistent
113
+ with one another, noting that:
114
+ scale_size.x = (scale_extent.x.max - scale_extent.x.min) / width
115
+ scale_size.y = (scale_extent.y.max - scale_extent.y.min) / height
116
+
117
+ Returns:
118
+ List of 3-element tuples. These are the file paths of:
119
+ - The output browse image
120
+ - Its associated ESRI world file (containing georeferencing information)
121
+ - The auxiliary XML file (containing duplicative georeferencing information)
122
+
123
+
124
+ Example Usage:
125
+ results = create_browse(
126
+ "/path/to/geotiff",
127
+ {
128
+ "mime": "image/png",
129
+ "crs": {"epsg": "EPSG:4326"},
130
+ "scale_extent": {
131
+ "x": {"min": -180, "max": 180},
132
+ "y": {"min": -90, "max": 90},
133
+ },
134
+ "scale_size": {"x": 10, "y": 10},
135
+ },
136
+ "https://remote-colortable",
137
+ logger,
138
+ )
139
+
140
+ """
141
+ ```
142
+
143
+ ### library installation
144
+
145
+ The hybig-py library can be installed from PyPI but has a prerequisite
146
+ dependency requirement on the GDAL libraries. Ensure you have an environment
147
+ with the libraries available. You can check on Linux/macOS:
148
+ ```bash
149
+ gdal-config --version
150
+ ```
151
+ on windows (if GDAL is in your PATH):
152
+ ```bash
153
+ gdalinfo --version
154
+ ```
155
+
156
+ Once verified, you can simply install the libary:
157
+
158
+ ```bash
159
+ pip install hybig-py
160
+ ```
161
+
162
+
163
+ ### Reprojection
164
+
165
+ GIBS expects to receive images in one of three Coordinate Reference System (CRS) projections.
166
+
167
+ | Region | Code | Name |
168
+ |-------------|-----------|-----------------------------------------------------------|
169
+ | north polar | EPSG:3413 | WGS 84 / NSIDC Sea Ice Polar Stereographic North |
170
+ | south polar | EPSG:3031 | WGS 84 / Antarctic Polar Stereographic |
171
+ | global | EPSG:4326 | WGS 84 -- WGS84 - World Geodetic System 1984, used in GPS |
172
+
173
+ HyBIG processing will attempt to choose a GIBS-suitable target CRS from the
174
+ input image or read it from the inputs. Reprojection is done by resampling via
175
+ nearest neighbor. It is important to note that HyBig outputs are not scientific
176
+ data, but browse imagery and should not be used for scientific analysis.
177
+
178
+
179
+ ### Tiling
180
+
181
+ Large output images are divided into smaller, more manageable tiles for
182
+ efficient handling and processing, as per agreement with GIBS. The maximum
183
+ untiled image size generated by HyBIG is 67,108,864 cells (8,192 x 8,192). If
184
+ the output image exceeds this threshold, HyBIG automatically tiles the output
185
+ into multiple 4,096 x 4,096 cell images.
186
+
187
+ Tiled images are labeled with the zero-based column and row numbers inserted
188
+ into the output filename before its
189
+ extension. For example, `VCF5KYR_1991001_001_2018224205008.r01c02.png` represents the
190
+ second row and third column of the output tiles. The tiles at the edges are
191
+ truncated to fit the overall image dimensions. Currently, you cannot override
192
+ this behavior.
193
+
194
+ ### Coloring
195
+
196
+ HyBIG images are colored in several ways. A palette can be included in the
197
+ input [STAC
198
+ Item](https://github.com/radiantearth/stac-spec/blob/master/item-spec/item-spec.md). If
199
+ an Item's asset contains a value with the role of `palette`, it is assumed to
200
+ be a reference to a remote color table, which is fetched from the asset's
201
+ `href` and parsed as a GDAL color table.
202
+
203
+ If the STAC Item lacks color information, the Harmony message source is
204
+ searched for a related URL with a "content type" of `VisualizationURL` and a
205
+ "type" of `Color Map`. If found, it is presumed to be a remote color table and
206
+ fetched from that location.
207
+
208
+ In the absence of remote color information, the input image itself is searched
209
+ for a color map, which is used if present.
210
+
211
+ If no color information can be found, grayscale is used.
212
+
213
+ ### Defaults
214
+
215
+ HyBIG tries to provide GIBS-appropriate default values for the browse image
216
+ outputs. When a user does not provide a target values for the output, HyBIG
217
+ will try to pick an appropriate default.
218
+
219
+ #### Coordinate Reference System (CRS)
220
+
221
+ HyBIG selects a default CRS from the list of GIBS preferred projections. The
222
+ steps followed are simple but effective:
223
+
224
+ 1. If the `proj` is `lonlat` use global (`EPSG:4326`)
225
+ 1. If the projection latitude of origin is above 80° N use northern (`EPSG:3413`)
226
+ 1. If the projection latitude of origin is below -80° N use southern (`EPSG:3031`)
227
+ 1. Otherwise use global (`EPGS:4326`)
228
+
229
+ #### Scale Extent (Image Bounds)
230
+
231
+ The default scale extent for an output image is computed by reprojecting the
232
+ input data boundary into the target CRS. It densifies the edges by adding 21
233
+ points ([rasterio's
234
+ default](https://rasterio.readthedocs.io/en/latest/api/rasterio.warp.html#rasterio.warp.transform_bounds))
235
+ to each edge before reprojection to account for non-linear edges produced by
236
+ the transformation ensuring inclusion of all data in the output image.
237
+
238
+ #### Dimensions / Scale Sizes
239
+
240
+ Output image dimensions can be explicitly included as `width` and `height` in
241
+ the harmony message or computed based on the scale extent and scale size
242
+ (resolution).
243
+
244
+ The dimension computations from the scale extent and scale size:
245
+ ```
246
+ height = round((scale_extent['ymax'] - scale_extent['ymin']) / scale_size.y)
247
+ width = round((scale_extent['xmax'] - scale_extent['xmin']) / scale_size.x)
248
+ ```
249
+
250
+ When a Harmony message contains neither `dimensions` nor `scaleSizes` a default
251
+ set of dimensions is computed.
252
+
253
+ For coarse input data, the resolution (scale size) is used with the scale
254
+ extent to compute the output dimensions. For high resolution data, finer than
255
+ 2km per gridcell, the input resolution is used to lookup the closest GIBS
256
+ preferred resolution (Table 4.1.8-1 and -2 from the ICD) and the preferred
257
+ resolution along with the scale extent is used to compute the output image
258
+ dimensions.
259
+
260
+ ### Customizations
261
+
262
+ Users can request customizations to the output images such as `crs`,
263
+ `scale_extents`, or `scale_sizes` and dimensions (`height` & `width`) in the
264
+ harmony request. However, the generated outputs may not be compatible with
265
+ GIBS.
266
+
267
+ When a user customizes `scale_extent` or `scale_size`, they must also include a
268
+ `crs` in the request. The units of the cusomized values must match the target
269
+ CRS. For example, specifying a bounding box in degrees requires a target CRS
270
+ also with units of degrees.
271
+
272
+
273
+ ## Repository structure:
274
+
275
+ ```
276
+ |- 📂 bin
277
+ |- 📂 docker
278
+ |- 📂 docs
279
+ |- 📂 hybig
280
+ |- 📂 harmony_service
281
+ |- 📂 tests
282
+ |- CHANGELOG.md
283
+ |- CONTRIBUTING.md
284
+ |- LICENSE
285
+ |- README.md
286
+ |- dev-requirements.txt
287
+ |- legacy-CHANGELOG.md
288
+ |- pip_requirements.txt
289
+ |- pip_requirements_skip_snyk.txt
290
+ |- pyproject.toml
291
+
292
+ ```
293
+
294
+ * `bin` - A directory containing utility scripts to build the service and test
295
+ images. A script to extract the release notes for the most recent version, as
296
+ contained in `CHANGELOG.md` is also in this directory.
297
+
298
+ * `docker` - A directory containing the Dockerfiles for the service and test
299
+ images. It also contains `service_version.txt`, which contains the semantic
300
+ version number of the library and service image. Update this file with a new
301
+ version to trigger a release.
302
+
303
+ * `docs` - A directory with example usage notebooks.
304
+
305
+ * `hybig` - A directory containing Python source code for the HyBIG library.
306
+ This directory contains the business logic for generating GIBS compatible
307
+ browse images.
308
+
309
+ * `harmony_service` - A directory containing the Harmony Service specific
310
+ python code. `adapter.py` contains the `BrowseImageGeneratorAdapter` class
311
+ that is invoked by calls to the Harmony service.
312
+
313
+ * `tests` - A directory containing the service unit test suite.
314
+
315
+ * `CHANGELOG.md` - This file contains a record of changes applied to each new
316
+ release of HyBIG. Any release of a new version should have a record
317
+ of what was changed in this file.
318
+
319
+ * `CONTRIBUTING.md` - This file contains guidance for making contributions to
320
+ HyBIG, including recommended git best practices.
321
+
322
+ * `LICENSE` - Required for distribution under NASA open-source
323
+ approval. Details conditions for use, reproduction and distribution.
324
+
325
+ * `README.md` - This file, containing guidance on developing the library and
326
+ service.
327
+
328
+ * `dev-requirements.txt` - list of packages required for library and service
329
+ development.
330
+
331
+ * `legacy-CHANGELOG.md` - Notes for each version that was previously released
332
+ internally to EOSDIS, prior to open-source publication of the code and Docker
333
+ image.
334
+
335
+ * `pip_requirements.txt` - A list of service Python package dependencies.
336
+
337
+ * `pip_requirements_skip_snyk.txt` - A list of service Python package
338
+ dependencies that are not scanned by snyk for vulnerabilities. This file
339
+ contains only the `GDAL` package. It is separated because snyk's scanning is
340
+ naive and cannot pre-install required libraries so that `pip install GDAL`
341
+ fails and we have no work around.
342
+
343
+ * `pyproject.toml` - Configuration file used by packaging tools, as well as
344
+ other tools such as linters, type checkers, etc.
345
+
346
+
347
+ ## Local development:
348
+
349
+ Local testing of service functionality can be achieved via a local instance of
350
+ [Harmony](https://github.com/nasa/harmony). Please see instructions there
351
+ regarding creation of a local Harmony instance.
352
+
353
+ For local development and testing of library modifications or small functions
354
+ independent of the main Harmony application:
355
+
356
+ 1. Create a Python virtual environment
357
+ 1. Ensure GDAL libraries are accessable in the virtual environment.
358
+ 1. Install the dependencies in `pip_requirements.txt`, `pip_requirements_skip_snyk.txt` and `dev-requirements.txt`
359
+ 1. Install the pre-commit hooks.
360
+
361
+
362
+ ```
363
+ > conda create --name hybig-env python==3.11
364
+ > pip install -r pip_requirements.txt -r pip_requirements_skip_snyk.txt
365
+ > pip install -r dev-requirements.txt
366
+
367
+ > pre-commit install
368
+ ```
369
+
370
+
371
+ ## Tests:
372
+
373
+ This service utilises the Python `unittest` package to perform unit tests on
374
+ classes and functions in the service. After local development is complete, and
375
+ test have been updated, they can be run in Docker via:
376
+
377
+ ```bash
378
+ $ ./bin/build-image
379
+ $ ./bin/build-test
380
+ $ ./bin/run-test
381
+ ```
382
+
383
+ The `tests/run_tests.sh` script will also generate a coverage report, rendered
384
+ in HTML, and scan the code with `pylint`.
385
+
386
+ Currently, the `unittest` suite is run automatically within a GitHub workflow
387
+ as part of a CI/CD pipeline. These tests are run for all changes made in a PR
388
+ against the `main` branch. The tests must pass in order to merge the PR.
389
+
390
+ Unit tests are executed automatically by github actions on each Pull Request.
391
+
392
+
393
+ ## Versioning:
394
+
395
+ Docker service images and the hybig-py package library adhere to semantic
396
+ version numbers: major.minor.patch.
397
+
398
+ * Major increments: These are non-backwards compatible API changes.
399
+ * Minor increments: These are backwards compatible API changes.
400
+ * Patch increments: These updates do not affect the API to the service.
401
+
402
+ ## CI/CD:
403
+
404
+ The CI/CD for HyBIG is run on github actions with the workflows in the
405
+ `.github/workflows` directory:
406
+
407
+ * `run_lib_tests.yml` - A reusable workflow that tests the library functions
408
+ against the supported python versions.
409
+ * `run_service_tests.yml` - A reusable workflow that builds the service and
410
+ test Docker images, then runs the Python unit test suite in an instance of
411
+ the test Docker container.
412
+ * `run_tests_on_pull_requests.yml` - Triggered for all PRs against the `main`
413
+ branch. It runs the workflow in `run_service_tests.yml` and
414
+ `run_lib_tests.yml` to ensure all tests pass for the new code.
415
+ * `publish_docker_image.yml` - Triggered either manually or for commits to the
416
+ `main` branch that contain changes to the `docker/service_version.txt` file.
417
+ * `publish_to_pypi.yml` - Triggered either manually or for commits to the
418
+ `main` branch that contain changes to the `docker/service_version.txt`file.
419
+ * `publish_release.yml`<a name="release-workflow"></a> - workflow runs automatically when there is a change to
420
+ the `docker/service_version.txt` file on the main branch. This workflow will:
421
+ * Run the full unit test suite, to prevent publication of broken code.
422
+ * Extract the semantic version number from `docker/service_version.txt`.
423
+ * Extract the released notes for the most recent version from `CHANGELOG.md`.
424
+ * Build and deploy a this service's docker image to `ghcr.io`.
425
+ * Build the library package to be published to PyPI.
426
+ * Publish the package to PyPI.
427
+ * Publish a GitHub release under the semantic version number, with associated
428
+ git tag.
429
+
430
+
431
+ ## Releasing
432
+
433
+ A release consists of a new version hybig-py library published to PyPI and a
434
+ new Docker service image published to github's container repository.
435
+
436
+ A release is made automatically when a commit to the main branch contains a
437
+ changes in the `docker/service_version.txt` file, see the [publish_release](#release-workflow) workflow in the CI/CD section above.
438
+
439
+ Before merging a PR that will trigger a release, ensure these two files are updated:
440
+
441
+ * `CHANGELOG.md` - Notes should be added to capture the changes to the service.
442
+ * `docker/service_version.txt` - The semantic version number should be updated.
443
+
444
+ The `CHANGELOG.md` file requires a specific format for a new release, as it
445
+ looks for the following string to define the newest release of the code
446
+ (starting at the top of the file).
447
+
448
+ ```
449
+ ## [vX.Y.Z] - YYYY-MM-DD
450
+ ```
451
+
452
+ Where the markdown reference needs to be updated at the bottom of the file following the existing pattern.
453
+ ```
454
+ [unreleased]:https://github.com/nasa/harmony-browse-image-generator/compare/X.Y.Z..HEAD
455
+ [vX.Y.Z]:https://github.com/nasa/harmony-browse-image-generator/compare/X.Y.Y..X.Y.Z
456
+ ```
457
+
458
+ ### pre-commit hooks:
459
+
460
+ This repository uses [pre-commit](https://pre-commit.com/) to enable pre-commit
461
+ checking the repository for some coding standard best practices. These include:
462
+
463
+ * Removing trailing whitespaces.
464
+ * Removing blank lines at the end of a file.
465
+ * JSON files have valid formats.
466
+ * [ruff](https://github.com/astral-sh/ruff) Python linting checks.
467
+ * [black](https://black.readthedocs.io/en/stable/index.html) Python code
468
+ formatting checks.
469
+
470
+ To enable these checks locally:
471
+
472
+ ```bash
473
+ # Install pre-commit Python package as part of test requirements:
474
+ pip install -r tests/pip_test_requirements.txt
475
+
476
+ # Install the git hook scripts:
477
+ pre-commit install
478
+
479
+ # (Optional) Run against all files:
480
+ pre-commit run --all-files
481
+ ```
482
+
483
+ When you try to make a new commit locally, `pre-commit` will automatically run.
484
+ If any of the hooks detect non-compliance (e.g., trailing whitespace), that
485
+ hook will state it failed, and also try to fix the issue. You will need to
486
+ review and `git add` the changes before you can make a commit.
487
+
488
+ It is planned to implement additional hooks, possibly including tools such as
489
+ `mypy`.
490
+
491
+ [pre-commit.ci](pre-commit.ci) is configured such that these same hooks will be
492
+ automatically run for every pull request.
493
+
494
+ ## Releasing a new version of the service:
495
+
496
+ Once a new Docker image has been published with a new semantic version tag,
497
+ that service version can be released to a Harmony environment by following the
498
+ directions in the [Harmony Managing Existing Services
499
+ Guide](https://github.com/nasa/harmony/blob/main/docs/guides/managing-existing-services.md).
500
+
501
+ ## Get in touch:
502
+
503
+ You can reach out to the maintainers of this repository via email:
504
+
505
+ * david.p.auty@nasa.gov
506
+ * matthew.savoie@colorado.edu
507
+ * owen.m.littlejohns@nasa.gov