opticallyshallowdeep 1.2.0__tar.gz → 1.2.2__tar.gz

{opticallyshallowdeep-1.2.0/opticallyshallowdeep.egg-info → opticallyshallowdeep-1.2.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: opticallyshallowdeep
-Version: 1.2.0
+Version: 1.2.2
 Summary: Identify optically shallow and deep waters in satellite imagery
 Author: Yulun Wu
 Author-email: yulunwu8@gmail.com
@@ -10,7 +10,7 @@ Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: geopandas
 Requires-Dist: rasterio
-Requires-Dist: tifffile
+Requires-Dist: tifffile==2023.8.12
 Requires-Dist: netCDF4
 Requires-Dist: pyproj
 Requires-Dist: joblib
@@ -23,12 +23,13 @@ Requires-Dist: tensorflow
 
 This python tool delineates optically shallow and deep waters in Sentinel-2 imagery. The tool uses a deep neural network (DNN) that was trained on a diverse set of global images.
 
-Supported input includes L1C SAFE files and ACOLITE-processed L2R netCDF files. The output geotiff contains probabilities of water pixels being optically shallow and deep. 
+Supported input includes Level-1C (L1C) SAFE files and ACOLITE-processed L2R netCDF files. The output geotiff contains probabilities of water pixels being optically shallow and deep. 
 
-Originally coded by by Galen Richardson and Anders Knudby, modified and packaged by Yulun Wu
+**Home page:** <a href="https://github.com/yulunwu8/Optically-Shallow-Deep" target="_blank">https://github.com/yulunwu8/Optically-Shallow-Deep</a>
 
-Home page: <a href="https://github.com/yulunwu8/Optically-Shallow-Deep" target="_blank">https://github.com/yulunwu8/Optically-Shallow-Deep</a>
+**Publication:** Richardson, G., Foreman, N., Knudby, A., Wu, Y., & Lin, Y. (2024). Global deep learning model for delineation of optically shallow and optically deep water in Sentinel-2 imagery. *Remote Sensing of Environment*, 311, 114302. <a href="https://doi.org/10.1016/j.rse.2024.114302" target="_blank">https://doi.org/10.1016/j.rse.2024.114302</a>
 
+Originally coded by by Galen Richardson and Anders Knudby, modified and packaged by Yulun Wu
 
  
 ## Installation 
@@ -47,26 +48,17 @@ For mac OS:
 ```bash
 conda install -c apple tensorflow-deps
 python -m pip install tensorflow-macos
-
 ```
 
 
-For windows:
+For Windows and Linux:
 
 ```bash
 pip3 install tensorflow==2.13.0
-
 ```
 
-In case of compatibility issues, please try the newest version of tensorflow: 
 
-```bash
-pip3 install --upgrade --force-reinstall tensorflow
-
-```
-
-
-For Linux and more on installing tensorflow: [https://www.tensorflow.org/install](https://www.tensorflow.org/install)
+More on installing tensorflow: [https://www.tensorflow.org/install](https://www.tensorflow.org/install)
 
 
 **3 - Install opticallyshallowdeep:**
@@ -106,16 +98,23 @@ file_L2R = 'test_folder_in/L2R.nc'
 folder_out = 'folder/test_folder_out'
 
 # Run the OSW/ODW classifier 
-osd.run(file_in, folder_out, file_L2R=file_L2R)
+osd.run(file_L1C, folder_out, file_L2R=file_L2R)
 ```
 
-The L1C file is always required as it contains a cloud mask. Pixels within 8 pixels of the cloud mask are masked to reduce the impact of clouds. 
+The L1C file is always required as it contains a built-in cloud mask. Pixels within 8 pixels of the cloud mask are masked to reduce the impact of clouds. 
 
 
-Output is a 1-band geotiff, with values of prediction probability of OSW (100 means most likely OSW, 0 means most likely ODW). Non-water pixels are masked. It is recommended to use pixels between 0 and 40 as ODW, and pixels between 60 and 100 as OSW (publication in review).
+Output is a 1-band geotiff, with values of prediction probability of optically shallow water (OSW): 100 means most likely OSW, 0 means most likely optically deep water (ODW). Non-water pixels are masked. It is recommended to treat pixels between 0 and 40 as ODW, and pixels between 60 and 100 as OSW (Richardson et al., 2024).
 
 A log file, an intermediate multi-band geotiff, and a preview PNG are also generated in the output folder. They can be deleted after the processing. 
 
+
+**Sample Sentinel-2 scene and output:**
+
+<img src="images/TOA.jpeg"  height="500">
+
+<img src="images/OSW.jpeg"  height="500">
+
 ## Training, test, and validation data 
 
 All annotated shapefiles used in training, testing, and validating the DNN model are in the annotated_shapefiles folder, grouped by Sentinel-2 Scene ID.

{opticallyshallowdeep-1.2.0 → opticallyshallowdeep-1.2.2}/README.md

@@ -2,12 +2,13 @@
 
 This python tool delineates optically shallow and deep waters in Sentinel-2 imagery. The tool uses a deep neural network (DNN) that was trained on a diverse set of global images.
 
-Supported input includes L1C SAFE files and ACOLITE-processed L2R netCDF files. The output geotiff contains probabilities of water pixels being optically shallow and deep. 
+Supported input includes Level-1C (L1C) SAFE files and ACOLITE-processed L2R netCDF files. The output geotiff contains probabilities of water pixels being optically shallow and deep. 
 
-Originally coded by by Galen Richardson and Anders Knudby, modified and packaged by Yulun Wu
+**Home page:** <a href="https://github.com/yulunwu8/Optically-Shallow-Deep" target="_blank">https://github.com/yulunwu8/Optically-Shallow-Deep</a>
 
-Home page: <a href="https://github.com/yulunwu8/Optically-Shallow-Deep" target="_blank">https://github.com/yulunwu8/Optically-Shallow-Deep</a>
+**Publication:** Richardson, G., Foreman, N., Knudby, A., Wu, Y., & Lin, Y. (2024). Global deep learning model for delineation of optically shallow and optically deep water in Sentinel-2 imagery. *Remote Sensing of Environment*, 311, 114302. <a href="https://doi.org/10.1016/j.rse.2024.114302" target="_blank">https://doi.org/10.1016/j.rse.2024.114302</a>
 
+Originally coded by by Galen Richardson and Anders Knudby, modified and packaged by Yulun Wu
 
  
 ## Installation 
@@ -26,26 +27,17 @@ For mac OS:
 ```bash
 conda install -c apple tensorflow-deps
 python -m pip install tensorflow-macos
-
 ```
 
 
-For windows:
+For Windows and Linux:
 
 ```bash
 pip3 install tensorflow==2.13.0
-
 ```
 
-In case of compatibility issues, please try the newest version of tensorflow: 
 
-```bash
-pip3 install --upgrade --force-reinstall tensorflow
-
-```
-
-
-For Linux and more on installing tensorflow: [https://www.tensorflow.org/install](https://www.tensorflow.org/install)
+More on installing tensorflow: [https://www.tensorflow.org/install](https://www.tensorflow.org/install)
 
 
 **3 - Install opticallyshallowdeep:**
@@ -85,16 +77,23 @@ file_L2R = 'test_folder_in/L2R.nc'
 folder_out = 'folder/test_folder_out'
 
 # Run the OSW/ODW classifier 
-osd.run(file_in, folder_out, file_L2R=file_L2R)
+osd.run(file_L1C, folder_out, file_L2R=file_L2R)
 ```
 
-The L1C file is always required as it contains a cloud mask. Pixels within 8 pixels of the cloud mask are masked to reduce the impact of clouds. 
+The L1C file is always required as it contains a built-in cloud mask. Pixels within 8 pixels of the cloud mask are masked to reduce the impact of clouds. 
 
 
-Output is a 1-band geotiff, with values of prediction probability of OSW (100 means most likely OSW, 0 means most likely ODW). Non-water pixels are masked. It is recommended to use pixels between 0 and 40 as ODW, and pixels between 60 and 100 as OSW (publication in review).
+Output is a 1-band geotiff, with values of prediction probability of optically shallow water (OSW): 100 means most likely OSW, 0 means most likely optically deep water (ODW). Non-water pixels are masked. It is recommended to treat pixels between 0 and 40 as ODW, and pixels between 60 and 100 as OSW (Richardson et al., 2024).
 
 A log file, an intermediate multi-band geotiff, and a preview PNG are also generated in the output folder. They can be deleted after the processing. 
 
+
+**Sample Sentinel-2 scene and output:**
+
+<img src="images/TOA.jpeg"  height="500">
+
+<img src="images/OSW.jpeg"  height="500">
+
 ## Training, test, and validation data 
 
 All annotated shapefiles used in training, testing, and validating the DNN model are in the annotated_shapefiles folder, grouped by Sentinel-2 Scene ID.

opticallyshallowdeep-1.2.2/opticallyshallowdeep/__init__.py

@@ -0,0 +1 @@
+from .run import *

{opticallyshallowdeep-1.2.0 → opticallyshallowdeep-1.2.2}/opticallyshallowdeep/check_transpose.py

@@ -1,10 +1,7 @@
-
 # Always output row, column, band 
-
 def check_transpose(img):
     #if the #of bands is greater than the number of x or y cords
     y,x,b=img.shape 
     if b>y or b>x:
         img=img.transpose(1,2,0)
-    return img    
-
+    return img

{opticallyshallowdeep-1.2.0 → opticallyshallowdeep-1.2.2}/opticallyshallowdeep/cloud_mask.py

@@ -1,7 +1,3 @@
-
-
-
-
 import os, sys
 import rasterio
 import numpy as np
@@ -99,5 +95,4 @@ def cloud_mask(file_L1C, buffer_size = 8):
     mask_cloud_buffered = ndimage.binary_dilation(mask_cloud, structure=struct1,iterations=buffer_size).astype(mask_cloud.dtype)
     
     print('Done')
-    return mask_cloud_buffered
-
+    return mask_cloud_buffered

{opticallyshallowdeep-1.2.0 → opticallyshallowdeep-1.2.2}/opticallyshallowdeep/find_epsg.py

@@ -1,5 +1,3 @@
-
-
 def find_epsg(data):
     
     # Use regex to extract the desired part
@@ -29,25 +27,8 @@ def find_epsg(data):
             # return f"The EPSG code for '{crs_name}' is {epsg_code}."
             return epsg_code
         
-        
         else:
             print( "EPSG code could not be found.")
         
-        
     except Exception as e:
         return f"An error occurred: {str(e)}"
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-

{opticallyshallowdeep-1.2.0 → opticallyshallowdeep-1.2.2}/opticallyshallowdeep/make_multiband_image.py

@@ -54,5 +54,3 @@ def make_multiband_image(file_in,folder_out):
         print('Done')
 
     return imageFile
-
-

{opticallyshallowdeep-1.2.0 → opticallyshallowdeep-1.2.2}/opticallyshallowdeep/make_vertical_strips.py

@@ -1,8 +1,4 @@
-
-
 import numpy as np
-
-
 def make_vertical_strips(full_img):
     '''use to save ram, process bigger images faster, and it overlaps so middle image is not 
     distorted from how edge pixels are handled'''
@@ -30,5 +26,4 @@ def make_vertical_strips(full_img):
         import sys
         sys.exit('Unknown dimension(s) of input imagery to be splited into strips')
           
-    return [strip1,strip2,strip3,strip4,strip5]
-
+    return [strip1,strip2,strip3,strip4,strip5]

{opticallyshallowdeep-1.2.0 → opticallyshallowdeep-1.2.2}/opticallyshallowdeep/netcdf_to_multiband_geotiff.py

@@ -1,4 +1,3 @@
-    
 import os
 import numpy as np
 import rasterio
@@ -7,9 +6,7 @@ from rasterio.crs import CRS
 from rasterio.transform import from_origin
 import netCDF4 as nc4
 # from pyproj import Proj, transform
-
 import pyproj
-
 from .find_epsg import find_epsg
 
 def netcdf_to_multiband_geotiff(netcdf_file, folder_out):
@@ -87,5 +84,4 @@ def netcdf_to_multiband_geotiff(netcdf_file, folder_out):
                 
         print('Done')
     
-    return output_geotiff_file
-    
+    return output_geotiff_file

{opticallyshallowdeep-1.2.0 → opticallyshallowdeep-1.2.2}/opticallyshallowdeep/parse_string.py

@@ -1,6 +1,4 @@
-
 import re
-
 def parse_string(s):
     if s.lower() in ["lat", "long", "lat_abs"]:
         return [s.lower()]#parses the column names for our model and makes into things that our scripts can interpret
@@ -19,4 +17,4 @@ def parse_string(s):
             first_group_int = first_group
         return [first_group_int, int(groups[2]), groups[4]] if groups[4] else [first_group_int, int(groups[2]), None]
     else:
-        return None
+        return None

{opticallyshallowdeep-1.2.0 → opticallyshallowdeep-1.2.2}/opticallyshallowdeep/process_as_strips.py

@@ -1,4 +1,3 @@
-
 import os, gc, warnings, math
 import numpy as np
 import pandas as pd
@@ -20,7 +19,6 @@ tf.compat.v1.logging.set_verbosity(tf.compat.v1.logging.ERROR)
 
 from .make_vertical_strips import make_vertical_strips
 
-
 def process_as_strips (full_img, image_path, if_SR, model_path, selected_columns, model_columns, file_in, cloud_list):
     striplist=make_vertical_strips(full_img) #create a list of strips with overlap
     RGBlist=[]
@@ -57,9 +55,7 @@ def process_img_to_rgb(img, file_path, if_SR, model_path, selected_columns, mode
         gc.collect()
         return RGB_img #this image is 0: OSW/ODW, 1:pred/prob, 2: Mask
 
-
 def correct_baseline(img,file_path, if_SR, file_in):
-    
     from xml.dom import minidom
     
     # if ACOLITE input 
@@ -160,7 +156,6 @@ def get_water_pix_coord(img,correction, if_SR, img_cloud):
     gc.collect()
     return common_coordinates_list
 
-
 def make_blank_img(img):
     Y_b, X_b, b = img.shape #sometimes the image is all no data or the correction value, in this instance, we make a blank image
     RGB_img = np.zeros((Y_b, X_b, 3), dtype=np.uint8)
@@ -171,7 +166,6 @@ def make_blank_img(img):
 def time_tracker(start_time):
     return "{}sec".format(round((datetime.now() - start_time).total_seconds(), 2))
 
-
 def process_image_with_filters(img, selected_columns):
     height, width, bands = img.shape#the output of this is an image of the filters required for this model
     filter_list = [value for value in selected_columns if value not in [["lat"], ["long"], ["lat_abs"]]]
@@ -179,13 +173,15 @@ def process_image_with_filters(img, selected_columns):
     for band, kernel_size, filter_type in filter_list:
         
         if filter_type is None:
-            filtered_band = img[:, :, band].astype(np.uint16)#this means it is a single pixel
+            filtered_band = img[:, :, band]
         else:
             with warnings.catch_warnings():
                 warnings.simplefilter("ignore", category=RuntimeWarning)
                 filtered_band = apply_filter(img[:, :, band].astype(np.float32), kernel_size, filter_type)
                 filtered_band[filtered_band==-32768] = 32768
-                filtered_band = filtered_band.astype(np.uint16)
+            
+        filtered_band[filtered_band<0] = 0
+        filtered_band = filtered_band.astype(np.uint16)#this means it is a single pixel
             
         output_bands.append(filtered_band)#append to list of filters
         del filtered_band
@@ -351,7 +347,6 @@ def load_model_and_predict_pixels(value_list, model_path, cord_list, if_SR):
 
     return cord_list.tolist(), pred_results.tolist(), con_1.tolist()
 
-
 def load_tf_model(model_path, if_SR):
     if if_SR == False:
         model=d6_model(32,0.01)
@@ -420,16 +415,4 @@ def plot_RGB_img(RGB_img, image_path):
     # plt.show()
     
     out_path = image_path.replace('.tif','.png')
-    plt.savefig(out_path)
-
-
-
-
-
-
-
-
-
-
-
-
+    plt.savefig(out_path)

{opticallyshallowdeep-1.2.0 → opticallyshallowdeep-1.2.2}/opticallyshallowdeep/run.py

@@ -1,4 +1,3 @@
-
 import sys, os, gc, time
 import tifffile as tif
 from importlib.metadata import version
@@ -14,9 +13,6 @@ from .netcdf_to_multiband_geotiff import netcdf_to_multiband_geotiff
 from .make_vertical_strips import make_vertical_strips
 from .cloud_mask import cloud_mask
 
-
-
-
 def run(file_L1C, folder_out, file_L2R = None, to_log=True):
     
     ### Check the two 
@@ -32,7 +28,11 @@ def run(file_L1C, folder_out, file_L2R = None, to_log=True):
         # Start logging in txt file
         orig_stdout = sys.stdout
         
-        log_base = os.path.basename(file_L1C).replace('.safe','.txt').replace('.SAFE','.txt')
+        if file_L2R is None: 
+            log_base = os.path.basename(file_L1C).replace('.safe','.txt').replace('.SAFE','.txt')
+        else:
+            log_base = os.path.basename(file_L2R).replace('.nc','.txt')
+        
         log_base = 'OSD_log_'+ log_base
         log_file = os.path.join(folder_out,log_base)
 

{opticallyshallowdeep-1.2.0 → opticallyshallowdeep-1.2.2}/opticallyshallowdeep/write_georef_image.py

@@ -1,4 +1,3 @@
-
 import rasterio, gc
 import numpy as np
 
@@ -23,5 +22,4 @@ def write_georef_image(image_path,RGB_img):
         dst.write(output_band, 1) 
         
     del raster_with_ref
-    gc.collect()
-    
+    gc.collect()

{opticallyshallowdeep-1.2.0 → opticallyshallowdeep-1.2.2/opticallyshallowdeep.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: opticallyshallowdeep
-Version: 1.2.0
+Version: 1.2.2
 Summary: Identify optically shallow and deep waters in satellite imagery
 Author: Yulun Wu
 Author-email: yulunwu8@gmail.com
@@ -10,7 +10,7 @@ Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: geopandas
 Requires-Dist: rasterio
-Requires-Dist: tifffile
+Requires-Dist: tifffile==2023.8.12
 Requires-Dist: netCDF4
 Requires-Dist: pyproj
 Requires-Dist: joblib
@@ -23,12 +23,13 @@ Requires-Dist: tensorflow
 
 This python tool delineates optically shallow and deep waters in Sentinel-2 imagery. The tool uses a deep neural network (DNN) that was trained on a diverse set of global images.
 
-Supported input includes L1C SAFE files and ACOLITE-processed L2R netCDF files. The output geotiff contains probabilities of water pixels being optically shallow and deep. 
+Supported input includes Level-1C (L1C) SAFE files and ACOLITE-processed L2R netCDF files. The output geotiff contains probabilities of water pixels being optically shallow and deep. 
 
-Originally coded by by Galen Richardson and Anders Knudby, modified and packaged by Yulun Wu
+**Home page:** <a href="https://github.com/yulunwu8/Optically-Shallow-Deep" target="_blank">https://github.com/yulunwu8/Optically-Shallow-Deep</a>
 
-Home page: <a href="https://github.com/yulunwu8/Optically-Shallow-Deep" target="_blank">https://github.com/yulunwu8/Optically-Shallow-Deep</a>
+**Publication:** Richardson, G., Foreman, N., Knudby, A., Wu, Y., & Lin, Y. (2024). Global deep learning model for delineation of optically shallow and optically deep water in Sentinel-2 imagery. *Remote Sensing of Environment*, 311, 114302. <a href="https://doi.org/10.1016/j.rse.2024.114302" target="_blank">https://doi.org/10.1016/j.rse.2024.114302</a>
 
+Originally coded by by Galen Richardson and Anders Knudby, modified and packaged by Yulun Wu
 
  
 ## Installation 
@@ -47,26 +48,17 @@ For mac OS:
 ```bash
 conda install -c apple tensorflow-deps
 python -m pip install tensorflow-macos
-
 ```
 
 
-For windows:
+For Windows and Linux:
 
 ```bash
 pip3 install tensorflow==2.13.0
-
 ```
 
-In case of compatibility issues, please try the newest version of tensorflow: 
 
-```bash
-pip3 install --upgrade --force-reinstall tensorflow
-
-```
-
-
-For Linux and more on installing tensorflow: [https://www.tensorflow.org/install](https://www.tensorflow.org/install)
+More on installing tensorflow: [https://www.tensorflow.org/install](https://www.tensorflow.org/install)
 
 
 **3 - Install opticallyshallowdeep:**
@@ -106,16 +98,23 @@ file_L2R = 'test_folder_in/L2R.nc'
 folder_out = 'folder/test_folder_out'
 
 # Run the OSW/ODW classifier 
-osd.run(file_in, folder_out, file_L2R=file_L2R)
+osd.run(file_L1C, folder_out, file_L2R=file_L2R)
 ```
 
-The L1C file is always required as it contains a cloud mask. Pixels within 8 pixels of the cloud mask are masked to reduce the impact of clouds. 
+The L1C file is always required as it contains a built-in cloud mask. Pixels within 8 pixels of the cloud mask are masked to reduce the impact of clouds. 
 
 
-Output is a 1-band geotiff, with values of prediction probability of OSW (100 means most likely OSW, 0 means most likely ODW). Non-water pixels are masked. It is recommended to use pixels between 0 and 40 as ODW, and pixels between 60 and 100 as OSW (publication in review).
+Output is a 1-band geotiff, with values of prediction probability of optically shallow water (OSW): 100 means most likely OSW, 0 means most likely optically deep water (ODW). Non-water pixels are masked. It is recommended to treat pixels between 0 and 40 as ODW, and pixels between 60 and 100 as OSW (Richardson et al., 2024).
 
 A log file, an intermediate multi-band geotiff, and a preview PNG are also generated in the output folder. They can be deleted after the processing. 
 
+
+**Sample Sentinel-2 scene and output:**
+
+<img src="images/TOA.jpeg"  height="500">
+
+<img src="images/OSW.jpeg"  height="500">
+
 ## Training, test, and validation data 
 
 All annotated shapefiles used in training, testing, and validating the DNN model are in the annotated_shapefiles folder, grouped by Sentinel-2 Scene ID.

{opticallyshallowdeep-1.2.0 → opticallyshallowdeep-1.2.2}/opticallyshallowdeep.egg-info/requires.txt

@@ -1,6 +1,6 @@
 geopandas
 rasterio
-tifffile
+tifffile==2023.8.12
 netCDF4
 pyproj
 joblib

{opticallyshallowdeep-1.2.0 → opticallyshallowdeep-1.2.2}/setup.py

@@ -5,7 +5,7 @@ with open("readme.md", "r") as fh:
 
 setup(
     name='opticallyshallowdeep',
-    version='1.2.0',
+    version='1.2.2',
     author='Yulun Wu',
     author_email='yulunwu8@gmail.com',
     description='Identify optically shallow and deep waters in satellite imagery',
@@ -18,7 +18,7 @@ setup(
         'Programming Language :: Python :: 3'
     ],
     python_requires='>=3.8',
-    install_requires=['geopandas','rasterio','tifffile','netCDF4','pyproj',
+    install_requires=['geopandas','rasterio','tifffile==2023.8.12','netCDF4','pyproj',
                       'joblib','scipy','matplotlib','imagecodecs','tensorflow']
 )
 

opticallyshallowdeep-1.2.0/opticallyshallowdeep/__init__.py

@@ -1,2 +0,0 @@
-from .run import *
-