goesgcp 2.0.2__tar.gz → 2.0.4__tar.gz

{goesgcp-2.0.2 → goesgcp-2.0.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.2
 Name: goesgcp
-Version: 2.0.2
+Version: 2.0.4
 Summary: A package to download and process GOES-16/17 data
 Home-page: https://github.com/helvecioneto/goesgcp
 Author: Helvecio B. L. Neto
@@ -41,18 +41,18 @@ Dynamic: summary
 <!-- badges: end -->
 
 
-`goesgcp` is a Python utility designed for downloading and reprojecting GOES-R satellite data. This script leverages the `google.cloud` library for accessing data from the Google Cloud Platform (GCP) and `pyproj` for reprojecting data to EPSG:4326, as well as cropping it to a user-defined bounding box.
+`goesgcp` is a Python utility designed for downloading and reprojecting GOES-R satellite data. This script leverages the `google.cloud` library for accessing data from the Google Cloud Platform (GCP) and `rioxarray` for reprojecting data to EPSG:4326 (rectangular grid), as well cropping it to a user-defined bounding box.
 
 ## Features
 
-- **Download GOES-R satellite data**: Supports GOES-16 and GOES-17.
+- **Download GOES-R satellite data**: Supports GOES-16 and GOES-18.
 - **Reprojection and cropping**: Reprojects data to EPSG:4326 and crops to a specified bounding box.
 - **Flexible command-line interface**: Customize download options, variables, channels, time range, and output format.
 - **Efficient processing**: Handles large datasets with optimized performance.
 
 ## Installation
 
-Install the necessary dependencies via `pip`:
+Install the package via `pip`:
 
 ```bash
 pip install goesgcp
@@ -61,7 +61,7 @@ pip install goesgcp
 
 ## Usage
 
-### Command-Line Arguments
+### Available Command-Line Arguments
 
 The script uses the `argparse` module for handling command-line arguments. Below are the available options:
 
@@ -97,14 +97,14 @@ A comprehensive list of available GOES products can be found at the following li
 In the example below, the command downloads the 3 most recent files from the GOES-16 satellite for the product ABI-L2-CMIPF. It focuses on the variable CMI (Cloud and Moisture Imagery) from channel 13, which is commonly used for infrared observations. The downloaded files are saved to the specified output directory output/.
 
 ```bash
-goesgcp --satellite goes-16 --product ABI-L2-CMIPF --recent 3 --output "output/"
+goesgcp --satellite goes-16 --recent 3 --product ABI-L2-CMIPF --var_name CMI --channel 13 --output "output/"
 ```
 
 #### Download Data for a Specific Time Range
-This command retrieves GOES-16 satellite data for the product ABI-L2-CMIPF within the date range 2022-12-15 00:00:00 to 2022-12-20 10:00:00, focusing on hours 5:00 and 6:00 AM. The data is cropped to the geographic bounds of -35° to 5° latitude and -80° to -30° longitude, reprojected with a resolution of 0.045 degrees, and saved in a by_date format for easy organization.
+This command retrieves GOES-16 satellite data for the product ABI-L2-CMIPF within the date range 2022-12-15 00:00:00 to 2022-12-15 12:00:00, focusing on hours 5:00 and 6:00 AM. The data is cropped to the geographic bounds of -35° to 5° latitude and -80° to -30° longitude, reprojected with a resolution of 0.045 degrees, and saved in a by_date format for easy organization.
 
 ```bash
-goesgcp --satellite goes-16 --product ABI-L2-CMIPF --start '2022-12-15 00:00:00' --end '2022-12-20 10:00:00' --bt_hour 5 6 --save_format by_date --resolution 0.045 --lat_min -35 --lat_max 5 --lon_min -80 --lon_max -30
+goesgcp --satellite goes-16 --product ABI-L2-CMIPF --start "2022-12-15 00:00:00" --end "2022-12-15 12:00:00" --bt_hour 5 6 --save_format by_date --resolution 0.045 --lat_min -35 --lat_max 5 --lon_min -80 --lon_max -30
 ```
 
 ### Contributing

{goesgcp-2.0.2 → goesgcp-2.0.4}/README.md

@@ -7,18 +7,18 @@
 <!-- badges: end -->
 
 
-`goesgcp` is a Python utility designed for downloading and reprojecting GOES-R satellite data. This script leverages the `google.cloud` library for accessing data from the Google Cloud Platform (GCP) and `pyproj` for reprojecting data to EPSG:4326, as well as cropping it to a user-defined bounding box.
+`goesgcp` is a Python utility designed for downloading and reprojecting GOES-R satellite data. This script leverages the `google.cloud` library for accessing data from the Google Cloud Platform (GCP) and `rioxarray` for reprojecting data to EPSG:4326 (rectangular grid), as well cropping it to a user-defined bounding box.
 
 ## Features
 
-- **Download GOES-R satellite data**: Supports GOES-16 and GOES-17.
+- **Download GOES-R satellite data**: Supports GOES-16 and GOES-18.
 - **Reprojection and cropping**: Reprojects data to EPSG:4326 and crops to a specified bounding box.
 - **Flexible command-line interface**: Customize download options, variables, channels, time range, and output format.
 - **Efficient processing**: Handles large datasets with optimized performance.
 
 ## Installation
 
-Install the necessary dependencies via `pip`:
+Install the package via `pip`:
 
 ```bash
 pip install goesgcp
@@ -27,7 +27,7 @@ pip install goesgcp
 
 ## Usage
 
-### Command-Line Arguments
+### Available Command-Line Arguments
 
 The script uses the `argparse` module for handling command-line arguments. Below are the available options:
 
@@ -63,14 +63,14 @@ A comprehensive list of available GOES products can be found at the following li
 In the example below, the command downloads the 3 most recent files from the GOES-16 satellite for the product ABI-L2-CMIPF. It focuses on the variable CMI (Cloud and Moisture Imagery) from channel 13, which is commonly used for infrared observations. The downloaded files are saved to the specified output directory output/.
 
 ```bash
-goesgcp --satellite goes-16 --product ABI-L2-CMIPF --recent 3 --output "output/"
+goesgcp --satellite goes-16 --recent 3 --product ABI-L2-CMIPF --var_name CMI --channel 13 --output "output/"
 ```
 
 #### Download Data for a Specific Time Range
-This command retrieves GOES-16 satellite data for the product ABI-L2-CMIPF within the date range 2022-12-15 00:00:00 to 2022-12-20 10:00:00, focusing on hours 5:00 and 6:00 AM. The data is cropped to the geographic bounds of -35° to 5° latitude and -80° to -30° longitude, reprojected with a resolution of 0.045 degrees, and saved in a by_date format for easy organization.
+This command retrieves GOES-16 satellite data for the product ABI-L2-CMIPF within the date range 2022-12-15 00:00:00 to 2022-12-15 12:00:00, focusing on hours 5:00 and 6:00 AM. The data is cropped to the geographic bounds of -35° to 5° latitude and -80° to -30° longitude, reprojected with a resolution of 0.045 degrees, and saved in a by_date format for easy organization.
 
 ```bash
-goesgcp --satellite goes-16 --product ABI-L2-CMIPF --start '2022-12-15 00:00:00' --end '2022-12-20 10:00:00' --bt_hour 5 6 --save_format by_date --resolution 0.045 --lat_min -35 --lat_max 5 --lon_min -80 --lon_max -30
+goesgcp --satellite goes-16 --product ABI-L2-CMIPF --start "2022-12-15 00:00:00" --end "2022-12-15 12:00:00" --bt_hour 5 6 --save_format by_date --resolution 0.045 --lat_min -35 --lat_max 5 --lon_min -80 --lon_max -30
 ```
 
 ### Contributing

{goesgcp-2.0.2 → goesgcp-2.0.4}/goesgcp/main.py

@@ -34,7 +34,7 @@ def get_directory_prefix(year, julian_day, hour):
 
 
 def get_files_period(connection, bucket_name, base_prefix, pattern, 
-                     start, end, bt_hour=[0, 23], bt_min=[0, 60], freq='10 min'):
+                     start, end, bt_hour=[], bt_min=[], freq=None):
     """
     Fetches files from a GCP bucket within a specified time period and returns them as a DataFrame.
 
@@ -82,14 +82,14 @@ def get_files_period(connection, bucket_name, base_prefix, pattern,
     # Create a DataFrame from the list of files
     df = pd.DataFrame(files_metadata)
 
+    if df.empty:
+        print("No files found matching the pattern and time range.")
+        print(prefix)
+        sys.exit(1)
+    
     # Transform file_name to datetime
     df['last_modified'] = pd.to_datetime(df['file_name'].str.extract(r'(\d{4}\d{3}\d{2}\d{2})').squeeze(), format='%Y%j%H%M')
 
-
-    if df.empty:
-        print("No files found matching the pattern.")
-        return pd.DataFrame()
-    
     # Ensure 'last_modified' is in the correct datetime format without timezone
     df['last_modified'] = pd.to_datetime(df['last_modified']).dt.tz_localize('UTC')
 
@@ -97,16 +97,19 @@ def get_files_period(connection, bucket_name, base_prefix, pattern,
     df = df[(df['last_modified'] >= start) & (df['last_modified'] <= end)]
 
     # Filter the DataFrame based on the hour range
-    df['hour'] = df['last_modified'].dt.hour
-    df = df[(df['hour'] >= bt_hour[0]) & (df['hour'] <= bt_hour[1])]
+    if len(bt_hour) > 1:
+        df['hour'] = df['last_modified'].dt.hour
+        df = df[(df['hour'] >= bt_hour[0]) & (df['hour'] <= bt_hour[1])]
 
     # Filter the DataFrame based on the minute range
-    df['minute'] = df['last_modified'].dt.minute
-    df = df[(df['minute'] >= bt_min[0]) & (df['minute'] <= bt_min[1])]
+    if len(bt_min) > 1:
+        df['minute'] = df['last_modified'].dt.minute
+        df = df[(df['minute'] >= bt_min[0]) & (df['minute'] <= bt_min[1])]
 
     # Filter the DataFrame based on the frequency
-    df['freq'] = df['last_modified'].dt.floor(freq)
-    df = df.groupby('freq').first().reset_index()
+    if freq is not None:
+        df['freq'] = df['last_modified'].dt.floor(freq)
+        df = df.groupby('freq').first().reset_index()
 
     return df['file_name'].tolist()
 
@@ -139,7 +142,7 @@ def get_recent_files(connection, bucket_name, base_prefix, pattern, min_files):
 
         # Filter blobs based on the pattern
         for blob in blobs:
-            if pattern in blob.name:  # You can use "re" here for more complex patterns
+            if pattern in blob.name:
                 files.append((blob.name, blob.updated))
 
         # Go back one hour
@@ -221,23 +224,27 @@ def crop_reproject(args):
     except:
         pass
 
-    # Reproject to EPSG:4326
-    ds = ds.rio.reproject("EPSG:4326", resolution=resolution)
+    try:
+        # Reproject to EPSG:4326
+        ds = ds.rio.reproject("EPSG:4326", resolution=resolution)
 
-    # Rename lat/lon coordinates
-    ds = ds.rename({"x": "lon", "y": "lat"})
+        # Rename lat/lon coordinates
+        ds = ds.rename({"x": "lon", "y": "lat"})
 
-    # Add resolution to attributes
-    for var in var_names:
-        ds[var].attrs['resolution'] = "x={:.2f} y={:.2f} degree".format(resolution, resolution)
-        ds[var].attrs['comments'] = 'Cropped and reprojected to EPSG:4326 by goesgcp'
+        # Add resolution to attributes
+        for var in var_names:
+            ds[var].attrs['resolution'] = "x={:.2f} y={:.2f} degree".format(resolution, resolution)
+            ds[var].attrs['comments'] = 'Cropped and reprojected to EPSG:4326 by goesgcp'
 
-    # Crop using lat/lon coordinates, in parallel
-    ds = ds.rio.clip_box(minx=lon_min, miny=lat_min, maxx=lon_max, maxy=lat_max)
+        # Crop using lat/lon coordinates, in parallel
+        ds = ds.rio.clip_box(minx=lon_min, miny=lat_min, maxx=lon_max, maxy=lat_max)
 
-    # Add global metadata comments
-    ds.attrs['comments'] = "Data processed by goesgcp, author: Helvecio B. L. Neto (helvecioblneto@gmail.com)"
-        
+        # Add global metadata comments
+        ds.attrs['comments'] = "Data processed by goesgcp, author: Helvecio B. L. Neto (helvecioblneto@gmail.com)"
+    except Exception as e:
+        print(f"Error processing file {file}: {e}")
+        pass    
+    
     if save_format == 'by_date':
         file_datetime = datetime.strptime(ds.time_coverage_start, 
                                           "%Y-%m-%dT%H:%M:%S.%fZ")
@@ -253,13 +260,14 @@ def crop_reproject(args):
         output_directory = f"{output}{year}/{julian_day}/"
     else:
         output_directory = output
+        
 
     # Create the output directory
     pathlib.Path(output_directory).mkdir(parents=True, exist_ok=True)
 
     # Save the file
     output_file = f"{output_directory}{file.split('/')[-1]}"
-    ds.to_netcdf(output_file, mode='w', format='NETCDF4_CLASSIC')
+    ds.to_netcdf(output_file, mode='w')
 
     ds.close()
 
@@ -313,7 +321,7 @@ def main():
 
     - To download files from the GOES-16 satellite for the ABI-L2-CMIPF product between 2022-12-15 and 2022-12-20:
 
-    goesgcp --satellite goes-16 --product ABI-L2-CMIPF --start '2022-12-15 09:00:00' --end '2022-12-15 09:50:00' --resolution 0.045 --lat_min -35 --lat_max 5 --lon_min -80 --lon_max -30
+    goesgcp --satellite goes-16 --product ABI-L2-CMIPF --start "2022-12-15 09:00:00" --end "2022-12-15 09:50:00" --resolution 0.045 --lat_min -35 --lat_max 5 --lon_min -80 --lon_max -30
 
     """
 
@@ -345,7 +353,7 @@ def main():
     parser.add_argument('--product', type=str, default='ABI-L2-CMIPF', help='Name of the satellite product', choices=product_names)
     parser.add_argument('--var_name', type=str, default=None, help='Variable name to extract (e.g., CMI)')
     parser.add_argument('--channel', type=int, default=13, help='Channel to use (e.g., 13)')
-    parser.add_argument('--op_mode', type=str, default='M6C', help='Operational mode to use (e.g., M6C)')
+    parser.add_argument('--op_mode', type=str, default='M6', help='Operational mode to use (e.g., M6C)')
 
     # Recent files settings
     parser.add_argument('--recent', type=int, help='Number of recent files to download (e.g., 3)')
@@ -385,7 +393,7 @@ def main():
     satellite = args.satellite
     product = args.product
     op_mode = args.op_mode
-    channel = str(args.channel).zfill(2)
+    channel = args.channel
     var_name = args.var_name
     lat_min = args.lat_min
     lat_max = args.lat_max
@@ -421,6 +429,13 @@ def main():
         print(f"Bucket {bucket_name} not found. Exiting...")
         sys.exit(1)
 
+    # Check if the channel exists
+    if not channel:
+        channel = ''
+    else:
+        channel = str(channel).zfill(2)
+        channel = f"C{channel}"
+
     # Set pattern for the files
     pattern = "OR_"+product+"-"+op_mode+channel+"_G" + satellite[-2:]
 

{goesgcp-2.0.2 → goesgcp-2.0.4}/goesgcp.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.2
 Name: goesgcp
-Version: 2.0.2
+Version: 2.0.4
 Summary: A package to download and process GOES-16/17 data
 Home-page: https://github.com/helvecioneto/goesgcp
 Author: Helvecio B. L. Neto
@@ -41,18 +41,18 @@ Dynamic: summary
 <!-- badges: end -->
 
 
-`goesgcp` is a Python utility designed for downloading and reprojecting GOES-R satellite data. This script leverages the `google.cloud` library for accessing data from the Google Cloud Platform (GCP) and `pyproj` for reprojecting data to EPSG:4326, as well as cropping it to a user-defined bounding box.
+`goesgcp` is a Python utility designed for downloading and reprojecting GOES-R satellite data. This script leverages the `google.cloud` library for accessing data from the Google Cloud Platform (GCP) and `rioxarray` for reprojecting data to EPSG:4326 (rectangular grid), as well cropping it to a user-defined bounding box.
 
 ## Features
 
-- **Download GOES-R satellite data**: Supports GOES-16 and GOES-17.
+- **Download GOES-R satellite data**: Supports GOES-16 and GOES-18.
 - **Reprojection and cropping**: Reprojects data to EPSG:4326 and crops to a specified bounding box.
 - **Flexible command-line interface**: Customize download options, variables, channels, time range, and output format.
 - **Efficient processing**: Handles large datasets with optimized performance.
 
 ## Installation
 
-Install the necessary dependencies via `pip`:
+Install the package via `pip`:
 
 ```bash
 pip install goesgcp
@@ -61,7 +61,7 @@ pip install goesgcp
 
 ## Usage
 
-### Command-Line Arguments
+### Available Command-Line Arguments
 
 The script uses the `argparse` module for handling command-line arguments. Below are the available options:
 
@@ -97,14 +97,14 @@ A comprehensive list of available GOES products can be found at the following li
 In the example below, the command downloads the 3 most recent files from the GOES-16 satellite for the product ABI-L2-CMIPF. It focuses on the variable CMI (Cloud and Moisture Imagery) from channel 13, which is commonly used for infrared observations. The downloaded files are saved to the specified output directory output/.
 
 ```bash
-goesgcp --satellite goes-16 --product ABI-L2-CMIPF --recent 3 --output "output/"
+goesgcp --satellite goes-16 --recent 3 --product ABI-L2-CMIPF --var_name CMI --channel 13 --output "output/"
 ```
 
 #### Download Data for a Specific Time Range
-This command retrieves GOES-16 satellite data for the product ABI-L2-CMIPF within the date range 2022-12-15 00:00:00 to 2022-12-20 10:00:00, focusing on hours 5:00 and 6:00 AM. The data is cropped to the geographic bounds of -35° to 5° latitude and -80° to -30° longitude, reprojected with a resolution of 0.045 degrees, and saved in a by_date format for easy organization.
+This command retrieves GOES-16 satellite data for the product ABI-L2-CMIPF within the date range 2022-12-15 00:00:00 to 2022-12-15 12:00:00, focusing on hours 5:00 and 6:00 AM. The data is cropped to the geographic bounds of -35° to 5° latitude and -80° to -30° longitude, reprojected with a resolution of 0.045 degrees, and saved in a by_date format for easy organization.
 
 ```bash
-goesgcp --satellite goes-16 --product ABI-L2-CMIPF --start '2022-12-15 00:00:00' --end '2022-12-20 10:00:00' --bt_hour 5 6 --save_format by_date --resolution 0.045 --lat_min -35 --lat_max 5 --lon_min -80 --lon_max -30
+goesgcp --satellite goes-16 --product ABI-L2-CMIPF --start "2022-12-15 00:00:00" --end "2022-12-15 12:00:00" --bt_hour 5 6 --save_format by_date --resolution 0.045 --lat_min -35 --lat_max 5 --lon_min -80 --lon_max -30
 ```
 
 ### Contributing

{goesgcp-2.0.2 → goesgcp-2.0.4}/setup.py

@@ -13,7 +13,7 @@ with open('requirements.txt') as f:
 
 setup(
     name="goesgcp",
-    version='2.0.2',
+    version='2.0.4',
     author="Helvecio B. L. Neto",
     author_email="helvecioblneto@gmail.com",
     description="A package to download and process GOES-16/17 data",