goesgcp 1.0.9__tar.gz → 2.0.1__tar.gz

goesgcp-2.0.1/PKG-INFO

@@ -0,0 +1,118 @@
+Metadata-Version: 2.2
+Name: goesgcp
+Version: 2.0.1
+Summary: A package to download and process GOES-16/17 data
+Home-page: https://github.com/helvecioneto/goesgcp
+Author: Helvecio B. L. Neto
+Author-email: helvecioblneto@gmail.com
+License: LICENSE
+Classifier: Programming Language :: Python
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering
+Classifier: Topic :: Software Development
+Classifier: Topic :: Utilities
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: google-cloud-storage
+Requires-Dist: pyproj
+Requires-Dist: xarray
+Requires-Dist: netcdf4
+Requires-Dist: rioxarray
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: license
+Dynamic: requires-dist
+Dynamic: summary
+
+# goesgcp
+<!-- badges: start -->
+[![pypi](https://badge.fury.io/py/goesgcp.svg)](https://pypi.python.org/pypi/goesgcp)
+[![Downloads](https://img.shields.io/pypi/dm/goesgcp.svg)](https://pypi.python.org/pypi/goesgcp)
+[![Contributors](https://img.shields.io/github/contributors/helvecioneto/goesgcp.svg)](https://github.com/helvecioneto/goesgcp/graphs/contributors)
+[![License](https://img.shields.io/pypi/l/goesgcp.svg)](https://github.com/helvecioneto/goesgcp/blob/main/LICENSE)
+<!-- badges: end -->
+
+
+`goesgcp` is a Python utility designed for downloading and reprojecting GOES-R satellite data. This script leverages the `google.cloud` library for accessing data from the Google Cloud Platform (GCP) and `pyproj` for reprojecting data to EPSG:4326, as well as cropping it to a user-defined bounding box.
+
+## Features
+
+- **Download GOES-R satellite data**: Supports GOES-16 and GOES-17.
+- **Reprojection and cropping**: Reprojects data to EPSG:4326 and crops to a specified bounding box.
+- **Flexible command-line interface**: Customize download options, variables, channels, time range, and output format.
+- **Efficient processing**: Handles large datasets with optimized performance.
+
+## Installation
+
+Install the necessary dependencies via `pip`:
+
+```bash
+pip install goesgcp
+```
+
+
+## Usage
+
+### Command-Line Arguments
+
+The script uses the `argparse` module for handling command-line arguments. Below are the available options:
+
+```bash
+goesgcp [OPTIONS]
+```
+
+| Option               | Description                                                                |
+|----------------------|----------------------------------------------------------------------------|
+| `--satellite`         | Name of the satellite (e.g., goes16).                                     |
+| `--product`           | Name of the satellite product (e.g., ABI-L2-CMIPF).                       |
+| `--var_name`          | Variable name to extract (e.g., CMI).                                     |
+| `--channel`           | Channel to use (e.g., 13).                                                |
+| `--output`            | Path for saving output files (default: `output/`).                        | 
+| `--lat_min`           | Minimum latitude of the bounding box (default: `-56`).                    |
+| `--lat_max`           | Maximum latitude of the bounding box (default: `35`).                     |
+| `--lon_min`           | Minimum longitude of the bounding box (default: `-116`).                  |
+| `--lon_max`           | Maximum longitude of the bounding box (default: `-25`).                   |
+| `--resolution`        | Set the reprojet data resolution in degree (default: `-0.045`).           |
+| `--recent`            | Number of most recent data to download (default: `1`).                    |
+| `--start`             | Start date for downloading data (default: `None`).                        |
+| `--end`               | End date for downloading data (default: `None`).                          |
+| `--bt_hour`           | Hour of the day to download data (default: [0, 1, ..., 23]).              |
+| `--bt_minute`         | Minute of the hour to download data (default: [0, 15, 30, 45]).           |
+| `--save_format`       | Format for saving output files (default: `by_date`).                      |
+
+#### Available GOES Products
+A comprehensive list of available GOES products can be found at the following link: [https://github.com/awslabs/open-data-docs/tree/main/docs/noaa/noaa-goes16](https://github.com/awslabs/open-data-docs/tree/main/docs/noaa/noaa-goes16)
+
+### Examples
+
+#### Download Recent Data
+In the example below, the command downloads the 3 most recent files from the GOES-16 satellite for the product ABI-L2-CMIPF. It focuses on the variable CMI (Cloud and Moisture Imagery) from channel 13, which is commonly used for infrared observations. The downloaded files are saved to the specified output directory output/.
+
+```bash
+goesgcp --satellite goes-16 --product ABI-L2-CMIPF --var_name CMI --channel 13 --recent 3 --output "output/"
+```
+
+#### Download Data for a Specific Time Range
+This command retrieves GOES-16 satellite data for the product ABI-L2-CMIPF within the date range 2022-12-15 00:00:00 to 2022-12-20 10:00:00, focusing on hours 5:00 and 6:00 AM. The data is cropped to the geographic bounds of -35° to 5° latitude and -80° to -30° longitude, reprojected with a resolution of 0.045 degrees, and saved in a by_date format for easy organization.
+
+```bash
+goesgcp --satellite goes-16 --product ABI-L2-CMIPF --start '2022-12-15 00:00:00' --end '2022-12-20 10:00:00' --bt_hour 5 6 --save_format by_date --resolution 0.045 --lat_min -35 --lat_max 5 --lon_min -80 --lon_max -30
+```
+
+### Contributing
+Contributions are welcome! If you encounter issues or have suggestions for improvements, please submit them via GitHub issues or pull requests.
+
+### Credits
+This project was developed and optimized by Helvecio Neto (2025).
+It builds upon NOAA GOES-R data and leverages resources provided by the Google Cloud Platform.
+
+### License
+This project is licensed under the MIT License. 

goesgcp-2.0.1/README.md

@@ -0,0 +1,84 @@
+# goesgcp
+<!-- badges: start -->
+[![pypi](https://badge.fury.io/py/goesgcp.svg)](https://pypi.python.org/pypi/goesgcp)
+[![Downloads](https://img.shields.io/pypi/dm/goesgcp.svg)](https://pypi.python.org/pypi/goesgcp)
+[![Contributors](https://img.shields.io/github/contributors/helvecioneto/goesgcp.svg)](https://github.com/helvecioneto/goesgcp/graphs/contributors)
+[![License](https://img.shields.io/pypi/l/goesgcp.svg)](https://github.com/helvecioneto/goesgcp/blob/main/LICENSE)
+<!-- badges: end -->
+
+
+`goesgcp` is a Python utility designed for downloading and reprojecting GOES-R satellite data. This script leverages the `google.cloud` library for accessing data from the Google Cloud Platform (GCP) and `pyproj` for reprojecting data to EPSG:4326, as well as cropping it to a user-defined bounding box.
+
+## Features
+
+- **Download GOES-R satellite data**: Supports GOES-16 and GOES-17.
+- **Reprojection and cropping**: Reprojects data to EPSG:4326 and crops to a specified bounding box.
+- **Flexible command-line interface**: Customize download options, variables, channels, time range, and output format.
+- **Efficient processing**: Handles large datasets with optimized performance.
+
+## Installation
+
+Install the necessary dependencies via `pip`:
+
+```bash
+pip install goesgcp
+```
+
+
+## Usage
+
+### Command-Line Arguments
+
+The script uses the `argparse` module for handling command-line arguments. Below are the available options:
+
+```bash
+goesgcp [OPTIONS]
+```
+
+| Option               | Description                                                                |
+|----------------------|----------------------------------------------------------------------------|
+| `--satellite`         | Name of the satellite (e.g., goes16).                                     |
+| `--product`           | Name of the satellite product (e.g., ABI-L2-CMIPF).                       |
+| `--var_name`          | Variable name to extract (e.g., CMI).                                     |
+| `--channel`           | Channel to use (e.g., 13).                                                |
+| `--output`            | Path for saving output files (default: `output/`).                        | 
+| `--lat_min`           | Minimum latitude of the bounding box (default: `-56`).                    |
+| `--lat_max`           | Maximum latitude of the bounding box (default: `35`).                     |
+| `--lon_min`           | Minimum longitude of the bounding box (default: `-116`).                  |
+| `--lon_max`           | Maximum longitude of the bounding box (default: `-25`).                   |
+| `--resolution`        | Set the reprojet data resolution in degree (default: `-0.045`).           |
+| `--recent`            | Number of most recent data to download (default: `1`).                    |
+| `--start`             | Start date for downloading data (default: `None`).                        |
+| `--end`               | End date for downloading data (default: `None`).                          |
+| `--bt_hour`           | Hour of the day to download data (default: [0, 1, ..., 23]).              |
+| `--bt_minute`         | Minute of the hour to download data (default: [0, 15, 30, 45]).           |
+| `--save_format`       | Format for saving output files (default: `by_date`).                      |
+
+#### Available GOES Products
+A comprehensive list of available GOES products can be found at the following link: [https://github.com/awslabs/open-data-docs/tree/main/docs/noaa/noaa-goes16](https://github.com/awslabs/open-data-docs/tree/main/docs/noaa/noaa-goes16)
+
+### Examples
+
+#### Download Recent Data
+In the example below, the command downloads the 3 most recent files from the GOES-16 satellite for the product ABI-L2-CMIPF. It focuses on the variable CMI (Cloud and Moisture Imagery) from channel 13, which is commonly used for infrared observations. The downloaded files are saved to the specified output directory output/.
+
+```bash
+goesgcp --satellite goes-16 --product ABI-L2-CMIPF --var_name CMI --channel 13 --recent 3 --output "output/"
+```
+
+#### Download Data for a Specific Time Range
+This command retrieves GOES-16 satellite data for the product ABI-L2-CMIPF within the date range 2022-12-15 00:00:00 to 2022-12-20 10:00:00, focusing on hours 5:00 and 6:00 AM. The data is cropped to the geographic bounds of -35° to 5° latitude and -80° to -30° longitude, reprojected with a resolution of 0.045 degrees, and saved in a by_date format for easy organization.
+
+```bash
+goesgcp --satellite goes-16 --product ABI-L2-CMIPF --start '2022-12-15 00:00:00' --end '2022-12-20 10:00:00' --bt_hour 5 6 --save_format by_date --resolution 0.045 --lat_min -35 --lat_max 5 --lon_min -80 --lon_max -30
+```
+
+### Contributing
+Contributions are welcome! If you encounter issues or have suggestions for improvements, please submit them via GitHub issues or pull requests.
+
+### Credits
+This project was developed and optimized by Helvecio Neto (2025).
+It builds upon NOAA GOES-R data and leverages resources provided by the Google Cloud Platform.
+
+### License
+This project is licensed under the MIT License. 

{goesgcp-1.0.9 → goesgcp-2.0.1}/goesgcp/main.py

@@ -5,6 +5,7 @@ import xarray as xr
 import argparse
 import sys
 import tqdm
+import pandas as pd
 from distutils.util import strtobool
 from multiprocessing import Pool
 from google.cloud import storage
@@ -28,6 +29,83 @@ def get_directory_prefix(year, julian_day, hour):
     """Generates the directory path based on year, Julian day, and hour."""
     return f"{year}/{julian_day}/{str(hour).zfill(2)}/"
 
+
+def get_files_period(connection, bucket_name, base_prefix, pattern, 
+                     start, end, bt_hour=[0, 23], bt_min=[0, 60], freq='10 min'):
+    """
+    Fetches files from a GCP bucket within a specified time period and returns them as a DataFrame.
+
+    :param connection: The GCP storage client connection.
+    :param bucket_name: Name of the GCP bucket.
+    :param base_prefix: Base directory prefix for the files.
+    :param pattern: Search pattern for file names.
+    :param start: Start datetime (inclusive).
+    :param end: End datetime (exclusive).
+    :return: DataFrame containing the file names and their metadata.
+    """
+
+    print(f"GOESGCP: Fetching files between {start} and {end}...")
+
+    # Ensure datetime objects
+    start = pd.to_datetime(start).tz_localize('UTC')
+    end = pd.to_datetime(end).tz_localize('UTC')
+
+    # Initialize list to store file metadata
+    files_metadata = []
+
+    # Generate the list of dates from start to end
+    current_time = start
+    while current_time < end:
+        year = current_time.year
+        julian_day = str(current_time.timetuple().tm_yday).zfill(3)  # Julian day
+        hour = current_time.hour
+
+        # Generate the directory prefix
+        prefix = f"{base_prefix}/{get_directory_prefix(year, julian_day, hour)}"
+
+        # List blobs in the bucket for the current prefix
+        blobs = list_blobs(connection, bucket_name, prefix)
+
+        # Filter blobs by pattern
+        for blob in blobs:
+            if pattern in blob.name:
+                files_metadata.append({
+                    'file_name': blob.name,
+                    'last_modified': blob.updated
+                })
+
+        # Move to the next hour
+        current_time += timedelta(hours=1)
+
+    # Create a DataFrame from the list of files
+    df = pd.DataFrame(files_metadata)
+
+    if df.empty:
+        print("No files found matching the pattern.")
+        return pd.DataFrame()
+    
+    # Ensure 'last_modified' is in the correct datetime format without timezone
+    df['last_modified'] = pd.to_datetime(df['last_modified']).dt.tz_localize(None)
+    start = pd.to_datetime(start).tz_localize(None)
+    end = pd.to_datetime(end).tz_localize(None)
+    
+    # Filter the DataFrame based on the date range
+    df = df[(df['last_modified'] >= start) & (df['last_modified'] < end)]
+
+    # Filter the DataFrame based on the hour range
+    df['hour'] = df['last_modified'].dt.hour
+    df = df[(df['hour'] >= bt_hour[0]) & (df['hour'] <= bt_hour[1])]
+
+    # Filter the DataFrame based on the minute range
+    df['minute'] = df['last_modified'].dt.minute
+    df = df[(df['minute'] >= bt_min[0]) & (df['minute'] <= bt_min[1])]
+
+    # Filter the DataFrame based on the frequency
+    df['freq'] = df['last_modified'].dt.floor(freq)
+    df = df.groupby('freq').first().reset_index()
+
+    return df['file_name'].tolist()
+
 def get_recent_files(connection, bucket_name, base_prefix, pattern, min_files):
     """
     Fetches the most recent files in a GCP bucket.
@@ -148,12 +226,31 @@ def crop_reproject(args):
     # Add global metadata comments
     ds.attrs['comments'] = "Data processed by goesgcp, author: Helvecio B. L. Neto (helvecioblneto@gmail.com)"
         
-    # Save as netcdf overwriting the original file
-    ds.to_netcdf(f'{output}{file.split("/")[-1]}', mode='w', format='NETCDF4_CLASSIC')
-
-    # Close the dataset
+    if save_format == 'by_date':
+        file_datetime = datetime.strptime(ds.time_coverage_start, 
+                                          "%Y-%m-%dT%H:%M:%S.%fZ")
+        year = file_datetime.strftime("%Y")
+        month = file_datetime.strftime("%m")
+        day = file_datetime.strftime("%d")
+        output_directory = f"{output}{year}/{month}/{day}/"
+    elif save_format == 'julian':
+        file_datetime = datetime.strptime(ds.time_coverage_start, 
+                                          "%Y-%m-%dT%H:%M:%S.%fZ")
+        year = file_datetime.strftime("%Y")
+        julian_day = file_datetime.timetuple().tm_yday
+        output_directory = f"{output}{year}/{julian_day}/"
+    else:
+        output_directory = output
+
+    # Create the output directory
+    pathlib.Path(output_directory).mkdir(parents=True, exist_ok=True)
+
+    # Save the file
+    output_file = f"{output_directory}{file.split('/')[-1]}"
+    ds.to_netcdf(output_file, mode='w', format='NETCDF4_CLASSIC')
+
+    # Fechar o dataset
     ds.close()
-
     return
 
 
@@ -197,16 +294,39 @@ def main():
 
     global output_path, var_name, \
           lat_min, lat_max, lon_min, lon_max, \
-          max_attempts, parallel, recent, resolution, storage_client
+          max_attempts, parallel, recent, resolution, storage_client, \
+            satellite, product, op_mode, channel, save_format
 
     epilog = """
     Example usage:
     
-    - To download recent 10 files from the GOES-16 satellite for the ABI-L2-CMIPF product:
+    - To download recent 3 files from the GOES-16 satellite for the ABI-L2-CMIPF product:
+
+    goesgcp --satellite goes16 --product ABI-L2-CMIP --recent 3
+
+    - To download files from the GOES-16 satellite for the ABI-L2-CMIPF product between 2022-12-15 and 2022-12-20:
+
+    goesgcp --start '2022-12-15 00:00:00' --end '2022-12-20 10:00:00' --bt_hour 5 6 --save_format by_date --resolution 0.045 --lat_min -35 --lat_max 5 --lon_min -80 --lon_max -30
 
-    goesgcp --satellite goes16 --product ABI-L2-CMIP --recent 10 --output_path "output/"
     """
 
+    product_names = [
+    "ABI-L1b-RadF", "ABI-L1b-RadC", "ABI-L1b-RadM", "ABI-L2-ACHAC", "ABI-L2-ACHAF", "ABI-L2-ACHAM",
+    "ABI-L2-ACHTF", "ABI-L2-ACHTM", "ABI-L2-ACMC", "ABI-L2-ACMF", "ABI-L2-ACMM", "ABI-L2-ACTPC",
+    "ABI-L2-ACTPF", "ABI-L2-ACTPM", "ABI-L2-ADPC", "ABI-L2-ADPF", "ABI-L2-ADPM", "ABI-L2-AICEF",
+    "ABI-L2-AITAF", "ABI-L2-AODC", "ABI-L2-AODF", "ABI-L2-BRFC", "ABI-L2-BRFF", "ABI-L2-BRFM",
+    "ABI-L2-CMIPC", "ABI-L2-CMIPF", "ABI-L2-CMIPM", "ABI-L2-CODC", "ABI-L2-CODF", "ABI-L2-CPSC",
+    "ABI-L2-CPSF", "ABI-L2-CPSM", "ABI-L2-CTPC", "ABI-L2-CTPF", "ABI-L2-DMWC", "ABI-L2-DMWF",
+    "ABI-L2-DMWM", "ABI-L2-DMWVC", "ABI-L2-DMWVF", "ABI-L2-DMWVF", "ABI-L2-DSIC", "ABI-L2-DSIF",
+    "ABI-L2-DSIM", "ABI-L2-DSRC", "ABI-L2-DSRF", "ABI-L2-DSRM", "ABI-L2-FDCC", "ABI-L2-FDCF",
+    "ABI-L2-FDCM", "ABI-L2-LSAC", "ABI-L2-LSAF", "ABI-L2-LSAM", "ABI-L2-LSTC", "ABI-L2-LSTF",
+    "ABI-L2-LSTM", "ABI-L2-LVMPC", "ABI-L2-LVMPF", "ABI-L2-LVMPM", "ABI-L2-LVTPC", "ABI-L2-LVTPF",
+    "ABI-L2-LVTPM", "ABI-L2-MCMIPC", "ABI-L2-MCMIPF", "ABI-L2-MCMIPM", "ABI-L2-RRQPEF",
+    "ABI-L2-RSRC", "ABI-L2-RSRF", "ABI-L2-SSTF", "ABI-L2-TPWC", "ABI-L2-TPWF", "ABI-L2-TPWM",
+    "ABI-L2-VAAF", "EXIS-L1b-SFEU", "EXIS-L1b-SFXR", "GLM-L2-LCFA", "MAG-L1b-GEOF", "SEIS-L1b-EHIS",
+    "SEIS-L1b-MPSH", "SEIS-L1b-MPSL", "SEIS-L1b-SGPS", "SUVI-L1b-Fe093", "SUVI-L1b-Fe131",
+    "SUVI-L1b-Fe171", "SUVI-L1b-Fe195", "SUVI-L1b-Fe284", "SUVI-L1b-He303"
+    ]
 
     # Set arguments
     parser = argparse.ArgumentParser(description='Converts GOES-16 L2 data to netCDF',
@@ -214,12 +334,21 @@ def main():
                                     formatter_class=argparse.RawDescriptionHelpFormatter)
     
     # Satellite and product settings
-    parser.add_argument('--satellite', type=str, default='goes-16', help='Name of the satellite (e.g., goes16)')
-    parser.add_argument('--product', type=str, default='ABI-L2-CMIP', help='Name of the satellite product')
+    parser.add_argument('--satellite', type=str, default='goes-16', choices=['goes-16', 'goes-18'], help='Name of the satellite (e.g., goes16)')
+    parser.add_argument('--product', type=str, default='ABI-L2-CMIP', help='Name of the satellite product', choices=product_names)
     parser.add_argument('--var_name', type=str, default='CMI', help='Variable name to extract (e.g., CMI)')
     parser.add_argument('--channel', type=int, default=13, help='Channel to use (e.g., 13)')
-    parser.add_argument('--domain', type=str, default='F', help='Domain to use (e.g., F or C)')
-    parser.add_argument('--recent', type=int, default=3, help='Number of recent files to download')
+    parser.add_argument('--op_mode', type=str, default='M6C', help='Operational mode to use (e.g., M6C)')
+
+    # Recent files settings
+    parser.add_argument('--recent', type=int, help='Number of recent files to download (e.g., 3)')
+
+    # Date and time settings
+    parser.add_argument('--start', type=str, help='Start date in YYYY-MM-DD format')
+    parser.add_argument('--end', type=str, help='End date in YYYY-MM-DD format')
+    parser.add_argument('--freq', type=str, default='10 min', help='Frequency for the time range (e.g., "10 min")')
+    parser.add_argument('--bt_hour', nargs=2, type=int, default=[0, 23], help='Filter data between these hours (e.g., 0 23)')
+    parser.add_argument('--bt_min', nargs=2, type=int, default=[0, 60], help='Filter data between these minutes (e.g., 0 60)')
 
     # Geographic bounding box
     parser.add_argument('--lat_min', type=float, default=-81.3282, help='Minimum latitude of the bounding box')
@@ -233,6 +362,9 @@ def main():
     parser.add_argument('--parallel', type=lambda x: bool(strtobool(x)), default=True, help='Use parallel processing')
     parser.add_argument('--processes', type=int, default=4, help='Number of processes for parallel execution')
     parser.add_argument('--max_attempts', type=int, default=3, help='Number of attempts to download a file')
+    parser.add_argument('--save_format', type=str, default='flat', choices=['flat', 'by_date','julian'],
+                    help="Save the files in a flat structure or by date")
+
 
     # Parse arguments
     args = parser.parse_args()
@@ -245,7 +377,7 @@ def main():
     output_path = args.output
     satellite = args.satellite
     product = args.product
-    domain = args.domain
+    op_mode = args.op_mode
     channel = str(args.channel).zfill(2)
     var_name = args.var_name
     lat_min = args.lat_min
@@ -255,11 +387,22 @@ def main():
     resolution = args.resolution
     max_attempts = args.max_attempts
     parallel = args.parallel
+    recent = args.recent
+    start = args.start
+    end = args.end
+    freq = args.freq
+    bt_hour = args.bt_hour
+    bt_min = args.bt_min
+    save_format = args.save_format
+
+
+    # Check mandatory arguments
+    if not args.recent and not (args.start and args.end):
+        print("You must provide either the --recent or --start and --end arguments. Exiting...")
+        sys.exit(1)
 
     # Set bucket name and pattern
     bucket_name = "gcp-public-data-" + satellite
-    pattern = "OR_"+product+domain+"-M6C"+channel+"_G" + satellite[-2:]
-    min_files = args.recent
 
     # Create output directory
     pathlib.Path(output_path).mkdir(parents=True, exist_ok=True)
@@ -274,11 +417,20 @@ def main():
         print(f"Bucket {bucket_name} not found. Exiting...")
         sys.exit(1)
 
-    # Search for recent files
-    recent_files = get_recent_files(storage_client, bucket_name, product + domain, pattern, min_files)
+    # Set pattern for the files
+    pattern = "OR_"+product+"-"+op_mode+channel+"_G" + satellite[-2:]
+
+    # Check operational mode if is recent or specific date
+    if start and end:
+        files_list = get_files_period(storage_client, bucket_name,
+                                       product, pattern, start, end,
+                                        bt_hour, bt_min, freq)
+    else:
+        # Get recent files
+        files_list = get_recent_files(storage_client, bucket_name, product, pattern, recent)
 
     # Check if any files were found
-    if not recent_files:
+    if not files_list:
         print(f"No files found with the pattern {pattern}. Exiting...")
         sys.exit(1)
 
@@ -286,14 +438,14 @@ def main():
     pathlib.Path('tmp/').mkdir(parents=True, exist_ok=True)
 
     # Download files
-    print(f"GOESGCP: Downloading and processing {len(recent_files)} files...")
-    loading_bar = tqdm.tqdm(total=len(recent_files), ncols=100, position=0, leave=True,
+    print(f"GOESGCP: Downloading and processing {len(files_list)} files...")
+    loading_bar = tqdm.tqdm(total=len(files_list), ncols=100, position=0, leave=True,
                         bar_format='{l_bar}{bar}| {n_fmt}/{total_fmt} + \
                         [Elapsed:{elapsed} Remaining:<{remaining}]')
     
     if parallel: # Run in parallel
         # Create a list of tasks
-        tasks = [(bucket_name, file, f"tmp/{file.split('/')[-1]}") for file in recent_files]
+        tasks = [(bucket_name, file, f"tmp/{file.split('/')[-1]}") for file in files_list]
 
         # Download files in parallel
         with Pool(processes=args.processes) as pool:
@@ -301,7 +453,7 @@ def main():
                 loading_bar.update(1)
         loading_bar.close()
     else: # Run in serial
-        for file in recent_files:
+        for file in files_list:
             local_path = f"tmp/{file.split('/')[-1]}"
             process_file((bucket_name, file, local_path))
             loading_bar.update(1)

goesgcp-2.0.1/goesgcp.egg-info/PKG-INFO

@@ -0,0 +1,118 @@
+Metadata-Version: 2.2
+Name: goesgcp
+Version: 2.0.1
+Summary: A package to download and process GOES-16/17 data
+Home-page: https://github.com/helvecioneto/goesgcp
+Author: Helvecio B. L. Neto
+Author-email: helvecioblneto@gmail.com
+License: LICENSE
+Classifier: Programming Language :: Python
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering
+Classifier: Topic :: Software Development
+Classifier: Topic :: Utilities
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: google-cloud-storage
+Requires-Dist: pyproj
+Requires-Dist: xarray
+Requires-Dist: netcdf4
+Requires-Dist: rioxarray
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: license
+Dynamic: requires-dist
+Dynamic: summary
+
+# goesgcp
+<!-- badges: start -->
+[![pypi](https://badge.fury.io/py/goesgcp.svg)](https://pypi.python.org/pypi/goesgcp)
+[![Downloads](https://img.shields.io/pypi/dm/goesgcp.svg)](https://pypi.python.org/pypi/goesgcp)
+[![Contributors](https://img.shields.io/github/contributors/helvecioneto/goesgcp.svg)](https://github.com/helvecioneto/goesgcp/graphs/contributors)
+[![License](https://img.shields.io/pypi/l/goesgcp.svg)](https://github.com/helvecioneto/goesgcp/blob/main/LICENSE)
+<!-- badges: end -->
+
+
+`goesgcp` is a Python utility designed for downloading and reprojecting GOES-R satellite data. This script leverages the `google.cloud` library for accessing data from the Google Cloud Platform (GCP) and `pyproj` for reprojecting data to EPSG:4326, as well as cropping it to a user-defined bounding box.
+
+## Features
+
+- **Download GOES-R satellite data**: Supports GOES-16 and GOES-17.
+- **Reprojection and cropping**: Reprojects data to EPSG:4326 and crops to a specified bounding box.
+- **Flexible command-line interface**: Customize download options, variables, channels, time range, and output format.
+- **Efficient processing**: Handles large datasets with optimized performance.
+
+## Installation
+
+Install the necessary dependencies via `pip`:
+
+```bash
+pip install goesgcp
+```
+
+
+## Usage
+
+### Command-Line Arguments
+
+The script uses the `argparse` module for handling command-line arguments. Below are the available options:
+
+```bash
+goesgcp [OPTIONS]
+```
+
+| Option               | Description                                                                |
+|----------------------|----------------------------------------------------------------------------|
+| `--satellite`         | Name of the satellite (e.g., goes16).                                     |
+| `--product`           | Name of the satellite product (e.g., ABI-L2-CMIPF).                       |
+| `--var_name`          | Variable name to extract (e.g., CMI).                                     |
+| `--channel`           | Channel to use (e.g., 13).                                                |
+| `--output`            | Path for saving output files (default: `output/`).                        | 
+| `--lat_min`           | Minimum latitude of the bounding box (default: `-56`).                    |
+| `--lat_max`           | Maximum latitude of the bounding box (default: `35`).                     |
+| `--lon_min`           | Minimum longitude of the bounding box (default: `-116`).                  |
+| `--lon_max`           | Maximum longitude of the bounding box (default: `-25`).                   |
+| `--resolution`        | Set the reprojet data resolution in degree (default: `-0.045`).           |
+| `--recent`            | Number of most recent data to download (default: `1`).                    |
+| `--start`             | Start date for downloading data (default: `None`).                        |
+| `--end`               | End date for downloading data (default: `None`).                          |
+| `--bt_hour`           | Hour of the day to download data (default: [0, 1, ..., 23]).              |
+| `--bt_minute`         | Minute of the hour to download data (default: [0, 15, 30, 45]).           |
+| `--save_format`       | Format for saving output files (default: `by_date`).                      |
+
+#### Available GOES Products
+A comprehensive list of available GOES products can be found at the following link: [https://github.com/awslabs/open-data-docs/tree/main/docs/noaa/noaa-goes16](https://github.com/awslabs/open-data-docs/tree/main/docs/noaa/noaa-goes16)
+
+### Examples
+
+#### Download Recent Data
+In the example below, the command downloads the 3 most recent files from the GOES-16 satellite for the product ABI-L2-CMIPF. It focuses on the variable CMI (Cloud and Moisture Imagery) from channel 13, which is commonly used for infrared observations. The downloaded files are saved to the specified output directory output/.
+
+```bash
+goesgcp --satellite goes-16 --product ABI-L2-CMIPF --var_name CMI --channel 13 --recent 3 --output "output/"
+```
+
+#### Download Data for a Specific Time Range
+This command retrieves GOES-16 satellite data for the product ABI-L2-CMIPF within the date range 2022-12-15 00:00:00 to 2022-12-20 10:00:00, focusing on hours 5:00 and 6:00 AM. The data is cropped to the geographic bounds of -35° to 5° latitude and -80° to -30° longitude, reprojected with a resolution of 0.045 degrees, and saved in a by_date format for easy organization.
+
+```bash
+goesgcp --satellite goes-16 --product ABI-L2-CMIPF --start '2022-12-15 00:00:00' --end '2022-12-20 10:00:00' --bt_hour 5 6 --save_format by_date --resolution 0.045 --lat_min -35 --lat_max 5 --lon_min -80 --lon_max -30
+```
+
+### Contributing
+Contributions are welcome! If you encounter issues or have suggestions for improvements, please submit them via GitHub issues or pull requests.
+
+### Credits
+This project was developed and optimized by Helvecio Neto (2025).
+It builds upon NOAA GOES-R data and leverages resources provided by the Google Cloud Platform.
+
+### License
+This project is licensed under the MIT License. 

{goesgcp-1.0.9 → goesgcp-2.0.1}/setup.py

@@ -13,7 +13,7 @@ with open('requirements.txt') as f:
 
 setup(
     name="goesgcp",
-    version='1.0.9',
+    version='2.0.1',
     author="Helvecio B. L. Neto",
     author_email="helvecioblneto@gmail.com",
     description="A package to download and process GOES-16/17 data",

goesgcp-1.0.9/PKG-INFO

@@ -1,71 +0,0 @@
-Metadata-Version: 2.1
-Name: goesgcp
-Version: 1.0.9
-Summary: A package to download and process GOES-16/17 data
-Home-page: https://github.com/helvecioneto/goesgcp
-Author: Helvecio B. L. Neto
-Author-email: helvecioblneto@gmail.com
-License: LICENSE
-Classifier: Programming Language :: Python
-Classifier: Development Status :: 5 - Production/Stable
-Classifier: Operating System :: OS Independent
-Classifier: Programming Language :: Python :: 3.10
-Classifier: Programming Language :: Python :: 3.11
-Classifier: Programming Language :: Python :: 3.12
-Classifier: Topic :: Scientific/Engineering
-Classifier: Topic :: Software Development
-Classifier: Topic :: Utilities
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Requires-Dist: google-cloud-storage
-Requires-Dist: pyproj
-Requires-Dist: xarray
-Requires-Dist: netcdf4
-Requires-Dist: rioxarray
-
-# goesgcp
-
-goesgcp is a utility script for downloading and reprojecting GOES-R satellite data. The script uses the `google.cloud` library to download data from the Google Cloud Platform (GCP) and the `pyproj` library to reproject the data to EPSG:4326 and crop it to a specified bounding box.
-
-
-## Installation
-
-You can install the necessary dependencies using `pip`:
-
-```bash
-pip install goesgcp
-```
-
-## Usage
-
-### Command-Line Arguments
-
-The script uses the `argparse` module for handling command-line arguments. Below are the available options:
-
-```bash
-goesgcp [OPTIONS]
-```
-
-| Option               | Description                                                                |
-|----------------------|----------------------------------------------------------------------------|
-| `--satellite`         | Name of the satellite (e.g., goes16).                                     |
-| `--product`           | Name of the satellite product (e.g., ABI-L2-CMIPF).                       |
-| `--var_name`          | Variable name to extract (e.g., CMI).                                     |
-| `--channel`           | Channel to use (e.g., 13).                                                |
-| `--output`            | Path for saving output files (default: `output/`).                        | 
-| `--lat_min`           | Minimum latitude of the bounding box (default: `-56`).                    |
-| `--lat_max`           | Maximum latitude of the bounding box (default: `35`).                     |
-| `--lon_min`           | Minimum longitude of the bounding box (default: `-116`).                  |
-| `--lon_max`           | Maximum longitude of the bounding box (default: `-25`).                   |
-| `--resolution`        | Set the reprojet data resolution in degree (default: `-0.045`).           |
-
-### Examples
-
-To download most 3 recent data for the GOES-16 satellite, ABI-L2-CMIPF product, variable CMI, and channel 13, run the following command:
-
-```bash
-goesgcp --satellite goes16 --product ABI-L2-CMIPF --var_name CMI --channel 13 --recent 3 --output "output/"
-```
-
-### Credits
-And this is a otimization by Helvecio Neto - 2025

goesgcp-1.0.9/README.md

@@ -1,46 +0,0 @@
-# goesgcp
-
-goesgcp is a utility script for downloading and reprojecting GOES-R satellite data. The script uses the `google.cloud` library to download data from the Google Cloud Platform (GCP) and the `pyproj` library to reproject the data to EPSG:4326 and crop it to a specified bounding box.
-
-
-## Installation
-
-You can install the necessary dependencies using `pip`:
-
-```bash
-pip install goesgcp
-```
-
-## Usage
-
-### Command-Line Arguments
-
-The script uses the `argparse` module for handling command-line arguments. Below are the available options:
-
-```bash
-goesgcp [OPTIONS]
-```
-
-| Option               | Description                                                                |
-|----------------------|----------------------------------------------------------------------------|
-| `--satellite`         | Name of the satellite (e.g., goes16).                                     |
-| `--product`           | Name of the satellite product (e.g., ABI-L2-CMIPF).                       |
-| `--var_name`          | Variable name to extract (e.g., CMI).                                     |
-| `--channel`           | Channel to use (e.g., 13).                                                |
-| `--output`            | Path for saving output files (default: `output/`).                        | 
-| `--lat_min`           | Minimum latitude of the bounding box (default: `-56`).                    |
-| `--lat_max`           | Maximum latitude of the bounding box (default: `35`).                     |
-| `--lon_min`           | Minimum longitude of the bounding box (default: `-116`).                  |
-| `--lon_max`           | Maximum longitude of the bounding box (default: `-25`).                   |
-| `--resolution`        | Set the reprojet data resolution in degree (default: `-0.045`).           |
-
-### Examples
-
-To download most 3 recent data for the GOES-16 satellite, ABI-L2-CMIPF product, variable CMI, and channel 13, run the following command:
-
-```bash
-goesgcp --satellite goes16 --product ABI-L2-CMIPF --var_name CMI --channel 13 --recent 3 --output "output/"
-```
-
-### Credits
-And this is a otimization by Helvecio Neto - 2025

goesgcp-1.0.9/goesgcp.egg-info/PKG-INFO

@@ -1,71 +0,0 @@
-Metadata-Version: 2.1
-Name: goesgcp
-Version: 1.0.9
-Summary: A package to download and process GOES-16/17 data
-Home-page: https://github.com/helvecioneto/goesgcp
-Author: Helvecio B. L. Neto
-Author-email: helvecioblneto@gmail.com
-License: LICENSE
-Classifier: Programming Language :: Python
-Classifier: Development Status :: 5 - Production/Stable
-Classifier: Operating System :: OS Independent
-Classifier: Programming Language :: Python :: 3.10
-Classifier: Programming Language :: Python :: 3.11
-Classifier: Programming Language :: Python :: 3.12
-Classifier: Topic :: Scientific/Engineering
-Classifier: Topic :: Software Development
-Classifier: Topic :: Utilities
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Requires-Dist: google-cloud-storage
-Requires-Dist: pyproj
-Requires-Dist: xarray
-Requires-Dist: netcdf4
-Requires-Dist: rioxarray
-
-# goesgcp
-
-goesgcp is a utility script for downloading and reprojecting GOES-R satellite data. The script uses the `google.cloud` library to download data from the Google Cloud Platform (GCP) and the `pyproj` library to reproject the data to EPSG:4326 and crop it to a specified bounding box.
-
-
-## Installation
-
-You can install the necessary dependencies using `pip`:
-
-```bash
-pip install goesgcp
-```
-
-## Usage
-
-### Command-Line Arguments
-
-The script uses the `argparse` module for handling command-line arguments. Below are the available options:
-
-```bash
-goesgcp [OPTIONS]
-```
-
-| Option               | Description                                                                |
-|----------------------|----------------------------------------------------------------------------|
-| `--satellite`         | Name of the satellite (e.g., goes16).                                     |
-| `--product`           | Name of the satellite product (e.g., ABI-L2-CMIPF).                       |
-| `--var_name`          | Variable name to extract (e.g., CMI).                                     |
-| `--channel`           | Channel to use (e.g., 13).                                                |
-| `--output`            | Path for saving output files (default: `output/`).                        | 
-| `--lat_min`           | Minimum latitude of the bounding box (default: `-56`).                    |
-| `--lat_max`           | Maximum latitude of the bounding box (default: `35`).                     |
-| `--lon_min`           | Minimum longitude of the bounding box (default: `-116`).                  |
-| `--lon_max`           | Maximum longitude of the bounding box (default: `-25`).                   |
-| `--resolution`        | Set the reprojet data resolution in degree (default: `-0.045`).           |
-
-### Examples
-
-To download most 3 recent data for the GOES-16 satellite, ABI-L2-CMIPF product, variable CMI, and channel 13, run the following command:
-
-```bash
-goesgcp --satellite goes16 --product ABI-L2-CMIPF --var_name CMI --channel 13 --recent 3 --output "output/"
-```
-
-### Credits
-And this is a otimization by Helvecio Neto - 2025