astrovello 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,136 @@
1
+ Metadata-Version: 2.4
2
+ Name: astrovello
3
+ Version: 0.1.0
4
+ Summary: A framework for galaxy morphology and non-parametric decomposition.
5
+ Author-email: André Almeida Trovello <seu-email@usp.br>
6
+ Requires-Python: >=3.10
7
+ Description-Content-Type: text/markdown
8
+ Requires-Dist: numpy>=2.0
9
+ Requires-Dist: astropy>=6.0
10
+ Requires-Dist: scipy>=1.10
11
+ Requires-Dist: matplotlib>=3.5
12
+ Requires-Dist: pandas>=2.0
13
+ Requires-Dist: photutils>=1.13
14
+ Requires-Dist: reproject>=0.14
15
+ Requires-Dist: regions>=0.10
16
+ Requires-Dist: scikit-image>=0.20
17
+ Requires-Dist: tqdm
18
+ Requires-Dist: psutil
19
+ Requires-Dist: pyyaml
20
+ Requires-Dist: pypher>=0.7.3
21
+ Requires-Dist: pyreadr>=0.5.4
22
+ Requires-Dist: rasterio
23
+ Requires-Dist: shapely
24
+ Requires-Dist: dask
25
+ Requires-Dist: zarr
26
+
27
+
28
+ <p align="center">
29
+ <img src="AsTrovello_logo.png" alt="AsTrovello Logo" width="300">
30
+ </p>
31
+
32
+ ## AsTrovello
33
+
34
+ **AsTrovello** is a Python-based astronomical data analysis framework designed to process and align multi-wavelength galaxy data. It focuses on integrating high-resolution photometry from **PHANGS (HST)** with mid-infrared data from **S4G (Spitzer/IRAC)**.
35
+
36
+ The pipeline performs image reprojection, PSF (Point Spread Function) homogenization through convolution, unit conversion to Jansky, and final 3D hypercube creation.
37
+
38
+ The respective data cubes can be found downloaded in:
39
+ **S4G:** https://irsa.ipac.caltech.edu/data/SPITZER/S4G/
40
+ ```text
41
+ ├── File: <target>.phot.1.fits and <target>.phot.2.fits
42
+ ```
43
+ **PHANGS (HST images):** https://archive.stsci.edu/hlsp/phangs
44
+ ```text
45
+ ├── File: File: HST science images (drz) -> hlsp_phangs-hst_hst_wfc3-uvis_<target>_<filter>_v1_exp-drc-sci.fits
46
+ ```
47
+
48
+ ---
49
+
50
+ ## Main Features
51
+
52
+ * **Image Alignment:** Reprojects S4G images onto the HST pixel grid (conserving surface brightness).
53
+ * **PSF Homogenization:** Calculates FWHM and generates convolution kernels via **PyPHER** to match the resolution of different filters to a common "Master" PSF.
54
+ * **Unit Standardization:** Automatically converts `ELECTRONS/S` (HST) and `MJy/sr` (Spitzer) to `Jy/pixel`.
55
+ * **Hypercube Creation:** Builds a 3D FITS hypercube (RA, Dec, Filter) with automated sky masking and spatial cropping (Bounding Box).
56
+
57
+ ---
58
+
59
+ ## Installation & Requirements
60
+
61
+ The pipeline runs on **Ubuntu Linux** and is optimized for use within a Conda environment.
62
+
63
+ 1. **Clone the repository:**
64
+ ```bash
65
+ git clone https://github.com/your-username/AsTrovello.git
66
+ cd AsTrovello
67
+ ```
68
+
69
+ 2. **Setup the environment:**
70
+ We recommend using the provided `environment.yml`
71
+ ```bash
72
+ conda env create -f environment.yml -n new_env_name
73
+ ```
74
+ or creating a dedicated environment with:
75
+ ```bash
76
+ conda create -n capivara python=3.10
77
+ conda activate capivara
78
+ pip install astropy reproject scipy tqdm photutils pypher pandas
79
+ ```
80
+
81
+ ---
82
+
83
+ ## How to Run
84
+
85
+ The main execution script is `AsTrovello_run.py`, located in the `Codes/` directory.
86
+
87
+ ### Basic Syntax
88
+ ```bash
89
+ python Codes/AsTrovello_run.py --galaxy [GALAXY_NAME] --mode [MODE] [FLAGS]
90
+ ```
91
+ ### Execution Modes
92
+
93
+ The pipeline can be executed in different stages depending on your needs. Use the `--mode` argument to select one of the following:
94
+
95
+ | Mode | Description |
96
+ | :--- | :--- |
97
+ | **`full`** | **Complete Pipeline:** Executes alignment, PSF homogenization, convolution, and hypercube creation in a single run. |
98
+ | **`alignment_only`** | **Spatial Alignment:** Only reprojects S4G images to the PHANGS pixel grid. Use this to check coordinate consistency. |
99
+ | **`conv_only`** | **Resolution Matching:** Performs PSF cleaning, kernel generation via PyPHER, and image convolution. |
100
+ | **`cube_only`** | **Data Integration:** Skips processing and builds the final 3D hypercube using existing convolved files. |
101
+
102
+ > **Note:** When using `conv_only` or `full`, remember to include the `--create_kernel` flag if you need to generate new homogenization kernels for the current galaxy.
103
+
104
+ ### Usage Examples
105
+
106
+ Here are the most common ways to run the **AsTrovello** pipeline:
107
+
108
+ #### 1. Full Processing (Standard)
109
+ Runs everything from alignment to the final hypercube. Ideal for a first-time run on a new galaxy.
110
+ ```bash
111
+ python Codes/AsTrovello_run.py --galaxy ngc1566 --mode full --create_kernel --apply_mask --sigma 1.5
112
+ ```
113
+ ## Project Structure
114
+
115
+ The repository is organized to separate source code, documentation, and data surveys. Below is the standard directory tree:
116
+
117
+ ```text
118
+ AsTrovello/
119
+ ├── Codes/
120
+ │ ├── AsTrovello_run.py # Main execution script (Master)
121
+ │ ├── AsTrovello_lib.py # Core functions library
122
+ │ └── galaxy_loop.py # Automation for multiple galaxies
123
+ ├── Input/
124
+ │ ├── PHANGS/ # HST images and PSF models
125
+ │ | ├── galaxies/
126
+ | | | ├── galaxies/
127
+ | | | ├── phangs_hst/
128
+ | | | ├── ngc.../
129
+ | | ├── PSF/
130
+ │ └── S4G/ # Spitzer/IRAC images and PSFs
131
+ │ ├── galaxies/
132
+ | | ├── galaxies/
133
+ | | ├── ngc.../
134
+ | ├── PSF/
135
+ └── Output/ # Processed FITS and Hypercubes
136
+ ```
@@ -0,0 +1,110 @@
1
+
2
+ <p align="center">
3
+ <img src="AsTrovello_logo.png" alt="AsTrovello Logo" width="300">
4
+ </p>
5
+
6
+ ## AsTrovello
7
+
8
+ **AsTrovello** is a Python-based astronomical data analysis framework designed to process and align multi-wavelength galaxy data. It focuses on integrating high-resolution photometry from **PHANGS (HST)** with mid-infrared data from **S4G (Spitzer/IRAC)**.
9
+
10
+ The pipeline performs image reprojection, PSF (Point Spread Function) homogenization through convolution, unit conversion to Jansky, and final 3D hypercube creation.
11
+
12
+ The respective data cubes can be found downloaded in:
13
+ **S4G:** https://irsa.ipac.caltech.edu/data/SPITZER/S4G/
14
+ ```text
15
+ ├── File: <target>.phot.1.fits and <target>.phot.2.fits
16
+ ```
17
+ **PHANGS (HST images):** https://archive.stsci.edu/hlsp/phangs
18
+ ```text
19
+ ├── File: File: HST science images (drz) -> hlsp_phangs-hst_hst_wfc3-uvis_<target>_<filter>_v1_exp-drc-sci.fits
20
+ ```
21
+
22
+ ---
23
+
24
+ ## Main Features
25
+
26
+ * **Image Alignment:** Reprojects S4G images onto the HST pixel grid (conserving surface brightness).
27
+ * **PSF Homogenization:** Calculates FWHM and generates convolution kernels via **PyPHER** to match the resolution of different filters to a common "Master" PSF.
28
+ * **Unit Standardization:** Automatically converts `ELECTRONS/S` (HST) and `MJy/sr` (Spitzer) to `Jy/pixel`.
29
+ * **Hypercube Creation:** Builds a 3D FITS hypercube (RA, Dec, Filter) with automated sky masking and spatial cropping (Bounding Box).
30
+
31
+ ---
32
+
33
+ ## Installation & Requirements
34
+
35
+ The pipeline runs on **Ubuntu Linux** and is optimized for use within a Conda environment.
36
+
37
+ 1. **Clone the repository:**
38
+ ```bash
39
+ git clone https://github.com/your-username/AsTrovello.git
40
+ cd AsTrovello
41
+ ```
42
+
43
+ 2. **Setup the environment:**
44
+ We recommend using the provided `environment.yml`
45
+ ```bash
46
+ conda env create -f environment.yml -n new_env_name
47
+ ```
48
+ or creating a dedicated environment with:
49
+ ```bash
50
+ conda create -n capivara python=3.10
51
+ conda activate capivara
52
+ pip install astropy reproject scipy tqdm photutils pypher pandas
53
+ ```
54
+
55
+ ---
56
+
57
+ ## How to Run
58
+
59
+ The main execution script is `AsTrovello_run.py`, located in the `Codes/` directory.
60
+
61
+ ### Basic Syntax
62
+ ```bash
63
+ python Codes/AsTrovello_run.py --galaxy [GALAXY_NAME] --mode [MODE] [FLAGS]
64
+ ```
65
+ ### Execution Modes
66
+
67
+ The pipeline can be executed in different stages depending on your needs. Use the `--mode` argument to select one of the following:
68
+
69
+ | Mode | Description |
70
+ | :--- | :--- |
71
+ | **`full`** | **Complete Pipeline:** Executes alignment, PSF homogenization, convolution, and hypercube creation in a single run. |
72
+ | **`alignment_only`** | **Spatial Alignment:** Only reprojects S4G images to the PHANGS pixel grid. Use this to check coordinate consistency. |
73
+ | **`conv_only`** | **Resolution Matching:** Performs PSF cleaning, kernel generation via PyPHER, and image convolution. |
74
+ | **`cube_only`** | **Data Integration:** Skips processing and builds the final 3D hypercube using existing convolved files. |
75
+
76
+ > **Note:** When using `conv_only` or `full`, remember to include the `--create_kernel` flag if you need to generate new homogenization kernels for the current galaxy.
77
+
78
+ ### Usage Examples
79
+
80
+ Here are the most common ways to run the **AsTrovello** pipeline:
81
+
82
+ #### 1. Full Processing (Standard)
83
+ Runs everything from alignment to the final hypercube. Ideal for a first-time run on a new galaxy.
84
+ ```bash
85
+ python Codes/AsTrovello_run.py --galaxy ngc1566 --mode full --create_kernel --apply_mask --sigma 1.5
86
+ ```
87
+ ## Project Structure
88
+
89
+ The repository is organized to separate source code, documentation, and data surveys. Below is the standard directory tree:
90
+
91
+ ```text
92
+ AsTrovello/
93
+ ├── Codes/
94
+ │ ├── AsTrovello_run.py # Main execution script (Master)
95
+ │ ├── AsTrovello_lib.py # Core functions library
96
+ │ └── galaxy_loop.py # Automation for multiple galaxies
97
+ ├── Input/
98
+ │ ├── PHANGS/ # HST images and PSF models
99
+ │ | ├── galaxies/
100
+ | | | ├── galaxies/
101
+ | | | ├── phangs_hst/
102
+ | | | ├── ngc.../
103
+ | | ├── PSF/
104
+ │ └── S4G/ # Spitzer/IRAC images and PSFs
105
+ │ ├── galaxies/
106
+ | | ├── galaxies/
107
+ | | ├── ngc.../
108
+ | ├── PSF/
109
+ └── Output/ # Processed FITS and Hypercubes
110
+ ```
@@ -0,0 +1,41 @@
1
+ [build-system]
2
+ requires = ["setuptools>=61.0"]
3
+ build-backend = "setuptools.build_meta"
4
+
5
+ [project]
6
+ name = "astrovello"
7
+ version = "0.1.0"
8
+ authors = [
9
+ { name="André Almeida Trovello", email="seu-email@usp.br" },
10
+ ]
11
+ description = "A framework for galaxy morphology and non-parametric decomposition."
12
+ readme = "README.md"
13
+ requires-python = ">=3.10"
14
+ dependencies = [
15
+ "numpy>=2.0",
16
+ "astropy>=6.0",
17
+ "scipy>=1.10",
18
+ "matplotlib>=3.5",
19
+ "pandas>=2.0",
20
+ "photutils>=1.13",
21
+ "reproject>=0.14",
22
+ "regions>=0.10",
23
+ "scikit-image>=0.20",
24
+ "tqdm",
25
+ "psutil",
26
+ "pyyaml",
27
+ "pypher>=0.7.3",
28
+ "pyreadr>=0.5.4",
29
+ "rasterio",
30
+ "shapely",
31
+ "dask",
32
+ "zarr"
33
+ ]
34
+
35
+ [tool.setuptools.packages.find]
36
+ where = ["src"]
37
+
38
+
39
+ [project.scripts]
40
+ astrovello = 'astrovello.astrovello_cli:main'
41
+ astrovello-batch = 'astrovello.batch_runner:main'
@@ -0,0 +1,4 @@
1
+ [egg_info]
2
+ tag_build =
3
+ tag_date = 0
4
+
@@ -0,0 +1,29 @@
1
+ # __init__.py
2
+
3
+ from .alignment import S4G2PHANGS_reproject
4
+
5
+ from .convolution import (
6
+ calculaFWHM_radial_profile,
7
+ final_clean_psf,
8
+ pypher_kernel_creation,
9
+ convolved_dict,
10
+ create_convolvedFITS,
11
+ )
12
+
13
+ from .units import convert2Jansky
14
+
15
+ from .mask import (
16
+ phangs_intersection_mask,
17
+ soma_img,
18
+ mask,
19
+ )
20
+
21
+ from .cube import (
22
+ create_data_cube,
23
+ create_cutouts,
24
+ )
25
+
26
+ from .utils import (
27
+ get_filters,
28
+ log_memory_usage,
29
+ )
@@ -0,0 +1,64 @@
1
+ import os
2
+ from astropy.io import fits
3
+ from astropy.wcs import WCS
4
+ from reproject import reproject_interp
5
+
6
+ # ----------------------------------------------------------------------------------------------------------------------
7
+ # --------------------------------------------- Image alignment -------------------------------------------------------
8
+
9
+ # Reproject S4G (Spitzer) images onto the PHANGS (HST) pixel grid
10
+ def S4G2PHANGS_reproject(s4g_file_path, phangs_ref_file_path, output_path):
11
+ """
12
+ Aligns a Spitzer/IRAC image to the HST pixel grid.
13
+ Returns an array with the same spatial dimensions as the PHANGS reference.
14
+ """
15
+ hdu_phangs = fits.open(phangs_ref_file_path)[0]
16
+ hdu_s4g = fits.open(s4g_file_path)[0]
17
+
18
+ sci_file_s4g = s4g_file_path.name
19
+ # Extract galaxy name and filter index (e.g., IRAC1) from filename
20
+ galaxy_name, filter_mode = sci_file_s4g.split('.')[0].lower(), sci_file_s4g.split('.')[-2]
21
+
22
+ # Initialize WCS (World Coordinate System) for both images
23
+ w_phangs = WCS(hdu_phangs.header)
24
+ w_s4g = WCS(hdu_s4g.header)
25
+
26
+ # Force SIP (Simple Imaging Polynomial) correction type for Spitzer headers
27
+ w_s4g.wcs.ctype = ["RA---TAN-SIP", "DEC--TAN-SIP"]
28
+ print('\n' + 100*'#' + '\nSIP correction added to the header!\n' + 100*'#')
29
+
30
+ # Perform the interpolation/reprojection using reproject_interp
31
+ # Surface brightness is preserved, but flux per pixel is not strictly conserved due to resampling
32
+ array, footprint = reproject_interp((hdu_s4g.data, w_s4g), w_phangs, shape_out=hdu_phangs.data.shape)
33
+
34
+ s4g_new_header = hdu_s4g.header.copy()
35
+
36
+ # Generate new WCS keywords based on the HST reference
37
+ wcs_phangs_header = w_phangs.to_header(relax=True)
38
+
39
+ # Clean old WCS keywords to prevent coordinate conflicts (especially CD vs PC matrices)
40
+ wcs_keys_to_remove = [
41
+ 'CRPIX1', 'CRPIX2', 'CRVAL1', 'CRVAL2', 'CDELT1', 'CDELT2',
42
+ 'CD1_1', 'CD1_2', 'CD2_1', 'CD2_2', 'CTYPE1', 'CTYPE2',
43
+ 'PC1_1', 'PC1_2', 'PC2_1', 'PC2_2'
44
+ ]
45
+ for key in wcs_keys_to_remove:
46
+ if key in s4g_new_header:
47
+ del s4g_new_header[key]
48
+
49
+ # Merge the new WCS information into the S4G header
50
+ s4g_new_header.update(wcs_phangs_header)
51
+ s4g_new_header['CTYPE1'] = 'RA---TAN-SIP'
52
+ s4g_new_header['CTYPE2'] = 'DEC--TAN-SIP'
53
+ s4g_new_header['COMMENT'] = 'Reprojected to PHANGS grid. Flux not conserved per pixel, surface brightness preserved.'
54
+
55
+ # Setup output directory
56
+ output_path = os.path.expanduser(output_path)
57
+ output_directory = os.path.join(output_path, galaxy_name)
58
+ if not os.path.exists(output_directory):
59
+ os.makedirs(output_directory, exist_ok=True)
60
+ print(f"📁 Directory created: {output_directory}")
61
+
62
+ output_name = f'{galaxy_name}_s4g_irac{filter_mode}_on_phangs_projection.fits'
63
+ fits.writeto(os.path.join(output_directory, output_name), array, s4g_new_header, overwrite=True)
64
+ print('\n' + 100*'#' + f'\nReprojected FITS file: {output_name}\n' + 100*'#')
@@ -0,0 +1,240 @@
1
+ import argparse
2
+ import gc
3
+ from pathlib import Path
4
+ import os
5
+ from tqdm import tqdm
6
+ from astropy.io import fits
7
+ import pandas as pd
8
+ import subprocess
9
+ import shutil
10
+ import astrovello as aat
11
+ import numpy as np
12
+
13
+ def main():
14
+ # --- COMMAND LINE ARGUMENT PARSING ---
15
+ # Setup the CLI (Command Line Interface) for the Capivara/AsTrovello pipeline
16
+ parser = argparse.ArgumentParser('Capivara Pipeline Control')
17
+
18
+ # 'mode' allows running specific parts of the pipeline (e.g., only alignment or only the cube)
19
+ parser.add_argument('--mode', type = str, choices = ['full', 'alignment_only', 'conv_only', 'cube_only'], \
20
+ default = 'full', help = 'Execution mode')
21
+ parser.add_argument('--galaxy', type = str, required = True, help = 'Galaxy name (e.g., ngc1566)')
22
+ parser.add_argument('--create_kernel', action='store_true', help='If set, triggers PSF cleaning and PyPHER kernel generation')
23
+ parser.add_argument('--apply_mask', action='store_true', help='If set, generates a signal-based sky mask for the final cube')
24
+ parser.add_argument('--sigma', type = float, default = 1.0, help = 'Sigma threshold for sky mask cutting')
25
+
26
+ args = parser.parse_args()
27
+
28
+ print(100*'#')
29
+ print(f'Executing AsTrovello for {args.galaxy}...\n')
30
+
31
+ # --- DYNAMIC PATH CONFIGURATION ---
32
+ # Identify the location of the running script to define the project root (BASE_DIR)
33
+ # This ensures the code runs correctly regardless of where the AsTrovello folder is placed
34
+ BASE_DIR = Path(os.getcwd())
35
+ print(f'Root Directory: {BASE_DIR}')
36
+
37
+ # Define Input hierarchy (PHANGS/HST and S4G/Spitzer)
38
+ input_dir = BASE_DIR / 'Input'
39
+ if not input_dir.exists():
40
+ print(f"❌ Erro: Pasta 'Input' não encontrada em {BASE_DIR}")
41
+ print("Certifique-se de estar na pasta correta do seu projeto.")
42
+ return
43
+ phangs_dir = input_dir / "PHANGS"
44
+ phangs_imgs = phangs_dir / 'galaxies' / "phangs_hst" / args.galaxy.lower()
45
+ phangs_psf_path = phangs_dir / 'PSF'
46
+
47
+ s4g_dir = input_dir / "S4G"
48
+ s4g_imgs = s4g_dir / 'galaxies' / args.galaxy.lower()
49
+ s4g_psf_path = s4g_dir / 'PSF'
50
+
51
+ # Define Output directory structure
52
+ output_dir = BASE_DIR / 'Output'
53
+ reprojected_path = output_dir / 'reprojected_files'
54
+ kernel_dir = output_dir / 'PSF_Kernels'
55
+ convolved_fits_path = output_dir / 'convolved_fits'
56
+
57
+ # --------------------------------------------------------------------------------------------------------------------
58
+ # ------------------------------------------------ Image alignment --------------------------------------------------
59
+ if args.mode == 'full' or args.mode == 'alignment_only':
60
+ print('Starting image reprojection and alignment...\n')
61
+
62
+ # List PHANGS (Reference) and S4G (To be reprojected) files
63
+ sci_files_phangs = list(phangs_imgs.glob('*exp-drc-sci.fits'))
64
+ sci_files_s4g = list(s4g_imgs.glob('*.fits'))
65
+
66
+ # Use the first HST image as the master grid reference
67
+ phangs_ref_file = sci_files_phangs[0]
68
+
69
+ for file in sci_files_s4g:
70
+ # Match Spitzer resolution/grid to HST grid
71
+ aat.S4G2PHANGS_reproject(file, phangs_ref_file, output_path = reprojected_path)
72
+
73
+ # --------------------------------------------------------------------------------------------------------------------
74
+ # ------------------------------------------------ Image convolution --------------------------------------------------
75
+ if args.mode == 'full' or args.mode == 'conv_only':
76
+ if args.create_kernel:
77
+ print('Initiating convolution process...\n')
78
+
79
+ # --- PSF RESOLUTION ANALYSIS ---
80
+ # Calculate FWHM for all available filters to find the lowest resolution (Master PSF)
81
+ fwhm_s4g, _, files_s4g = aat.calculaFWHM_radial_profile(s4g_psf_path)
82
+ fwhm_phangs, _, files_phangs = aat.calculaFWHM_radial_profile(phangs_psf_path)
83
+
84
+ todos_fwhm = {**fwhm_s4g, **fwhm_phangs}
85
+ df_fwhm = pd.DataFrame(list(todos_fwhm.items()), columns=['Filtro', 'FWHM_pixels'])
86
+ df_fwhm = df_fwhm.sort_values(by='FWHM_pixels').reset_index(drop=True)
87
+
88
+ print("\nResolutions Table:\n", df_fwhm)
89
+
90
+ # The Master PSF is the one with the largest FWHM (coarsest resolution)
91
+ psf_master_name = df_fwhm.iloc[-1]['Filtro']
92
+ print(f"\n⭐ Recommended PSF (master): {psf_master_name}")
93
+
94
+ # --- PSF STANDARDIZATION ---
95
+ survey_data = {
96
+ 'PHANGS': {'path': phangs_psf_path, 'files': files_phangs},
97
+ 'S4G': {'path': s4g_psf_path, 'files': files_s4g}
98
+ }
99
+
100
+ print('\nInitiating PSF cleaning...')
101
+ for s_name, s_info in survey_data.items():
102
+ print(f'\nCleaning {s_name} PSFs...')
103
+ input_p = os.path.expanduser(s_info['path'])
104
+ output_p = os.path.expanduser(str(s_info['path']) + '_LIMPAS')
105
+ os.makedirs(output_p, exist_ok=True)
106
+
107
+ for f_name in s_info['files']:
108
+ # Clean headers and fix parity for PyPHER compatibility
109
+ aat.final_clean_psf(os.path.join(input_p, f_name), os.path.join(output_p, f_name))
110
+
111
+ # Locate the Master PSF file after cleaning
112
+ if psf_master_name.upper().startswith('F'):
113
+ psf_master_path = phangs_dir / 'PSF_LIMPAS' / f'PSFSTD_WFC3UV_{psf_master_name.upper()}.fits'
114
+ elif psf_master_name.upper().startswith('I'):
115
+ psf_master_path = s4g_dir / 'PSF_LIMPAS' / f'{psf_master_name.upper()}_col129_row129.fits'
116
+
117
+ # --- KERNEL GENERATION ---
118
+ # Create shell commands for PyPHER to generate homogenization kernels
119
+ comandos_pypher = aat.pypher_kernel_creation(todos_fwhm, psf_master_path, input_dir, kernel_dir)
120
+
121
+ print(f"\n--- Creating {len(comandos_pypher)} kernels via PyPHER ---")
122
+ for c in comandos_pypher:
123
+ print(f"----- Running: {c} -----")
124
+ try:
125
+ # Execute PyPHER in the shell; check=True raises an error if it fails
126
+ subprocess.run(c, shell=True, check=True)
127
+ print("==> Kernel generated successfully!")
128
+ except subprocess.CalledProcessError as e:
129
+ print(f"==> PyPHER error: {e}")
130
+ continue
131
+
132
+ print("\nKernel processing completed!")
133
+
134
+ else:
135
+ # If kernels already exist, skip generation and identify the current Master filter
136
+ print('Matching kernels already exist. Proceeding to image convolution...')
137
+ kernel_files = list(kernel_dir.glob('*.fits'))
138
+ psf_master_name = str(kernel_files[0].stem).split('_')[-1]
139
+
140
+ # --- IMAGE CONVOLUTION ---
141
+ convolved_fits_path_gal = convolved_fits_path / args.galaxy
142
+ if os.path.exists(convolved_fits_path_gal):
143
+ shutil.rmtree(convolved_fits_path_gal)
144
+
145
+ # Pair images with their specific kernels
146
+ fftconvolve_dict = aat.convolved_dict(path_phangs = phangs_imgs, \
147
+ path_s4g_reprojected = reprojected_path / args.galaxy, path_kernels = kernel_dir)
148
+
149
+ for k in fftconvolve_dict.keys():
150
+ original_fits = fftconvolve_dict[k]['img']['path']
151
+ kernel_fits = fftconvolve_dict[k]['kernel']['path']
152
+ # Run the convolution (FFT based)
153
+ galaxy_name = aat.create_convolvedFITS(original_fits , kernel_fits, output_dir = convolved_fits_path, GAL_NAME = True)
154
+
155
+ # Handle the Master image (it doesn't need convolution, just a copy to the final folder)
156
+ master_source_dir = reprojected_path / galaxy_name
157
+ master_source_name = f'{galaxy_name}_s4g_{psf_master_name}_on_phangs_projection.fits'
158
+ master_source_path = master_source_dir / master_source_name
159
+ master_dest_path = convolved_fits_path / args.galaxy / f'{galaxy_name}_s4g_irac2_master.fits'
160
+
161
+ os.makedirs(convolved_fits_path, exist_ok=True)
162
+ shutil.copy2(master_source_path, master_dest_path)
163
+
164
+ print(100 * '#')
165
+ print(f'Master file {psf_master_name} from s4g survey:\nFITS saved to: {master_dest_path}\n' + 100 * '#')
166
+
167
+ # -------------------------------------------------------------------------------------------------------------------------
168
+ # ----------------------------------------------- FITS unit conversion-----------------------------------------------------
169
+ if args.mode == 'full' or args.mode == 'conv_only':
170
+ print('\n ----- Starting unit conversion -----' )
171
+ convolved_fits_path_gal = convolved_fits_path / args.galaxy
172
+ conv_files = list(convolved_fits_path_gal.glob('*.fits'))
173
+
174
+ for file in conv_files:
175
+ filter_name = file.name.split('_')[-2]
176
+ print(f'Converting {filter_name} image units to Jy/pixel:')
177
+ # Perform astronomical unit conversion (e/s or MJy/sr -> Jy/pix)
178
+ new_data, new_header = aat.convert2Jansky(file)
179
+
180
+ new_hdu = fits.PrimaryHDU(new_data, new_header)
181
+ file_path = file.parent / f'{file.stem}_Jy_per_pixel{file.suffix}'
182
+ new_hdu.writeto(file_path, overwrite=True)
183
+ print(f'Saved file in: {file_path}\n' + 100 * '#')
184
+
185
+ # -------------------------------------------------------------------------------------------------------------------------
186
+ # ------------------------------------------------ Hypercube creation -----------------------------------------------------
187
+ if args.mode == 'full' or args.mode == 'cube_only':
188
+ # Hypercube configuration parameters
189
+ VALOR_N_SIGMA = args.sigma
190
+ APLICAR_MASCARA = args.apply_mask
191
+
192
+ output_dir_cube = output_dir / 'datacubes' / args.galaxy
193
+ loc = convolved_fits_path / args.galaxy
194
+ os.makedirs(output_dir_cube, exist_ok=True)
195
+
196
+ # Collect all processed Jy/pixel files
197
+ file_list = list(loc.glob('*_Jy_per_pixel.fits'))
198
+ ref_file = list(phangs_imgs.glob('*f275w*sci.fits'))
199
+
200
+ if not file_list:
201
+ print("Error: File list empty! Verify paths.")
202
+ return
203
+
204
+ # Sort files by filter name for consistency in the cube layers
205
+ fits_files = sorted([(f, f.name.split('_')[2]) for f in file_list])
206
+ ref_header = fits.getheader(fits_files[0][0], ext=0)
207
+
208
+ aligned_images, filter_names = [], []
209
+ for file, filt in tqdm(fits_files, desc="Reading FITS for hypercube"):
210
+ aligned_images.append(fits.getdata(file, ext=0).astype(np.float32))
211
+ filter_names.append(filt)
212
+
213
+ if aligned_images:
214
+ # --- HYPERCUBE ASSEMBLY ---
215
+ temp_name = output_dir_cube / 'temp_cube.fits'
216
+ cube, cube_header = aat.create_data_cube(
217
+ aligned_images, filter_names, ref_file, ref_header,
218
+ temp_name, aplicar_mask=APLICAR_MASCARA, N_SIGMA=VALOR_N_SIGMA
219
+ )
220
+
221
+ # --- DYNAMIC RENAMING ---
222
+ # Rename based on actual pixel dimensions after BBox cutout
223
+ _, final_ny, final_nx = cube.shape
224
+ final_name = output_dir_cube / f'{args.galaxy}_datacube_sci_{final_nx}x{final_ny}_Jy_per_pixel.fits'
225
+ temp_name.rename(final_name)
226
+ print(f"==> Cube created: {final_name.name}")
227
+
228
+ # Memory cleanup
229
+ del aligned_images
230
+ gc.collect()
231
+
232
+ # --- AUTOMATED ZOOM (Cutout) ---
233
+ # Create a 300x300 pixel zoom centered on the reference coordinate (CRPIX)
234
+ cx, cy = int(cube_header['CRPIX1']), int(cube_header['CRPIX2'])
235
+ regions = [(output_dir_cube / f'{args.galaxy}_datacube_sci_{final_nx}x{final_ny}_zoomed.fits',
236
+ cx-150, cx+150, cy-150, cy+150)]
237
+ aat.create_cutouts(cube, cube_header, regions)
238
+
239
+ if __name__ == "__main__":
240
+ main()
@@ -0,0 +1,57 @@
1
+ import subprocess
2
+ from pathlib import Path
3
+ import os
4
+
5
+ def main():
6
+ # --- PATH CONFIGURATION ---
7
+ # Identify the root directory of the project to locate Input folders
8
+ BASE_DIR = Path(os.getcwd())
9
+ input_dir = BASE_DIR / 'Input'
10
+
11
+ # Define paths for the galaxy folders in both surveys
12
+ input_phangs = input_dir / 'PHANGS' / 'galaxies' / 'phangs_hst'
13
+ input_s4g = input_dir / 'S4G' / 'galaxies'
14
+
15
+ # --- GALAXY LISTING ---
16
+ # Generate sorted lists of directory names (galaxy names like ngc1566)
17
+ # iterdir() scans the folder and x.is_dir() ensures we only get folders, not files
18
+ gal_names_phangs = sorted([x.name for x in input_phangs.iterdir() if x.is_dir()])
19
+ gal_names_s4g = sorted([x.name for x in input_s4g.iterdir() if x.is_dir()])
20
+
21
+ # --- EXECUTION PARAMETERS ---
22
+ mode = 'full'
23
+ apply_mask = '--apply_mask'
24
+ create_kernel = '--create_kernel'
25
+ sigma = '1.0'
26
+
27
+ # --- BATCH PROCESSING LOGIC ---
28
+ # Only proceed if the list of galaxies is identical across both surveys
29
+ if gal_names_phangs == gal_names_s4g:
30
+ print(f'Galaxies found: {len(gal_names_phangs)} - {gal_names_phangs}')
31
+ print(100*'*')
32
+
33
+ # Iterate through the list of galaxies
34
+ for i, galaxy in enumerate(gal_names_phangs):
35
+
36
+ # LOGIC: Only generate kernels for the first galaxy (i == 0)
37
+ # This prevents the pipeline from recalculating the same kernels (e.g., F275W to IRAC2)
38
+ # multiple times, since kernels are often instrument-dependent, not galaxy-dependent.
39
+ extra_args = f"{apply_mask} {create_kernel}" if i == 0 else f"{apply_mask}"
40
+
41
+ # Shell command
42
+ cmd = (
43
+ f"python AsTrovello_run.py --mode {mode} --galaxy {galaxy} "
44
+ f"{extra_args} --sigma {sigma}"
45
+ )
46
+
47
+ # Execute the main pipeline script for the current galaxy
48
+ # shell=True allows running the command as a string
49
+ # check=True stops the entire batch if one galaxy fails
50
+ subprocess.run(cmd, shell=True, check=True)
51
+ else:
52
+ # Safety check to avoid mismatched data between HST and Spitzer folders
53
+ print('Lists do not match!. Guarantee that all surveys have the same galaxies')
54
+
55
+
56
+ if __name__ == '__main__':
57
+ main()
@@ -0,0 +1,210 @@
1
+ import numpy as np
2
+ import os
3
+ from astropy.io import fits
4
+ import shutil
5
+ from scipy.signal import fftconvolve
6
+ from collections import defaultdict
7
+ from pathlib import Path
8
+ # ----------------------------------------------------------------------------------------------------------------------
9
+ # -------------------------------------------- Image convolution ------------------------------------------------------
10
+
11
+ def get_fwhm_simple(data):
12
+ """
13
+ Estimates the Full Width at Half Maximum (FWHM) using spatial moments.
14
+ Suitable for centered point sources (PSFs).
15
+ FWHM = 2.355 * sigma (for a Gaussian profile).
16
+ """
17
+ y, x = np.indices(data.shape)
18
+ center_y, center_x = np.array(data.shape) // 2
19
+
20
+ # Calculate brightness-weighted spatial variance
21
+ r = np.sqrt((x - center_x)**2 + (y - center_y)**2)
22
+ variance = np.sum(data * r**2) / np.sum(data)
23
+ sigma = np.sqrt(variance)
24
+
25
+ return 2.355 * sigma
26
+
27
+ def radial_profile(data, center):
28
+ """Generates a 1D radial brightness profile from a 2D image."""
29
+ y, x = np.indices(data.shape)
30
+ r = np.sqrt((x - center[0])**2 + (y - center[1])**2).astype(int)
31
+ tbin = np.bincount(r.ravel(), data.ravel())
32
+ nr = np.bincount(r.ravel())
33
+ return tbin / nr
34
+
35
+ def calculaFWHM_radial_profile(files_path):
36
+ """
37
+ Iterates through a folder of PSF files, filters them by survey,
38
+ and returns dictionaries containing their FWHM and radial profiles.
39
+ """
40
+ FWHM_dict, dict_profiles, valid_files = {}, {}, []
41
+ file_list = list(files_path.glob('*.fits'))
42
+
43
+ for file in file_list:
44
+ # Survey-specific filename parsing
45
+ if 'S4G' in str(files_path):
46
+ if file.name == 'IRAC1_col129_row129.fits': filter_name = 'irac1'
47
+ elif file.name == 'IRAC2_col129_row129.fits': filter_name = 'irac2'
48
+ else: continue
49
+ elif 'PHANGS' in str(files_path):
50
+ filter_name = file.name.replace('.fits', '').split('_')[-1].lower()
51
+ else: continue
52
+
53
+ try:
54
+ with fits.open(file, ignore_missing_end=True) as hdu:
55
+ # Dynamically find the data HDU
56
+ data = next((h.data for h in hdu if h.data is not None), None)
57
+ if data is not None:
58
+ if data.ndim == 3: data = data[0] # Flatten 3D PSF cubes
59
+
60
+ FWHM_dict[filter_name] = get_fwhm_simple(data)
61
+ dict_profiles[filter_name] = radial_profile(data, (data.shape[1]//2, data.shape[0]//2))
62
+ valid_files.append(file.name)
63
+ print(f"Succesfully read: {filter_name}")
64
+ except Exception as e:
65
+ print(f"Processing error {file.name}: {e}")
66
+
67
+ return FWHM_dict, dict_profiles, valid_files
68
+
69
+ def final_clean_psf(input_file, output_file):
70
+ """
71
+ Standardizes PSF headers for PyPHER compatibility.
72
+ Calculates pixel scales and ensures correct centering and coordinate keywords.
73
+ """
74
+ if 'WFC3UV' in input_file:
75
+ # HST scale: native 0.04"/pix, oversampled 4x
76
+ pixel_scale_arcsec = 0.0395 / 4.0
77
+ pixel_scale_deg = pixel_scale_arcsec / 3600.0
78
+
79
+ with fits.open(input_file, ignore_missing_end=True) as hdu:
80
+ # Average the PSF cube to get a 2D representative PSF
81
+ data_2d = np.mean(hdu[0].data, axis=0)
82
+ new_hdu = fits.PrimaryHDU(data_2d)
83
+
84
+ # Inject WCS keywords required by PyPHER/Astropy
85
+ new_hdu.header.update({
86
+ 'CTYPE1': 'RA---TAN', 'CTYPE2': 'DEC--TAN',
87
+ 'CRVAL1': 0.0, 'CRVAL2': 0.0,
88
+ 'CRPIX1': 51.0, 'CRPIX2': 51.0,
89
+ 'CDELT1': -pixel_scale_deg, 'CDELT2': pixel_scale_deg,
90
+ 'PIXSCALE': pixel_scale_arcsec
91
+ })
92
+ new_hdu.writeto(output_file, overwrite=True)
93
+ print(f"==> File ready to be applied in PyPHER: {os.path.basename(output_file)}")
94
+
95
+ elif any(x in input_file for x in ['IRAC1', 'IRAC2']):
96
+ # Spitzer scale: native ~1.22"/pix, oversampled 5x
97
+ pixel_scale_arcsec = 1.221/5.0 if 'IRAC1' in input_file else 1.213/5.0
98
+ pixel_scale_deg = pixel_scale_arcsec / 3600.0
99
+
100
+ with fits.open(input_file) as hdu:
101
+ data_raw = hdu[0].data
102
+ # Force odd parity: PyPHER prefers kernels/PSFs with an odd number of pixels
103
+ data_2d = data_raw[:-1, :-1] if data_raw.shape[0] % 2 == 0 else data_raw
104
+
105
+ new_hdu = fits.PrimaryHDU(data_2d)
106
+ new_hdu.header.update({
107
+ 'CTYPE1': 'RA---TAN', 'CTYPE2': 'DEC--TAN',
108
+ 'CRVAL1': 0.0, 'CRVAL2': 0.0,
109
+ 'CRPIX1': (data_2d.shape[1] // 2) + 1, 'CRPIX2': (data_2d.shape[0] // 2) + 1,
110
+ 'CDELT1': -pixel_scale_deg, 'CDELT2': pixel_scale_deg,
111
+ 'PIXSCALE': pixel_scale_arcsec
112
+ })
113
+ new_hdu.writeto(output_file, overwrite=True)
114
+ print(f"==> File ready to be applied in PyPHER: {os.path.basename(output_file)}")
115
+
116
+ def pypher_kernel_creation(todos_fwhm, psf_master_path, input_dir, output_dir):
117
+ """
118
+ Prepares a list of shell commands for the PyPHER library to generate homogenization kernels.
119
+ It identifies which PSF belongs to which survey to locate files in the 'PSF_LIMPAS' folders.
120
+ """
121
+ psf_path_phangs_clean = input_dir / 'PHANGS' / 'PSF_LIMPAS'
122
+ psf_path_s4g_clean = input_dir / 'S4G' / 'PSF_LIMPAS'
123
+ psf_master_name = psf_master_path.stem.split('_')[0].lower()
124
+
125
+ if os.path.exists(output_dir):
126
+ print(f"==> Removing previous directory: {os.path.basename(output_dir)}")
127
+ shutil.rmtree(output_dir)
128
+ os.makedirs(output_dir)
129
+
130
+ comandos_pypher = []
131
+ for filtro in todos_fwhm.keys():
132
+ if filtro == psf_master_name: continue
133
+
134
+ # Determine High-Resolution PSF path
135
+ if filtro.startswith('f'):
136
+ psf_high_res = os.path.join(psf_path_phangs_clean, f"PSFSTD_WFC3UV_{filtro.upper()}.fits")
137
+ elif filtro.startswith('i'):
138
+ psf_high_res = os.path.join(psf_path_s4g_clean, f"{filtro.upper()}_col129_row129.fits")
139
+ else: continue
140
+
141
+ kernel_name = os.path.join(output_dir, f"kernel_{filtro}_to_{psf_master_name}.fits")
142
+ # Format command: pypher [HR_PSF] [Target_PSF] [Output_Kernel]
143
+ comandos_pypher.append(f"pypher {psf_high_res} {psf_master_path} {kernel_name}")
144
+
145
+ return comandos_pypher
146
+
147
+ def convolved_dict(path_phangs, path_s4g_reprojected, path_kernels):
148
+ """
149
+ Organizes all images and kernels into a nested dictionary indexed by filter.
150
+ Used to pair the correct kernel with its corresponding image for convolution.
151
+ """
152
+ phangs_files = list(path_phangs.glob('*exp-drc-sci.fits'))
153
+ s4g_files = list(path_s4g_reprojected.glob('*.fits'))
154
+ kernel_files = list(path_kernels.glob('*.fits'))
155
+
156
+ all_files = phangs_files + s4g_files + kernel_files
157
+ filter_info = [f.name.split('_')[1] for f in sorted(kernel_files)]
158
+
159
+ fftconvolve_dict = defaultdict(lambda: {'kernel': {}, 'img': {}})
160
+ for f in filter_info:
161
+ for file in all_files:
162
+ file_string = str(file).lower()
163
+ if f in file_string:
164
+ key = 'kernel' if 'kernel' in file_string else 'img'
165
+ fftconvolve_dict[f][key]['path'] = file
166
+ fftconvolve_dict[f][key]['name'] = file.name
167
+ return fftconvolve_dict
168
+
169
+ def create_convolvedFITS(original_fits, kernel_fits, output_dir, GAL_NAME=False):
170
+ """
171
+ Applies the convolution kernel to an image using FFT (Fast Fourier Transform).
172
+ Standardizes data to handle NaNs and saves the degraded resolution image.
173
+ """
174
+ output_dir = Path(output_dir).expanduser()
175
+ if not os.path.exists(output_dir): os.makedirs(output_dir)
176
+
177
+ with fits.open(kernel_fits) as hdu_k, fits.open(original_fits) as hdu_i:
178
+ kernel_data = np.nan_to_num(hdu_k[0].data)
179
+ img_data = hdu_i[0].data
180
+ img_header = hdu_i[0].header
181
+
182
+ # Normalize kernel to preserve flux
183
+ if np.sum(kernel_data) != 0:
184
+ kernel_norm = kernel_data / np.sum(kernel_data)
185
+
186
+ original_file_name = original_fits.name
187
+ info = original_file_name.split('_')
188
+
189
+ # Identify survey to apply specific NaN handling/naming
190
+ if 'phangs-hst' in info:
191
+ convolved_img = fftconvolve(img_data, kernel_norm, mode='same')
192
+ gal_name, survey, filt = info[4].lower(), 'phangs', info[5].lower()
193
+ elif 's4g' in original_file_name:
194
+ # IRAC images often have NaNs that break FFT; convert to zero
195
+ img_data_limpa = np.nan_to_num(img_data, nan=0.0)
196
+ convolved_img = fftconvolve(img_data_limpa, kernel_norm, mode='same')
197
+ gal_name, survey, filt = info[0].lower(), 's4g', info[2].lower()
198
+
199
+ convolved_fits = fits.PrimaryHDU(data=convolved_img, header=img_header)
200
+ print(100*'#' + f'\nConvolving {filt} filter from {survey} survey:')
201
+
202
+ output_path = os.path.join(output_dir, gal_name)
203
+ if not os.path.exists(output_path): os.makedirs(output_path)
204
+
205
+ out_file = os.path.join(output_path, f'{gal_name}_{survey}_{filt}_convolved.fits')
206
+ convolved_fits.writeto(out_file, overwrite=True)
207
+ print(f'Convolved FITS saved to: {out_file}\n' + 100*'#')
208
+
209
+ if GAL_NAME: return gal_name
210
+
@@ -0,0 +1,78 @@
1
+ import numpy as np
2
+ from .mask import soma_img, mask, phangs_intersection_mask
3
+ from astropy.wcs import WCS
4
+ from astropy.io import fits
5
+
6
+
7
+
8
+ # ----------------------------------------------------------------------------------------------------------------------
9
+ # -------------------------------------------- Data Cube creation-------------------------------------------------------
10
+
11
+ def create_data_cube(aligned_images, filter_names, ref_file, ref_header, output_filename,
12
+ aplicar_mask=True, N_SIGMA=3, padding=50):
13
+ """
14
+ Constructs a 3D FITS Hypercube (RA, DEC, Filter).
15
+ Includes automatic sky masking, background subtraction, and Bounding Box cutout.
16
+ Updates WCS to 3D.
17
+ """
18
+ print(f'\nInitiating hypercube creation (N_SIGMA={N_SIGMA})...')
19
+ ny, nx = aligned_images[0].shape
20
+ cubo = np.empty((len(filter_names), ny, nx), dtype=np.float32)
21
+
22
+ # 1. Determine final processing mask (Signal-based or Border-based)
23
+ if aplicar_mask:
24
+ summed = soma_img(aligned_images, ref_file)
25
+ _, mask_final = mask(summed, N_SIGMA=N_SIGMA)
26
+ else:
27
+ mask_final = phangs_intersection_mask(ref_file)
28
+
29
+ # 2. Fill the cube layers, setting non-mask regions to NaN
30
+ for i, img_atual in enumerate(aligned_images):
31
+ cubo[i, :, :] = np.where(mask_final, img_atual, np.nan) if mask_final is not None else img_atual
32
+
33
+ # 3. Bounding Box Cutout: Shrink the cube to the relevant area plus padding
34
+ y_off, x_off = 0, 0
35
+ if mask_final is not None:
36
+ coords = np.argwhere(mask_final)
37
+ if coords.size > 0:
38
+ y_min, x_min = coords.min(axis=0); y_max, x_max = coords.max(axis=0)
39
+ y_min, y_max = max(0, y_min - padding), min(ny, y_max + padding)
40
+ x_min, x_max = max(0, x_min - padding), min(nx, x_max + padding)
41
+ cubo = cubo[:, y_min:y_max, x_min:x_max]
42
+ y_off, x_off = y_min, x_min
43
+ print(f"✂️ Bounding box cutout: {ny}x{nx} -> {cubo.shape[1]}x{cubo.shape[2]}")
44
+
45
+ # 4. Construct 3D WCS Header
46
+ # Adjust Reference Pixels (CRPIX) to reflect the BBox shift
47
+ w_2d = WCS(ref_header, naxis=2)
48
+ w_3d = WCS(naxis=3)
49
+ for i in [0, 1]:
50
+ for p in ['crpix', 'crval', 'cdelt', 'ctype', 'cunit']:
51
+ try:
52
+ val = getattr(w_2d.wcs, p)[i]
53
+ if p == 'crpix': getattr(w_3d.wcs, p)[i] = val - (x_off if i == 0 else y_off)
54
+ else: getattr(w_3d.wcs, p)[i] = val
55
+ except: continue
56
+
57
+ # Set the 3rd axis (Filters)
58
+ w_3d.wcs.crpix[2], w_3d.wcs.crval[2], w_3d.wcs.cdelt[2], w_3d.wcs.ctype[2] = 1, 0, 1, 'FILTER'
59
+ cube_header = w_3d.to_header()
60
+ cube_header['BUNIT'] = 'Jy/pixel'
61
+ for i, filt in enumerate(filter_names): cube_header[f'FILT{i+1:03d}'] = filt
62
+
63
+ fits.writeto(output_filename, cubo, header=cube_header, overwrite=True)
64
+ return cubo, cube_header
65
+
66
+ def create_cutouts(cube, cube_header, regions):
67
+ """Extracts spatial sub-cubes (zooms) from the main hypercube."""
68
+ for path, x_start, x_end, y_start, y_end in regions:
69
+ if x_end > cube.shape[2] or y_end > cube.shape[1] or x_start < 0 or y_start < 0:
70
+ print(f"Warning: Zoom '{path.name}' out of bounds. Continuing.")
71
+ continue
72
+
73
+ cube_cut = cube[:, y_start:y_end, x_start:x_end]
74
+ cut_h = cube_header.copy()
75
+ cut_h['CRPIX1'] -= x_start
76
+ cut_h['CRPIX2'] -= y_start
77
+ fits.writeto(path, cube_cut, header=cut_h, overwrite=True)
78
+ print(f"==> Cutout saved to: {path.name}")
@@ -0,0 +1,45 @@
1
+ from astropy.io import fits
2
+ import tqdm
3
+ import numpy as np
4
+
5
+
6
+ # ----------------------------------------------------------------------------------------------------------------------
7
+ # --------------------------------------------------- Mask ----------------------------------------------------------
8
+
9
+ def phangs_intersection_mask(ref_file):
10
+ """Detects the valid observation area by checking where data exists (!= 0)."""
11
+ print('Finding intersection area between surveys...')
12
+ if not ref_file: return None
13
+ ref_data = fits.getdata(ref_file[0], ext=0)
14
+ return (ref_data != 0)
15
+
16
+ def soma_img(aligned_images, ref_file):
17
+ """Integrates all images into a single 2D plane to create a signal-based mask."""
18
+ res, inter_mask = None, phangs_intersection_mask(ref_file)
19
+ for data_orig in tqdm(aligned_images, desc="Integrating for mask"):
20
+ data = data_orig.copy()
21
+ if inter_mask is not None and data.shape == inter_mask.shape:
22
+ data[~inter_mask] = 0 # Zero out regions outside the observation footprint
23
+ res = data if res is None else res + data
24
+ return res
25
+
26
+ def mask(data, N_SIGMA=3):
27
+ """
28
+ Performs sky subtraction and generates a signal mask using Median Absolute Deviation (MAD).
29
+ Identifies 'objects' as pixels N_SIGMA above the background noise level.
30
+ """
31
+ data_filtrada = data[data != 0]
32
+ if data_filtrada.size == 0: return np.zeros_like(data), np.zeros_like(data, dtype=bool)
33
+
34
+ local_bg = np.nanmedian(data_filtrada)
35
+ data_subtraida = data - local_bg
36
+ residuo_filtrado = data_subtraida[data != 0]
37
+
38
+ noise_median = np.nanmedian(residuo_filtrado)
39
+ mad = np.nanmedian(np.abs(residuo_filtrado - noise_median))
40
+ sigma_bg = 1.4826 * mad # Conversion factor from MAD to Sigma
41
+
42
+ mask_res = (data_subtraida > (N_SIGMA * sigma_bg)) if sigma_bg > 0 else np.zeros_like(data, dtype=bool)
43
+ if sigma_bg > 0: data_subtraida[data_subtraida < 0] = 0
44
+
45
+ return data_subtraida, mask_res
@@ -0,0 +1,33 @@
1
+ import numpy as np
2
+ from astropy.io import fits
3
+
4
+ # -------------------------------------------------------------------------------------------------------------------------
5
+ # ----------------------------------------------- FITS unit conversion-----------------------------------------------------
6
+
7
+ def convert2Jansky(fits_file):
8
+ """
9
+ Converts image units to Jansky per pixel.
10
+ Handles HST (Electrons/s to Jy via PHOTFNU) and Spitzer (MJy/sr to Jy/pixel via pixel area).
11
+ """
12
+ with fits.open(fits_file) as hdu:
13
+ data, header = hdu[0].data, hdu[0].header
14
+
15
+ new_data, new_header = data.copy(), header.copy()
16
+
17
+ # HST Case: Using Photometric Flux density (Jy*s/e)
18
+ if header.get('BUNIT') == 'ELECTRONS/S':
19
+ new_data *= header['PHOTFNU']
20
+ new_header['BUNIT'] = 'Jy/pixel'
21
+ print("HST: Converted using PHOTFNU")
22
+
23
+ # Spitzer Case: Surface brightness to flux per pixel
24
+ elif header.get('BUNIT') == 'MJy/sr':
25
+ pixel_area_arcsec2 = np.abs(header['PXSCAL1'] * header['PXSCAL2'])
26
+ # 1 arcsec² ≈ 2.35e-11 steradians
27
+ pixel_area_sr = pixel_area_arcsec2 * 2.3504430539e-11
28
+ # Convert MJy -> Jy (1e6) and sr -> pixel area
29
+ new_data = new_data * 1e6 * pixel_area_sr
30
+ new_header['BUNIT'] = 'Jy/pixel'
31
+ print(f"S4G: Converted using pixel area ({pixel_area_arcsec2} arcsec2)")
32
+
33
+ return new_data, new_header
@@ -0,0 +1,14 @@
1
+ import psutil
2
+ import os
3
+
4
+ # ----------------------------------------------------------------------------------------------------------------------
5
+ # -------------------------------------------- Utility Functions -----------------------------------------------------
6
+
7
+ def get_filters(file_list, start, position):
8
+ """Extracts unique filter names from a list of filenames based on naming conventions."""
9
+ return list(set(f.split('_')[position] for f in file_list if f.startswith(start)))
10
+
11
+ def log_memory_usage():
12
+ """Prints the current resident set size (RSS) memory consumption of the script."""
13
+ process = psutil.Process(os.getpid())
14
+ print(f"Uso de memória: {process.memory_info().rss / 1024 ** 2:.2f} MB")
@@ -0,0 +1,136 @@
1
+ Metadata-Version: 2.4
2
+ Name: astrovello
3
+ Version: 0.1.0
4
+ Summary: A framework for galaxy morphology and non-parametric decomposition.
5
+ Author-email: André Almeida Trovello <seu-email@usp.br>
6
+ Requires-Python: >=3.10
7
+ Description-Content-Type: text/markdown
8
+ Requires-Dist: numpy>=2.0
9
+ Requires-Dist: astropy>=6.0
10
+ Requires-Dist: scipy>=1.10
11
+ Requires-Dist: matplotlib>=3.5
12
+ Requires-Dist: pandas>=2.0
13
+ Requires-Dist: photutils>=1.13
14
+ Requires-Dist: reproject>=0.14
15
+ Requires-Dist: regions>=0.10
16
+ Requires-Dist: scikit-image>=0.20
17
+ Requires-Dist: tqdm
18
+ Requires-Dist: psutil
19
+ Requires-Dist: pyyaml
20
+ Requires-Dist: pypher>=0.7.3
21
+ Requires-Dist: pyreadr>=0.5.4
22
+ Requires-Dist: rasterio
23
+ Requires-Dist: shapely
24
+ Requires-Dist: dask
25
+ Requires-Dist: zarr
26
+
27
+
28
+ <p align="center">
29
+ <img src="AsTrovello_logo.png" alt="AsTrovello Logo" width="300">
30
+ </p>
31
+
32
+ ## AsTrovello
33
+
34
+ **AsTrovello** is a Python-based astronomical data analysis framework designed to process and align multi-wavelength galaxy data. It focuses on integrating high-resolution photometry from **PHANGS (HST)** with mid-infrared data from **S4G (Spitzer/IRAC)**.
35
+
36
+ The pipeline performs image reprojection, PSF (Point Spread Function) homogenization through convolution, unit conversion to Jansky, and final 3D hypercube creation.
37
+
38
+ The respective data cubes can be found downloaded in:
39
+ **S4G:** https://irsa.ipac.caltech.edu/data/SPITZER/S4G/
40
+ ```text
41
+ ├── File: <target>.phot.1.fits and <target>.phot.2.fits
42
+ ```
43
+ **PHANGS (HST images):** https://archive.stsci.edu/hlsp/phangs
44
+ ```text
45
+ ├── File: File: HST science images (drz) -> hlsp_phangs-hst_hst_wfc3-uvis_<target>_<filter>_v1_exp-drc-sci.fits
46
+ ```
47
+
48
+ ---
49
+
50
+ ## Main Features
51
+
52
+ * **Image Alignment:** Reprojects S4G images onto the HST pixel grid (conserving surface brightness).
53
+ * **PSF Homogenization:** Calculates FWHM and generates convolution kernels via **PyPHER** to match the resolution of different filters to a common "Master" PSF.
54
+ * **Unit Standardization:** Automatically converts `ELECTRONS/S` (HST) and `MJy/sr` (Spitzer) to `Jy/pixel`.
55
+ * **Hypercube Creation:** Builds a 3D FITS hypercube (RA, Dec, Filter) with automated sky masking and spatial cropping (Bounding Box).
56
+
57
+ ---
58
+
59
+ ## Installation & Requirements
60
+
61
+ The pipeline runs on **Ubuntu Linux** and is optimized for use within a Conda environment.
62
+
63
+ 1. **Clone the repository:**
64
+ ```bash
65
+ git clone https://github.com/your-username/AsTrovello.git
66
+ cd AsTrovello
67
+ ```
68
+
69
+ 2. **Setup the environment:**
70
+ We recommend using the provided `environment.yml`
71
+ ```bash
72
+ conda env create -f environment.yml -n new_env_name
73
+ ```
74
+ or creating a dedicated environment with:
75
+ ```bash
76
+ conda create -n capivara python=3.10
77
+ conda activate capivara
78
+ pip install astropy reproject scipy tqdm photutils pypher pandas
79
+ ```
80
+
81
+ ---
82
+
83
+ ## How to Run
84
+
85
+ The main execution script is `AsTrovello_run.py`, located in the `Codes/` directory.
86
+
87
+ ### Basic Syntax
88
+ ```bash
89
+ python Codes/AsTrovello_run.py --galaxy [GALAXY_NAME] --mode [MODE] [FLAGS]
90
+ ```
91
+ ### Execution Modes
92
+
93
+ The pipeline can be executed in different stages depending on your needs. Use the `--mode` argument to select one of the following:
94
+
95
+ | Mode | Description |
96
+ | :--- | :--- |
97
+ | **`full`** | **Complete Pipeline:** Executes alignment, PSF homogenization, convolution, and hypercube creation in a single run. |
98
+ | **`alignment_only`** | **Spatial Alignment:** Only reprojects S4G images to the PHANGS pixel grid. Use this to check coordinate consistency. |
99
+ | **`conv_only`** | **Resolution Matching:** Performs PSF cleaning, kernel generation via PyPHER, and image convolution. |
100
+ | **`cube_only`** | **Data Integration:** Skips processing and builds the final 3D hypercube using existing convolved files. |
101
+
102
+ > **Note:** When using `conv_only` or `full`, remember to include the `--create_kernel` flag if you need to generate new homogenization kernels for the current galaxy.
103
+
104
+ ### Usage Examples
105
+
106
+ Here are the most common ways to run the **AsTrovello** pipeline:
107
+
108
+ #### 1. Full Processing (Standard)
109
+ Runs everything from alignment to the final hypercube. Ideal for a first-time run on a new galaxy.
110
+ ```bash
111
+ python Codes/AsTrovello_run.py --galaxy ngc1566 --mode full --create_kernel --apply_mask --sigma 1.5
112
+ ```
113
+ ## Project Structure
114
+
115
+ The repository is organized to separate source code, documentation, and data surveys. Below is the standard directory tree:
116
+
117
+ ```text
118
+ AsTrovello/
119
+ ├── Codes/
120
+ │ ├── AsTrovello_run.py # Main execution script (Master)
121
+ │ ├── AsTrovello_lib.py # Core functions library
122
+ │ └── galaxy_loop.py # Automation for multiple galaxies
123
+ ├── Input/
124
+ │ ├── PHANGS/ # HST images and PSF models
125
+ │ | ├── galaxies/
126
+ | | | ├── galaxies/
127
+ | | | ├── phangs_hst/
128
+ | | | ├── ngc.../
129
+ | | ├── PSF/
130
+ │ └── S4G/ # Spitzer/IRAC images and PSFs
131
+ │ ├── galaxies/
132
+ | | ├── galaxies/
133
+ | | ├── ngc.../
134
+ | ├── PSF/
135
+ └── Output/ # Processed FITS and Hypercubes
136
+ ```
@@ -0,0 +1,17 @@
1
+ README.md
2
+ pyproject.toml
3
+ src/astrovello/__init__.py
4
+ src/astrovello/alignment.py
5
+ src/astrovello/astrovello_cli.py
6
+ src/astrovello/batch_runner.py
7
+ src/astrovello/convolution.py
8
+ src/astrovello/cube.py
9
+ src/astrovello/mask.py
10
+ src/astrovello/units.py
11
+ src/astrovello/utils.py
12
+ src/astrovello.egg-info/PKG-INFO
13
+ src/astrovello.egg-info/SOURCES.txt
14
+ src/astrovello.egg-info/dependency_links.txt
15
+ src/astrovello.egg-info/entry_points.txt
16
+ src/astrovello.egg-info/requires.txt
17
+ src/astrovello.egg-info/top_level.txt
@@ -0,0 +1,3 @@
1
+ [console_scripts]
2
+ astrovello = astrovello.astrovello_cli:main
3
+ astrovello-batch = astrovello.batch_runner:main
@@ -0,0 +1,18 @@
1
+ numpy>=2.0
2
+ astropy>=6.0
3
+ scipy>=1.10
4
+ matplotlib>=3.5
5
+ pandas>=2.0
6
+ photutils>=1.13
7
+ reproject>=0.14
8
+ regions>=0.10
9
+ scikit-image>=0.20
10
+ tqdm
11
+ psutil
12
+ pyyaml
13
+ pypher>=0.7.3
14
+ pyreadr>=0.5.4
15
+ rasterio
16
+ shapely
17
+ dask
18
+ zarr
@@ -0,0 +1 @@
1
+ astrovello