tglc 0.5.7__tar.gz → 0.5.8__tar.gz

{tglc-0.5.7/tglc.egg-info → tglc-0.5.8}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: tglc
-Version: 0.5.7
+Version: 0.5.8
 Summary: TESS-Gaia Light Curve
 Home-page: https://github.com/TeHanHunter/TESS_Gaia_Light_Curve
 Author: Te Han
@@ -24,9 +24,9 @@ Usage
 There are four fluxes in each FITS file: aperture flux, PSF flux, calibrated aperture flux, and calibrated PSF flux.
 If you are uncertain which to use:
 
-* Calibrated aperture flux is the most robust in transit depth. Use this if you are doing transit science.
-* Calibrated psf flux is better in deblending targets. Use this if you need to deblend a target near a variable source. The best deblending can be achieved with tglc package by setting a non-zero prior.
-* The aperture flux and PSF flux are not detrended or normalized. Use this if you are doing stellar variability science. Or, if the detrending is not optimal (default detrending has a window length of 1 day), start with the aperture flux or PSF flux and detrend carefully!
+* Calibrated psf flux is better in **deblending** targets. Use this if you need to deblend a target near a variable source. The best deblending can be achieved with tglc package by setting a non-zero prior. It also gives the more accurate **transit depth** in most cases, especially when fitting with an optimized prior.
+* Calibrated aperture flux usually has slightly **higher SNR**. The transit depth (or variation amplitude), however, can be imperfect since the normalization depends on the PSF fitting which is imperfect. This imperfection can be minimized by using a bigger aperture than the default aperture (3*3). One need to use the tglc package and set tglc_lc(save_aper=True) to access the 5*5 aperture. In the presence of a bright but "constant" contamination (several magnitudes brighter), the calibrated aperture flux is better in removing the constant contamination. 
+* The aperture flux and PSF flux are not detrended or normalized. Use this if you are doing stellar variability science with long baseline. Or, if the detrending is not optimal (default detrending has a window length of 1 day; see Known Problems below), start with the aperture flux or PSF flux and detrend carefully!
 * **If you are uncertain, start with calibrated aperture flux!**
 
 The `tutorial <tutorial/TGLC_tutorial.ipynb>`_ shows the syntaxes and differences among these light curves in several examples.
@@ -60,7 +60,8 @@ for the latest tglc release. After installation, follow the `tutorial <tutorial/
 Known Problems
 ==================================
 There are several imperfections we noticed in the MAST TGLC light curves and tglc package:
-* If the star is very dim (~< 15 Tmag) near a variable source, it can make the aperture and/or PSF light curve negative for some cadences. This is a very rare scenario, but could be important. The detrending algorithm could malfunction and result in bad cal_aper_flux and/or cal_psf_flux. This is now fixed for tglc package, but this problem remains for the primary mission light curves published on MAST. Please detrend again if necessary. We will also correct for the extended mission light curves on MAST.
+
+* If the star is very dim (~< 15 Tmag) near a variable source, it can make the aperture and/or PSF light curve negative for some cadences. The detrending algorithm could malfunction and result in bad cal_aper_flux and/or cal_psf_flux. This is now fixed for tglc package, but this problem remains for the primary mission light curves published on MAST. Please detrend again if necessary.The extended mission light curves on MAST will not be affected. This is a very rare scenario, but could be important.
 
 ==================================
 Reference

{tglc-0.5.7 → tglc-0.5.8}/README.rst

@@ -10,9 +10,9 @@ Usage
 There are four fluxes in each FITS file: aperture flux, PSF flux, calibrated aperture flux, and calibrated PSF flux.
 If you are uncertain which to use:
 
-* Calibrated aperture flux is the most robust in transit depth. Use this if you are doing transit science.
-* Calibrated psf flux is better in deblending targets. Use this if you need to deblend a target near a variable source. The best deblending can be achieved with tglc package by setting a non-zero prior.
-* The aperture flux and PSF flux are not detrended or normalized. Use this if you are doing stellar variability science. Or, if the detrending is not optimal (default detrending has a window length of 1 day), start with the aperture flux or PSF flux and detrend carefully!
+* Calibrated psf flux is better in **deblending** targets. Use this if you need to deblend a target near a variable source. The best deblending can be achieved with tglc package by setting a non-zero prior. It also gives the more accurate **transit depth** in most cases, especially when fitting with an optimized prior.
+* Calibrated aperture flux usually has slightly **higher SNR**. The transit depth (or variation amplitude), however, can be imperfect since the normalization depends on the PSF fitting which is imperfect. This imperfection can be minimized by using a bigger aperture than the default aperture (3*3). One need to use the tglc package and set tglc_lc(save_aper=True) to access the 5*5 aperture. In the presence of a bright but "constant" contamination (several magnitudes brighter), the calibrated aperture flux is better in removing the constant contamination. 
+* The aperture flux and PSF flux are not detrended or normalized. Use this if you are doing stellar variability science with long baseline. Or, if the detrending is not optimal (default detrending has a window length of 1 day; see Known Problems below), start with the aperture flux or PSF flux and detrend carefully!
 * **If you are uncertain, start with calibrated aperture flux!**
 
 The `tutorial <tutorial/TGLC_tutorial.ipynb>`_ shows the syntaxes and differences among these light curves in several examples.
@@ -46,7 +46,8 @@ for the latest tglc release. After installation, follow the `tutorial <tutorial/
 Known Problems
 ==================================
 There are several imperfections we noticed in the MAST TGLC light curves and tglc package:
-* If the star is very dim (~< 15 Tmag) near a variable source, it can make the aperture and/or PSF light curve negative for some cadences. This is a very rare scenario, but could be important. The detrending algorithm could malfunction and result in bad cal_aper_flux and/or cal_psf_flux. This is now fixed for tglc package, but this problem remains for the primary mission light curves published on MAST. Please detrend again if necessary. We will also correct for the extended mission light curves on MAST.
+
+* If the star is very dim (~< 15 Tmag) near a variable source, it can make the aperture and/or PSF light curve negative for some cadences. The detrending algorithm could malfunction and result in bad cal_aper_flux and/or cal_psf_flux. This is now fixed for tglc package, but this problem remains for the primary mission light curves published on MAST. Please detrend again if necessary.The extended mission light curves on MAST will not be affected. This is a very rare scenario, but could be important.
 
 ==================================
 Reference

{tglc-0.5.7 → tglc-0.5.8}/setup.py

@@ -6,7 +6,7 @@ with open("README.rst", "r", encoding="utf-8") as fh:
     long_description = fh.read()
 setuptools.setup(
     name="tglc",
-    version='0.5.7',
+    version='0.5.8',
     author="Te Han",
     author_email="tehanhunter@gmail.com",
     description="TESS-Gaia Light Curve",

tglc-0.5.8/tglc/__init__.py

@@ -0,0 +1,3 @@
+__version__ = "0.5.8"
+__author__ = 'Te Han, Timothy Brandt'
+__credits__ = 'University of California, Santa Barbara'

{tglc-0.5.7 → tglc-0.5.8}/tglc/effective_psf.py

@@ -200,6 +200,7 @@ def fit_lc(A, source, star_info=None, x=0., y=0., star_num=0, factor=2, psf_size
     for j in range(len(source.time)):
         aperture[j] = np.array(source.flux[j][down:up, left:right]).flatten() - np.dot(A_cut, e_psf[j])
     aperture = aperture.reshape((len(source.time), up - down, right - left))
+    # np.save(f'_residual_{source.sector}.npy', aperture)
 
     # psf_lc
     over_size = psf_size * factor + 1
@@ -259,6 +260,8 @@ def fit_lc(A, source, star_info=None, x=0., y=0., star_num=0, factor=2, psf_size
             aper_flat = np.delete(aper_flat, edge_pixel[outliers])
             psf_lc[j] = np.linalg.lstsq(a, aper_flat)[0][0]
     portion = np.nansum(psf_shape[:, 4:7, 4:7]) / np.nansum(psf_shape)
+    # print(np.nansum(psf_shape[:, 5, 5]) / np.nansum(psf_shape))
+    # np.save(f'toi-5344_psf_{source.sector}.npy', psf_shape)
     return aperture, psf_lc, y - down, x - left, portion
 
 
@@ -427,12 +430,14 @@ def bg_mod(source, q=None, aper_lc=None, psf_lc=None, portion=None, star_num=0,
     :return: local background, modified aperture light curve, modified PSF light curve
     '''
     bar = 15000 * 10 ** ((source.gaia['tess_mag'][star_num] - 10) / -2.5)
+    # print(bar)
     # med_epsf = np.nanmedian(e_psf[:, :23 ** 2].reshape(len(source.time), 23, 23), axis=0)
     # centroid_to_aper_ratio = 4/9 * np.sum(med_epsf[10:13, 10:13]) / np.sum(med_epsf)
     # centroid_to_aper_ratio = np.nanmedian(ratio)
     # flux_bar = aperture_bar * centroid_to_aper_ratio
     # lightcurve = lightcurve + (flux_bar - np.nanmedian(lightcurve[q]))
     aperture_bar = bar * portion
+    # print(bar)
     local_bg = np.nanmedian(aper_lc[q]) - aperture_bar
     if np.isnan(local_bg):
         local_bg = 0

{tglc-0.5.7 → tglc-0.5.8}/tglc/ffi.py

@@ -62,14 +62,14 @@ def mast_json2table(json_obj):
     return data_table
 
 
-def tic_advanced_search_position_rows(ra=1., dec=1., radius=0.5):
+def tic_advanced_search_position_rows(ra=1., dec=1., radius=0.5, limit_mag=16):
     request = {"service": "Mast.Catalogs.Filtered.Tic.Position.Rows",
                "format": "json",
                "params": {
                    "columns": 'ID, GAIA',
                    "filters": [
                        {"paramName": "Tmag",
-                        "values": [{"min": -10., "max": 16.5}]}],
+                        "values": [{"min": -10., "max": (limit_mag + 0.5)}]}],
                    "ra": ra,
                    "dec": dec,
                    "radius": radius
@@ -89,7 +89,7 @@ def convert_gaia_id(catalogdata_tic):
     gaia_array = np.array(catalogdata_tic['GAIA'])
     gaia_array = gaia_array[gaia_array != 'None']
     # np.save('gaia_array.npy', gaia_array)
-    segment = len(gaia_array) // 10000
+    segment = (len(gaia_array) - 1) // 10000
     gaia_tuple = tuple(gaia_array[:10000])
     results = Gaia.launch_job_async(query.format(gaia_ids=gaia_tuple)).get_results()
     # np.save('result.npy', np.array(results))

{tglc-0.5.7 → tglc-0.5.8}/tglc/ffi_cut.py

@@ -23,7 +23,7 @@ Gaia.MAIN_GAIA_TABLE = "gaiadr3.gaia_source"
 
 
 class Source_cut(object):
-    def __init__(self, name, size=50, sector=None, cadence=None):
+    def __init__(self, name, size=50, sector=None, cadence=None, limit_mag=None):
         """
         Source_cut object that includes all data from TESS and Gaia DR3
         :param name: str, required
@@ -62,7 +62,7 @@ class Source_cut(object):
                                              columns=['DESIGNATION', 'phot_g_mean_mag', 'phot_bp_mean_mag',
                                                       'phot_rp_mean_mag', 'ra', 'dec', 'pmra', 'pmdec']).get_results()
         print(f'Found {len(catalogdata)} Gaia DR3 objects.')
-        catalogdata_tic = tic_advanced_search_position_rows(ra=ra, dec=dec, radius=(self.size + 2) * 21 * 0.707 / 3600)
+        catalogdata_tic = tic_advanced_search_position_rows(ra=ra, dec=dec, radius=(self.size + 2) * 21 * 0.707 / 3600, limit_mag=limit_mag)
         print(f'Found {len(catalogdata_tic)} TIC objects.')
         self.tic = convert_gaia_id(catalogdata_tic)
         sector_table = Tesscut.get_sectors(coordinates=coord)
@@ -71,9 +71,12 @@ class Source_cut(object):
         print(sector_table)
         if sector is None:
             hdulist = Tesscut.get_cutouts(coordinates=coord, size=self.size)
-        elif sector is True:
+        elif sector == 'first':
             hdulist = Tesscut.get_cutouts(coordinates=coord, size=self.size, sector=sector_table['sector'][0])
             sector = sector_table['sector'][0]
+        elif sector == 'last':
+            hdulist = Tesscut.get_cutouts(coordinates=coord, size=self.size, sector=sector_table['sector'][-1])
+            sector = sector_table['sector'][-1]
         else:
             hdulist = Tesscut.get_cutouts(coordinates=coord, size=self.size, sector=sector)
         self.catalogdata = catalogdata
@@ -234,7 +237,7 @@ class Source_cut_pseudo(object):
         self.gaia = gaia_targets
 
 
-def ffi_cut(target='', local_directory='', size=90, sector=None):
+def ffi_cut(target='', local_directory='', size=90, sector=None, limit_mag=None):
     """
     Function to generate Source_cut objects
     :param target: string, required
@@ -249,8 +252,10 @@ def ffi_cut(target='', local_directory='', size=90, sector=None):
     """
     if sector is None:
         source_name = f'source_{target}'
-    elif sector is True:
+    elif sector == 'first':
         source_name = f'source_{target}_earliest_sector'
+    elif sector == 'last':
+        source_name = f'source_{target}_last_sector'
     else:
         source_name = f'source_{target}_sector_{sector}'
     source_exists = exists(f'{local_directory}source/{source_name}.pkl')
@@ -261,6 +266,6 @@ def ffi_cut(target='', local_directory='', size=90, sector=None):
         print('Loaded ffi_cut from directory. ')
     else:
         with open(f'{local_directory}source/{source_name}.pkl', 'wb') as output:
-            source = Source_cut(target, size=size, sector=sector)
+            source = Source_cut(target, size=size, sector=sector, limit_mag=limit_mag)
             pickle.dump(source, output, pickle.HIGHEST_PROTOCOL)
     return source

{tglc-0.5.7 → tglc-0.5.8}/tglc/mast.py

@@ -38,12 +38,12 @@ def zip_folder(i, sector=1, do_zip=True, lc_num_per_zip=1e6):
     original_file = f'/home/tehan/data/sector{sector:04d}/lc/{cam}-{ccd}/'
     files = glob(f'{original_file}*.fits')
     if do_zip:
-        num_zips = int(len(files) // lc_num_per_zip + 1)
-        for i in range(num_zips):
-            with zipfile.ZipFile(f'{zip_file}_{i:02d}.zip', 'w') as zipMe:
-                for file in files[int(i * lc_num_per_zip):int((i + 1) * lc_num_per_zip)]:
-                    zipMe.write(file, compress_type=zipfile.ZIP_DEFLATED)
-        # shutil.make_archive(zip_file, 'zip', original_file)
+        # num_zips = int(len(files) // lc_num_per_zip + 1)
+        # for i in range(num_zips):
+            # with zipfile.ZipFile(f'{zip_file}_{i:02d}.zip', 'w') as zipMe:
+            #     for file in files[int(i * lc_num_per_zip):int((i + 1) * lc_num_per_zip)]:
+            #         zipMe.write(file, compress_type=zipfile.ZIP_DEFLATED)
+        shutil.make_archive(zip_file, 'zip', original_file)
         return
     else:
         ftps = ftplib.FTP_TLS('archive.stsci.edu')
@@ -108,8 +108,9 @@ def star_spliter(server=1,  # or 2
 
 
 if __name__ == '__main__':
-    # sector = 1
-    # filter_no_tic(sector=sector)
-    # hlsp_transfer(sector=sector, do_zip=True)
+    sector = 1
+    filter_no_tic(sector=sector)
+    hlsp_transfer(sector=sector, do_zip=True)
     # hlsp_transfer(sector=sector, do_zip=False)
-    star_spliter(server=1)  # star_list='/home/tehan/Documents/tglc/dominic_EB/eb_cat.txt'
+    # star_spliter(server=1)
+    # star_list='/home/tehan/Documents/tglc/dominic_EB/eb_cat.txt'

{tglc-0.5.7 → tglc-0.5.8}/tglc/quick_lc.py

@@ -21,7 +21,7 @@ controller = ThreadpoolController()
 
 @controller.wrap(limits=1, user_api='blas')
 def tglc_lc(target='TIC 264468702', local_directory='', size=90, save_aper=True, limit_mag=16, get_all_lc=False,
-            first_sector_only=False, sector=None, prior=None):
+            first_sector_only=False, last_sector_only=False, sector=None, prior=None):
     '''
     Generate light curve for a single target.
 
@@ -37,8 +37,11 @@ def tglc_lc(target='TIC 264468702', local_directory='', size=90, save_aper=True,
     os.makedirs(local_directory + f'epsf/', exist_ok=True)
     os.makedirs(local_directory + f'source/', exist_ok=True)
     if first_sector_only:
-        sector = True
-    source = ffi_cut(target=target, size=size, local_directory=local_directory, sector=sector)  # sector
+        sector = 'first'
+    elif last_sector_only:
+        sector = 'last'
+    source = ffi_cut(target=target, size=size, local_directory=local_directory, sector=sector,
+                     limit_mag=limit_mag)  # sector
     if get_all_lc:
         name = None
     else:
@@ -54,14 +57,20 @@ def tglc_lc(target='TIC 264468702', local_directory='', size=90, save_aper=True,
         source.select_sector(sector=sector)
         epsf(source, factor=2, sector=source.sector, target=target, local_directory=local_directory,
              name=name, limit_mag=limit_mag, save_aper=save_aper, prior=prior)
+    elif first_sector_only:
+        source.select_sector(sector=source.sector_table['sector'][0])
+        epsf(source, factor=2, sector=source.sector, target=target, local_directory=local_directory,
+             name=name, limit_mag=limit_mag, save_aper=save_aper, prior=prior)
+    elif last_sector_only:
+        source.select_sector(sector=source.sector_table['sector'][-1])
+        epsf(source, factor=2, sector=source.sector, target=target, local_directory=local_directory,
+             name=name, limit_mag=limit_mag, save_aper=save_aper, prior=prior)
     else:
         for j in range(len(source.sector_table)):
             # try:
             source.select_sector(sector=source.sector_table['sector'][j])
             epsf(source, factor=2, sector=source.sector, target=target, local_directory=local_directory,
                  name=name, limit_mag=limit_mag, save_aper=save_aper, prior=prior)
-            if first_sector_only:
-                break
 
 
 def search_stars(i, sector=1, tics=None, local_directory=None):
@@ -86,25 +95,72 @@ def star_spliter(server=1,  # or 2
     return
 
 
-def plot_lc(local_directory=None):
+def plot_lc(local_directory=None, type='cal_aper_flux'):
     files = glob(f'{local_directory}*.fits')
     os.makedirs(f'{local_directory}plots/', exist_ok=True)
     for i in range(len(files)):
         with fits.open(files[i], mode='denywrite') as hdul:
             q = [a and b for a, b in zip(list(hdul[1].data['TESS_flags'] == 0), list(hdul[1].data['TGLC_flags'] == 0))]
             plt.figure(constrained_layout=False, figsize=(8, 4))
-            plt.plot(hdul[1].data['time'], hdul[1].data['cal_aper_flux'], '.', c='silver', label='cal_aper')
-            plt.plot(hdul[1].data['time'][q], hdul[1].data['cal_aper_flux'][q], '.k', label='cal_aper_flagged')
+            plt.plot(hdul[1].data['time'], hdul[1].data[type], '.', c='silver', label=type)
+            plt.plot(hdul[1].data['time'][q], hdul[1].data[type][q], '.k', label=f'{type}_flagged')
             # plt.xlim(2845, 2855)
+            plt.ylim(0.5, 1.5)
+            plt.title(f'TIC_{hdul[0].header["TICID"]}_sector_{hdul[0].header["SECTOR"]:04d}_{type}')
+            plt.legend()
+            # plt.show()
+            plt.savefig(
+                f'{local_directory}plots/TIC_{hdul[0].header["TICID"]}_sector_{hdul[0].header["SECTOR"]:04d}.png',
+                dpi=300)
+            plt.close()
+
+
+def plot_aperture(local_directory=None, type='cal_aper_flux'):
+    files = glob(f'{local_directory}*.fits')
+    os.makedirs(f'{local_directory}plots/', exist_ok=True)
+    portion = [0.9361215204370542, 0.9320709087810205]
+    data = np.empty((3, 0))
+
+    for i in range(len(files)):
+        with fits.open(files[i], mode='denywrite') as hdul:
+            print(files[i], portion[i])
+            q = [a and b for a, b in zip(list(hdul[1].data['TESS_flags'] == 0), list(hdul[1].data['TGLC_flags'] == 0))]
+            plt.figure(constrained_layout=False, figsize=(8, 4))
+            plt.plot(hdul[1].data['time'] % 3.79262026, hdul[1].data[type], '.', c='silver', label=type)
+            plt.plot(hdul[1].data['time'][q] % 3.79262026, hdul[1].data[type][q], '.k', label=f'{type}_flagged')
+            aperture_bar = 709.5512462444653 * portion[i]
+            aper_lc = np.nansum(hdul[0].data, axis=(1, 2))
+            local_bg = np.nanmedian(aper_lc) - aperture_bar
+            aper_lc = (aper_lc - local_bg) / portion[i]
+            cal_aper_lc = aper_lc / np.nanmedian(aper_lc)
+            cal_aper_lc[np.where(cal_aper_lc > 100)] = np.nan
+            _, trend = flatten(hdul[1].data['time'], cal_aper_lc - np.nanmin(cal_aper_lc) + 1000,
+                               window_length=1, method='biweight', return_trend=True)
+            cal_aper_lc = (cal_aper_lc - np.nanmin(cal_aper_lc) + 1000 - trend) / np.nanmedian(cal_aper_lc) + 1
+            non_outliers = np.where(cal_aper_lc[q] > 0.6)[0]
+            plt.plot(hdul[1].data['time'][q][non_outliers] % 3.79262026, cal_aper_lc[q][non_outliers], '.r',
+                     label=f'5_5_pixel_flagged')
+            plt.xlim(0.5, 1.0)
+            plt.ylim(0.95, 1.1)
             plt.title(f'TIC_{hdul[0].header["TICID"]}_sector_{hdul[0].header["SECTOR"]:04d}')
             plt.legend()
             # plt.show()
             plt.savefig(
                 f'{local_directory}plots/TIC_{hdul[0].header["TICID"]}_sector_{hdul[0].header["SECTOR"]:04d}.png',
                 dpi=300)
+            time = hdul[1].data['time'][q][non_outliers]
+            flux = cal_aper_lc[q][non_outliers]
+            f_err = 1.4826 * np.nanmedian(np.abs(flux - np.nanmedian(flux)))
+            not_nan = np.invert(np.isnan(flux))
+            data_ = np.array([time[not_nan],
+                              flux[not_nan],
+                              np.array([f_err] * len(time[not_nan]))
+                              ])
+            data = np.append(data, data_, axis=1)
+    np.savetxt(f'{local_directory}TESS_TOI-5344_5_5_aper.csv', data, delimiter=',')
 
 
-def plot_pf_lc(local_directory=None, period=None):
+def plot_pf_lc(local_directory=None, period=None, type='cal_aper_flux'):
     files = glob(f'{local_directory}*.fits')
     os.makedirs(f'{local_directory}plots/', exist_ok=True)
     fig = plt.figure(figsize=(13, 5))
@@ -116,20 +172,21 @@ def plot_pf_lc(local_directory=None, period=None):
             if len(hdul[1].data['cal_aper_flux']) == len(hdul[1].data['time']):
                 if hdul[0].header["SECTOR"] <= 26:
                     t = hdul[1].data['time'][q]
-                    f = hdul[1].data['cal_psf_flux'][q]
-                else:
+                    f = hdul[1].data[type][q]
+                elif hdul[0].header["SECTOR"] <= 55:
                     t = np.mean(hdul[1].data['time'][q][:len(hdul[1].data['time'][q]) // 3 * 3].reshape(-1, 3), axis=1)
                     f = np.mean(
-                        hdul[1].data['cal_aper_flux'][q][:len(hdul[1].data['cal_aper_flux'][q]) // 3 * 3].reshape(-1,
-                                                                                                                  3),
-                        axis=1)
-                plt.plot(hdul[1].data['time'] % period / period, hdul[1].data['cal_aper_flux'], '.', c=f'C{j}', ms=2)
-                plt.errorbar(t % period / period, f, hdul[1].header['CAPE_ERR'], c=f'C{j}', ls='', elinewidth=0,
+                        hdul[1].data[type][q][:len(hdul[1].data[type][q]) // 3 * 3].reshape(-1, 3), axis=1)
+                else:
+                    t = np.mean(hdul[1].data['time'][q][:len(hdul[1].data['time'][q]) // 9 * 9].reshape(-1, 9), axis=1)
+                    f = np.mean(
+                        hdul[1].data[type][q][:len(hdul[1].data[type][q]) // 9 * 9].reshape(-1, 9), axis=1)
+                plt.plot(hdul[1].data['time'] % period / period, hdul[1].data[type], '.', c='silver', ms=2)
+                plt.errorbar(t % period / period, f, hdul[1].header['CAPE_ERR'], c=f'C{j}', ls='', elinewidth=1,
                              marker='.', ms=2, zorder=2, label=f'Sector {hdul[0].header["sector"]}')
-                #
             else:
                 not_plotted_num += 1
-            title = f'TIC_{hdul[0].header["TICID"]} with {len(files) - not_plotted_num} sector(s) of data'
+            title = f'TIC_{hdul[0].header["TICID"]} with {len(files) - not_plotted_num} sector(s) of data, {type}'
     # PDCSAP_files = glob('/home/tehan/Documents/GEMS/TIC 172370679/PDCSAP/*.txt')
     # for i in range(len(files)):
     #     PDCSAP = ascii.read(PDCSAP_files[i])
@@ -141,7 +198,9 @@ def plot_pf_lc(local_directory=None, period=None):
     # plt.xlim(0.84, 0.86)
     plt.legend()
     plt.title(title)
-    plt.xlabel('Phase (days)')
+    plt.xlim(0.16, 0.26)
+    plt.ylim(0.9, 1.1)
+    plt.xlabel('Phase')
     plt.ylabel('Normalized flux')
     plt.savefig(f'{local_directory}/plots/{title}.png', dpi=300)
     plt.close(fig)
@@ -149,6 +208,7 @@ def plot_pf_lc(local_directory=None, period=None):
 
 def plot_contamination(local_directory=None, gaia_dr3=None):
     files = glob(f'{local_directory}lc/*.fits')
+    os.makedirs(f'{local_directory}lc/plots/', exist_ok=True)
     for i in range(len(files)):
         with open(glob(f'{local_directory}source/*.pkl')[0], 'rb') as input_:
             with fits.open(files[i], mode='denywrite') as hdul:
@@ -159,8 +219,8 @@ def plot_contamination(local_directory=None, gaia_dr3=None):
                 # print(source.gaia[891])
                 # print(source.gaia[140])
                 nearby_stars = np.argsort(
-                    (source.gaia[f'sector_{sector}_x'][:1000] - source.gaia[star_num][f'sector_{sector}_x']) ** 2 +
-                    (source.gaia[f'sector_{sector}_y'][:1000] - source.gaia[star_num][f'sector_{sector}_y']) ** 2)[1:10]
+                    (source.gaia[f'sector_{sector}_x'][:500] - source.gaia[star_num][f'sector_{sector}_x']) ** 2 +
+                    (source.gaia[f'sector_{sector}_y'][:500] - source.gaia[star_num][f'sector_{sector}_y']) ** 2)[0:5]
                 # print(f'sector = {source.sector}')
                 star_x = source.gaia[star_num][f'sector_{sector}_x'][0]
                 star_y = source.gaia[star_num][f'sector_{sector}_y'][0]
@@ -173,7 +233,7 @@ def plot_contamination(local_directory=None, gaia_dr3=None):
                 ax0 = fig.add_subplot(gs[:5, :5])
                 ax0.imshow(source.flux[0], cmap='RdBu', vmin=-max_flux, vmax=max_flux, origin='lower')
 
-                ax0.scatter(source.gaia[f'sector_{sector}_x'][:1000], source.gaia[f'sector_{sector}_y'][:1000], s=50,
+                ax0.scatter(source.gaia[f'sector_{sector}_x'][:500], source.gaia[f'sector_{sector}_y'][:500], s=50,
                             c='r', label='background stars')
                 ax0.scatter(source.gaia[f'sector_{sector}_x'][nearby_stars],
                             source.gaia[f'sector_{sector}_y'][nearby_stars], s=50,
@@ -181,12 +241,31 @@ def plot_contamination(local_directory=None, gaia_dr3=None):
                 for l in range(len(nearby_stars)):
                     index = np.where(
                         source.tic['dr3_source_id'] == int(source.gaia['DESIGNATION'][nearby_stars[l]].split(' ')[-1]))
+                    gaia_targets = source.gaia
+                    median_time = np.median(source.time)
+                    interval = (median_time - 388.5) / 365.25 + 3000
+                    ra = gaia_targets['ra'][nearby_stars[l]]
+                    dec = gaia_targets['dec'][nearby_stars[l]]
+                    if not np.isnan(gaia_targets['pmra'][nearby_stars[l]]):
+                        ra += gaia_targets['pmra'][nearby_stars[l]] * np.cos(np.deg2rad(dec)) * interval / 1000 / 3600
+                    if not np.isnan(gaia_targets['pmdec'][nearby_stars[l]]):
+                        dec += gaia_targets['pmdec'][nearby_stars[l]] * interval / 1000 / 3600
+                    pixel = source.wcs.all_world2pix(np.array([ra, dec]).reshape((1, 2)), 0)
+                    x_gaia = pixel[0][0]
+                    y_gaia = pixel[0][1]
+                    ax0.arrow(source.gaia[f'sector_{sector}_x'][nearby_stars[l]],
+                              source.gaia[f'sector_{sector}_y'][nearby_stars[l]],
+                              x_gaia - source.gaia[f'sector_{sector}_x'][nearby_stars[l]],
+                              y_gaia - source.gaia[f'sector_{sector}_y'][nearby_stars[l]],
+                              width=0.02, color='r', edgecolor=None, head_width=0.1)
                     try:
                         ax0.text(source.gaia[f'sector_{sector}_x'][nearby_stars[l]] - 0.1,
-                                    source.gaia[f'sector_{sector}_y'][nearby_stars[l]] + 0.2,
-                                    f'TIC {int(source.tic["TIC"][index])}', rotation=90)
+                                 source.gaia[f'sector_{sector}_y'][nearby_stars[l]] + 0.3,
+                                 f'TIC {int(source.tic["TIC"][index])}', rotation=90)
                     except TypeError:
-                        pass
+                        ax0.text(source.gaia[f'sector_{sector}_x'][nearby_stars[l]] - 0.1,
+                                 source.gaia[f'sector_{sector}_y'][nearby_stars[l]] + 0.2,
+                                 f'{source.gaia[f"DESIGNATION"][nearby_stars[l]]}', rotation=90)
                 ax0.scatter(star_x, star_y, s=300, c='r', marker='*', label='target star')
 
                 # ax0.legend()
@@ -214,47 +293,38 @@ def plot_contamination(local_directory=None, gaia_dr3=None):
                     for k in range(x_):
                         ax_ = fig.add_subplot(gs[(4 - j), (5 + k)])
                         ax_.patch.set_facecolor('C0')
-                        ax_.patch.set_alpha(min(1, max(0, 3 * np.median(hdul[0].data[:, j, k]) / max_flux)))
+                        ax_.patch.set_alpha(min(1, max(0, 5 * np.nanmedian(hdul[0].data[:, j, k]) / max_flux)))
                         q = [a and b for a, b in
                              zip(list(hdul[1].data['TESS_flags'] == 0), list(hdul[1].data['TGLC_flags'] == 0))]
 
                         _, trend = flatten(hdul[1].data['time'][q],
-                                           hdul[0].data[:, j, k][q] - np.min(hdul[0].data[:, j, k][q]) + 1000,
+                                           hdul[0].data[:, j, k][q] - np.nanmin(hdul[0].data[:, j, k][q]) + 1000,
                                            window_length=1, method='biweight', return_trend=True)
-                        cal_aper = (hdul[0].data[:, j, k][q] - np.min(
-                            hdul[0].data[:, j, k][q]) + 1000 - trend) / np.median(
+                        cal_aper = (hdul[0].data[:, j, k][q] - np.nanmin(
+                            hdul[0].data[:, j, k][q]) + 1000 - trend) / np.nanmedian(
                             hdul[0].data[:, j, k][q]) + 1
                         ax_.plot(hdul[1].data['time'][q], cal_aper, '.k', ms=1, label='center pixel')
+                        ax_.set_ylim(0.95, 1.05)
                 plt.savefig(f'{local_directory}lc/plots/contamination_sector_{hdul[0].header["SECTOR"]:04d}.pdf',
                             dpi=300)
                 plt.show()
 
 
-def choose_prior(local_directory=None, priors=np.logspace(-5, 0, 100)):
-    mad = np.zeros((5, 100))
+def choose_prior(tics, local_directory=None, priors=np.logspace(-5, 0, 100)):
+    mad = np.zeros((2, 100))
     for i in trange(len(priors)):
-        get_tglc_lc(tics=tics, method='query', server=1, directory=local_directory, prior=priors[i])
-        with fits.open(
-                '/home/tehan/data/cosmos/GEMS/TIC 172370679/lc/hlsp_tglc_tess_ffi_gaiaid-2073530190996615424-s0014-cam1-ccd1_tess_v1_llc.fits',
-                mode='denywrite') as hdul:
-            mad[0, i] = np.median(abs(hdul[1].data['cal_psf_flux'] - np.median(hdul[1].data['cal_psf_flux'])))
-        with fits.open(
-                '/home/tehan/data/cosmos/GEMS/TIC 172370679/lc/hlsp_tglc_tess_ffi_gaiaid-2073530190996615424-s0015-cam1-ccd2_tess_v1_llc.fits',
-                mode='denywrite') as hdul:
-            mad[1, i] = np.median(abs(hdul[1].data['cal_psf_flux'] - np.median(hdul[1].data['cal_psf_flux'])))
-        with fits.open(
-                '/home/tehan/data/cosmos/GEMS/TIC 172370679/lc/hlsp_tglc_tess_ffi_gaiaid-2073530190996615424-s0041-cam1-ccd1_tess_v1_llc.fits',
-                mode='denywrite') as hdul:
-            mad[2, i] = np.median(abs(hdul[1].data['cal_psf_flux'] - np.median(hdul[1].data['cal_psf_flux'])))
-        with fits.open(
-                '/home/tehan/data/cosmos/GEMS/TIC 172370679/lc/hlsp_tglc_tess_ffi_gaiaid-2073530190996615424-s0054-cam3-ccd2_tess_v1_llc.fits',
-                mode='denywrite') as hdul:
-            mad[3, i] = np.median(abs(hdul[1].data['cal_psf_flux'] - np.median(hdul[1].data['cal_psf_flux'])))
-        with fits.open(
-                '/home/tehan/data/cosmos/GEMS/TIC 172370679/lc/hlsp_tglc_tess_ffi_gaiaid-2073530190996615424-s0055-cam3-ccd1_tess_v1_llc.fits',
-                mode='denywrite') as hdul:
-            mad[4, i] = np.median(abs(hdul[1].data['cal_psf_flux'] - np.median(hdul[1].data['cal_psf_flux'])))
-    np.save('/home/tehan/data/cosmos/GEMS/TIC 172370679/mad.npy', mad)
+        resid = get_tglc_lc(tics=tics, method='query', server=1, directory=local_directory, prior=priors[i])
+        print(resid)
+        mad[:, i] = resid
+        # with fits.open(
+        #         '/home/tehan/data/cosmos/GEMS/TIC 16005254/lc/hlsp_tglc_tess_ffi_gaiaid-52359538285081728-s0043-cam3-ccd3_tess_v1_llc.fits',
+        #         mode='denywrite') as hdul:
+        #     mad[0, i] = np.nanmedian(abs(hdul[1].data['cal_psf_flux'] - np.nanmedian(hdul[1].data['cal_psf_flux'])))
+        # with fits.open(
+        #         '/home/tehan/data/cosmos/GEMS/TIC 16005254/lc/hlsp_tglc_tess_ffi_gaiaid-52359538285081728-s0044-cam1-ccd1_tess_v1_llc.fits',
+        #         mode='denywrite') as hdul:
+        #     mad[1, i] = np.nanmedian(abs(hdul[1].data['cal_psf_flux'] - np.nanmedian(hdul[1].data['cal_psf_flux'])))
+    np.save('/home/tehan/Documents/GEMS/TIC 16005254/mad.npy', mad)
     # plt.plot(priors, mad)
     # plt.xscale('log')
     # plt.title(f'best prior = {priors[np.argmin(mad)]:04d}')
@@ -267,19 +337,19 @@ def get_tglc_lc(tics=None, method='query', server=1, directory=None, prior=None)
             target = f'TIC {tics[i]}'
             local_directory = f'{directory}{target}/'
             os.makedirs(local_directory, exist_ok=True)
-            tglc_lc(target=target, local_directory=local_directory, size=90, save_aper=True, limit_mag=16,
-                    get_all_lc=False, first_sector_only=False, sector=None, prior=prior)
+            tglc_lc(target=target, local_directory=local_directory, size=90, save_aper=False, limit_mag=16,
+                    get_all_lc=False, first_sector_only=False, last_sector_only=True, sector=None, prior=prior)
     if method == 'search':
         star_spliter(server=server, tics=tics, local_directory=directory)
 
 
 if __name__ == '__main__':
-    tics = [119585136]
+    tics = [11893637]
     directory = f'/home/tehan/data/cosmos/GEMS/'
     os.makedirs(directory, exist_ok=True)
     get_tglc_lc(tics=tics, method='query', server=1, directory=directory)
-    # plot_contamination(local_directory=f'{directory}TIC 27858644/', gaia_dr3=2091177593123254016)
-    # plot_contamination(local_directory=f'{directory}TIC 172370679/', gaia_dr3=2073530190996615424)
-    # plot_lc(local_directory=f'{directory}TIC 135272255/lc/')
-    # plot_pf_lc(local_directory=f'{directory}TIC 27858644/lc/', period=384)
-    # choose_prior(local_directory=directory)
+    plot_lc(local_directory=f'{directory}TIC {tics[0]}/lc/', type='cal_aper_flux')
+    # plot_aperture(local_directory=f'{directory}TIC {tics[0]}/lc/', type='cal_aper_flux')
+    # plot_contamination(local_directory=f'{directory}TIC {tics[0]}/', gaia_dr3=52359538285081728)
+    # plot_pf_lc(local_directory=f'{directory}TIC {tics[0]}/lc/', period=3.7926244)
+    # choose_prior(tics, local_directory=directory)

{tglc-0.5.7 → tglc-0.5.8}/tglc/target_lightcurve.py

@@ -278,9 +278,11 @@ def epsf(source, psf_size=11, factor=2, local_directory='', target=None, cut_x=0
                                  str(source.tic['dr3_source_id'][np.where(source.tic['TIC'] == name)][0]))[0][0])
             end = start + 1
         except IndexError:
-            print(f'Target not found in the requested sector (Sector {sector}). It is possible that this target does '
-                  f'not have a Gaia ID. ')
-            return
+            print(f'Target not found in the requested sector (Sector {sector}). This can be caused by a lack of Gaia '
+                  f'ID or an incomplete TESS to Gaia crossmatch table. Please check whether the output light curve Gaia'
+                  f' DR3 ID agrees with your target.')
+            start = 0
+            end = 1
     for i in trange(start, end, desc='Fitting lc', disable=no_progress_bar):
         if x_left <= x_round[i] < source.size - x_right and y_left <= y_round[i] < source.size - y_right:
             if type(source) == Source:

{tglc-0.5.7 → tglc-0.5.8/tglc.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: tglc
-Version: 0.5.7
+Version: 0.5.8
 Summary: TESS-Gaia Light Curve
 Home-page: https://github.com/TeHanHunter/TESS_Gaia_Light_Curve
 Author: Te Han
@@ -24,9 +24,9 @@ Usage
 There are four fluxes in each FITS file: aperture flux, PSF flux, calibrated aperture flux, and calibrated PSF flux.
 If you are uncertain which to use:
 
-* Calibrated aperture flux is the most robust in transit depth. Use this if you are doing transit science.
-* Calibrated psf flux is better in deblending targets. Use this if you need to deblend a target near a variable source. The best deblending can be achieved with tglc package by setting a non-zero prior.
-* The aperture flux and PSF flux are not detrended or normalized. Use this if you are doing stellar variability science. Or, if the detrending is not optimal (default detrending has a window length of 1 day), start with the aperture flux or PSF flux and detrend carefully!
+* Calibrated psf flux is better in **deblending** targets. Use this if you need to deblend a target near a variable source. The best deblending can be achieved with tglc package by setting a non-zero prior. It also gives the more accurate **transit depth** in most cases, especially when fitting with an optimized prior.
+* Calibrated aperture flux usually has slightly **higher SNR**. The transit depth (or variation amplitude), however, can be imperfect since the normalization depends on the PSF fitting which is imperfect. This imperfection can be minimized by using a bigger aperture than the default aperture (3*3). One need to use the tglc package and set tglc_lc(save_aper=True) to access the 5*5 aperture. In the presence of a bright but "constant" contamination (several magnitudes brighter), the calibrated aperture flux is better in removing the constant contamination. 
+* The aperture flux and PSF flux are not detrended or normalized. Use this if you are doing stellar variability science with long baseline. Or, if the detrending is not optimal (default detrending has a window length of 1 day; see Known Problems below), start with the aperture flux or PSF flux and detrend carefully!
 * **If you are uncertain, start with calibrated aperture flux!**
 
 The `tutorial <tutorial/TGLC_tutorial.ipynb>`_ shows the syntaxes and differences among these light curves in several examples.
@@ -60,7 +60,8 @@ for the latest tglc release. After installation, follow the `tutorial <tutorial/
 Known Problems
 ==================================
 There are several imperfections we noticed in the MAST TGLC light curves and tglc package:
-* If the star is very dim (~< 15 Tmag) near a variable source, it can make the aperture and/or PSF light curve negative for some cadences. This is a very rare scenario, but could be important. The detrending algorithm could malfunction and result in bad cal_aper_flux and/or cal_psf_flux. This is now fixed for tglc package, but this problem remains for the primary mission light curves published on MAST. Please detrend again if necessary. We will also correct for the extended mission light curves on MAST.
+
+* If the star is very dim (~< 15 Tmag) near a variable source, it can make the aperture and/or PSF light curve negative for some cadences. The detrending algorithm could malfunction and result in bad cal_aper_flux and/or cal_psf_flux. This is now fixed for tglc package, but this problem remains for the primary mission light curves published on MAST. Please detrend again if necessary.The extended mission light curves on MAST will not be affected. This is a very rare scenario, but could be important.
 
 ==================================
 Reference

tglc-0.5.7/tglc/__init__.py

@@ -1,3 +0,0 @@
-__version__ = "0.5.7"
-__author__ = 'Te Han, Timothy Brandt'
-__credits__ = 'University of California, Santa Barbara'