timewise 0.5.4__py3-none-any.whl → 1.0.0a1__py3-none-any.whl

timewise-1.0.0a1.dist-info/METADATA

@@ -0,0 +1,205 @@
+Metadata-Version: 2.4
+Name: timewise
+Version: 1.0.0a1
+Summary: Download WISE infrared data for many objects and process them with AMPEL
+License: MIT
+License-File: LICENSE
+Author: Jannis Necker
+Author-email: jannis.necker@gmail.com
+Requires-Python: >=3.11,<3.12
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Provides-Extra: dev
+Provides-Extra: docs
+Requires-Dist: ampel-alerts (==0.10.3a5)
+Requires-Dist: ampel-core (>=0.10.4.post0,<0.11.0)
+Requires-Dist: ampel-photometry (>=0.10.1,<0.11.0)
+Requires-Dist: ampel-plot (>=0.9.1,<0.10.0)
+Requires-Dist: astropy (>=5.1,<6.0.0)
+Requires-Dist: autodoc_pydantic[erdantic] (>=2.2.0,<3.0.0) ; extra == "docs"
+Requires-Dist: backoff (>=2.1.2,<3.0.0)
+Requires-Dist: coveralls (>=3.3.1,<4.0.0) ; extra == "dev"
+Requires-Dist: jupyter[jupyter] (>=1.0.0,<2.0.0)
+Requires-Dist: jupyterlab[jupyter] (>=4.0.6,<5.0.0)
+Requires-Dist: matplotlib (>=3.5.3,<4.0.0)
+Requires-Dist: mypy (>=1.18.2,<2.0.0) ; extra == "dev"
+Requires-Dist: myst-parser (>=1,<3) ; extra == "docs"
+Requires-Dist: numpy (>=1.23.2,<2.0.0)
+Requires-Dist: pandas (>=1.4.3,<3.0.0)
+Requires-Dist: pandas-stubs (>=2.3.2.250926,<3.0.0.0) ; extra == "dev"
+Requires-Dist: pydantic (>=2.0.0,<3.0.0)
+Requires-Dist: pytest (>=7.2.2,<8.0.0) ; extra == "dev"
+Requires-Dist: pyvo (>=1.7.0,<2.0.0)
+Requires-Dist: requests (>=2.28.1,<3.0.0)
+Requires-Dist: ruff (>=0.13.0,<0.14.0) ; extra == "dev"
+Requires-Dist: scikit-image (>=0.19.3,<0.22.0)
+Requires-Dist: scikit-learn (>=1.3.0,<2.0.0)
+Requires-Dist: scipy-stubs (>=1.16.2.0,<2.0.0.0) ; extra == "dev"
+Requires-Dist: seaborn (>=0.11.2,<0.14.0)
+Requires-Dist: sphinx-rtd-theme (>=1.3.0,<2.0.0) ; extra == "docs"
+Requires-Dist: tqdm (>=4.64.0,<5.0.0)
+Requires-Dist: typer (>=0.19.2,<0.20.0)
+Requires-Dist: types-pyyaml (>=6.0.12.20250915,<7.0.0.0) ; extra == "dev"
+Requires-Dist: types-requests (>=2.32.4.20250913,<3.0.0.0) ; extra == "dev"
+Requires-Dist: urllib3 (>=2.5.0,<3.0.0)
+Requires-Dist: virtualenv (>=20.16.3,<21.0.0)
+Project-URL: Bug Tracker, https://github.com/JannisNe/timewise/issues
+Project-URL: Homepage, https://github.com/JannisNe/timewise
+Description-Content-Type: text/markdown
+
+[![CI](https://github.com/JannisNe/timewise/actions/workflows/continous_integration.yml/badge.svg)](https://github.com/JannisNe/timewise/actions/workflows/continous_integration.yml)
+[![Coverage Status](https://coveralls.io/repos/github/JannisNe/timewise/badge.svg?branch=main)](https://coveralls.io/github/JannisNe/timewise?branch=main)
+[![PyPI version](https://badge.fury.io/py/timewise.svg)](https://badge.fury.io/py/timewise)
+[![DOI](https://zenodo.org/badge/449677569.svg)](https://zenodo.org/badge/latestdoi/449677569)
+
+
+![](timewise.png)
+# Infrared light curves from WISE data
+
+This package downloads WISE data for positions on the sky and stacks single-exposure photometry per visit
+
+## Prerequisites
+
+`timewise` makes use of [AMPEL](https://ampelproject.github.io/ampelastro/) and needs a running [MongoDB](https://www.mongodb.com/).
+
+## Installation
+The package can be installed via `pip`:
+```bash
+pip install timewise
+```
+
+To tell AMPEL which modules, aka units, to use, build the corresponding configuration file:
+```bash
+ampel config build -distributions ampel timewise -stop-on-errors 0 -out <path-to-ampel-config-file>
+```
+
+## Usage
+
+### Command line interface
+
+```
+ Usage: timewise [OPTIONS] COMMAND [ARGS]...                                                                
+                                                                                                            
+ Timewsie CLI                                                                                               
+                                                                                                            
+╭─ Options ────────────────────────────────────────────────────────────────────────────────────────────────╮
+│ --log-level           -l      TEXT  Logging level (DEBUG, INFO, WARNING, ERROR, CRITICAL)                │
+│                                     [default: INFO]                                                      │
+│ --install-completion                Install completion for the current shell.                            │
+│ --show-completion                   Show completion for the current shell, to copy it or customize the   │
+│                                     installation.                                                        │
+│ --help                              Show this message and exit.                                          │
+╰──────────────────────────────────────────────────────────────────────────────────────────────────────────╯
+╭─ Commands ───────────────────────────────────────────────────────────────────────────────────────────────╮
+│ download        Download WISE photometry from IRSA                                                       │
+│ prepare-ampel   Prepares the AMPEL job file so AMPEL can be run manually                                 │
+│ process         Processes the lightcurves using AMPEL                                                    │
+│ export          Write stacked lightcurves to disk                                                        │
+│ run-chain       Run download, process and export                                                         │
+│ plot            Make diagnostic plots                                                                    │
+╰──────────────────────────────────────────────────────────────────────────────────────────────────────────╯
+
+```
+
+The input is a CSV file with at least three columns:  
+- `orig_id`: an original identifier that **must** be an integer (for now)
+- `ra`, `dec`: Right Ascension and Declination
+
+
+
+`timewise` is configured with a YAML file. This is a sensible default which will use all single exposure photometry from AllWISE and NEOWISE:
+```yaml
+download:
+  input_csv: <path-to-input>
+
+  backend:
+    type: filesystem
+    base_path: <path-to-working-directory>
+
+  queries:
+    - type: positional
+      radius_arcsec: 6
+      table:
+        name: allwise_p3as_mep
+      columns:
+        - ra
+        - dec
+        - mjd
+        - cntr_mf
+        - w1mpro_ep
+        - w1sigmpro_ep
+        - w2mpro_ep
+        - w2sigmpro_ep
+        - w1flux_ep
+        - w1sigflux_ep
+        - w2flux_ep
+        - w2sigflux_ep
+
+    - type: positional
+      radius_arcsec: 6
+      table:
+        name: neowiser_p1bs_psd
+      columns:
+        - ra
+        - dec
+        - mjd
+        - allwise_cntr
+        - w1mpro
+        - w1sigmpro
+        - w2mpro
+        - w2sigmpro
+        - w1flux
+        - w1sigflux
+        - w2flux
+        - w2sigflux
+
+ampel:
+  mongo_db_name: <mongodb-name>
+```
+
+This configuration file will be the input to all subcommands. Downloading and stacking can be run together or separate.
+
+
+#### All-in-one:
+Run download, stacking, and export:
+```bash
+timewise run-chain <path-to-config-file> <path-to-ampel-config-file> <output-directory>
+```
+
+#### Separate download and processing:
+To only download the data:
+```bash
+timewise download <path-to-config-file>
+```
+
+To execute the stacking:
+```bash
+timewise process <path-to-config-file> <path-to-ampel-config-file>
+```
+
+#### Run AMPEL manually
+Prepare an AMPEL job file for stacking the single-exposure data:
+```bash
+timewise prepare-ampel <path-to-config-file>
+```
+The result will contain the path to the prepared AMPEL job file that can be run with
+```bash
+ampel job -config <path-to-ampel-config-file> -schema <path-to-ampel-job-file>
+```
+
+#### Make some diagnostic plots
+To check the datapoint selection and binning, take a quick look at the data:
+```bash
+timewise plot <path-to-config-file> <indices-to-plot> <output-directory>
+```
+
+
+## Citation
+If you use `timewise` please make sure to cite [Necker et al. A&A 695, A228 (2025)](https://www.aanda.org/articles/aa/abs/2025/03/aa51340-24/aa51340-24.html).
+Additionally, you might want to include a reference to the specific version you are using: [![DOI](https://zenodo.org/badge/449677569.svg)](https://zenodo.org/badge/latestdoi/449677569)
+
+## Difference lightcurves
+Make sure to check out `timewise-sup`, the Timewise Subtraction Pipeline: 
+[link](https://gitlab.desy.de/jannisnecker/timewise_sup).
+

timewise-1.0.0a1.dist-info/RECORD

@@ -0,0 +1,39 @@
+timewise/__init__.py,sha256=pHLUWHD0-i3VhQGA7h59p8IgmMU_fyxItpb1wjckTmY,24
+timewise/backend/__init__.py,sha256=w79nWfCw8n9g98CkHWJELmb4j9xblWC4DGZOV3_XhH4,134
+timewise/backend/base.py,sha256=dHxRzu2q3uQ0wdGmDxnn-p68Tp19qChue7HMEu56wNA,1080
+timewise/backend/filesystem.py,sha256=GQ4Hrb6_7Q7fKOn6QUl8bqAihAmyeZoTRxElNIwPQ1Y,2465
+timewise/chunking.py,sha256=q7njvTSD84gdcvIk54SC-Ob863MsR79RPec8HS-bm4U,1668
+timewise/cli.py,sha256=l6j-M2-x1LeDEAEnuDv9tadTNmVoxxMNgfyOaS-0oAw,3641
+timewise/config.py,sha256=ZTSokRZMZDqBqVFV9DxvO-47yE9E9xWF48Rcjb0QG10,1044
+timewise/io/__init__.py,sha256=S7vb0glKJnw6ztOlrD-0Wma2bQZ2RwpmXDLFJLKBMVo,35
+timewise/io/config.py,sha256=aizLxt9l4aeWQNsvcemtQdr_fW1vLmpRSofcgA3Bgvk,2239
+timewise/io/download.py,sha256=rKjeW7nh7xUtHh8llvsg_qmkx71Wzn2LggQnyHwqJOI,10719
+timewise/io/stable_tap.py,sha256=jukCkBi2d7WACOo_kXTMCppzWUsN-pLVg9EDqHi3qd0,3478
+timewise/plot/__init__.py,sha256=cc00UenWC_8zAkBH-Ylhs3yCF49tAqZ2Al9MfOoXYDI,120
+timewise/plot/diagnostic.py,sha256=CKxOYd030CRz-JBkxW5lo0GoQ3nNpEWKh3WlxVdio20,8272
+timewise/plot/lightcurve.py,sha256=oK0y6RFzv7QSrO1Qqyc1wNghsTILC9QAfIjuoA8i92I,3757
+timewise/plot/panstarrs.py,sha256=X2ZULm7QT91cp4qociG0fVeI0saGLJwyKzL0141Vqis,8014
+timewise/plot/sdss.py,sha256=cc1zU-4XFkqc8xH5yqCyMsDJf9w_54B_6NeRMjr9Pt8,2622
+timewise/process/__init__.py,sha256=Yk-j1B1MnBuuaM6eFi43TxdWmFKbwFHvDsuQZt4yB_c,70
+timewise/process/config.py,sha256=0TtlxbZs5FNEUt_tv8DKpOxjxp7OoTVunLRUOSw0zPM,831
+timewise/process/interface.py,sha256=ZTUreyu_WkFTs8pOBNqFmCqgVs5OXEtPHjwpGVV1X_s,4929
+timewise/process/keys.py,sha256=0TEVn-BwfzHGlScU-N8AtxgkhA2mUO0wBu4_ol_ylH4,197
+timewise/process/stacking.py,sha256=X5GTkCV5TMn4JmAu8WVM0tQR6mocxNbxrMdR4OVx40Y,12256
+timewise/process/template.yml,sha256=U_xKmygDl3E-viTgZEI8pQIJwWduB52SdI2X9vy61Yo,1037
+timewise/query/__init__.py,sha256=1OA_FlLI7O0aIDOXHpBKMOyMvYLCd0kQVkzoqbxouyE,242
+timewise/query/base.py,sha256=LzG207uIQOEE_RucEKBI4-sHR_EwyILGSlSk10HlEeU,973
+timewise/query/positional.py,sha256=6fLfRdzat1jMqJAY7k7dEdnAZRiWyRKpWkbqpwzDKLo,1255
+timewise/tables/__init__.py,sha256=5efKk4YEhHLA_QTMYc8CQp0nQWXG-F3EAPE6935ManU,263
+timewise/tables/allwise_p3as_mep.py,sha256=NfCnTOdj_6DSFUWGmrwDyrrxIWvJFHktHWF40K_w5wQ,632
+timewise/tables/base.py,sha256=thF5X8gqlmIapxJTkxgktuBwI_wSEeQVHIglL7YcC_M,222
+timewise/tables/neowiser_p1bs_psd.py,sha256=LytaKMLfc-iRfMiTzlLjYgs3_1vWrUjZj7hKIW76lnI,616
+timewise/types.py,sha256=MB-aqpyos1aCS58QWBas34WcwFHiHOkrDj_d9_ZxuVc,667
+timewise/util/backoff.py,sha256=bU5yhsBO4U53XEPJ_32tgql9rq_Rzrv7w32nVQYHr64,272
+timewise/util/csv_utils.py,sha256=5i3Jd4c58-doPs1N_hyYZ8Uc2nvuk9nGwgPNZoNrlu0,298
+timewise/util/error_threading.py,sha256=uyV1Ri-wf87lpa17Xlp520B1V8DWHh3v9Mk97QrPmv0,2264
+timewise/util/visits.py,sha256=3intUo1iiSovu7PB7uUrT-IAxyuOxl58aL0mKj5dAMI,1039
+timewise-1.0.0a1.dist-info/METADATA,sha256=2KeU_-cjdIZePzr7sKNiu-nY97FeuBq087UrzXj_aI8,9172
+timewise-1.0.0a1.dist-info/WHEEL,sha256=zp0Cn7JsFoX2ATtOhtaFYIiE2rmFAD4OcMhtUki8W3U,88
+timewise-1.0.0a1.dist-info/entry_points.txt,sha256=mYh1HsUFbV7KT8kxiGqVtR3Pk0oEk6Bd-2c5FsYVhG4,45
+timewise-1.0.0a1.dist-info/licenses/LICENSE,sha256=sVoNJWiTlH-NarJx0wdsob468Pg3JE6vIIgll4lCa3E,1070
+timewise-1.0.0a1.dist-info/RECORD,,

timewise-1.0.0a1.dist-info/entry_points.txt

@@ -0,0 +1,3 @@
+[console_scripts]
+timewise=timewise.cli:app
+

timewise/big_parent_sample.py

@@ -1,106 +0,0 @@
-import gc
-import pickle
-import threading
-import time
-import logging
-
-from timewise.parent_sample_base import ParentSampleBase
-
-logger = logging.getLogger(__name__)
-
-
-class BigParentSampleBase(ParentSampleBase):
-    """
-    This should not be used. It was a bad idea. The better way would be to implement a
-    ParentSample class that splits the sample file in separate files and then maybe use Dask or similar.
-    """
-
-    def __init__(self, base_name, keep_file_in_memory=30*60):
-        """
-        See doc of ParentSampleBase
-
-        :param base_name: base name for storage directories
-        :type base_name: str
-        :param keep_file_in_memory: time in seconds to keep the parent sample file in the memory, after that gets written to a cache file on disk
-        :type keep_file_in_memory: float
-        """
-        super().__init__(base_name=base_name)
-
-        self._keep_df_in_memory = keep_file_in_memory
-        self._time_when_df_was_used_last = time.time()
-        self._df = None
-        self._cache_file = self.cache_dir / "cache.pkl"
-        self._lock_cache_file = False
-
-        self._clean_thread = threading.Thread(target=self._periodically_drop_df_to_disk, daemon=True, name='ParentSampleCleanThread').start()
-        self._stop_thread = False
-
-    def _wait_for_unlock_cache_file(self):
-        if self._lock_cache_file:
-            logger.debug('cache file locked, waiting')
-            while self._lock_cache_file:
-                pass
-
-        logger.debug('cache file unlocked')
-
-    @property
-    def df(self):
-        self._time_when_df_was_used_last = time.time()
-
-        if isinstance(self._df, type(None)):
-
-            if self._cache_file.is_file():
-                logger.debug(f'loading from {self._cache_file}')
-                self._wait_for_unlock_cache_file()
-                self._lock_cache_file = True
-
-                with open(self._cache_file, "rb") as f:
-                    self._df = pickle.load(f)
-
-                self._lock_cache_file = False
-
-            else:
-                logger.debug(f'No file {self._cache_file}')
-
-        return self._df
-
-    @df.setter
-    def df(self, value):
-        self._time_when_df_was_used_last = time.time()
-        self._df = value
-
-    def _periodically_drop_df_to_disk(self):
-        logger.debug(f'starting cleaning thread for parent sample')
-
-        while True:
-            if self._stop_thread:
-                break
-
-            if (
-                    ((time.time() - self._time_when_df_was_used_last) > self._keep_df_in_memory)
-                    and not isinstance(self._df, type(None))
-            ):
-                logger.debug(f'writing DataFrame to {self._cache_file}')
-                self._wait_for_unlock_cache_file()
-                self._lock_cache_file = True
-
-                with open(self._cache_file, "wb") as f:
-                    pickle.dump(self._df, f)
-
-                self._lock_cache_file = False
-                self._df = None
-                gc.collect()
-
-            time.sleep(self._keep_df_in_memory / 2)
-
-        logger.debug('stopped clean thread')
-
-    def __del__(self):
-        if hasattr(self, "_cache_file") and self._cache_file.is_file():
-            logger.debug(f'removing {self._cache_file}')
-            self._cache_file.unlink()
-
-        if hasattr(self, "clean_thread"):
-            logger.debug(f'stopping clean thread')
-            self._stop_thread = True
-            self._clean_thread.join()

timewise/config_loader.py

@@ -1,157 +0,0 @@
-import logging
-import yaml
-import json
-import inspect
-from pydantic import BaseModel, validator
-import pandas as pd
-import importlib
-from pathlib import Path
-
-from timewise.parent_sample_base import ParentSampleBase
-from timewise.wise_data_base import WISEDataBase
-
-
-logger = logging.getLogger(__name__)
-
-
-class TimewiseConfig(BaseModel):
-
-    wise_data: WISEDataBase
-    timewise_instructions: list[dict] = list()
-
-    class Config:
-        arbitrary_types_allowed = True
-
-    @validator("timewise_instructions")
-    def validate_instructions(cls, v: list[dict], values: dict):
-        # get the WiseData class
-        wise_data = values["wise_data"]
-        wise_data_class_name = type(wise_data).__name__
-        # collect its members
-        members = inspect.getmembers(wise_data)
-        # loop through the methods and the corresponding arguments that wre given in the instructions
-        for instructions in v:
-            for method, arguments in instructions.items():
-                found = False
-                # check each member for a fit
-                for member_name, member in members:
-                    if member_name == method:
-                        found = True
-                        # get the call signature of the member and see if it fits the given arguments
-                        signature = inspect.signature(member)
-                        param_list = list(signature.parameters)
-                        # check if the member is a normal method, i.e. if the first arguments is 'self'
-                        is_method = param_list[0] == "self"
-                        _arguments = arguments or dict()
-                        try:
-                            if is_method:
-                                signature.bind(WISEDataBase, **_arguments)
-                            else:
-                                signature.bind(**_arguments)
-                        except TypeError as e:
-                            raise ValueError(f"{wise_data_class_name}.{method}: {e}!")
-
-                if not found:
-                    raise ValueError(f"{wise_data_class_name} does not have a method {method}!")
-
-        return v
-
-    def run_config(self):
-        logger.info("running config")
-        for instructions in self.timewise_instructions:
-            for method, arguments in instructions.items():
-                _arguments = arguments or dict()
-                logger.info(f"running {method} with arguments {_arguments}")
-                self.wise_data.__getattribute__(method)(**_arguments)
-        logger.info("successfully ran config")
-
-
-class TimewiseConfigLoader(BaseModel):
-
-    base_name: str
-    filename: str = None
-    class_module: str = "timewise"
-    class_name: str = "WiseDataByVisit"
-    min_sep_arcsec: float = 6.
-    n_chunks: int = 1
-    default_keymap: dict = {k: k for k in ["ra", "dec", "id"]}
-    timewise_instructions: list[dict] = list()
-
-    @validator("filename")
-    def validate_file(cls, v: str):
-        if v is not None:
-            if not Path(v).is_file():
-                raise ValueError(f"No file {v}!")
-        return v
-
-    @validator("class_module")
-    def validate_class_module(cls, v: str):
-        try:
-            importlib.import_module(v)
-        except (ImportError, ModuleNotFoundError):
-            raise ValueError(f"Could not import {v}")
-        return v
-
-    @validator("class_name")
-    def validate_class_name(cls, v: str, values: dict):
-        if "class_module" not in values:
-            raise ValueError("class_module must be given before class_name!")
-        class_module = values["class_module"]
-        try:
-            getattr(importlib.import_module(class_module), v)
-        except (ImportError, AttributeError):
-            raise ValueError(f"Could not find {v} in {class_module}")
-        return v
-
-    @validator("default_keymap")
-    def validate_keymap(cls, v: dict):
-        for k in ["ra", "dec", "id"]:
-            if k not in v:
-                raise ValueError(f"Keymap is missing key {k}!")
-        return v
-
-    def parse_config(self):
-
-        logger.info(f"Parsing config")
-        logger.debug(json.dumps(self.dict(), indent=4))
-
-        _default_keymap = self.default_keymap
-        _base_name = self.base_name
-        _filename = self.filename
-        _class_name = self.class_name
-
-        class DynamicParentSample(ParentSampleBase):
-            default_keymap = _default_keymap
-
-            def __init__(self):
-                super().__init__(_base_name)
-                self.df = pd.read_csv(_filename)
-
-                for k, v in self.default_keymap.items():
-                    if v not in self.df.columns:
-                        raise KeyError(f"Can not map '{v}' to '{k}': '{v}' not in table columns! Adjust keymap")
-
-        wise_data_config = {
-            "base_name": _base_name,
-            "parent_sample_class": DynamicParentSample,
-            "n_chunks": self.n_chunks,
-            "min_sep_arcsec": self.min_sep_arcsec
-        }
-        wise_data_class = getattr(importlib.import_module(self.class_module), self.class_name)
-        wise_data = wise_data_class(**wise_data_config)
-
-        return TimewiseConfig(wise_data=wise_data, timewise_instructions=self.timewise_instructions)
-
-    @classmethod
-    def read_yaml(cls, filename):
-        logger.debug(f"reading {filename}")
-        with open(filename, "r") as f:
-            config_dict = yaml.safe_load(f)
-        logger.debug(f"config: {json.dumps(config_dict, indent=4)}")
-        return cls(**config_dict)
-
-    @classmethod
-    def run_yaml(cls, filename):
-        logger.info(f"running {filename}")
-        config = cls.read_yaml(filename).parse_config()
-        config.run_config()

timewise/general.py

@@ -1,52 +0,0 @@
-import logging
-import os
-from pathlib import Path
-
-
-# Setting up the Logger
-main_logger = logging.getLogger('timewise')
-logger_format = logging.Formatter(
-    '%(levelname)s:%(name)s:%(funcName)s - [%(threadName)s] - %(asctime)s: \n\t%(message)s', "%H:%M:%S"
-)
-stream_handler = logging.StreamHandler()
-stream_handler.setFormatter(logger_format)
-main_logger.addHandler(stream_handler)
-main_logger.propagate = False  # do not propagate to root logger
-
-logger = logging.getLogger(__name__)
-
-
-def get_directories() -> dict[str, Path | None]:
-    # Setting up data directory
-    DATA_DIR_KEY = 'TIMEWISE_DATA'
-    if DATA_DIR_KEY in os.environ:
-        data_dir = Path(os.environ[DATA_DIR_KEY]).expanduser()
-    else:
-        logger.warning(f'{DATA_DIR_KEY} not set! Using home directory.')
-        data_dir = Path('~/').expanduser()
-
-    BIGDATA_DIR_KEY = 'TIMEWISE_BIGDATA'
-    if BIGDATA_DIR_KEY in os.environ:
-        bigdata_dir = Path(os.environ[BIGDATA_DIR_KEY]).expanduser()
-        logger.info(f"Using bigdata directory {bigdata_dir}")
-    else:
-        bigdata_dir = None
-        logger.info(f"No bigdata directory set.")
-
-    output_dir = data_dir / 'output'
-    plots_dir = output_dir / 'plots'
-    cache_dir = data_dir / 'cache'
-
-    return {
-        'data_dir': data_dir,
-        'bigdata_dir': bigdata_dir,
-        'output_dir': output_dir,
-        'plots_dir': plots_dir,
-        'cache_dir': cache_dir
-    }
-
-
-def backoff_hndlr(details):
-    logger.info("Backing off {wait:0.1f} seconds after {tries} tries "
-                "calling function {target} with args {args} and kwargs "
-                "{kwargs}".format(**details))

timewise/parent_sample_base.py

@@ -1,89 +0,0 @@
-import abc, os
-import pandas as pd
-import numpy as np
-import logging
-
-from timewise.general import get_directories
-from timewise.utils import plot_sdss_cutout, plot_panstarrs_cutout
-
-
-logger = logging.getLogger(__name__)
-
-
-class ParentSampleBase(abc.ABC):
-    """
-    Base class for parent sample.
-    Any subclass must implement
-
-    - `ParentSample.df`: A `pandas.DataFrame` consisting of minimum three columns: two columns holding the sky positions of each object in the form of right ascension and declination and one row with a unique identifier.
-    - `ParentSample.default_keymap`: a dictionary, mapping the column in `ParentSample.df` to 'ra', 'dec' and 'id'
-
-    :param base_name: determining the location of any data in the `timewise` data directory.
-    """
-
-    df = pd.DataFrame()
-    default_keymap = dict()
-
-    def __init__(self, base_name):
-        # set up directories
-        d = get_directories()
-        self.cache_dir = d["cache_dir"] / base_name
-        self.plots_dir = d["plots_dir"] / base_name
-
-        for d in [self.cache_dir, self.plots_dir]:
-            d.parent.mkdir(parents=True, exist_ok=True)
-
-        self.local_sample_copy = self.cache_dir / 'sample.csv'
-
-    def plot_cutout(self, ind, arcsec=20, interactive=False, **kwargs):
-        """
-        Plot the coutout images in all filters around the position of object with index i
-
-        :param ind: the index in the sample
-        :type ind: int or list-like
-        :param arcsec: the radius of the cutout
-        :type arcsec: float
-        :param interactive: interactive mode
-        :type interactive: bool
-        :param kwargs: any additional kwargs will be passed to `matplotlib.pyplot.subplots()`
-        :return: figure and axes if `interactive=True`
-        """
-        sel = self.df.loc[np.atleast_1d(ind)]
-        ra, dec = sel[self.default_keymap["ra"]], sel[self.default_keymap["dec"]]
-        title = [r[self.default_keymap["id"]] for i, r in sel.iterrows()]
-
-        fn = kwargs.pop(
-            "fn",
-            [self.plots_dir / f"{i}_{r[self.default_keymap['id']]}.pdf"
-             for i, r in sel.iterrows()]
-        )
-        self.plots_dir.mkdir(parents=True, exist_ok=True)
-
-        logger.debug(f"\nRA: {ra}\nDEC: {dec}\nTITLE: {title}\nFN: {fn}")
-        ou = list()
-
-        ras = np.atleast_1d(ra)
-        decs = np.atleast_1d(dec)
-        title = np.atleast_1d(title) if title else [None] * len(ras)
-        fn = np.atleast_1d(fn) if fn else [None] * len(ras)
-
-        for _ra, _dec, _title, _fn in zip(ras, decs, title, fn):
-            ou.append(self._plot_cutout(_ra, _dec, arcsec, interactive, title=_title, fn=_fn, **kwargs))
-
-        if len(ou) == 1:
-            ou = ou[0]
-        return ou
-
-    @staticmethod
-    def _plot_cutout(ra, dec, arcsec, interactive, which="sdss", **kwargs):
-        if which == "sdss":
-            return plot_sdss_cutout(ra, dec, arcsec=arcsec, interactive=interactive, **kwargs)
-        elif which == "panstarrs":
-            return plot_panstarrs_cutout(ra, dec, arcsec=arcsec, interactive=interactive, **kwargs)
-        else:
-            raise ValueError(f"{which} not an implemented survey! Choose one of 'sdss' or 'panstarrs'.")
-
-    def save_local(self):
-        logger.debug(f"saving under {self.local_sample_copy}")
-        self.local_sample_copy.parent.mkdir(parents=True, exist_ok=True)
-        self.df.to_csv(self.local_sample_copy)