stixdcpy 3.0__tar.gz → 4.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (34) hide show
  1. {stixdcpy-3.0/stixdcpy.egg-info → stixdcpy-4.0}/PKG-INFO +25 -8
  2. {stixdcpy-3.0 → stixdcpy-4.0}/README.md +1 -2
  3. stixdcpy-4.0/setup.py +30 -0
  4. {stixdcpy-3.0 → stixdcpy-4.0}/stixdcpy/algorithms.py +5 -0
  5. {stixdcpy-3.0 → stixdcpy-4.0}/stixdcpy/auxiliary.py +22 -0
  6. stixdcpy-4.0/stixdcpy/cfl.py +0 -0
  7. {stixdcpy-3.0 → stixdcpy-4.0}/stixdcpy/continuous_spectrogram.py +9 -0
  8. {stixdcpy-3.0 → stixdcpy-4.0}/stixdcpy/energylut.py +10 -4
  9. {stixdcpy-3.0 → stixdcpy-4.0}/stixdcpy/imgspec.py +18 -10
  10. {stixdcpy-3.0 → stixdcpy-4.0}/stixdcpy/instrument.py +54 -5
  11. stixdcpy-4.0/stixdcpy/integer_compression.py +125 -0
  12. {stixdcpy-3.0 → stixdcpy-4.0}/stixdcpy/io.py +16 -0
  13. {stixdcpy-3.0 → stixdcpy-4.0}/stixdcpy/net.py +1 -0
  14. {stixdcpy-3.0 → stixdcpy-4.0}/stixdcpy/science.py +62 -19
  15. stixdcpy-4.0/stixdcpy/time_util.py +117 -0
  16. {stixdcpy-3.0 → stixdcpy-4.0}/stixdcpy/transmission.py +69 -7
  17. {stixdcpy-3.0 → stixdcpy-4.0/stixdcpy.egg-info}/PKG-INFO +25 -8
  18. {stixdcpy-3.0 → stixdcpy-4.0}/stixdcpy.egg-info/SOURCES.txt +2 -0
  19. stixdcpy-3.0/setup.py +0 -17
  20. stixdcpy-3.0/stixdcpy/time_util.py +0 -40
  21. {stixdcpy-3.0 → stixdcpy-4.0}/LICENSE.md +0 -0
  22. {stixdcpy-3.0 → stixdcpy-4.0}/setup.cfg +0 -0
  23. {stixdcpy-3.0 → stixdcpy-4.0}/stixdcpy/__init__.py +0 -0
  24. stixdcpy-3.0/stixdcpy/cfl.py → stixdcpy-4.0/stixdcpy/ancillary.py +0 -0
  25. {stixdcpy-3.0 → stixdcpy-4.0}/stixdcpy/detector_view.py +0 -0
  26. {stixdcpy-3.0 → stixdcpy-4.0}/stixdcpy/housekeeping.py +0 -0
  27. {stixdcpy-3.0 → stixdcpy-4.0}/stixdcpy/logger.py +0 -0
  28. {stixdcpy-3.0 → stixdcpy-4.0}/stixdcpy/quicklook.py +0 -0
  29. {stixdcpy-3.0 → stixdcpy-4.0}/stixdcpy/spectrogram.py +0 -0
  30. {stixdcpy-3.0 → stixdcpy-4.0}/stixdcpy/stix_map.py +0 -0
  31. {stixdcpy-3.0 → stixdcpy-4.0}/stixdcpy/utils.py +0 -0
  32. {stixdcpy-3.0 → stixdcpy-4.0}/stixdcpy.egg-info/dependency_links.txt +0 -0
  33. {stixdcpy-3.0 → stixdcpy-4.0}/stixdcpy.egg-info/requires.txt +6 -6
  34. {stixdcpy-3.0 → stixdcpy-4.0}/stixdcpy.egg-info/top_level.txt +0 -0
@@ -1,20 +1,40 @@
1
- Metadata-Version: 2.1
1
+ Metadata-Version: 2.4
2
2
  Name: stixdcpy
3
- Version: 3.0
3
+ Version: 4.0
4
4
  Summary: STIX data center APIs and data analysis tools
5
5
  Home-page: https://github.com/i4ds/stixdcpy
6
6
  Author: Hualin Xiao
7
7
  Author-email: hualin.xiao@fhnw.ch
8
- License: UNKNOWN
9
- Platform: UNKNOWN
10
8
  Requires-Python: >=3.7
11
9
  Description-Content-Type: text/markdown
12
10
  License-File: LICENSE.md
11
+ Requires-Dist: numpy
12
+ Requires-Dist: requests
13
+ Requires-Dist: python-dateutil
14
+ Requires-Dist: ipython
15
+ Requires-Dist: astropy
16
+ Requires-Dist: matplotlib
17
+ Requires-Dist: tqdm
18
+ Requires-Dist: pandas
19
+ Requires-Dist: joblib
20
+ Requires-Dist: roentgen
21
+ Requires-Dist: simplejson
22
+ Requires-Dist: sunpy
23
+ Requires-Dist: wget
24
+ Dynamic: author
25
+ Dynamic: author-email
26
+ Dynamic: description
27
+ Dynamic: description-content-type
28
+ Dynamic: home-page
29
+ Dynamic: license-file
30
+ Dynamic: requires-dist
31
+ Dynamic: requires-python
32
+ Dynamic: summary
13
33
 
14
34
 
15
35
  [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.7180433.svg)](https://doi.org/10.5281/zenodo.7180433)
16
36
 
17
- stixdcpy is a Python package designed to simplify the retrieval and analysis of STIX data. It offers convenient APIs for querying and downloading data from the STIX data center (https://datacenter.stix.i4ds.net/), as well as a collection of tools for visualizing data and conducting typical analysis tasks. stixdcpy empowers users to effortlessly retrieve and download various types of data from the STIX data center, including:
37
+ stixdcpy is a Python library for STIX data users to retrieve and analyze STIX data. It provides user-friendly APIs for querying and fetching data from the STIX data center (https://datacenter.stix.i4ds.net/), along with a suite of utilities for visualizing data and conducting common analysis tasks. With stixdcpy, users can effortlessly access and manipulate various data types from the STIX data center, including:
18
38
 
19
39
  * Quick-look light curves
20
40
  * Housekeeping data
@@ -24,7 +44,6 @@ stixdcpy is a Python package designed to simplify the retrieval and analysis of
24
44
  * STIX solar flare list
25
45
  * STIX operations events
26
46
  * Ephemeris data
27
-
28
47
  # Installation
29
48
 
30
49
 
@@ -47,5 +66,3 @@ If you use the stixdcpy in your research and publications, we would definitely a
47
66
 
48
67
  # Reporting Issues and Contributing
49
68
  Open an issue on GitHub to report a problem. Pull requests welcome.
50
-
51
-
@@ -1,7 +1,7 @@
1
1
 
2
2
  [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.7180433.svg)](https://doi.org/10.5281/zenodo.7180433)
3
3
 
4
- stixdcpy is a Python package designed to simplify the retrieval and analysis of STIX data. It offers convenient APIs for querying and downloading data from the STIX data center (https://datacenter.stix.i4ds.net/), as well as a collection of tools for visualizing data and conducting typical analysis tasks. stixdcpy empowers users to effortlessly retrieve and download various types of data from the STIX data center, including:
4
+ stixdcpy is a Python library for STIX data users to retrieve and analyze STIX data. It provides user-friendly APIs for querying and fetching data from the STIX data center (https://datacenter.stix.i4ds.net/), along with a suite of utilities for visualizing data and conducting common analysis tasks. With stixdcpy, users can effortlessly access and manipulate various data types from the STIX data center, including:
5
5
 
6
6
  * Quick-look light curves
7
7
  * Housekeeping data
@@ -11,7 +11,6 @@ stixdcpy is a Python package designed to simplify the retrieval and analysis of
11
11
  * STIX solar flare list
12
12
  * STIX operations events
13
13
  * Ephemeris data
14
-
15
14
  # Installation
16
15
 
17
16
 
stixdcpy-4.0/setup.py ADDED
@@ -0,0 +1,30 @@
1
+ from setuptools import setup, find_packages
2
+
3
+ setup(
4
+ name="stixdcpy",
5
+ description="STIX data center APIs and data analysis tools",
6
+ version="4.0",
7
+ author="Hualin Xiao",
8
+ author_email="hualin.xiao@fhnw.ch",
9
+ long_description=open("README.md").read(),
10
+ install_requires=[
11
+ "numpy",
12
+ "requests",
13
+ "python-dateutil",
14
+ "ipython",
15
+ "astropy",
16
+ "matplotlib",
17
+ "tqdm",
18
+ "pandas",
19
+ "joblib",
20
+ "roentgen",
21
+ "simplejson",
22
+ "sunpy",
23
+ "wget",
24
+ ],
25
+ long_description_content_type="text/markdown",
26
+ # packages=find_packages(where='stixdcpy'),
27
+ url="https://github.com/i4ds/stixdcpy",
28
+ packages=["stixdcpy"],
29
+ python_requires=">=3.7",
30
+ )
@@ -50,6 +50,11 @@ class IO(object):
50
50
  return pd.DataFrame(self.data)
51
51
  return None
52
52
  def dataframe(self):
53
+ """_summary_
54
+
55
+ Returns:
56
+ _type_: _description_
57
+ """
53
58
  if self.data:
54
59
  return pd.DataFrame(self.data)
55
60
  return None
@@ -10,6 +10,11 @@ from stixdcpy.net import Request as jreq
10
10
 
11
11
 
12
12
  class Ephemeris(sio.IO):
13
+ """_summary_
14
+
15
+ Args:
16
+ sio (_type_): _description_
17
+ """
13
18
  def __init__(self, start_utc, end_utc, data):
14
19
  self.start_utc = start_utc
15
20
  self.end_utc = end_utc
@@ -83,6 +88,15 @@ class Ephemeris(sio.IO):
83
88
  return self.data
84
89
 
85
90
  def peek(self, ax=None):
91
+ """_summary_
92
+
93
+ Args:
94
+ ax (_type_, optional): _description_. Defaults to None.
95
+
96
+ Returns:
97
+ _type_: _description_
98
+ """
99
+
86
100
  if not self.data:
87
101
  logger.error(f'Data not loaded. ')
88
102
  return None
@@ -135,6 +149,14 @@ class Pointing(Ephemeris):
135
149
  return cls(utc, data)
136
150
 
137
151
  def peek(self, ax=None):
152
+ """
153
+ Plot the solar ephemeris data
154
+
155
+ Args:
156
+ ax (_type_, optional): _description_. Defaults to None.
157
+ Returns:
158
+ ax: matplotlib axes
159
+ """
138
160
  if not self.data:
139
161
  logger.error(f'Data not loaded. ')
140
162
  return None
File without changes
@@ -1,4 +1,13 @@
1
1
  def fetch_spectrogram(start_utc, end_utc):
2
+ """
3
+ Fetch spectrogram from the STIX data center
4
+ Args:
5
+ start_utc (_type_): _description_
6
+ end_utc (_type_): _description_
7
+
8
+ Returns:
9
+ a diect containing the spectrogram
10
+ """
2
11
  url='https://datacenter.stix.i4ds.net/request/bsd/spectrograms'
3
12
  res=requests.post(url, data={'begin':start_utc, 'end':end_utc})
4
13
  data=res.json()
@@ -44,10 +44,6 @@ class EnergyLUT(sio.IO):
44
44
  def info(self):
45
45
  try:
46
46
  pprint(self.data['info'])
47
- # pprint('Pixel 0 true energy bin edges: ')
48
- # pprint(self.get_pixel_true_ebins(0))
49
- # pprint('...')
50
-
51
47
  except KeyError as e:
52
48
  logger.error(e)
53
49
 
@@ -72,6 +68,16 @@ class EnergyLUT(sio.IO):
72
68
  return None
73
69
 
74
70
  def get_onboard_elut(self):
71
+ """
72
+ Retrieve the onboard energy lookup table (ELUT) data.
73
+
74
+ Returns:
75
+ onboard_elut (dict): The onboard ELUT data, or None if not available.
76
+
77
+ Raises:
78
+ KeyError: If the onboard ELUT data is not present in the STI-1 data.
79
+ TypeError: If the STI-1 data does not contain a 'data' key.
80
+ """
75
81
  try:
76
82
  return self.data['data']['onboard']
77
83
  except (KeyError, TypeError):
@@ -69,13 +69,19 @@ class ImgSpecArchive(sio.IO):
69
69
  """
70
70
  Downloads flare meta data from the STIX data center.
71
71
 
72
- Parameters:
73
- flare_id (int): Flare unique id.
74
- start_utc (str): Start UTC to search for flares.
75
- end_utc (str): End UTC to search for flares.
76
-
77
- Returns:
78
- ImgSpecArchive object: The queried image archive.
72
+ Parameters
73
+ ----------
74
+ flare_id : int
75
+ Flare unique id.
76
+ start_utc : str
77
+ Start UTC to search for flares.
78
+ end_utc : str
79
+ End UTC to search for flares.
80
+
81
+ Returns
82
+ -------
83
+ ImgSpecArchive
84
+ The queried image archive.
79
85
  """
80
86
  flare_id = None
81
87
  start_utc = None
@@ -118,7 +124,7 @@ class ImgSpecArchive(sio.IO):
118
124
 
119
125
  # The implementation of this method goes here (omitted for brevity).
120
126
  def animate(self, image_type='CLEAN'):
121
- pass
127
+ pass
122
128
 
123
129
  def peek(self, image_type='CLEAN'):
124
130
  """
@@ -133,7 +139,9 @@ class ImgSpecArchive(sio.IO):
133
139
  ishown = 0
134
140
  for data in self.imgspec_results:
135
141
  display(HTML(f'<h3>Image ID # {data["_id"]}</h3>'))
136
- uid = data.get('unique_id', None)
142
+ uid = data['unique_id']
143
+ start_utc = data['utc_range'][0]
144
+ work_dir=data['work_dir']
137
145
  try:
138
146
  peak= data['peak_utc']
139
147
  display(
@@ -152,7 +160,7 @@ class ImgSpecArchive(sio.IO):
152
160
  for key, val in report.items():
153
161
  if image_type not in val['title']:
154
162
  continue
155
- url = f'{net.HOST}/image-archive/{uid}/{val["filename"]}'
163
+ url = f'{net.HOST}/image-archive/{work_dir}/{val["filename"]}'
156
164
  display(HTML(f'<b>{val.get("title","")}</b>'))
157
165
  display(Image(url=url, width=400, height=300))
158
166
  ishown += 1
@@ -1,7 +1,9 @@
1
+ #!/usr/bin/env python
2
+ # -*- coding: utf8 -*-
3
+
1
4
  import math
2
5
  import numpy as np
3
6
 
4
-
5
7
  nominal_grid_parameters = {
6
8
  "1": {
7
9
  "Label": "3c",
@@ -1518,19 +1520,66 @@ def get_trigger_acc_detector_ids():
1518
1520
 
1519
1521
 
1520
1522
  def get_sci_ebins(a, b):
1523
+ """
1524
+ Returns the indices of the energy bins that correspond to the specified energy range.
1525
+
1526
+ Parameters
1527
+ ----------
1528
+ a : float
1529
+ Lower energy bound (inclusive).
1530
+ b : float
1531
+ Upper energy bound (exclusive).
1532
+
1533
+ Returns
1534
+ -------
1535
+ tuple
1536
+ A tuple containing the indices of the energy bins that correspond to the specified energy range.
1537
+
1538
+ Raises
1539
+ ------
1540
+ ValueError
1541
+ If the specified energy range is outside the range of valid energy bins.
1542
+
1543
+ """
1521
1544
  low = np.where(ebin_low_edges == a)[0][0]
1522
1545
  up = np.where(ebin_low_edges == b)[0][0] - 1
1523
1546
 
1547
+ if up >= len(ebin_low_edges) or low < 0:
1548
+ raise ValueError(
1549
+ "Invalid energy range specified. Energy range must be within the range of valid energy bins."
1550
+ )
1551
+
1524
1552
  return (low, up)
1525
1553
 
1526
- def get_spectrogram_energy_bins(elow, ehigh, eunit):
1554
+ def get_spectrogram_energy_bins(elow: float, ehigh: float, eunit: int):
1527
1555
  """
1528
1556
  get spectrogram energy bands of science bins
1557
+
1558
+ Parameters
1559
+ ----------
1560
+ elow : float
1561
+ Lower energy bound (inclusive).
1562
+ ehigh : float
1563
+ Upper energy bound (exclusive).
1564
+ eunit : int
1565
+ Energy unit.
1566
+
1567
+ Returns
1568
+ -------
1569
+ List[Tuple[float, float]]
1570
+ A list of tuples containing the lower and upper energy bounds of the energy bins.
1571
+
1572
+ Raises
1573
+ ------
1574
+ ValueError
1575
+ If the specified energy range is outside the range of valid energy bins.
1576
+
1529
1577
  """
1530
1578
  eunit += 1
1531
- nbins=int((ehigh-elow+1)/eunit)
1532
-
1533
- ebins=[ (ebin_low_edges[int(elow+i*eunit)], ebin_low_edges[int(elow+(i+1)*eunit)]) for i in range(nbins)]
1579
+ nbins = int((ehigh - elow + 1) / eunit)
1580
+
1581
+ ebins = [(ebin_low_edges[int(elow + i * eunit)], ebin_low_edges[int(elow + (i + 1) * eunit)])
1582
+ for i in range(nbins)]
1534
1583
 
1535
1584
  return ebins
1536
1585
 
@@ -0,0 +1,125 @@
1
+ """
2
+ This script provides functions for decompressing integer values and creating error lookup tables.
3
+
4
+ It includes the following functions:
5
+
6
+ - decompress(x, K, M): Decompresses a compressed integer value.
7
+ - make_lut(k, m): Creates a lookup table for error calculation based on k and m values.
8
+
9
+ Additionally, it defines error lookup tables for specific combinations of k and m values.
10
+
11
+ Author: Hualin Xiao(hualin.xiao@fhnw.ch)
12
+
13
+ """
14
+
15
+ import numpy as np
16
+ from pprint import pprint
17
+
18
+ MAX_STORED_INTEGER = 1e8
19
+
20
+
21
+
22
+ class Compression(object):
23
+ def __init__(self,s, k,m, include_stat_error=True):
24
+ self.skm=(s,k,m)
25
+ self.lut=Compression.get_error_lut(s,k,m, include_stat_error)
26
+
27
+ def get_errors(self, counts: np.ndarray) -> np.ndarray:
28
+ """
29
+ Calculates the errors for a given set of counts.
30
+
31
+ Parameters:
32
+ counts (np.ndarray): The counts to calculate the errors for.
33
+
34
+ Returns:
35
+ np.ndarray: The calculated errors.
36
+ """
37
+ # create a vectorized function that calculates the errors for a single count
38
+ vectorized_function = np.vectorize(self.get_error, otypes=[np.float])
39
+
40
+ # call the vectorized function on the input counts and return the result
41
+ return vectorized_function(counts)
42
+
43
+
44
+ def get_error(self,counts):
45
+ if counts==0:
46
+ return 0
47
+ try:
48
+ return self.lut[counts]
49
+ except KeyError:
50
+ s,k,m=self.skm
51
+ raise Exception(f'Failed to error of {counts}! Could the compression scheme ({s=},{k=},{m=}) wrong?')
52
+
53
+ @staticmethod
54
+ def decompress(x,S, K, M):
55
+ """
56
+ Decompresses a compressed integer value.
57
+
58
+ Parameters:
59
+ x (int): The compressed integer value to decompress.
60
+ K (int): The number of bits reserved for the exponent.
61
+ M (int): The number of bits reserved for the mantissa.
62
+
63
+ Returns:
64
+ tuple: A tuple containing the error and the decompressed value.
65
+ The error represents the uncertainty in the decompressed value.
66
+ The decompressed value is the original integer value.
67
+ """
68
+ if S + K + M > 8 or S not in (0, 1) or K > 7 or M > 7:
69
+ return None, None
70
+ if K == 0 or M == 0:
71
+ return None, None
72
+
73
+ sign = 1
74
+ if S == 1: # signed
75
+ MSB = x & (1 << 7)
76
+ if MSB != 0:
77
+ sign = -1
78
+ x = x & ((1 << 7) - 1)
79
+
80
+ x0 = 1 << (M + 1)
81
+ if x < x0:
82
+ return x,0
83
+ mask1 = (1 << M) - 1
84
+ mask2 = (1 << M)
85
+ mantissa1 = x & mask1
86
+ exponent = (x >> M) - 1
87
+ # number of shifted bits
88
+ mantissa2 = mask2 | mantissa1 # add 1 before mantissa
89
+ low = mantissa2 << exponent # minimal possible value
90
+ high = low | ((1 << exponent) - 1) # maximal possible value
91
+ mean = (low + high) >> 1 # mean value
92
+ error = np.sqrt((high - low) ** 2 / 12)
93
+ #error of a flat distribution
94
+
95
+ if mean > MAX_STORED_INTEGER:
96
+ return float(mean),error
97
+
98
+ return sign * mean, error
99
+
100
+ @staticmethod
101
+ def get_error_lut(s, k, m, include_stat_error=True):
102
+ """
103
+ Creates a lookup table for error calculation based on k and m values.
104
+
105
+ Parameters:
106
+ k (int): The number of bits reserved for the exponent.
107
+ m (int): The number of bits reserved for the mantissa.
108
+
109
+ Returns:
110
+ dict: A dictionary mapping decompressed values to their respective errors.
111
+ """
112
+ res = {}
113
+ for i in range(256):
114
+ val,err = Compression.decompress(i,s, k, m)
115
+ if err is not None:
116
+ res[val] = np.sqrt(err ** 2 + np.abs(val)) if include_stat_error else err
117
+ return res
118
+
119
+
120
+ if __name__ == '__main__':
121
+ from pprint import pprint
122
+ c=Compression(0,4,4)
123
+ error_luts=c.get_error_lut(0,4,4)
124
+ pprint(error_luts)
125
+
@@ -46,6 +46,12 @@ class IO(object):
46
46
  # pass
47
47
 
48
48
  def to_pandas(self):
49
+ """
50
+ Returns a pandas dataframe representation of the data.
51
+
52
+ Returns:
53
+ pandas.DataFrame: A pandas dataframe representation of the data.
54
+ """
49
55
  if self.data:
50
56
  return pd.DataFrame(self.data)
51
57
  return None
@@ -53,3 +59,13 @@ class IO(object):
53
59
  if self.data:
54
60
  return pd.DataFrame(self.data)
55
61
  return None
62
+ def dataframe(self):
63
+ """
64
+ Returns a pandas dataframe representation of the data.
65
+
66
+ Returns:
67
+ pandas.DataFrame: A pandas dataframe representation of the data.
68
+ """
69
+ if self.data:
70
+ return pd.DataFrame(self.data)
71
+ return None
@@ -338,6 +338,7 @@ class Request(object):
338
338
  @staticmethod
339
339
  def post(url, form, result_type = 'object'):
340
340
  response = requests.post(url, data=form)
341
+
341
342
  try:
342
343
  data = response.json()
343
344
  if result_type == 'object':
@@ -13,8 +13,11 @@ from stixdcpy import time_util as sdt
13
13
  from stixdcpy.logger import logger
14
14
  from stixdcpy import io as sio
15
15
  from stixdcpy.net import FitsQuery as freq
16
+ from stixdcpy import integer_compression as sic
17
+
16
18
  from stixdcpy import net as net
17
19
  from stixdcpy import instrument as inst
20
+
18
21
  from pathlib import PurePath
19
22
  from datetime import datetime as dt
20
23
  from datetime import timedelta as td
@@ -52,6 +55,7 @@ class ScienceData(sio.IO):
52
55
  link = f'{net.HOST}/view/list/bsd/uid/{req_id}'
53
56
  return f'<a href="{link}">{link}</a>'
54
57
 
58
+
55
59
  @property
56
60
  def trigger_error(self):
57
61
  try:
@@ -66,6 +70,12 @@ class ScienceData(sio.IO):
66
70
  def filename(self):
67
71
  return self.fname
68
72
 
73
+
74
+
75
+
76
+
77
+
78
+
69
79
  def read_fits(self, light_time_correction=True):
70
80
  """
71
81
  Read data L1 compression level FITS files
@@ -94,10 +104,9 @@ class ScienceData(sio.IO):
94
104
  self.timedel = self.data['timedel']
95
105
  self.time = self.data['time']
96
106
 
97
- if self.is_time_bin_shifted(self.T0_unix) and len(self.timedel) > 1:
107
+ if self.is_time_bin_shifted() and len(self.timedel) > 1:
98
108
  self.timedel = self.timedel[:-1]
99
109
  self.time = self.time[1:]
100
- logger.info('Shifted time bins have been corrected automatically!')
101
110
  if self.data_type == 'PixelData':
102
111
  self.counts = self.counts[1:, :, :, :]
103
112
  self.triggers = self.triggers[1:, :]
@@ -106,6 +115,7 @@ class ScienceData(sio.IO):
106
115
  self.counts = self.counts[1:, :]
107
116
  self.triggers = self.triggers[1:]
108
117
  # self.rcr = self.rcr[1:]
118
+ logger.info('Shifted time bins have been corrected automatically!')
109
119
 
110
120
  self.request_id = self.hdul['CONTROL'].data['request_id']
111
121
 
@@ -153,7 +163,7 @@ class ScienceData(sio.IO):
153
163
  self.count_rates = self.counts / self.timedel[:, None]
154
164
  self.trigger_rates = self.triggers / self.timedel
155
165
 
156
- def is_time_bin_shifted(self, unix_time):
166
+ def is_time_bin_shifted(self):
157
167
  """
158
168
  Time bins are shifted in the data collected before 2021-12-09 due a bug in the flight software
159
169
 
@@ -164,8 +174,7 @@ class ScienceData(sio.IO):
164
174
  is_shifted: bool
165
175
  True if time bin is shifted else False
166
176
  """
167
-
168
- return (unix_time < sdt.utc2unix('2021-12-09T14:00:00'))
177
+ return (self.T0_unix< sdt.utc2unix('2021-12-09T14:00:00'))
169
178
 
170
179
  @classmethod
171
180
  def from_sdc(cls, request_id, level='L1'):
@@ -260,7 +269,9 @@ class PixelData(ScienceData):
260
269
  self.pixel_count_rates = None
261
270
  self.correct_pixel_count_rates = None
262
271
  self.read_fits(light_time_correction=ltc)
272
+ self.pixel_counts_comp_stat_err= None
263
273
  self.make_spectra()
274
+
264
275
 
265
276
  def make_spectra(self, pixel_counts=None):
266
277
  if pixel_counts is None:
@@ -279,15 +290,46 @@ class PixelData(ScienceData):
279
290
 
280
291
  self.pixel_total_counts = np.sum(self.pixel_counts, axis=(0, 3))
281
292
 
293
+ def compute_errors_from_counts(self, skm=None):
294
+ """
295
+ compute errors using counts
296
+ Args
297
+ include_stat_error: include statistical errors in the final results
298
+
299
+ """
300
+
301
+ if skm is None:
302
+ try:
303
+ skm=self.hdul['CONTROL'].data['compression_scheme_counts_skm']
304
+ s,k,m=skm.flatten()
305
+ except (KeyError,TypeError, ValueError):
306
+ raise Exception("Couldn't find SKM found in the FITS file!")
307
+ else:
308
+ s,k,m=skm
309
+ comp = sic.Compression(s,k,m, include_stat_error=True)
310
+ self.pixel_counts_comp_stat_err=comp.get_errors(self.counts)
311
+
312
+
282
313
  @property
283
314
  def pixel_counts_error(self):
284
- try:
285
- counts_err=self.hdul['data'].data['counts_err']
286
- except KeyError:
287
- counts_err=self.hdul['data'].data['counts_comp_err']
288
- return error_computation(counts_err,
315
+ """
316
+ compute or read counts errors from FITS file
317
+ """
318
+
319
+ if self.pixel_counts_comp_stat_err is not None:
320
+ #if already computed
321
+ return self.pixel_counts_comp_stat_err
322
+ else:
323
+ try:
324
+ counts_err=self.hdul['data'].data['counts_err']
325
+ except KeyError:
326
+ counts_err=self.hdul['data'].data['counts_comp_err']
327
+ #in some FITS files compression errors are zeros
328
+ return error_computation(counts_err,
289
329
  self.pixel_counts)
290
330
 
331
+
332
+
291
333
 
292
334
 
293
335
  def correct_dead_time(self):
@@ -336,7 +378,6 @@ class PixelData(ScienceData):
336
378
 
337
379
  count_rate = counts_arr / time_bins
338
380
  count_rate_err = counts_err_arr / time_bins
339
- # print(counts_arr.shape)
340
381
  for det in range(32):
341
382
  trig_idx = inst.detector_id_to_trigger_index(det)
342
383
  nin = photons_in[:, trig_idx]
@@ -347,13 +388,16 @@ class PixelData(ScienceData):
347
388
 
348
389
  #live_ratio=np.zeros_like(live_ratio)+1
349
390
  #disable live time correction
391
+ live_ratio=live_ratio[:, :, None, None]
350
392
 
351
- corrected_rate = count_rate / live_ratio[:, :, None, None]
393
+ corrected_rate = count_rate / live_ratio
352
394
 
353
- corrected_rate_err = count_rate_err / live_ratio[:, :, None, None]
395
+ corrected_rate_err =count_rate_err/ live_ratio
354
396
 
355
397
  corrected_counts = corrected_rate * time_bins
356
398
  corrected_counts_err = corrected_rate_err * time_bins
399
+ #errors of live ratio not taken into account yet
400
+
357
401
 
358
402
  return {
359
403
  'corrected_rates': corrected_rate,
@@ -450,8 +494,7 @@ class PixelData(ScienceData):
450
494
  + sum_counts['bottom_err']**2 )
451
495
 
452
496
  sum_counts['total'] = sum_counts['top']+sum_counts['bottom']+sum_counts['small']
453
- sum_counts['total_err'] = np.sqrt(sum_counts['top_err']**2
454
- + sum_counts['bottom_err']**2 + sum_counts['small_err']**2)
497
+ sum_counts['total_err'] = np.sqrt(sum_counts['big_err']**2 + sum_counts['small_err']**2)
455
498
 
456
499
  return sum_counts
457
500
 
@@ -602,7 +645,6 @@ class BackgroundSubtraction(object):
602
645
  ])
603
646
  # set counts beyond the signal energy range to 0
604
647
  self.subtracted_counts = (self.l1sig.counts - self.pixel_bkg_counts)
605
- #print(self.l1sig.inversed_energy_bin_mask)
606
648
  self.subtracted_counts *= self.l1sig.energy_bin_mask
607
649
 
608
650
  # Dead time correction needs to be included in the future
@@ -774,7 +816,6 @@ class Spectrogram(ScienceData):
774
816
  def correct(triggers, counts_arr, time_bins, num_detectors):
775
817
  ''' correct dead time using triggers '''
776
818
  tau_conv_const = 1e-6
777
-
778
819
  photons_in = triggers / (time_bins * num_detectors -
779
820
  TRIG_TAU * triggers)
780
821
  # photon rate approximated using triggers
@@ -865,7 +906,8 @@ def fits_time_corrections(primary_header, tstart, tend):
865
906
  primary_header.set('DATE_EAR', date_ear)
866
907
  primary_header.set('DATE_SUN', date_sun)
867
908
  primary_header.set('MJDREF', Time(tstart).mjd)
868
- # Note that MJDREF in these files is NOT 1979-01-01 but rather the observation start time! Affects time calculation later, but further processing of the file corrects this
909
+ # Note that MJDREF in these files is NOT 1979-01-01 but rather the
910
+ # observation start time! Affects time calculation later, but further processing of the file corrects this
869
911
 
870
912
  return primary_header
871
913
 
@@ -967,7 +1009,8 @@ def time_select_indices(tstart, tend, primary_header, data_table, factor=1.):
967
1009
 
968
1010
 
969
1011
  def spec_fits_crop(fitsfile, tstart, tend, outfilename=None):
970
- """ Crop a STIX science data product (L1A, L1, or L4) to contain only the data within a given time interval. Create a new FITS file containing this data. The new file will be of the same processing level as the input file.
1012
+ """ Crop a STIX science data product (L1A, L1, or L4) to contain only the data within a given time interval.
1013
+ Create a new FITS file containing this data. The new file will be of the same processing level as the input file.
971
1014
 
972
1015
  Inputs:
973
1016
  fitsfile : str
@@ -0,0 +1,117 @@
1
+ #!/usr/bin/python3
2
+
3
+ from datetime import datetime
4
+ import pandas as pd
5
+ from astropy.time import Time
6
+
7
+ def anytime(dt, fm='iso'):
8
+ """Converts various date/time formats to specified format.
9
+
10
+ Args:
11
+ dt (str, Time, datetime, Timestamp): Input date/time.
12
+ fm (str): Output format, 'iso' (default), 'unix', or 'datetime'.
13
+
14
+ Returns:
15
+ str or float or datetime: Converted date/time.
16
+
17
+ Raises:
18
+ ValueError: If the input format is not recognized.
19
+ """
20
+ if isinstance(dt, Time):
21
+ dt = dt.to_datetime()
22
+ t = pd.to_datetime(dt, utc=True)
23
+ if fm == 'iso':
24
+ return t.strftime('%Y-%m-%dT%H:%M:%S.%fZ')
25
+ elif fm == 'unix':
26
+ return t.timestamp()
27
+ elif fm == 'datetime':
28
+ return t.to_pydatetime()
29
+ else:
30
+ raise ValueError("Invalid output format. Use 'iso', 'unix', or 'datetime'.")
31
+
32
+
33
+ def utc2unix(dt):
34
+ """Converts UTC date/time to Unix timestamp.
35
+
36
+ Args:
37
+ dt (str): UTC date/time string.
38
+
39
+ Returns:
40
+ float: Unix timestamp.
41
+ """
42
+ t = pd.to_datetime(dt, utc=True)
43
+ return t.timestamp()
44
+
45
+
46
+ def utc2datetime(t):
47
+ """Converts UTC date/time to datetime object.
48
+
49
+ Args:
50
+ t (str): UTC date/time string.
51
+
52
+ Returns:
53
+ datetime: Datetime object in UTC.
54
+ """
55
+ return pd.to_datetime(t, utc=True).to_pydatetime()
56
+
57
+
58
+ def datetime2unix(t):
59
+ """Converts datetime object to Unix timestamp.
60
+
61
+ Args:
62
+ t (datetime): Datetime object.
63
+
64
+ Returns:
65
+ float: Unix timestamp.
66
+ """
67
+ t = pd.to_datetime(t, utc=True)
68
+ return t.timestamp()
69
+
70
+
71
+ def utc2filepath(t):
72
+ """Converts UTC date/time to filepath format 'year/month/day'.
73
+
74
+ Args:
75
+ t (str): UTC date/time string.
76
+
77
+ Returns:
78
+ str: Filepath in 'year/month/day' format.
79
+ """
80
+ dt = utc2datetime(t)
81
+ return dt.strftime("%Y/%m/%d")
82
+
83
+
84
+ def unix2utc(ts):
85
+ """Converts Unix timestamp to UTC date/time string.
86
+
87
+ Args:
88
+ ts (float): Unix timestamp.
89
+
90
+ Returns:
91
+ str: UTC date/time string.
92
+ """
93
+ return datetime.utcfromtimestamp(ts).isoformat(timespec='milliseconds')
94
+
95
+
96
+ def unix2datetime(unix_timestamp):
97
+ """Converts Unix timestamp to datetime object.
98
+
99
+ Args:
100
+ unix_timestamp (float): Unix timestamp.
101
+
102
+ Returns:
103
+ datetime: Datetime object in UTC.
104
+ """
105
+ return datetime.utcfromtimestamp(unix_timestamp)
106
+
107
+ # Example usage and testing:
108
+ # Uncomment and use the functions as needed.
109
+ # For example:
110
+ # print(anytime('2023-04-05T00:00:00', 'iso'))
111
+ # print(utc2unix('2023-04-05T00:00:00'))
112
+ # print(utc2datetime('2023-04-05T00:00:00'))
113
+ # print(datetime2unix(datetime.now()))
114
+ # print(utc2filepath('2023-04-05T00:00:00'))
115
+ # print(unix2utc(1616812800))
116
+ # print(unix2datetime(1616812800))
117
+
@@ -35,8 +35,8 @@ COMPONENTS = OrderedDict([
35
35
  ('calibration_foil', [('al', 4 * 1000 * u.angstrom),
36
36
  ('kapton', 4 * 2 * MIL_SI)]),
37
37
  ('dead_layer', [('te_o2', 392 * u.nm)]),
38
- ('single_grid', [('tungsten', 0.4 * u.mm)]),
39
- ('double_grid', [('tungsten', 0.8 * u.mm)]),
38
+ ('single_grid', [('tungsten', 0.40 * u.mm)]),
39
+ ('double_grid', [('tungsten', 0.80 * u.mm)]),
40
40
  ('caliste', [('cdte', 1 * u.mm)]),
41
41
  ]) #old materials, not valid anymore
42
42
 
@@ -50,7 +50,7 @@ MATERIALS = OrderedDict([
50
50
  }, 5.85 * u.g / u.cm**3)),
51
51
  ('tungsten', ({
52
52
  'W': 1.0
53
- }, 19.28 * u.g / u.cm**3)),
53
+ }, 19.3 * u.g / u.cm**3)),
54
54
  (
55
55
  'alum7075-max',
56
56
  (
@@ -485,15 +485,61 @@ class Transmission:
485
485
  material.mass_attenuation_coefficient.func = material_func
486
486
  return material
487
487
 
488
+ def get_transmission(composition: dict, thickness: float, density: float, energies: np.array):
489
+ """
490
+ Calculates the transmission of a material for given energies based on its composition, thickness, and density.
491
+
492
+ Parameters:
493
+ - composition (dict): A dictionary representing the composition of the material, where keys are chemical symbols
494
+ and values are the fractional masses of the corresponding elements.
495
+ - thickness (float): The thickness of the material in millimeters (mm).
496
+ - density (float): The density of the material in grams per cubic centimeter (g/cm^3).
497
+ - energies (np.array): An array containing the energies in kiloelectronvolts (keV) for which transmission is to be calculated.
498
+
499
+ Returns:
500
+ energies, transmission
501
+
502
+ Note:
503
+ The function internally uses the 'Transmission' class to create a material and calculate its transmission.
504
+ Example usage:
505
+ composition = {'Te': 0.531644, 'Cd': 0.4683554}
506
+ thickness = 2.5 # mm
507
+ density = 8.9 # g/cm^3
508
+ energies = np.array([30, 50, 100]) # keV
509
+ get_transmission(composition, thickness, density, energies)
510
+ """
511
+ density = density * u.g / u.cm ** 3
512
+ thickness = thickness * u.mm
513
+ energies_keV = energies * u.keV
514
+ material = Material(composition,thickness,density=density)
515
+ trans = material.transmission(energies_keV)
516
+ return energies, trans
488
517
 
489
518
  def get_detector_absorption(energies=None):
519
+ """
520
+ Calculates the absorption of a detector material for given energies.
521
+
522
+ Parameters:
523
+ - energies (array-like, optional): An array containing the energies in kiloelectronvolts (keV) for which absorption
524
+ is to be calculated. If not provided, energies will be generated from 2 to 150 keV
525
+ with 1001 points.
526
+
527
+ Returns:
528
+ Tuple: A tuple containing two arrays:
529
+ - energies (array): An array containing the energies in keV.
530
+ - absorption (array): An array containing the corresponding absorption values.
531
+
532
+ Note:
533
+ The function internally uses the 'Transmission' class to create a material and calculate its absorption.
534
+ Example usage:
535
+ get_detector_absorption()
536
+ get_detector_absorption(energies=np.linspace(2, 200, 500))
537
+ """
490
538
  mass_fraction = {'Te': 0.531644, 'Cd': 0.4683554}
491
539
  density = 6.2 * u.g / u.cm**3
492
540
  thickness = 1 * u.mm
493
- material = Transmission.create_material(name='cdte',
494
- fractional_masses=mass_fraction,
495
- thickness=thickness,
496
- density=density)
541
+ material = Material(mass_fraction,thickness,density=density)
542
+
497
543
  if energies is None:
498
544
  energies = np.linspace(2, 150, 1001)
499
545
  energies_keV = energies * u.keV
@@ -501,6 +547,22 @@ def get_detector_absorption(energies=None):
501
547
  return energies, absorption
502
548
 
503
549
  def get_component_absorption(name, energies):
550
+ """
551
+ Calculates the absorption of a specific component material for given energies.
552
+
553
+ Parameters:
554
+ - name (str): The name of the component material.
555
+ - energies (array-like): An array containing the energies in kiloelectronvolts (keV) for which absorption
556
+ is to be calculated.
557
+
558
+ Returns:
559
+ array: An array containing the absorption values corresponding to the given energies.
560
+
561
+ Note:
562
+ The function internally uses the 'Transmission' class to calculate the transmission of the component material.
563
+ Example usage:
564
+ get_component_absorption(name='my_material', energies=np.linspace(2, 150, 1001))
565
+ """
504
566
  t=Transmission()
505
567
  trans=t.get_transmission_of_component(name, energies)
506
568
  return trans
@@ -1,20 +1,40 @@
1
- Metadata-Version: 2.1
1
+ Metadata-Version: 2.4
2
2
  Name: stixdcpy
3
- Version: 3.0
3
+ Version: 4.0
4
4
  Summary: STIX data center APIs and data analysis tools
5
5
  Home-page: https://github.com/i4ds/stixdcpy
6
6
  Author: Hualin Xiao
7
7
  Author-email: hualin.xiao@fhnw.ch
8
- License: UNKNOWN
9
- Platform: UNKNOWN
10
8
  Requires-Python: >=3.7
11
9
  Description-Content-Type: text/markdown
12
10
  License-File: LICENSE.md
11
+ Requires-Dist: numpy
12
+ Requires-Dist: requests
13
+ Requires-Dist: python-dateutil
14
+ Requires-Dist: ipython
15
+ Requires-Dist: astropy
16
+ Requires-Dist: matplotlib
17
+ Requires-Dist: tqdm
18
+ Requires-Dist: pandas
19
+ Requires-Dist: joblib
20
+ Requires-Dist: roentgen
21
+ Requires-Dist: simplejson
22
+ Requires-Dist: sunpy
23
+ Requires-Dist: wget
24
+ Dynamic: author
25
+ Dynamic: author-email
26
+ Dynamic: description
27
+ Dynamic: description-content-type
28
+ Dynamic: home-page
29
+ Dynamic: license-file
30
+ Dynamic: requires-dist
31
+ Dynamic: requires-python
32
+ Dynamic: summary
13
33
 
14
34
 
15
35
  [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.7180433.svg)](https://doi.org/10.5281/zenodo.7180433)
16
36
 
17
- stixdcpy is a Python package designed to simplify the retrieval and analysis of STIX data. It offers convenient APIs for querying and downloading data from the STIX data center (https://datacenter.stix.i4ds.net/), as well as a collection of tools for visualizing data and conducting typical analysis tasks. stixdcpy empowers users to effortlessly retrieve and download various types of data from the STIX data center, including:
37
+ stixdcpy is a Python library for STIX data users to retrieve and analyze STIX data. It provides user-friendly APIs for querying and fetching data from the STIX data center (https://datacenter.stix.i4ds.net/), along with a suite of utilities for visualizing data and conducting common analysis tasks. With stixdcpy, users can effortlessly access and manipulate various data types from the STIX data center, including:
18
38
 
19
39
  * Quick-look light curves
20
40
  * Housekeeping data
@@ -24,7 +44,6 @@ stixdcpy is a Python package designed to simplify the retrieval and analysis of
24
44
  * STIX solar flare list
25
45
  * STIX operations events
26
46
  * Ephemeris data
27
-
28
47
  # Installation
29
48
 
30
49
 
@@ -47,5 +66,3 @@ If you use the stixdcpy in your research and publications, we would definitely a
47
66
 
48
67
  # Reporting Issues and Contributing
49
68
  Open an issue on GitHub to report a problem. Pull requests welcome.
50
-
51
-
@@ -3,6 +3,7 @@ README.md
3
3
  setup.py
4
4
  stixdcpy/__init__.py
5
5
  stixdcpy/algorithms.py
6
+ stixdcpy/ancillary.py
6
7
  stixdcpy/auxiliary.py
7
8
  stixdcpy/cfl.py
8
9
  stixdcpy/continuous_spectrogram.py
@@ -11,6 +12,7 @@ stixdcpy/energylut.py
11
12
  stixdcpy/housekeeping.py
12
13
  stixdcpy/imgspec.py
13
14
  stixdcpy/instrument.py
15
+ stixdcpy/integer_compression.py
14
16
  stixdcpy/io.py
15
17
  stixdcpy/logger.py
16
18
  stixdcpy/net.py
stixdcpy-3.0/setup.py DELETED
@@ -1,17 +0,0 @@
1
- from setuptools import setup, find_packages
2
-
3
- setup(
4
- name='stixdcpy',
5
- description='STIX data center APIs and data analysis tools',
6
- version='3.0',
7
- author='Hualin Xiao',
8
- author_email='hualin.xiao@fhnw.ch',
9
- long_description=open('README.md').read(),
10
- install_requires=['numpy', 'requests', 'python-dateutil', 'ipython',
11
- 'astropy', 'matplotlib','tqdm','pandas','joblib', 'roentgen', 'simplejson', 'sunpy','wget'],
12
- long_description_content_type='text/markdown',
13
- #packages=find_packages(where='stixdcpy'),
14
- url='https://github.com/i4ds/stixdcpy',
15
- packages=['stixdcpy'],
16
- python_requires='>=3.7'
17
- )
@@ -1,40 +0,0 @@
1
- #!/usr/bin/python3
2
-
3
- from datetime import datetime
4
- import pandas as pd
5
- from astropy.time import Time
6
-
7
- def anytime(dt, fm='iso'):
8
- if isinstance(dt, Time):
9
- dt=dt.to_datetime()
10
- t=pd.to_datetime(dt,utc=True)
11
- if fm=='iso':
12
- return t.strftime('%Y-%m-%dT%H:%M:%S.%fZ')
13
- elif fm=='unix':
14
- return t.timestamp()
15
- elif fm=='datetime':
16
- return t.to_pydatetime()
17
- return t
18
-
19
-
20
-
21
- def utc2unix(dt):
22
- print(dt)
23
- t=pd.to_datetime(dt,utc=True)
24
- return t.timestamp()
25
-
26
-
27
- def utc2datetime(t):
28
- return pd.to_datetime(t, utc=True).to_pydatetime()
29
-
30
- def datetime2unix(t):
31
- t=pd.to_datetime(t, utc=True)
32
- return t.timestamp()
33
-
34
-
35
- def unix2utc(ts):
36
- return datetime.utcfromtimestamp(ts).isoformat(timespec='milliseconds')
37
-
38
-
39
- def unix2datetime(unix_timestamp):
40
- return datetime.utcfromtimestamp(unix_timestamp)
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
@@ -1,13 +1,13 @@
1
- astropy
1
+ numpy
2
+ requests
3
+ python-dateutil
2
4
  ipython
3
- joblib
5
+ astropy
4
6
  matplotlib
5
- numpy
7
+ tqdm
6
8
  pandas
7
- python-dateutil
8
- requests
9
+ joblib
9
10
  roentgen
10
11
  simplejson
11
12
  sunpy
12
- tqdm
13
13
  wget