mlso-api-client 0.3.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2025 NSF National Center for Atmospheric Research
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,165 @@
1
+ Metadata-Version: 2.4
2
+ Name: mlso-api-client
3
+ Version: 0.3.0
4
+ Summary: Python package providing a client to access MLSO data
5
+ Author-email: Michael Galloy <mgalloy@ucar.edu>
6
+ License-Expression: BSD-3-Clause
7
+ Project-URL: homepage, https://www2.hao.ucar.edu/mlso
8
+ Project-URL: repository, https://github.com/NCAR/mlso-api-client
9
+ Project-URL: documentation, https://mlso-api-client.readthedocs.io/en/latest/
10
+ Project-URL: issues, https://github.com/NCAR/mlso-api-client/issues
11
+ Project-URL: changelog, https://github.com/NCAR/mlso-api-client/blob/master/CHANGELOG.md
12
+ Keywords: webservice,Mauna Loa Solar Observatory,MLSO,solar
13
+ Classifier: Development Status :: 4 - Beta
14
+ Classifier: Intended Audience :: Developers
15
+ Classifier: Intended Audience :: Science/Research
16
+ Classifier: Natural Language :: English
17
+ Classifier: Programming Language :: Python
18
+ Requires-Python: >=3.7
19
+ Description-Content-Type: text/markdown
20
+ License-File: LICENSE
21
+ Requires-Dist: requests
22
+ Requires-Dist: tqdm
23
+ Provides-Extra: dev
24
+ Requires-Dist: pytest; extra == "dev"
25
+ Requires-Dist: pre-commit; extra == "dev"
26
+ Requires-Dist: tox; extra == "dev"
27
+ Requires-Dist: wheel; extra == "dev"
28
+ Requires-Dist: watchdog; extra == "dev"
29
+ Requires-Dist: Sphinx; extra == "dev"
30
+ Requires-Dist: twine; extra == "dev"
31
+ Requires-Dist: coverage; extra == "dev"
32
+ Requires-Dist: flake8; extra == "dev"
33
+ Requires-Dist: pytest-runner; extra == "dev"
34
+ Requires-Dist: black; extra == "dev"
35
+ Requires-Dist: sphinx_rtd_theme; extra == "dev"
36
+ Requires-Dist: myst-parser; extra == "dev"
37
+ Dynamic: license-file
38
+
39
+ # mlso-api-client
40
+
41
+ This package contains Python and IDL clients for accessing MLSO data via the
42
+ MLSO data web API.
43
+
44
+ ## Installation
45
+
46
+ ### Installing from PyPI
47
+
48
+ The easiest way to install the MLSO API client is via the released versions on
49
+ PyPI. This is the recommended method for most users.
50
+
51
+ ```console
52
+ pip install mlso-api-client
53
+ ```
54
+
55
+ If you want to upgrade an existing installation, do:
56
+
57
+ ```console
58
+ pip install -U mlso-api-client
59
+ ```
60
+
61
+
62
+ ### Installing from source
63
+
64
+ The source code can be found on the [repo's GitHub page]. Use git or download
65
+ a ZIP file with contents of the source.
66
+
67
+ [repo's GitHub page]: https://github.com/NCAR/mlso-api-client
68
+
69
+ Once you have the source code, install the Python portion of the package:
70
+
71
+ ```console
72
+ cd mlso-api-client
73
+ pip install .
74
+ ```
75
+
76
+ If you intend to make changes to the code, install the dev requirements and
77
+ allow changes to the code to automatically be used:
78
+
79
+ ```console
80
+ pip install -e .[dev]
81
+ ```
82
+
83
+ For IDL, simply put the `idl/` directory in your `IDL_PATH`.
84
+
85
+
86
+ ## Usage
87
+
88
+ ### Command-line interface
89
+
90
+ Installing the Python package, should install a command-line utility to query
91
+ and download MLSO data, the `mlsoapi` script.
92
+
93
+ ```console
94
+ $ mlsoapi --help
95
+ usage: mlsoapi [-h] [-v] [-u BASE_URL] [--verbose] [-q] {instruments,products,files} ...
96
+
97
+ MLSO API command line interface (mlso-api-client 1.0.0)
98
+
99
+ positional arguments:
100
+ {instruments,products,files}
101
+ sub-command help
102
+ instruments MLSO instruments
103
+ products MLSO instruments
104
+ files MLSO data files
105
+
106
+ options:
107
+ -h, --help show this help message and exit
108
+ -v, --version show program's version number and exit
109
+ -u BASE_URL, --base-url BASE_URL
110
+ base URL for MLSO API
111
+ --verbose output warnings
112
+ -q, --quiet surpress informational messages
113
+ ```
114
+
115
+ To determine the instruments with data available via the API, use the
116
+ "instruments" sub-command:
117
+
118
+ ```console
119
+ $ mlsoapi instruments
120
+ ID Instrument name Dates available
121
+ -------- -------------------------------------------- -----------------------
122
+ kcor COSMO K-Coronagraph (KCor) 2013-09-30...2025-03-24
123
+ ucomp Upgraded Coronal Multi-Polarimeter (UCoMP) 2021-07-15...2025-03-24
124
+ ```
125
+
126
+ New data for existing and new instruments will be added to the API as possible.
127
+ Submit requests via the [Issues].
128
+
129
+ [Issues]: https://github.com/NCAR/mlso-api-client/issues
130
+
131
+ Each instrument has various products available:
132
+
133
+ ```console
134
+ $ mlsoapi products --instrument ucomp
135
+ ID Title Description
136
+ ------------- ---------------------- -------------------------------------------------------
137
+ l1 Level 1 IQUV and backgrounds for various wavelengths
138
+ intensity Level 1 intensity intensity-only level 1
139
+ mean Level 1 mean mean of level 1 files
140
+ median Level 1 median median of level 1 files
141
+ sigma Level 1 sigma standard deviation of level 1 files
142
+ l2 Level 2 level 2 products
143
+ l2average Level 2 average mean, median, standard deviation of level 2 files
144
+ density Density density
145
+ dynamics Dynamics level 2 dynamics products
146
+ polarization Polarization level 2 polarization products
147
+ all All all products
148
+ ```
149
+
150
+ ### Python API
151
+
152
+ TODO: example of using the Python API
153
+
154
+
155
+ ### IDL API
156
+
157
+ TODO: example using the IDL routines
158
+
159
+
160
+ ### API endpoints
161
+
162
+ To use the webservice API directly from any language, see the [API Endpoints] wiki
163
+ page.
164
+
165
+ [API Endpoints]: https://github.com/NCAR/mlso-api-client/wiki/API-endpoints
@@ -0,0 +1,127 @@
1
+ # mlso-api-client
2
+
3
+ This package contains Python and IDL clients for accessing MLSO data via the
4
+ MLSO data web API.
5
+
6
+ ## Installation
7
+
8
+ ### Installing from PyPI
9
+
10
+ The easiest way to install the MLSO API client is via the released versions on
11
+ PyPI. This is the recommended method for most users.
12
+
13
+ ```console
14
+ pip install mlso-api-client
15
+ ```
16
+
17
+ If you want to upgrade an existing installation, do:
18
+
19
+ ```console
20
+ pip install -U mlso-api-client
21
+ ```
22
+
23
+
24
+ ### Installing from source
25
+
26
+ The source code can be found on the [repo's GitHub page]. Use git or download
27
+ a ZIP file with contents of the source.
28
+
29
+ [repo's GitHub page]: https://github.com/NCAR/mlso-api-client
30
+
31
+ Once you have the source code, install the Python portion of the package:
32
+
33
+ ```console
34
+ cd mlso-api-client
35
+ pip install .
36
+ ```
37
+
38
+ If you intend to make changes to the code, install the dev requirements and
39
+ allow changes to the code to automatically be used:
40
+
41
+ ```console
42
+ pip install -e .[dev]
43
+ ```
44
+
45
+ For IDL, simply put the `idl/` directory in your `IDL_PATH`.
46
+
47
+
48
+ ## Usage
49
+
50
+ ### Command-line interface
51
+
52
+ Installing the Python package, should install a command-line utility to query
53
+ and download MLSO data, the `mlsoapi` script.
54
+
55
+ ```console
56
+ $ mlsoapi --help
57
+ usage: mlsoapi [-h] [-v] [-u BASE_URL] [--verbose] [-q] {instruments,products,files} ...
58
+
59
+ MLSO API command line interface (mlso-api-client 1.0.0)
60
+
61
+ positional arguments:
62
+ {instruments,products,files}
63
+ sub-command help
64
+ instruments MLSO instruments
65
+ products MLSO instruments
66
+ files MLSO data files
67
+
68
+ options:
69
+ -h, --help show this help message and exit
70
+ -v, --version show program's version number and exit
71
+ -u BASE_URL, --base-url BASE_URL
72
+ base URL for MLSO API
73
+ --verbose output warnings
74
+ -q, --quiet surpress informational messages
75
+ ```
76
+
77
+ To determine the instruments with data available via the API, use the
78
+ "instruments" sub-command:
79
+
80
+ ```console
81
+ $ mlsoapi instruments
82
+ ID Instrument name Dates available
83
+ -------- -------------------------------------------- -----------------------
84
+ kcor COSMO K-Coronagraph (KCor) 2013-09-30...2025-03-24
85
+ ucomp Upgraded Coronal Multi-Polarimeter (UCoMP) 2021-07-15...2025-03-24
86
+ ```
87
+
88
+ New data for existing and new instruments will be added to the API as possible.
89
+ Submit requests via the [Issues].
90
+
91
+ [Issues]: https://github.com/NCAR/mlso-api-client/issues
92
+
93
+ Each instrument has various products available:
94
+
95
+ ```console
96
+ $ mlsoapi products --instrument ucomp
97
+ ID Title Description
98
+ ------------- ---------------------- -------------------------------------------------------
99
+ l1 Level 1 IQUV and backgrounds for various wavelengths
100
+ intensity Level 1 intensity intensity-only level 1
101
+ mean Level 1 mean mean of level 1 files
102
+ median Level 1 median median of level 1 files
103
+ sigma Level 1 sigma standard deviation of level 1 files
104
+ l2 Level 2 level 2 products
105
+ l2average Level 2 average mean, median, standard deviation of level 2 files
106
+ density Density density
107
+ dynamics Dynamics level 2 dynamics products
108
+ polarization Polarization level 2 polarization products
109
+ all All all products
110
+ ```
111
+
112
+ ### Python API
113
+
114
+ TODO: example of using the Python API
115
+
116
+
117
+ ### IDL API
118
+
119
+ TODO: example using the IDL routines
120
+
121
+
122
+ ### API endpoints
123
+
124
+ To use the webservice API directly from any language, see the [API Endpoints] wiki
125
+ page.
126
+
127
+ [API Endpoints]: https://github.com/NCAR/mlso-api-client/wiki/API-endpoints
@@ -0,0 +1,84 @@
1
+ [project]
2
+ name = "mlso-api-client"
3
+ version = "0.3.0"
4
+ description = "Python package providing a client to access MLSO data"
5
+ authors = [
6
+ { name="Michael Galloy", email="mgalloy@ucar.edu" },
7
+ ]
8
+ license = "BSD-3-Clause"
9
+ license-files = ["LICENSE"]
10
+ readme = "README.md"
11
+ requires-python = ">=3.7"
12
+ # see classifiers at https://pypi.org/classifiers/
13
+ classifiers = [
14
+ "Development Status :: 4 - Beta",
15
+ # Development Status :: 5 - Production/Stable
16
+ "Intended Audience :: Developers",
17
+ "Intended Audience :: Science/Research",
18
+ "Natural Language :: English",
19
+ "Programming Language :: Python",
20
+ ]
21
+ keywords = ["webservice", "Mauna Loa Solar Observatory", "MLSO", "solar"]
22
+ dependencies = [
23
+ "requests",
24
+ "tqdm"
25
+ ]
26
+
27
+ [project.urls]
28
+ homepage = "https://www2.hao.ucar.edu/mlso"
29
+ repository = "https://github.com/NCAR/mlso-api-client"
30
+ documentation = "https://mlso-api-client.readthedocs.io/en/latest/"
31
+ issues = "https://github.com/NCAR/mlso-api-client/issues"
32
+ changelog = "https://github.com/NCAR/mlso-api-client/blob/master/CHANGELOG.md"
33
+
34
+ [project.optional-dependencies]
35
+ dev = [
36
+ "pytest",
37
+ "pre-commit",
38
+ "tox",
39
+ "wheel",
40
+ "watchdog",
41
+ "Sphinx",
42
+ "twine",
43
+ "coverage",
44
+ "flake8",
45
+ "pytest-runner",
46
+ "black",
47
+ "sphinx_rtd_theme",
48
+ "myst-parser",
49
+ ]
50
+
51
+ [project.scripts]
52
+ mlsoapi = "mlso.api:client.main"
53
+
54
+ [tool.black]
55
+ line-length = 88
56
+ target-version = ['py37']
57
+ include = '\.pyi?$'
58
+ exclude = '''
59
+
60
+ (
61
+ /(
62
+ \.eggs # exclude a few common directories in the
63
+ | \.git # root of the project
64
+ | \.hg
65
+ | \.mypy_cache
66
+ | \.tox
67
+ | \.venv
68
+ | _build
69
+ | buck-out
70
+ | build
71
+ | dist
72
+ )/
73
+ | foo.py # also separately exclude a file named foo.py in
74
+ # the root of the project
75
+ )
76
+ '''
77
+
78
+ [tool.pytest.ini_options]
79
+ # increment the --cov-fail-under as we increase test coverage
80
+ addopts = "--cov-report html:coverage_html --cov-report term-missing --cov-fail-under 80"
81
+
82
+ [build-system]
83
+ requires = ["setuptools >= 77.0.3"]
84
+ build-backend = "setuptools.build_meta"
@@ -0,0 +1,4 @@
1
+ [egg_info]
2
+ tag_build =
3
+ tag_date = 0
4
+
@@ -0,0 +1,9 @@
1
+ # encoding: utf-8
2
+
3
+ """This package contains Python and IDL clients for accessing MLSO data via the
4
+ MLSO data web API.
5
+ """
6
+
7
+ import importlib.metadata
8
+
9
+ __version__ = importlib.metadata.version("mlso-api-client")
@@ -0,0 +1,553 @@
1
+ # -*- coding: utf-8 -*-
2
+
3
+ """Module defining the Python and command line client.
4
+ """
5
+
6
+ import argparse
7
+ import datetime
8
+ import logging
9
+ import math
10
+ import os
11
+ from pathlib import Path
12
+ from pprint import pformat
13
+ import sys
14
+ import textwrap
15
+
16
+ import requests
17
+ import tqdm
18
+
19
+
20
+ from . import __version__
21
+
22
+ BASE_URL = "http://api.mlso.ucar.edu:5000"
23
+ API_VERSION = "v1"
24
+ SIGNUP_URL = "https://registration.hao.ucar.edu"
25
+
26
+
27
+ # chunk size for downloading files, 1-10M is probably the most efficient size
28
+ # for good speed and reasonable memory usage
29
+ CHUNK_SIZE = 1024 * 1024
30
+
31
+ session = requests.Session()
32
+
33
+ # setup default logging, users of this library can set this to their own logger
34
+ # or modify this one to suit their needs
35
+ logger = logging.getLogger(__name__)
36
+ logger.addHandler(logging.StreamHandler())
37
+ logger.setLevel(logging.DEBUG)
38
+
39
+
40
+ # API
41
+
42
+
43
+ class UserNotFound(Exception):
44
+ """Exception raised when a username is found in the registration database."""
45
+
46
+ pass
47
+
48
+
49
+ class ServerError(Exception):
50
+ """Exception raised when there is a problem with the server not returning a
51
+ valid result.
52
+ """
53
+
54
+ pass
55
+
56
+
57
+ def about(
58
+ base_url: str = BASE_URL, api_version: str = API_VERSION, verbose: bool = False
59
+ ):
60
+ """Retrieve the results of the `/about` endpoint."""
61
+ url = f"{base_url}/{api_version}/about"
62
+ if verbose:
63
+ logger.debug(f"URL: {url}")
64
+
65
+ try:
66
+ r = requests.get(url)
67
+ except requests.exceptions.ConnectionError as e:
68
+ raise ServerError(f"Connection error reaching {url}")
69
+
70
+ if not r.ok:
71
+ j = r.json()
72
+ msg = j["message"]
73
+ raise ServerError(f"Server response: {r.status_code} {r.reason} ({msg})")
74
+
75
+ j = r.json()
76
+
77
+ if verbose:
78
+ logger.debug(pformat(j))
79
+
80
+ return j
81
+
82
+
83
+ def instruments(
84
+ base_url: str = BASE_URL, api_version: str = API_VERSION, verbose: bool = False
85
+ ):
86
+ """Retrieve list of instruments from the `/instruments/{instrument}`
87
+ endpoint with some of their properties.
88
+ """
89
+ url = f"{base_url}/{api_version}/instruments"
90
+ if verbose:
91
+ logger.debug(f"URL: {url}")
92
+
93
+ try:
94
+ r = requests.get(url)
95
+ except requests.exceptions.ConnectionError as e:
96
+ raise ServerError(f"Connection error reaching {url}")
97
+
98
+ if not r.ok:
99
+ raise ServerError(f"Server response: {r.status_code} {r.reason}")
100
+
101
+ instruments = r.json()
102
+ if verbose:
103
+ logger.debug(pformat(instruments))
104
+
105
+ instruments = sorted(instruments)
106
+
107
+ results = []
108
+
109
+ for instrument in instruments:
110
+ url = f"{base_url}/{api_version}/instruments/{instrument}/"
111
+ if verbose:
112
+ logger.debug(f"URL: {url}")
113
+
114
+ try:
115
+ r = requests.get(url)
116
+ except requests.exceptions.ConnectionError as e:
117
+ raise ServerError(f"Connection error reaching {url}")
118
+
119
+ j = r.json()
120
+
121
+ if verbose:
122
+ logger.debug(pformat(j))
123
+
124
+ dates = j["dates"]
125
+ full_name = j["name"]
126
+ start_date = dates["start-date"]
127
+ end_date = dates["end-date"]
128
+
129
+ i = {
130
+ "id": instrument,
131
+ "start-date": start_date,
132
+ "end-date": end_date,
133
+ "name": j["name"],
134
+ }
135
+ results.append(i)
136
+
137
+ return results
138
+
139
+
140
+ def products(
141
+ instrument,
142
+ base_url: str = BASE_URL,
143
+ api_version: str = API_VERSION,
144
+ verbose: bool = False,
145
+ ):
146
+ """Handle retrieving the `/instruments/{instrument}/products` endpoint
147
+ results. Can raise ServerError if there is a problem with the web request.
148
+ """
149
+ url = f"{base_url}/{api_version}/instruments/{instrument}/products"
150
+ if verbose:
151
+ logger.debug(f"URL: {url}")
152
+
153
+ try:
154
+ r = requests.get(url)
155
+ except requests.exceptions.ConnectionError as e:
156
+ raise ServerError(f"Connection error reaching {url}")
157
+
158
+ if not r.ok:
159
+ j = r.json()
160
+ msg = j["message"]
161
+ raise ServerError(f"Server response: {r.status_code} {r.reason} ({msg})")
162
+
163
+ j = r.json()
164
+
165
+ if verbose:
166
+ logger.debug(pformat(j))
167
+
168
+ return j
169
+
170
+
171
+ def authenticate(
172
+ username: str = None,
173
+ base_url: str = BASE_URL,
174
+ api_version: str = API_VERSION,
175
+ verbose: bool = False,
176
+ ):
177
+ """Authenticate username within the session. This must be called before
178
+ `download_file`.
179
+ """
180
+ if username is None:
181
+ msg = f"username required, sign up at {SIGNUP_URL}"
182
+ raise UserNotFound(msg)
183
+ else:
184
+ url = f"{base_url}/{api_version}/authenticate?username={username}"
185
+ if verbose:
186
+ logger.debug(pformat(f"URL: {url}"))
187
+ try:
188
+ r = session.get(url)
189
+ except requests.exceptions.ConnectionError as e:
190
+ raise ServerError(f"Connection error reaching {url}")
191
+
192
+ if not r.ok:
193
+ if r.status_code == 404:
194
+ info = r.json()
195
+ if verbose:
196
+ logger.debug(pformat(info))
197
+ raise UserNotFound(info["message"])
198
+ else:
199
+ j = r.json()
200
+ msg = j["message"]
201
+ raise ServerError(
202
+ f"Server response: {r.status_code} {r.reason} ({msg})"
203
+ )
204
+
205
+
206
+ def download_file(file, output_dir):
207
+ """Download a single file to the given output directory. The `file` argument
208
+ is a dict with fields "url" and "filename". `output_dir` is simply the
209
+ directory to put the downloaded file. Can raise ServerError if there is a
210
+ problem with the web request.
211
+ """
212
+ url = file["url"]
213
+ try:
214
+ r = session.get(url, stream=True, cookies=session.cookies.get_dict())
215
+ except requests.exceptions.ConnectionError as e:
216
+ raise ServerError(f"Connection error reaching {url}")
217
+
218
+ if not r.ok:
219
+ raise ServerError(f"Server response: {r.status_code} {r.reason}")
220
+
221
+ path = Path(output_dir) / file["filename"]
222
+
223
+ with open(path, "wb") as handle:
224
+ for data in r.iter_content(chunk_size=CHUNK_SIZE):
225
+ handle.write(data)
226
+ return path
227
+
228
+
229
+ def files(
230
+ instrument: str,
231
+ product: str,
232
+ filters: list[dict[str, str]] | None = None,
233
+ base_url: str = BASE_URL,
234
+ api_version: str = API_VERSION,
235
+ verbose=False,
236
+ ):
237
+ """Handle retrieving the `/instruments/{instrument}/products/{product}`
238
+ endpoint results. Can raise ServerError if there is a problem with the web
239
+ request. Use `download_file` to download the file(s) returned with routine.
240
+ """
241
+ url = f"{base_url}/{api_version}/instruments/{instrument}/products/{product}"
242
+
243
+ if len(filters) > 0:
244
+ url += "?" + "&".join([f"{f}={filters[f]}" for f in filters])
245
+
246
+ if verbose:
247
+ logger.debug(f"URL: {url}")
248
+
249
+ try:
250
+ r = requests.get(url)
251
+ except requests.exceptions.ConnectionError as e:
252
+ raise ServerError(f"Connection error reaching {url}")
253
+
254
+ if not r.ok:
255
+ j = r.json()
256
+ msg = j["message"]
257
+ raise ServerError(f"Server response: {r.status_code} {r.reason} ({msg})")
258
+
259
+ j = r.json()
260
+ if verbose:
261
+ logger.debug(pformat(j))
262
+
263
+ return j
264
+
265
+
266
+ # Command line interface sub-command handlers
267
+
268
+
269
+ def _about(args):
270
+ """Handle printing the `/about` endpoint results for the command line
271
+ interface.
272
+ """
273
+ try:
274
+ about_response = about(args.base_url, verbose=args.verbose)
275
+ server_version = about_response["version"]
276
+ documentation_url = about_response["documentation"]
277
+ print(
278
+ f"server version: {server_version}, client version: {__version__}"
279
+ )
280
+ print(f"documentation: {documentation_url}")
281
+ except ServerError as e:
282
+ print(e)
283
+
284
+
285
+ def _instruments(args):
286
+ """Handle printing the `/instruments` endpoint results for the command line
287
+ interface.
288
+ """
289
+ try:
290
+ instruments_response = instruments(args.base_url, verbose=args.verbose)
291
+ except ServerError as e:
292
+ print(e)
293
+ return
294
+
295
+ print(f"{'ID':8s} {'Instrument name':44s} Dates available")
296
+ print(f"{'-' * 8} {'-' * 44} {'-' * 23}")
297
+
298
+ for i in instruments_response:
299
+ instrument = i["id"]
300
+ instrument_name = i["name"]
301
+ start_date = i["start-date"][:10]
302
+ end_date = i["end-date"][:10]
303
+
304
+ print(f"{instrument:8s} {instrument_name:44s} {start_date}...{end_date}")
305
+
306
+
307
+ def _products(args):
308
+ """Handle printing the `/instruments/{instrument}/products` endpoint
309
+ results for the command line interface.
310
+ """
311
+ try:
312
+ products_response = products(
313
+ args.instrument, base_url=args.base_url, verbose=args.verbose
314
+ )
315
+ except ServerError as e:
316
+ print(e)
317
+ return
318
+
319
+ print(f"{'ID':13s} {'Title':22s} {'Description'}")
320
+ print(f"{'-' * 13} {'-' * 22} {'-' * 55}")
321
+ for p in products_response["products"]:
322
+ title = p["title"]
323
+ product_id = p["id"]
324
+ description = textwrap.wrap(p["description"], width=55)
325
+ if len(description) == 0:
326
+ description = [""]
327
+ for description_line in description:
328
+ print(f"{product_id:13s} {title:22s} {description_line}")
329
+ product_id = ""
330
+ title = ""
331
+ name = ""
332
+
333
+
334
+ def _download_files(
335
+ base_url: str,
336
+ filelist: list,
337
+ output_dir: Path,
338
+ username: str,
339
+ verbose: bool = False,
340
+ quiet: bool = False,
341
+ ):
342
+ """Download the given files to an output directory. The `files` argument is
343
+ a list of dicts with fields "url" and "filename".
344
+ """
345
+ if not output_dir.is_dir():
346
+ if verbose:
347
+ print(f"creating output path {output_dir}")
348
+ os.makedirs(output_dir)
349
+
350
+ try:
351
+ authenticate(username, base_url=base_url, verbose=verbose)
352
+ except UserNotFound as e:
353
+ print(e)
354
+ sys.exit(1)
355
+ except ServerError as e:
356
+ print(e)
357
+ sys.exit(1)
358
+
359
+ if quiet:
360
+ iterable_files = filelist
361
+ message = print
362
+ else:
363
+ iterable_files = tqdm.tqdm(filelist)
364
+ message = tqdm.tqdm.write
365
+
366
+ n_failed = 0
367
+ for f in iterable_files:
368
+ try:
369
+ filepath = download_file(f, output_dir)
370
+ except ServerError as e:
371
+ message(f"{f['url']} failed")
372
+ n_failed += 1
373
+
374
+ if n_failed > 0:
375
+ print(f"{n_failed} failed downloads")
376
+
377
+
378
+ unit_list = list(zip(["B", "KB", "MB", "GB", "TB", "PB"], [0, 0, 1, 2, 2, 2]))
379
+
380
+
381
+ def sizeof_fmt(n_bytes: int) -> str:
382
+ """Human friendly file size"""
383
+ if n_bytes == 0:
384
+ return "0 B"
385
+ if n_bytes >= 1:
386
+ exponent = min(int(math.log(n_bytes, 1024)), len(unit_list) - 1)
387
+ quotient = float(n_bytes) / 1024**exponent
388
+ unit, num_decimals = unit_list[exponent]
389
+ format_string = "{:.%sf} {}" % (num_decimals)
390
+ return format_string.format(quotient, unit)
391
+
392
+
393
+ def _files(args):
394
+ """Handle printing the `/instruments/{instrument}/products/{product}`
395
+ endpoint results, optionally downloading the files.
396
+ """
397
+ filters = {}
398
+
399
+ if args.wave_region is not None:
400
+ filters["wave-region"] = args.wave_region
401
+
402
+ if args.obs_plan is not None:
403
+ filters["obs-plan"] = args.obs_plan
404
+
405
+ if args.start_date is not None:
406
+ filters["start-date"] = args.start_date
407
+
408
+ if args.end_date is not None:
409
+ filters["end-date"] = args.end_date
410
+
411
+ if args.carrington_rotation is not None:
412
+ filters["cr"] = args.carrington_rotation
413
+
414
+ if args.every is not None:
415
+ filters["every"] = args.every
416
+
417
+ try:
418
+ files_response = files(
419
+ args.instrument,
420
+ args.product,
421
+ filters,
422
+ base_url=args.base_url,
423
+ verbose=args.verbose,
424
+ )
425
+ except ServerError as e:
426
+ print(e)
427
+ return
428
+
429
+ if args.verbose:
430
+ instrument = files_response["instrument"]
431
+ product = files_response["product"]
432
+ start_date = files_response["start-date"]
433
+ end_date = files_response["end-date"]
434
+ filesize = sizeof_fmt(files_response["total_filesize"])
435
+ print(f"Instrument : {instrument}")
436
+ print(f"Product : {product}")
437
+ print(f"Start date : {start_date}")
438
+ print(f"End date : {end_date}")
439
+ print(f"Filesize : {filesize}")
440
+
441
+ filelist = files_response["files"]
442
+ if args.download:
443
+ _download_files(
444
+ args.base_url,
445
+ filelist,
446
+ Path(args.output_dir),
447
+ args.username,
448
+ verbose=args.verbose,
449
+ quiet=args.quiet,
450
+ )
451
+ else:
452
+ if len(filelist) > 0:
453
+ if args.verbose:
454
+ print()
455
+ max_filename_len = max([len(f["filename"]) for f in filelist])
456
+ print(
457
+ f"{'Date/time':20s} {'Instrument':10s} {'Product':13s} {'Filesize':10s} {'Filename'}"
458
+ )
459
+ print(
460
+ f"{'-' * 20} {'-' * 10} {'-' * 13} {'-' * 10} {'-' * max_filename_len}"
461
+ )
462
+ total_filesize = 0
463
+ for f in filelist:
464
+ total_filesize += f["filesize"]
465
+ print(
466
+ f"{f['date-obs']:20s} {f['instrument']:10s} {f['product']:13s} {sizeof_fmt(f['filesize']):>10s} {f['filename']}"
467
+ )
468
+ if len(filelist) > 1:
469
+ print(
470
+ f"{'-' * 20} {'-' * 10} {'-' * 13} {'-' * 10} {'-' * max_filename_len}"
471
+ )
472
+ n_files = f"{len(filelist)} files"
473
+ print(f"{n_files:45s} {sizeof_fmt(total_filesize):>10s} {''}")
474
+
475
+
476
+ def _print_help(args):
477
+ """Print the usage help for the command-line utility."""
478
+ args.parser.print_help()
479
+
480
+
481
+ def main():
482
+ """Entry point for MLSO API command-line utility."""
483
+ name = f"MLSO API command line interface (mlso-api-client {__version__})"
484
+ parser = argparse.ArgumentParser(description=name)
485
+
486
+ parser.add_argument("-v", "--version", action="version", version=name)
487
+
488
+ # show help if no sub-command given
489
+ parser.set_defaults(func=_print_help, parser=parser)
490
+
491
+ parser.add_argument(
492
+ "-u", "--base-url", help="base URL for MLSO API", default=BASE_URL
493
+ )
494
+ parser.add_argument("--verbose", help="output warnings", action="store_true")
495
+ parser.add_argument(
496
+ "-q", "--quiet", help="surpress informational messages", action="store_true"
497
+ )
498
+
499
+ subparsers = parser.add_subparsers(help="sub-command help")
500
+
501
+ instruments_parser = subparsers.add_parser("instruments", help="MLSO instruments")
502
+ instruments_parser.set_defaults(func=_instruments, parser=instruments_parser)
503
+
504
+ products_parser = subparsers.add_parser("products", help="MLSO instruments")
505
+ products_parser.add_argument("-i", "--instrument", help="instrument", default=None)
506
+ products_parser.set_defaults(func=_products, parser=products_parser)
507
+
508
+ files_parser = subparsers.add_parser("files", help="MLSO data files")
509
+ files_parser.add_argument("-i", "--instrument", help="instrument", default=None)
510
+ files_parser.add_argument("-p", "--product", help="product", default=None)
511
+ files_parser.add_argument(
512
+ "--wave-region", help="wave region, e.g., 1074, 1079, etc.", default=None
513
+ )
514
+ files_parser.add_argument(
515
+ "--obs-plan", help="observing plan: synoptic or waves", default=None
516
+ )
517
+ files_parser.add_argument("-s", "--start-date", help="start date", default=None)
518
+ files_parser.add_argument("-e", "--end-date", help="end date", default=None)
519
+ files_parser.add_argument(
520
+ "-c", "--carrington-rotation", help="Carrington Rotation number", default=None
521
+ )
522
+ files_parser.add_argument(
523
+ "--every", help="time to choose 1 file from", default=None
524
+ )
525
+ files_parser.add_argument(
526
+ "-d", "--download", help="download the displayed files", action="store_true"
527
+ )
528
+ files_parser.add_argument(
529
+ "-u", "--username", help="email already registered at HAO website", default=None
530
+ )
531
+ files_parser.add_argument(
532
+ "-o", "--output-dir", help="output directory for downloaded files", default="."
533
+ )
534
+ files_parser.set_defaults(func=_files, parser=files_parser)
535
+
536
+ # parse args and call appropriate sub-command
537
+ args = parser.parse_args()
538
+
539
+ if args.verbose:
540
+ _about(args)
541
+ print()
542
+
543
+ if parser.get_default("func"):
544
+ try:
545
+ args.func(args)
546
+ except KeyboardInterrupt:
547
+ print()
548
+ else:
549
+ parser.print_help()
550
+
551
+
552
+ if __name__ == "__main__":
553
+ main()
@@ -0,0 +1,165 @@
1
+ Metadata-Version: 2.4
2
+ Name: mlso-api-client
3
+ Version: 0.3.0
4
+ Summary: Python package providing a client to access MLSO data
5
+ Author-email: Michael Galloy <mgalloy@ucar.edu>
6
+ License-Expression: BSD-3-Clause
7
+ Project-URL: homepage, https://www2.hao.ucar.edu/mlso
8
+ Project-URL: repository, https://github.com/NCAR/mlso-api-client
9
+ Project-URL: documentation, https://mlso-api-client.readthedocs.io/en/latest/
10
+ Project-URL: issues, https://github.com/NCAR/mlso-api-client/issues
11
+ Project-URL: changelog, https://github.com/NCAR/mlso-api-client/blob/master/CHANGELOG.md
12
+ Keywords: webservice,Mauna Loa Solar Observatory,MLSO,solar
13
+ Classifier: Development Status :: 4 - Beta
14
+ Classifier: Intended Audience :: Developers
15
+ Classifier: Intended Audience :: Science/Research
16
+ Classifier: Natural Language :: English
17
+ Classifier: Programming Language :: Python
18
+ Requires-Python: >=3.7
19
+ Description-Content-Type: text/markdown
20
+ License-File: LICENSE
21
+ Requires-Dist: requests
22
+ Requires-Dist: tqdm
23
+ Provides-Extra: dev
24
+ Requires-Dist: pytest; extra == "dev"
25
+ Requires-Dist: pre-commit; extra == "dev"
26
+ Requires-Dist: tox; extra == "dev"
27
+ Requires-Dist: wheel; extra == "dev"
28
+ Requires-Dist: watchdog; extra == "dev"
29
+ Requires-Dist: Sphinx; extra == "dev"
30
+ Requires-Dist: twine; extra == "dev"
31
+ Requires-Dist: coverage; extra == "dev"
32
+ Requires-Dist: flake8; extra == "dev"
33
+ Requires-Dist: pytest-runner; extra == "dev"
34
+ Requires-Dist: black; extra == "dev"
35
+ Requires-Dist: sphinx_rtd_theme; extra == "dev"
36
+ Requires-Dist: myst-parser; extra == "dev"
37
+ Dynamic: license-file
38
+
39
+ # mlso-api-client
40
+
41
+ This package contains Python and IDL clients for accessing MLSO data via the
42
+ MLSO data web API.
43
+
44
+ ## Installation
45
+
46
+ ### Installing from PyPI
47
+
48
+ The easiest way to install the MLSO API client is via the released versions on
49
+ PyPI. This is the recommended method for most users.
50
+
51
+ ```console
52
+ pip install mlso-api-client
53
+ ```
54
+
55
+ If you want to upgrade an existing installation, do:
56
+
57
+ ```console
58
+ pip install -U mlso-api-client
59
+ ```
60
+
61
+
62
+ ### Installing from source
63
+
64
+ The source code can be found on the [repo's GitHub page]. Use git or download
65
+ a ZIP file with contents of the source.
66
+
67
+ [repo's GitHub page]: https://github.com/NCAR/mlso-api-client
68
+
69
+ Once you have the source code, install the Python portion of the package:
70
+
71
+ ```console
72
+ cd mlso-api-client
73
+ pip install .
74
+ ```
75
+
76
+ If you intend to make changes to the code, install the dev requirements and
77
+ allow changes to the code to automatically be used:
78
+
79
+ ```console
80
+ pip install -e .[dev]
81
+ ```
82
+
83
+ For IDL, simply put the `idl/` directory in your `IDL_PATH`.
84
+
85
+
86
+ ## Usage
87
+
88
+ ### Command-line interface
89
+
90
+ Installing the Python package, should install a command-line utility to query
91
+ and download MLSO data, the `mlsoapi` script.
92
+
93
+ ```console
94
+ $ mlsoapi --help
95
+ usage: mlsoapi [-h] [-v] [-u BASE_URL] [--verbose] [-q] {instruments,products,files} ...
96
+
97
+ MLSO API command line interface (mlso-api-client 1.0.0)
98
+
99
+ positional arguments:
100
+ {instruments,products,files}
101
+ sub-command help
102
+ instruments MLSO instruments
103
+ products MLSO instruments
104
+ files MLSO data files
105
+
106
+ options:
107
+ -h, --help show this help message and exit
108
+ -v, --version show program's version number and exit
109
+ -u BASE_URL, --base-url BASE_URL
110
+ base URL for MLSO API
111
+ --verbose output warnings
112
+ -q, --quiet surpress informational messages
113
+ ```
114
+
115
+ To determine the instruments with data available via the API, use the
116
+ "instruments" sub-command:
117
+
118
+ ```console
119
+ $ mlsoapi instruments
120
+ ID Instrument name Dates available
121
+ -------- -------------------------------------------- -----------------------
122
+ kcor COSMO K-Coronagraph (KCor) 2013-09-30...2025-03-24
123
+ ucomp Upgraded Coronal Multi-Polarimeter (UCoMP) 2021-07-15...2025-03-24
124
+ ```
125
+
126
+ New data for existing and new instruments will be added to the API as possible.
127
+ Submit requests via the [Issues].
128
+
129
+ [Issues]: https://github.com/NCAR/mlso-api-client/issues
130
+
131
+ Each instrument has various products available:
132
+
133
+ ```console
134
+ $ mlsoapi products --instrument ucomp
135
+ ID Title Description
136
+ ------------- ---------------------- -------------------------------------------------------
137
+ l1 Level 1 IQUV and backgrounds for various wavelengths
138
+ intensity Level 1 intensity intensity-only level 1
139
+ mean Level 1 mean mean of level 1 files
140
+ median Level 1 median median of level 1 files
141
+ sigma Level 1 sigma standard deviation of level 1 files
142
+ l2 Level 2 level 2 products
143
+ l2average Level 2 average mean, median, standard deviation of level 2 files
144
+ density Density density
145
+ dynamics Dynamics level 2 dynamics products
146
+ polarization Polarization level 2 polarization products
147
+ all All all products
148
+ ```
149
+
150
+ ### Python API
151
+
152
+ TODO: example of using the Python API
153
+
154
+
155
+ ### IDL API
156
+
157
+ TODO: example using the IDL routines
158
+
159
+
160
+ ### API endpoints
161
+
162
+ To use the webservice API directly from any language, see the [API Endpoints] wiki
163
+ page.
164
+
165
+ [API Endpoints]: https://github.com/NCAR/mlso-api-client/wiki/API-endpoints
@@ -0,0 +1,11 @@
1
+ LICENSE
2
+ README.md
3
+ pyproject.toml
4
+ src/mlso/api/__init__.py
5
+ src/mlso/api/client.py
6
+ src/mlso_api_client.egg-info/PKG-INFO
7
+ src/mlso_api_client.egg-info/SOURCES.txt
8
+ src/mlso_api_client.egg-info/dependency_links.txt
9
+ src/mlso_api_client.egg-info/entry_points.txt
10
+ src/mlso_api_client.egg-info/requires.txt
11
+ src/mlso_api_client.egg-info/top_level.txt
@@ -0,0 +1,2 @@
1
+ [console_scripts]
2
+ mlsoapi = mlso.api:client.main
@@ -0,0 +1,17 @@
1
+ requests
2
+ tqdm
3
+
4
+ [dev]
5
+ pytest
6
+ pre-commit
7
+ tox
8
+ wheel
9
+ watchdog
10
+ Sphinx
11
+ twine
12
+ coverage
13
+ flake8
14
+ pytest-runner
15
+ black
16
+ sphinx_rtd_theme
17
+ myst-parser