hapiclient 0.2.6__tar.gz → 0.2.8__tar.gz

{hapiclient-0.2.6 → hapiclient-0.2.8}/CHANGES.txt

@@ -10,7 +10,7 @@ v0.0.6:
 		2018-10-04 -- Improve dateticks.py
 		2018-10-04 -- Add hapi_demo.ipynb
 		2018-10-04 -- Other documentation improvements
-v0.0.7: 
+v0.0.7:
 		2018-10-06 -- Improve documentation
 		2018-10-06 -- Add reader tests
 		2018-10-23 -- Improve dateticks and tests
@@ -103,4 +103,20 @@ v0.2.6b1:
 	2023-02-04 -- 2ef7cff Unicode fix; Log string fix
 	2024-05-11 -- fba4ad3 Testing Updates
 v0.2.6:
-
+	2023-05-24 -- Use token for PyPi in Makefile
+v0.2.7b1:
+	2025-03-01 -- Unrecognized ISO 8601 time format: '00:00:00.Z' https://github.com/hapi-server/client-python/issues/76
+	2025-03-01 -- 'infer_datetime_format' is deprecated ... https://github.com/hapi-server/client-python/issues/78
+	2026-04-08 -- Improve NaN handling https://github.com/hapi-server/client-python/issues/88
+	2026-05-02 -- Logging and testing https://github.com/hapi-server/client-python/issues/89
+v0.2.7:
+	2026-05-02 -- Support for Python 2.7 dropped. This is last release with support for 3.5 through 3.8.
+v0.2.8b0:
+	2026-05-02 -- Documentation
+v0.2.8b1:
+	2026-05-02 -- Versioning updates
+	2026-05-19 -- Better response error messages
+v0.2.8b2:
+	2026-05-20 -- Refactor
+v0.2.8:
+	2026-06-04 -- Compat. fixes for hapiplot package.

{hapiclient-0.2.6 → hapiclient-0.2.8}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.1
+Metadata-Version: 2.4
 Name: hapiclient
-Version: 0.2.6
+Version: 0.2.8
 Summary: Interface to Heliophysics data server API
 Home-page: http://pypi.python.org/pypi/hapiclient/
 Author: Bob Weigel
@@ -8,13 +8,29 @@ Author-email: rweigel@gmu.edu
 License: LICENSE.txt
 Description-Content-Type: text/markdown
 License-File: LICENSE.txt
+Requires-Dist: urllib3
+Requires-Dist: joblib
+Requires-Dist: isodate>=0.6.0
+Requires-Dist: pandas>=0.23
+Requires-Dist: numpy>=1.14.3
+Dynamic: author
+Dynamic: author-email
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: license
+Dynamic: license-file
+Dynamic: requires-dist
+Dynamic: summary
 
 [![DOI](https://zenodo.org/badge/93170857.svg)](https://zenodo.org/badge/latestdoi/93170857)
 [![Build Status](https://app.travis-ci.com/hapi-server/client-python.svg?branch=master)](https://app.travis-ci.com/hapi-server/client-python)
 
 **HAPI Client for Python**
 
-Basic usage examples for various HAPI servers are given in [hapi_demo.py](https://github.com/hapi-server/client-python/blob/master/hapi_demo.py) and the Examples section of a Jupyter Notebook hosted on Google Colab: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/hapi-server/client-python-notebooks/blob/master/hapi_demo.ipynb#examples).
+Basic usage examples for various HAPI servers are given in [hapi_demo.py](https://github.com/hapi-server/client-python/blob/master/hapi_demo.py). See also
+* Examples section of a Jupyter Notebook hosted on Google Colab: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/hapi-server/client-python-notebooks/blob/master/hapi_demo.ipynb#examples).
+* [PyHC Summer School tutorials](https://github.com/hapi-server/tutorial-python)
 
 # Installation
 
@@ -50,6 +66,7 @@ start      = '2003-09-01T00:00:00'
 stop       = '2003-12-01T00:00:00'
 parameters = 'DST1800'
 opts       = {'logging': True}
+# See misc/hapi_logging_demo.py for example of using conventional Python logging.
 
 # Get data
 data, meta = hapi(server, dataset, parameters, start, stop, **opts)
@@ -92,7 +109,41 @@ returns the [Numpy N-D array](https://docs.scipy.org/doc/numpy-1.15.1/user/quick
 
 The HAPI client data model is intentionally basic. There is an ongoing discussion of a data model for Heliophysics data among the [PyHC community](https://heliopython.org/). When this data model is complete, a function that converts `data` and `meta` to that data model will be included in the `hapiclient` package.
 
-# Development
+# Development and Testing
+
+## Testing Python versions available with tox-env
+
+As of 2026-04-14, Python 3.8+ is available with tox-env.
+
+```bash
+git clone https://github.com/hapi-server/client-python
+cd client-python; python -m pip install -e .
+
+# Run a specific test on current Python version (default is log_level=INFO
+# for hapiclient logger and test logger when executed this way).
+python test/test_hapi_data_requests.py
+
+# Run a specific test on a specific Python version with all logging
+tox -e py311 -- test/test_hapi_data_requests.py
+# Set log level for test logger
+tox -e py311 -- test/test_hapi_data_requests.py --log-level=INFO
+
+# Run a specific test on all Python versions set by tox.ini `envlist`
+tox -- test/test_hapi_data_requests.py
+
+# Run all tests in ./test
+tox
+
+# Run long-running tests
+tox -e long-test
+
+# Run long-running tests on a specific Python version
+tox -e long-test --override testenv:long-test.basepython=python3.11
+```
+
+## Testing Python versions available with Anaconda
+
+As of 2026-04-14, Python 3.5+ is available with Anaconda.
 
 ```bash
 git clone https://github.com/hapi-server/client-python
@@ -117,12 +168,13 @@ make repository-test
 
 To run an individual unit test in a Python session, use, e.g.,
 
-```python
-from hapiclient.test.test_hapi import test_reader_short
-test_reader_short()
+```bash
+cd test; python test_hapitime_reformat.py
 ```
 
 # Contact
 
 Submit bug reports and feature requests on the [repository issue
 tracker](https://github.com/hapi-server/client-python/issues>).
+
+Before submitting a pull request, please post an issue with the proposed changes for discussion prior to developing the pull request.

hapiclient-0.2.6/hapiclient.egg-info/PKG-INFO → hapiclient-0.2.8/README.md

@@ -1,20 +1,11 @@
-Metadata-Version: 2.1
-Name: hapiclient
-Version: 0.2.6
-Summary: Interface to Heliophysics data server API
-Home-page: http://pypi.python.org/pypi/hapiclient/
-Author: Bob Weigel
-Author-email: rweigel@gmu.edu
-License: LICENSE.txt
-Description-Content-Type: text/markdown
-License-File: LICENSE.txt
-
 [![DOI](https://zenodo.org/badge/93170857.svg)](https://zenodo.org/badge/latestdoi/93170857)
 [![Build Status](https://app.travis-ci.com/hapi-server/client-python.svg?branch=master)](https://app.travis-ci.com/hapi-server/client-python)
 
 **HAPI Client for Python**
 
-Basic usage examples for various HAPI servers are given in [hapi_demo.py](https://github.com/hapi-server/client-python/blob/master/hapi_demo.py) and the Examples section of a Jupyter Notebook hosted on Google Colab: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/hapi-server/client-python-notebooks/blob/master/hapi_demo.ipynb#examples).
+Basic usage examples for various HAPI servers are given in [hapi_demo.py](https://github.com/hapi-server/client-python/blob/master/hapi_demo.py). See also
+* Examples section of a Jupyter Notebook hosted on Google Colab: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/hapi-server/client-python-notebooks/blob/master/hapi_demo.ipynb#examples).
+* [PyHC Summer School tutorials](https://github.com/hapi-server/tutorial-python)
 
 # Installation
 
@@ -50,6 +41,7 @@ start      = '2003-09-01T00:00:00'
 stop       = '2003-12-01T00:00:00'
 parameters = 'DST1800'
 opts       = {'logging': True}
+# See misc/hapi_logging_demo.py for example of using conventional Python logging.
 
 # Get data
 data, meta = hapi(server, dataset, parameters, start, stop, **opts)
@@ -92,7 +84,41 @@ returns the [Numpy N-D array](https://docs.scipy.org/doc/numpy-1.15.1/user/quick
 
 The HAPI client data model is intentionally basic. There is an ongoing discussion of a data model for Heliophysics data among the [PyHC community](https://heliopython.org/). When this data model is complete, a function that converts `data` and `meta` to that data model will be included in the `hapiclient` package.
 
-# Development
+# Development and Testing
+
+## Testing Python versions available with tox-env
+
+As of 2026-04-14, Python 3.8+ is available with tox-env.
+
+```bash
+git clone https://github.com/hapi-server/client-python
+cd client-python; python -m pip install -e .
+
+# Run a specific test on current Python version (default is log_level=INFO
+# for hapiclient logger and test logger when executed this way).
+python test/test_hapi_data_requests.py
+
+# Run a specific test on a specific Python version with all logging
+tox -e py311 -- test/test_hapi_data_requests.py
+# Set log level for test logger
+tox -e py311 -- test/test_hapi_data_requests.py --log-level=INFO
+
+# Run a specific test on all Python versions set by tox.ini `envlist`
+tox -- test/test_hapi_data_requests.py
+
+# Run all tests in ./test
+tox
+
+# Run long-running tests
+tox -e long-test
+
+# Run long-running tests on a specific Python version
+tox -e long-test --override testenv:long-test.basepython=python3.11
+```
+
+## Testing Python versions available with Anaconda
+
+As of 2026-04-14, Python 3.5+ is available with Anaconda.
 
 ```bash
 git clone https://github.com/hapi-server/client-python
@@ -117,12 +143,13 @@ make repository-test
 
 To run an individual unit test in a Python session, use, e.g.,
 
-```python
-from hapiclient.test.test_hapi import test_reader_short
-test_reader_short()
+```bash
+cd test; python test_hapitime_reformat.py
 ```
 
 # Contact
 
 Submit bug reports and feature requests on the [repository issue
 tracker](https://github.com/hapi-server/client-python/issues>).
+
+Before submitting a pull request, please post an issue with the proposed changes for discussion prior to developing the pull request.

{hapiclient-0.2.6 → hapiclient-0.2.8}/hapiclient/__init__.py

@@ -1,6 +1,9 @@
 # Allow "from hapiclient import hapi"
 from hapiclient.hapi import hapi
 
+# Allow "from hapiclient import request2path"
+from hapiclient.hapi import request2path
+
 # Allow "from hapiclient import hapitime2datetime"
 from hapiclient.hapitime import hapitime2datetime
 
@@ -10,9 +13,15 @@ from hapiclient.hapitime import datetime2hapitime
 # Allow "from hapiclient import HAPIError"
 from hapiclient.util import HAPIError
 
-__version__ = '0.2.6'
+__version__ = '0.2.8'
 
 import sys
+import platform
+
+if sys.version_info < (3, 10) and platform.system() == "Darwin":
+    import warnings
+    warnings.filterwarnings("ignore", message=".*urllib3.*OpenSSL.*")
+
 if sys.version_info[0] < 3:
     # Python 2.7
     reload(sys)

hapiclient-0.2.8/hapiclient/cache.py

@@ -0,0 +1,255 @@
+def server2dirname(server):
+  """Convert a server URL to a directory name."""
+
+  import re
+
+  urld = re.sub(r"https*://", "", server)
+  urld = re.sub(r'/', '_', urld)
+
+  return urld
+
+
+def cachedir(*args):
+  """HAPI cache directory.
+
+  cachedir() returns os.path.join(tempfile.gettempdir(), 'hapi-data')
+
+  cachedir(basedir, server) returns os.path.join(basedir, server2dirname(server))
+  """
+
+  import os
+  import tempfile
+
+  if len(args) != 0 and len(args) != 2:
+    raise ValueError('cachedir() takes either 0 or 2 arguments.')
+
+  if len(args) == 0:
+    # cachedir()
+    return os.path.join(tempfile.gettempdir(), 'hapi-data')
+
+  if len(args) == 2:
+    # cachedir(base_dir, server)
+    return os.path.join(args[0], server2dirname(args[1]))
+
+
+def request2path(*args):
+  # request2path(server, dataset, parameters, start, stop)
+  # request2path(server, dataset, parameters, start, stop, cachedir)
+  import os
+  import re
+  import platform
+
+  if len(args) == 5:
+    # Use default if cachedir not given.
+    cachedirectory = cachedir()
+  else:
+    cachedirectory = args[5]
+
+  args = list(args)
+
+  # Replace forbidden characters in directory and filename
+  # Replacements assume that there will be no name collisions,
+  # e.g., one parameter named abc-< and another abc-@lt@.
+  # This also introduces an incompatibility between caches on Windows
+  # Unix.
+  if platform.system() == 'Windows':
+    # List and code from responses in
+    # https://stackoverflow.com/q/1976007
+    reps = (
+              ('<', '@lt@'),
+              ('>', '@gt@'),
+              (':', '@colon@'),
+              ('"', '@doublequote@'),
+              ('/', '@forwardslash@'),
+              ('/', '@backslash@'),
+              ('\\|', '@pipe@'),
+              ('\\?', '@questionmark@'),
+              ('\\*', '@asterisk@')
+          )
+
+    for element in reps:
+      args[1] = re.sub(element[0], element[1], args[1])
+      args[2] = re.sub(element[0], element[1], args[2])
+
+  else:
+    args[1] = re.sub('/','@forwardslash@',args[1])
+    args[2] = re.sub('/','@forwardslash@',args[2])
+
+  # To shorten filenames.
+  args[3] = re.sub(r'-|:|\.|Z', '', args[3])
+  args[4] = re.sub(r'-|:|\.|Z', '', args[4])
+
+  # URL subdirectory
+  urldirectory = server2dirname(args[0])
+  fname = '%s_%s_%s_%s' % (args[1], args[2], args[3], args[4])
+
+  return os.path.join(cachedirectory, urldirectory, fname)
+
+
+def meta_cache_paths(SERVER, DATASET, cachedir):
+  """Return dict with metadata cache directory and file names."""
+
+  fname_root = request2path(SERVER, DATASET, '', '', '', cachedir)
+
+  return {
+    'json': fname_root + '.json',
+    'pkl': fname_root + '.pkl'
+  }
+
+
+def meta_cache_read(SERVER, DATASET, opts):
+  """Read metadata from PKL cache. Returns meta dict or None."""
+
+  import os
+  import pickle
+
+  from hapiclient.log import log
+
+  if not opts["usecache"]:
+    log('Not checking metadata cache because usecache is False.')
+    return None
+
+  fnamepkl = meta_cache_paths(SERVER, DATASET, opts['cachedir'])['pkl']
+  if os.path.isfile(fnamepkl):
+    log('Reading %s' % os.path.basename(fnamepkl))
+    with open(fnamepkl, 'rb') as f:
+      return pickle.load(f)
+
+  if opts["usecache"]:
+    log('No metadata cache file found: %s' % os.path.basename(fnamepkl))
+
+  return None
+
+
+def meta_cache_write(meta, SERVER, DATASET, opts):
+  """Write metadata to JSON and PKL cache files."""
+
+  import os
+  import json
+  import pickle
+
+  from hapiclient.log import log
+
+  if not opts["cache"]:
+    return
+
+  paths = meta_cache_paths(SERVER, DATASET, opts['cachedir'])
+  fnamejson, fnamepkl = paths['json'], paths['pkl']
+
+  server_dir = cachedir(opts["cachedir"], SERVER)
+  os.makedirs(server_dir, exist_ok=True)
+
+  log('Writing %s ' % os.path.basename(fnamejson))
+  with open(fnamejson, 'w') as f:
+    json.dump(meta, f, indent=4)
+
+  log('Writing %s ' % os.path.basename(fnamepkl))
+  with open(fnamepkl, 'wb') as f:
+    # protocol=2 used for Python 2.7 compatibility.
+    pickle.dump(meta, f, protocol=2)
+
+
+def data_cache_paths(SERVER, DATASET, PARAMETERS, START, STOP, cachedir):
+  """Return dict with data cache file names."""
+
+  fname_root = request2path(SERVER, DATASET, PARAMETERS, START, STOP, cachedir)
+
+  return {
+    'csv': fname_root + '.csv',
+    'bin': fname_root + '.bin',
+    'npy': fname_root + '.npy',
+    'pkl': fname_root + '.pkl'
+  }
+
+
+def data_cache_read_metax(SERVER, DATASET, PARAMETERS, START, STOP, opts):
+  """Read extended request metadata from PKL cache. Returns meta dict or None."""
+
+  import os
+  import pickle
+
+  from hapiclient.log import log
+
+  if not opts["usecache"]:
+    log('Not checking data cache because usecache is False.')
+    return None
+
+  fnamepklx = data_cache_paths(SERVER, DATASET, PARAMETERS, START, STOP, opts['cachedir'])['pkl']
+  if os.path.isfile(fnamepklx):
+    log('Reading subsetted metadata cache %s' % os.path.basename(fnamepklx))
+    with open(fnamepklx, 'rb') as f:
+      return pickle.load(f)
+  if opts["usecache"]:
+    log('No subsetted metadata cache file found: %s' % os.path.basename(fnamepklx))
+
+  return None
+
+
+def data_cache_read_npy(SERVER, DATASET, PARAMETERS, START, STOP, opts):
+  """Read cached numpy data array. Returns None if not cached."""
+
+  import os
+  import numpy as np
+
+  from hapiclient.log import log
+
+  if not opts["usecache"]:
+    return None
+
+  fnamenpy = data_cache_paths(SERVER, DATASET, PARAMETERS, START, STOP, opts['cachedir'])['npy']
+
+  if not os.path.isfile(fnamenpy):
+    return None
+
+  log('Reading %s ' % os.path.basename(fnamenpy))
+  with open(fnamenpy, 'rb') as f:
+    data = np.load(f)
+
+  return data
+
+
+def data_cache_write(data_result, meta, SERVER, DATASET, PARAMETERS, START, STOP, opts):
+  """Write data array and extended metadata to cache files.
+
+  Also updates meta with file-related x_ fields before writing.
+  """
+
+  import os
+  import pickle
+  import warnings
+  import numpy as np
+
+  from hapiclient.log import log
+
+  data_paths = data_cache_paths(SERVER, DATASET, PARAMETERS, START, STOP, opts['cachedir'])
+  fnamecsv, fnamebin, fnamenpy, fnamepklx = data_paths['csv'], data_paths['bin'], data_paths['npy'], data_paths['pkl']
+
+  meta_paths = meta_cache_paths(SERVER, DATASET, opts['cachedir'])
+  fnamejson, fnamepkl = meta_paths['json'], meta_paths['pkl']
+
+  meta.update({"x_metaFileParsed": fnamepkl})
+  meta.update({"x_dataFileParsed": fnamenpy})
+  meta.update({"x_metaFile": fnamejson})
+  meta.update({"x_dataFile": fnamebin if opts['format'] == 'binary' else fnamecsv})
+
+  if not opts["cache"]:
+    # Need to return after meta is updated.
+    return
+
+  server_dir = cachedir(opts["cachedir"], SERVER)
+  os.makedirs(server_dir, exist_ok=True)
+
+  log('Writing %s' % os.path.basename(fnamepklx))
+  with open(fnamepklx, 'wb') as f:
+    pickle.dump(meta, f, protocol=2)
+
+  log('Writing %s' % os.path.basename(fnamenpy))
+  with warnings.catch_warnings():
+    # Ignore warning that occurs when saving Unicode data.
+    kwargs = {
+        'message': r"Stored array in format 3\.0.*",
+        'category': UserWarning,
+        'module': r"numpy\.lib\.format"
+    }
+    warnings.filterwarnings("ignore", **kwargs)
+    np.save(fnamenpy, data_result)

hapiclient-0.2.8/hapiclient/capabilities.py

@@ -0,0 +1,42 @@
+
+def capabilities(SERVER):
+  """Return the capabilities of a HAPI server.
+
+  Args:
+    SERVER (str): The base URL of the HAPI server.
+
+  Returns:
+    dict: A dictionary containing the capabilities of the server.
+  """
+  from hapiclient.util import urlopen
+
+  caps = urlopen(SERVER + '/capabilities', parse_json=True)
+
+  return caps
+
+
+def get_format(SERVER, format):
+  """Return the transport format to use, accounting for server capabilities.
+
+  If the requested format is not supported by the server, falls back to 'csv'
+  with a warning.
+  """
+
+  from hapiclient.util import error, warning
+
+  cformats = ['csv', 'binary']  # client formats
+  if format not in cformats:
+    msg = 'This client does not handle transport format "%s". Available options: %s'
+    error(msg % (format, ', '.join(cformats)))
+
+  if format != 'csv':
+    caps = capabilities(SERVER)
+    sformats = caps["outputFormats"]  # Server formats
+    if format not in sformats:
+      msg = 'Requested transport format "%s" not avaiable from %s. Will use "csv". Available options: %s'
+      warning(msg % (format, SERVER, ', '.join(sformats)))
+      format = 'csv'
+    if 'binary' not in sformats:
+      format = 'csv'
+
+  return format

hapiclient-0.2.8/hapiclient/catalog.py

@@ -0,0 +1,10 @@
+from hapiclient.log import log
+from hapiclient.util import urlopen
+
+
+def catalog(SERVER):
+  # TODO: Cache
+  url = SERVER + '/catalog'
+  meta = urlopen(url, parse_json=True)
+
+  return meta