hapiclient 0.2.5__tar.gz → 0.2.7__tar.gz

{hapiclient-0.2.5 → hapiclient-0.2.7}/CHANGES.txt

@@ -10,7 +10,7 @@ v0.0.6:
 		2018-10-04 -- Improve dateticks.py
 		2018-10-04 -- Add hapi_demo.ipynb
 		2018-10-04 -- Other documentation improvements
-v0.0.7: 
+v0.0.7:
 		2018-10-06 -- Improve documentation
 		2018-10-06 -- Add reader tests
 		2018-10-23 -- Improve dateticks and tests
@@ -94,3 +94,20 @@ v0.2.4:
 v0.2.5:
 	2022-06-30 -- 10df388 Update docstring for hapi()
 	2022-07-12 -- 460c30b Final Unicode/Windows fixes
+v0.2.6b:
+	2022-08-08 -- 508fb20-3f1168c Add datetime2hapitime
+	2022-08-08 -- 8bfef2f Allow start and stop to be None; https://github.com/hapi-server/client-python/issues/10
+	2022-09-21 -- bc70f3c CSV read failure; https://github.com/hapi-server/client-python/issues/62
+	2022-09-22 -- 373fdca Catch other CSV read failures
+v0.2.6b1:
+	2023-02-04 -- 2ef7cff Unicode fix; Log string fix
+	2024-05-11 -- fba4ad3 Testing Updates
+v0.2.6:
+	2023-05-24 -- Use token for PyPi in Makefile
+v0.2.7b1:
+	2025-03-01 -- Unrecognized ISO 8601 time format: '00:00:00.Z' https://github.com/hapi-server/client-python/issues/76
+	2025-03-01 -- 'infer_datetime_format' is deprecated ... https://github.com/hapi-server/client-python/issues/78
+	2026-04-08 -- Improve NaN handling https://github.com/hapi-server/client-python/issues/88
+	2026-05-02 -- Logging and testing https://github.com/hapi-server/client-python/issues/89
+v0.2.7:
+	2026--05-02-- Support for Python 2.7 dropped. This is last release with support for 3.5 through 3.8.

{hapiclient-0.2.5 → hapiclient-0.2.7}/PKG-INFO

@@ -1,14 +1,27 @@
-Metadata-Version: 2.1
+Metadata-Version: 2.4
 Name: hapiclient
-Version: 0.2.5
+Version: 0.2.7
 Summary: Interface to Heliophysics data server API
 Home-page: http://pypi.python.org/pypi/hapiclient/
 Author: Bob Weigel
 Author-email: rweigel@gmu.edu
 License: LICENSE.txt
-Platform: UNKNOWN
 Description-Content-Type: text/markdown
 License-File: LICENSE.txt
+Requires-Dist: urllib3
+Requires-Dist: joblib
+Requires-Dist: isodate>=0.6.0
+Requires-Dist: pandas>=0.23
+Requires-Dist: numpy>=1.14.3
+Dynamic: author
+Dynamic: author-email
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: license
+Dynamic: license-file
+Dynamic: requires-dist
+Dynamic: summary
 
 [![DOI](https://zenodo.org/badge/93170857.svg)](https://zenodo.org/badge/latestdoi/93170857)
 [![Build Status](https://app.travis-ci.com/hapi-server/client-python.svg?branch=master)](https://app.travis-ci.com/hapi-server/client-python)
@@ -30,9 +43,9 @@ The optional [hapiplot package](https://github.com/hapi-server/plot-python) prov
 To install `hapiplot`, use
 
 ```bash
-pip install hapiplot --upgrade
+python -m pip install hapiplot --upgrade
 # or
-pip install 'git+https://github.com/hapi-server/plot-python' --upgrade
+python -m pip install 'git+https://github.com/hapi-server/plot-python' --upgrade
 ```
 
 See the [Appendix](#appendix) for a fail-safe installation method.
@@ -51,6 +64,7 @@ start      = '2003-09-01T00:00:00'
 stop       = '2003-12-01T00:00:00'
 parameters = 'DST1800'
 opts       = {'logging': True}
+# See misc/hapi_logging_demo.py for example of using conventional Python logging.
 
 # Get data
 data, meta = hapi(server, dataset, parameters, start, stop, **opts)
@@ -93,11 +107,45 @@ returns the [Numpy N-D array](https://docs.scipy.org/doc/numpy-1.15.1/user/quick
 
 The HAPI client data model is intentionally basic. There is an ongoing discussion of a data model for Heliophysics data among the [PyHC community](https://heliopython.org/). When this data model is complete, a function that converts `data` and `meta` to that data model will be included in the `hapiclient` package.
 
-# Development
+# Development and Testing
+
+## Testing Python versions available with tox-env
+
+As of 2026-04-14, Python 3.8+ is available with tox-env.
 
 ```bash
 git clone https://github.com/hapi-server/client-python
-cd client-python; pip install -e .
+cd client-python; python -m pip install -e .
+
+# Run a specific test on current Python version (default is log_level=INFO
+# for hapiclient logger and test logger when executed this way).
+python test/test_hapi_data_requests.py
+
+# Run a specific test on a specific Python version with all logging
+tox -e py311 -- test/test_hapi_data_requests.py
+# Set log level for test logger
+tox -e py311 -- test/test_hapi_data_requests.py --log-level=INFO
+
+# Run a specific test on all Python versions set by tox.ini `envlist`
+tox -- test/test_hapi_data_requests.py
+
+# Run all tests in ./test
+tox
+
+# Run long-running tests
+tox -e long-test
+
+# Run long-running tests on a specific Python version
+tox -e long-test --override testenv:long-test.basepython=python3.11
+```
+
+## Testing Python versions available with Anaconda
+
+As of 2026-04-14, Python 3.5+ is available with Anaconda.
+
+```bash
+git clone https://github.com/hapi-server/client-python
+cd client-python; python -m pip install -e .
 ```
 
 or, create an isolated Anaconda installation (downloads and installs latest Miniconda3) using
@@ -108,7 +156,7 @@ make install PYTHON=python3.6
 ```
 
 The command `pip install -e .` creates symlinks so that the local package is
-used instead of an installed package. You may need to execute `pip uninstall hapiclient` to ensure the local package is used. To check the version installed, use `pip list | grep hapiclient`.
+used instead of an installed package. You may need to execute `python -m pip uninstall hapiclient` to ensure the local package is used. To check the version installed, use `python -m pip list | grep hapiclient`.
 
 To run tests before a commit, execute
 
@@ -118,9 +166,8 @@ make repository-test
 
 To run an individual unit test in a Python session, use, e.g.,
 
-```python
-from hapiclient.test.test_hapi import test_reader_short
-test_reader_short()
+```bash
+cd test; python test_hapitime_reformat.py
 ```
 
 # Contact
@@ -128,27 +175,4 @@ test_reader_short()
 Submit bug reports and feature requests on the [repository issue
 tracker](https://github.com/hapi-server/client-python/issues>).
 
-# Appendix
-
-Fail-safe installation
-
-Python command line:
-
-```python
-import os
-print(os.popen("pip install hapiclient").read())
-```
-
-The above executes and displays the output of the operating system
-command `pip install hapiclient` using the shell environment
-associated with that installation of Python.
-
-This method addresses a problem that is sometimes encountered when
-attempting to use `pip` packages in Anaconda. To use a `pip` package
-in Anaconda, one must use the version of `pip` installed with Anaconda
-(it is usually under a subdirectory with the name `anaconda/`) as
-opposed to the one installed with the operating system. To see the
-location of ``pip`` used in a given Python session, enter
-`print(os.popen("which pip").read())`.
-
-
+Before submitting a pull request, please post an issue with the proposed changes for discussion prior to developing the pull request.

hapiclient-0.2.5/hapiclient.egg-info/PKG-INFO → hapiclient-0.2.7/README.md

@@ -1,15 +1,3 @@
-Metadata-Version: 2.1
-Name: hapiclient
-Version: 0.2.5
-Summary: Interface to Heliophysics data server API
-Home-page: http://pypi.python.org/pypi/hapiclient/
-Author: Bob Weigel
-Author-email: rweigel@gmu.edu
-License: LICENSE.txt
-Platform: UNKNOWN
-Description-Content-Type: text/markdown
-License-File: LICENSE.txt
-
 [![DOI](https://zenodo.org/badge/93170857.svg)](https://zenodo.org/badge/latestdoi/93170857)
 [![Build Status](https://app.travis-ci.com/hapi-server/client-python.svg?branch=master)](https://app.travis-ci.com/hapi-server/client-python)
 
@@ -30,9 +18,9 @@ The optional [hapiplot package](https://github.com/hapi-server/plot-python) prov
 To install `hapiplot`, use
 
 ```bash
-pip install hapiplot --upgrade
+python -m pip install hapiplot --upgrade
 # or
-pip install 'git+https://github.com/hapi-server/plot-python' --upgrade
+python -m pip install 'git+https://github.com/hapi-server/plot-python' --upgrade
 ```
 
 See the [Appendix](#appendix) for a fail-safe installation method.
@@ -51,6 +39,7 @@ start      = '2003-09-01T00:00:00'
 stop       = '2003-12-01T00:00:00'
 parameters = 'DST1800'
 opts       = {'logging': True}
+# See misc/hapi_logging_demo.py for example of using conventional Python logging.
 
 # Get data
 data, meta = hapi(server, dataset, parameters, start, stop, **opts)
@@ -93,11 +82,45 @@ returns the [Numpy N-D array](https://docs.scipy.org/doc/numpy-1.15.1/user/quick
 
 The HAPI client data model is intentionally basic. There is an ongoing discussion of a data model for Heliophysics data among the [PyHC community](https://heliopython.org/). When this data model is complete, a function that converts `data` and `meta` to that data model will be included in the `hapiclient` package.
 
-# Development
+# Development and Testing
+
+## Testing Python versions available with tox-env
+
+As of 2026-04-14, Python 3.8+ is available with tox-env.
+
+```bash
+git clone https://github.com/hapi-server/client-python
+cd client-python; python -m pip install -e .
+
+# Run a specific test on current Python version (default is log_level=INFO
+# for hapiclient logger and test logger when executed this way).
+python test/test_hapi_data_requests.py
+
+# Run a specific test on a specific Python version with all logging
+tox -e py311 -- test/test_hapi_data_requests.py
+# Set log level for test logger
+tox -e py311 -- test/test_hapi_data_requests.py --log-level=INFO
+
+# Run a specific test on all Python versions set by tox.ini `envlist`
+tox -- test/test_hapi_data_requests.py
+
+# Run all tests in ./test
+tox
+
+# Run long-running tests
+tox -e long-test
+
+# Run long-running tests on a specific Python version
+tox -e long-test --override testenv:long-test.basepython=python3.11
+```
+
+## Testing Python versions available with Anaconda
+
+As of 2026-04-14, Python 3.5+ is available with Anaconda.
 
 ```bash
 git clone https://github.com/hapi-server/client-python
-cd client-python; pip install -e .
+cd client-python; python -m pip install -e .
 ```
 
 or, create an isolated Anaconda installation (downloads and installs latest Miniconda3) using
@@ -108,7 +131,7 @@ make install PYTHON=python3.6
 ```
 
 The command `pip install -e .` creates symlinks so that the local package is
-used instead of an installed package. You may need to execute `pip uninstall hapiclient` to ensure the local package is used. To check the version installed, use `pip list | grep hapiclient`.
+used instead of an installed package. You may need to execute `python -m pip uninstall hapiclient` to ensure the local package is used. To check the version installed, use `python -m pip list | grep hapiclient`.
 
 To run tests before a commit, execute
 
@@ -118,9 +141,8 @@ make repository-test
 
 To run an individual unit test in a Python session, use, e.g.,
 
-```python
-from hapiclient.test.test_hapi import test_reader_short
-test_reader_short()
+```bash
+cd test; python test_hapitime_reformat.py
 ```
 
 # Contact
@@ -128,27 +150,4 @@ test_reader_short()
 Submit bug reports and feature requests on the [repository issue
 tracker](https://github.com/hapi-server/client-python/issues>).
 
-# Appendix
-
-Fail-safe installation
-
-Python command line:
-
-```python
-import os
-print(os.popen("pip install hapiclient").read())
-```
-
-The above executes and displays the output of the operating system
-command `pip install hapiclient` using the shell environment
-associated with that installation of Python.
-
-This method addresses a problem that is sometimes encountered when
-attempting to use `pip` packages in Anaconda. To use a `pip` package
-in Anaconda, one must use the version of `pip` installed with Anaconda
-(it is usually under a subdirectory with the name `anaconda/`) as
-opposed to the one installed with the operating system. To see the
-location of ``pip`` used in a given Python session, enter
-`print(os.popen("which pip").read())`.
-
-
+Before submitting a pull request, please post an issue with the proposed changes for discussion prior to developing the pull request.

{hapiclient-0.2.5 → hapiclient-0.2.7}/hapiclient/__init__.py

@@ -4,12 +4,21 @@ from hapiclient.hapi import hapi
 # Allow "from hapiclient import hapitime2datetime"
 from hapiclient.hapitime import hapitime2datetime
 
+# Allow "from hapiclient import datetime2hapitime"
+from hapiclient.hapitime import datetime2hapitime
+
 # Allow "from hapiclient import HAPIError"
 from hapiclient.util import HAPIError
 
-__version__ = '0.2.5'
+__version__ = '0.2.7'
 
 import sys
+import platform
+
+if sys.version_info < (3, 10) and platform.system() == "Darwin":
+    import warnings
+    warnings.filterwarnings("ignore", message=".*urllib3.*OpenSSL.*")
+
 if sys.version_info[0] < 3:
     # Python 2.7
     reload(sys)

{hapiclient-0.2.5 → hapiclient-0.2.7}/hapiclient/hapi.py

@@ -4,6 +4,7 @@ import sys
 import json
 import time
 import pickle
+import logging as _logging
 import warnings
 from datetime import datetime, timedelta
 
@@ -13,7 +14,7 @@ import numpy as np
 from joblib import Parallel, delayed
 
 from hapiclient.hapitime import hapitime2datetime, hapitime_reformat
-from hapiclient.util import setopts, log, warning, error
+from hapiclient.util import setopts, log, warning, error, configure_logging
 from hapiclient.util import urlopen, urlretrieve, jsonparse, unicode_error_message
 
 
@@ -176,7 +177,7 @@ def hapiopts():
 def hapi(*args, **kwargs):
     """Request data from a HAPI server.
 
-    Version: 0.2.5
+    Version: 0.2.7
 
 
     Examples
@@ -194,10 +195,10 @@ def hapi(*args, **kwargs):
         A comma-separated list of parameters in `dataset`
     start: str
         The start time of the requested data
-    stop: str
+    stop: str or None
         The end time of the requested data; end times are exclusive - the
         last data record returned by a HAPI server should have a timestamp
-        before `start`.
+        before `start`. If `None`, `stopDate` is used.
     options: dict
             `logging` (``False``) - Log to console
 
@@ -250,7 +251,7 @@ def hapi(*args, **kwargs):
                     Cadence = PT1M and request for
 
                         start/stop=1999-11-12T00:10:00/stop=1999-11-12T12:09:00
-                        
+
                         Chunk size is P1D and requested time range < 1/2 of this
                         =>  Default behavior
 
@@ -337,14 +338,18 @@ def hapi(*args, **kwargs):
             START = START + 'Z'
     if nin > 4:
         STOP = args[4]
-        if STOP[-1] != 'Z':
+        if STOP is not None and STOP[-1] != 'Z':
             # TODO: Consider warning.
             STOP = STOP + 'Z'
 
     # Override defaults
     opts = setopts(hapiopts(), kwargs)
 
-    assert (opts['logging'] in [True, False]), "logging keyword must be True of False"
+    assert (isinstance(opts['logging'], bool) or hasattr(opts['logging'], 'write')), \
+        "logging keyword must be True, False, or a file-like object"
+
+    configure_logging(opts)
+
     assert (opts['cache'] in [True, False]), "cache keyword must be True of False"
     assert (opts['usecache'] in [True, False]), "usecache keyword must be True of False"
     assert (opts['format'] in ['binary', 'csv']), "format keyword must be 'csv' or 'binary'"
@@ -408,7 +413,7 @@ def hapi(*args, **kwargs):
         # urld = url subdirectory of cachedir to store files from SERVER
         urld = cachedir(opts["cachedir"], SERVER)
 
-        if opts["cachedir"]: log('file directory = %s' % urld, opts)
+        if opts["cachedir"]: log('cache subdirectory = %s' % urld, opts)
 
         urljson = SERVER + '/info?id=' + DATASET
 
@@ -422,6 +427,12 @@ def hapi(*args, **kwargs):
 
         if nin == 5:  # Data requested
 
+            if STOP is None:
+                log('STOP was given as None. Getting stopDate for dataset.', opts)
+                meta = hapi(SERVER, DATASET)
+                STOP = meta['stopDate']
+                log('Using STOP = {STOP}', opts)
+
             tic_totalTime = time.time()
 
             # URL to get CSV (will be used if binary response is not available)
@@ -552,10 +563,13 @@ def hapi(*args, **kwargs):
 
             backend = 'sequential'
             if opts['parallel']:
+                # Note that this does not often lead to significant speed-up.
+                # Needs further testing and optimization.
                 backend = 'threading'
-                # multiprocessing was not tested. It may work, but will
-                # need a speed comparison with threading.
-                # backend = 'multiprocessing'
+                # multiprocessing not working.
+                #backend = 'multiprocessing'
+                # loky works, but not speed-up.
+                #backend = 'loky'
 
             log('backend = {}'.format(backend), opts)
 
@@ -657,14 +671,18 @@ def hapi(*args, **kwargs):
         if opts["usecache"] and os.path.isfile(fnamenpy):
             # Read cached data file.
             log('Reading %s ' % fnamenpy.replace(urld + '/', ''), opts)
+            tic = time.time()
             f = open(fnamenpy, 'rb')
             data = np.load(f)
             f.close()
+            toc = time.time() - tic
             # There is a possibility that the fnamenpy file existed but
             # fnamepklx was not found (b/c removed). In this case, the meta
             # returned will not have all of the "x_" information inserted below.
             # Code that uses this information needs to account for this.
             meta['x_totalTime'] = time.time() - tic_totalTime
+            meta['x_readTime'] = tic - tic_totalTime
+            meta['x_downloadTime'] = 0
             return data, meta
 
         cformats = ['csv', 'binary']  # client formats
@@ -728,7 +746,7 @@ def hapi(*args, **kwargs):
                 try:
                     data = np.fromfile(fnamebin, dtype=dt)
                 except:
-                    error('Could not read response from {}'.format(urlbin))
+                    error('Malformed response? Could not read: {}'.format(urlbin))
             else:
                 from io import BytesIO
                 log('Writing %s to buffer' % urlbin.replace(urld + '/', ''), opts)
@@ -740,7 +758,8 @@ def hapi(*args, **kwargs):
                 try:
                     data = np.frombuffer(buff.read(), dtype=dt)
                 except:
-                    error('Could not read response from {}'.format(urlbin))
+                    error('Malformed response? Could not read: {}'.format(urlbin))
+
 
             # Handle Unicode
             time_name = meta['parameters'][0]['name']
@@ -753,7 +772,7 @@ def hapi(*args, **kwargs):
                     # with Unicode, it automatically converts Unicode chars to
                     # slash encoded ASCII.
                     name = str(name)
-                if isinstance(dt[i][1], str) and 'U' in dto[i][1]:
+                if data[name].size > 0 and isinstance(dt[i][1], str) and 'U' in dto[i][1]:
                     # Decode data.
                     datanew[name] = np.char.decode(data[name])
                 else:
@@ -763,6 +782,8 @@ def hapi(*args, **kwargs):
         else:
             # HAPI CSV
 
+            file_empty = False
+
             if opts["cache"]:
                 log('Writing %s to %s' % (urlcsv, fnamecsv.replace(urld + '/', '')), opts)
                 tic0 = time.time()
@@ -770,38 +791,78 @@ def hapi(*args, **kwargs):
                 toc0 = time.time() - tic0
                 log('Reading and parsing %s' % fnamecsv.replace(urld + '/', ''), opts)
                 tic = time.time()
+                if os.path.getsize(fnamecsv) == 0:
+                    file_empty = True
+                    data = np.array([], dtype=dt)
             else:
                 from io import StringIO
-                #log('Writing %s to buffer' % urlcsv.replace(urld + '/', ''), opts)
+                log('Writing %s to buffer' % urlcsv.replace(urld + '/', ''), opts)
                 tic0 = time.time()
                 fnamecsv = StringIO(urlopen(urlcsv).read().decode())
+                fnamecsv.seek(0, os.SEEK_END)
+                if fnamecsv.tell() == 0:
+                    file_empty = True
+                    data = np.array([], dtype=dt)
+                else:
+                    fnamecsv.seek(0)
                 toc0 = time.time() - tic0
                 log('Parsing StringIO buffer.', opts)
                 tic = time.time()
 
-            if not missing_length:
-                # All string and isotime parameters have a length in metadata.
-                if opts['method'] == 'numpy':
-                    data = np.genfromtxt(fnamecsv, dtype=dt, delimiter=',',
-                                            replace_space=' ',
-                                            deletechars='', encoding='utf-8')
-                if opts['method'] == '' or opts['method'] == 'pandas':
-                    # Read file into Pandas DataFrame
-                    df = pandas.read_csv(fnamecsv, sep=',', header=None,
-                                            encoding='utf-8')
-                    # Allocate output N-D array (It is not possible to pass dtype=dt
-                    # as computed to pandas.read_csv; pandas dtype is different
-                    # from numpy's dtype.)
-                    data = np.ndarray(shape=(len(df)), dtype=dt)
-                    # Insert data from dataframe 'df' columns into N-D array 'data'
-                    for i in range(0, len(pnames)):
-                        shape = np.append(len(data), psizes[i])
-                        # In numpy 1.8.2 and Python 2.7, this throws an error
-                        # for no apparent reason. Works as expected in numpy 1.10.4
-                        data[pnames[i]] = np.squeeze(
-                            np.reshape(df.values[:, np.arange(cols[i][0], cols[i][1] + 1)], shape))
-            else:
-                data = parse_missing_length(fnamecsv, dt, cols, psizes, pnames, ptypes, opts)
+            if file_empty == False:
+                if not missing_length:
+                    # All string and isotime parameters have a length in metadata.
+                    if opts['method'] == 'numpy':
+                        try:
+                            data = np.genfromtxt(fnamecsv,
+                                                 dtype=dt,
+                                                 delimiter=',',
+                                                 replace_space=' ',
+                                                 deletechars='',
+                                                 encoding='utf-8')
+                        except:
+                            error('Malformed response? Could not read response: {}'.format(urlcsv))
+                    if opts['method'] == '' or opts['method'] == 'pandas':
+                        # Read file into Pandas DataFrame
+                        csv_kwargs = {
+                            'sep': ',',
+                            'header': None,
+                            'encoding': 'utf-8',
+                            'skipinitialspace': True,
+                            'keep_default_na': False,
+                            'na_values': ['NaN', 'nan', 'Nan', 'naN', ' "NaN"', ' "nan"', ' "Nan"', ' "naN"', '"NaN"', '"nan"', '"Nan"', '"naN"']
+                        }
+                        """
+                        Note that this does not handle trailing whitespace after
+                        any of the na_values. (There is no skiptrailingspace option).
+                        Stripping trailing whitespace would require adding something
+                        like
+                            def strip_field(x):
+                                # Strip whitespace and normalize NaN values.
+                                x = x.strip()
+                                return np.nan if x.lower() == "nan" else x
+
+                            ncols = cols[-1][1] + 1
+                            csv_kwargs["converters"] = {i: strip_field for i in range(ncols)}
+                        """
+                        try:
+                            df = pandas.read_csv(fnamecsv, **csv_kwargs)
+                        except:
+                            error('Malformed response? Could not read response: {}'.format(urlcsv))
+
+                        # Allocate output N-D array (It is not possible to pass dtype=dt
+                        # as computed to pandas.read_csv; pandas dtype is different
+                        # from numpy's dtype.)
+                        data = np.ndarray(shape=(len(df)), dtype=dt)
+                        # Insert data from dataframe 'df' columns into N-D array 'data'
+                        for i in range(0, len(pnames)):
+                            shape = np.append(len(data), psizes[i])
+                            # In numpy 1.8.2 and Python 2.7, this throws an error
+                            # for no apparent reason. Works as expected in numpy 1.10.4
+                            data[pnames[i]] = np.squeeze(
+                                np.reshape(df.values[:, np.arange(cols[i][0], cols[i][1] + 1)], shape))
+                else:
+                    data = parse_missing_length(fnamecsv, dt, cols, psizes, pnames, ptypes, opts)
 
         toc = time.time() - tic
 
@@ -835,7 +896,14 @@ def hapi(*args, **kwargs):
                 pickle.dump(meta, f, protocol=2)
 
             log('Writing %s' % fnamenpy, opts)
-            np.save(fnamenpy, data)
+            with warnings.catch_warnings():
+                # Ignore warning that occurs when saving Unicode data.
+                warnings.filterwarnings("ignore",
+                    message=r"Stored array in format 3\.0.*",
+                    category=UserWarning,
+                    module=r"numpy\.lib\.format",
+                )
+                np.save(fnamenpy, data)
 
         meta['x_totalTime'] = time.time() - tic_totalTime
 
@@ -933,9 +1001,28 @@ def parse_missing_length(fnamecsv, dt, cols, psizes, pnames, ptypes, opts):
     if opts['method'] == 'numpy' or opts['method'] == 'numpynolength':
         # If requested method was numpy, use numpynolength method.
 
-        # With dtype='None', the data type is determined automatically
-        table = np.genfromtxt(fnamecsv, dtype=None, deletechars='',
-                                delimiter=',', encoding='utf-8')
+        ncols = cols[-1][1] + 1
+
+        def normalize_field(value):
+            if isinstance(value, bytes):
+                value = value.decode('utf-8')
+            value = value.strip()
+            if len(value) >= 2 and value[0] == '"' and value[-1] == '"':
+                value = value[1:-1].strip()
+            if value.lower() == 'nan':
+                return 'nan'
+            return value
+
+        converters = {i: normalize_field for i in range(ncols)}
+
+        table = np.genfromtxt(fnamecsv,
+                              dtype=None,
+                              deletechars='',
+                              replace_space=' ',
+                              delimiter=',',
+                              encoding='utf-8',
+                              converters=converters)
+
         # table is a 1-D array. Each element is a row in the file.
         # - If the data types are not the same for each column,
         # the elements are tuples with length equal to the number
@@ -980,8 +1067,17 @@ def parse_missing_length(fnamecsv, dt, cols, psizes, pnames, ptypes, opts):
     if opts['method'] == '' or opts['method'] == 'pandas' or opts['method'] == 'pandasnolength':
         # If requested method was pandas, use pandasnolength method.
 
+        # TODO: Duplicate code.
         # Read file into Pandas DataFrame
-        df = pandas.read_csv(fnamecsv, sep=',', header=None, encoding='utf-8')
+        csv_kwargs = {
+            'sep': ',',
+            'header': None,
+            'encoding': 'utf-8',
+            'skipinitialspace': True,
+            'keep_default_na': False,
+            'na_values': ['NaN', 'nan', 'Nan', 'naN', ' "NaN"', ' "nan"', ' "Nan"', ' "naN"', '"NaN"', '"nan"', '"Nan"', '"naN"']
+        }
+        df = pandas.read_csv(fnamecsv, **csv_kwargs)
 
         # Allocate output N-D array (It is not possible to pass dtype=dt
         # as computed to pandas.read_csv, so need to create new ND array.)