shipgrav 1.0.2__py2.py3-none-any.whl → 1.0.4__py2.py3-none-any.whl

shipgrav/__init__.py

@@ -9,9 +9,9 @@ DGS gravimeters output two types of files: serial, or 'raw' files; and 'laptop'
 Installation
 ------------
 
-shipgrav can be installed from `PyPI <https://pypi.org/project/shipgrav/>`_ using ``pip``. we recommend using an environment management tool like `conda <https://anaconda.org>`_. An exemplary set of commands to make a conda enviroment with shipgrav would be: ::
+shipgrav can be installed from `PyPI <https://pypi.org/project/shipgrav/>`_ using ``pip``. We recommend using an environment management tool like `conda <https://anaconda.org>`_. An exemplary set of commands to make a conda enviroment with shipgrav would be: ::
 
-    conda create --name shipgrav numpy scipy pandas statsmodels tomli pyyaml matplotlib
+    conda create --name shipgrav numpy scipy pandas statsmodels tomli pyyaml tqdm pooch matplotlib geographiclib
     conda activate shipgrav
     pip install shipgrav
 
@@ -24,7 +24,14 @@ shipgrav's dependencies are
 * statsmodels
 * tomli
 * pyyaml
+* tqdm
+* pooch (optional, needed to run the example scripts)
 * matplotlib (optional, needed to run some of the example scripts)
+* geographiclib (optional, needed to run one of the example scripts)
+* jupyterlab (optional, if you want to run example scripts in jupyter)
+* jupytext (optional, if you want to run example scripts in jupyter)
+
+If you install shipgrav with ``pip``, it will also install any of the required dependencies that are missing. To make ``pip`` include the optional dependencies, run ``pip install shipgrav[examples]``.
 
 The example scripts are `available on github <https://github.com/PFPE/shipgrav>`_. They are not packaged with the PyPI package and must be downloaded separately.
 
@@ -43,7 +50,7 @@ shipgrav consists of the modules ``io``, ``nav``, ``grav``, and ``utils``, along
 Data directories
 ----------------
 
-You can organize your data however you like; shipgrav does not care as long as you tell it where to look. However, to run the example scripts, you will need to download some data and ensure that the scripts are pointed to the paths where you put that data. A shell script ``download_data.sh`` is provided in ``example-scripts/`` to help download and organize the data.
+You can organize your data however you like; shipgrav does not care as long as you tell it where to look. The example scripts are set up to download data files from public repositories using ``pooch``. The ``pooch.retrieve()`` function returns lists of file paths for files that have been downloaded and unpacked, so to adapt the example workflows for other data files, you will need to replace those lists of paths with the paths to your data.
 
 Ties and bias
 -------------
@@ -62,7 +69,9 @@ The database file included in shipgrav lists the navigation talkers that we expe
 Example scripts
 ---------------
 
-The scripts in the ``example-scripts/`` directory use publicly available data files to run through some common workflows for marine gravity processing. The data files can be downloaded from R2R and Zenodo. The ``download_data.sh`` shell script in ``example-scripts/`` uses ``curl``, ``tar``, and ``unzip`` to download all the data files for you and unpack them into a directory structure that matches what the examples expect. You can also download and unpack the files on your own from the sources listed below:
+The scripts in the ``example-scripts/`` directory use publicly available data files to run through some common workflows for marine gravity processing. All of the examples can be run as scripts (ie, with ``python -m <script-name>.py``). All except ``interactive_line_pick.py`` can also be opened in ``jupyter`` as notebooks thanks to ``jupytext``. To run the examples in ``jupyter``, start ``jupyter lab``, right-click on the script file name, and select `open with -> notebook`.
+
+The data files can be downloaded from R2R and Zenodo, and the scripts will do this automatically using ``pooch``. The sources are:
 
 * https://doi.org/10.7284/151470 - TN400 BGM3 data
 * https://doi.org/10.7284/151457 - TN400 nav data
@@ -71,8 +80,6 @@ The scripts in the ``example-scripts/`` directory use publicly available data fi
 * https://doi.org/10.7284/157177 - SR2312 mru data
 * https://doi.org/10.5281/zenodo.12733929 - TN400 DGS raw and laptop data, SR2312 DGS raw data, R/V Ride DGS meter and Hydrins metadata, satellite FAA tracks for comparison, example file for RMBA calculation
 
-If you are not using the download script provided, we recommend looking at what it does and mimicking the directory structure it sets up for the data files. Otherwise, you will have to edit the examples to point to the correct filepaths for wherever you put the data in your system.
-
 ``dgs_bgm_comp.py`` reads data from DGS and BGM gravimeter files from R/V Thompson cruise TN400. The files are lightly processed to obtain the FAA (including syncing with navigation data for more accurate locations), and the FAA is plotted alongside corresponding satellite-derived FAA.
 
 .. image:: _static/TN400_FAA.png
@@ -101,7 +108,7 @@ If you are not using the download script provided, we recommend looking at what
    :height: 250px
    :align: center
 
-``interactive_line_pick.py`` reads laptop data and navigation data from R/V Sally Ride cruise SR2312. The script generates an interactive plot with a cursor for users to select segments of the time series data based on mapped locations, in order to extract straight line segments from a cruise track. The selected segments are written to files that can be re-read by the next script...
+``interactive_line_pick.py`` reads laptop data and navigation data from R/V Sally Ride cruise SR2312. The script generates an interactive plot with a cursor for users to select segments of the time series data based on mapped locations, in order to extract straight line segments from a cruise track. `This script cannot be run in jupyter`. The selected segments are written to files that can be re-read by the next script...
 
 .. image:: _static/cursor.png
    :alt: Interactive line segment picker window.
@@ -141,4 +148,6 @@ Contributing
 
 Do you have ideas for making this software better? Go ahead and `raise an issue <https://github.com/PFPE/shipgrav/issues>`_ on the github page or, if you're a savvy Python programmer, submit a pull request. You can also email PFPE at pfpe-interal(at)whoi.edu.
 
+If you raise an issue on github, please include as much detail as possible, such as the text of error messages. If there are no visible errors but you think the code is behaving oddly, provide a description of what the code is doing and what you think it *should* be doing instead. PFPE may ask for additional details or copies of data files in order to reproduce and diagnose an issue.
+
 """

shipgrav/database.toml

@@ -13,6 +13,7 @@ NBP = 'GPGGA'
 Ride = 'INGGA'
 Revelle = 'GPGGA'
 Langseth = 'INGGA'
+Sikuliaq = 'GPGGA'
 
 [BGM-scale]
 Atlantis = 4.994070552

shipgrav/grav.py

@@ -1,11 +1,12 @@
-import numpy as np
-from scipy.signal import lfilter, firwin
-from pandas import DataFrame
-from statsmodels.api import OLS, add_constant
+from copy import copy
 from datetime import datetime, timezone
 from math import factorial
+
+import numpy as np
+from pandas import DataFrame
+from scipy.signal import firwin, lfilter
 from scipy.special import erf, erfcinv
-from copy import copy
+from statsmodels.api import OLS, add_constant
 
 # impulse response of 10th order Taylor series differentiator
 tay10 = [1/1260, -5/504, 5/84, -5/21, 5/6,
@@ -579,10 +580,6 @@ def _calc_up_vecs(ctilt, ltilt):
     :return: **up_vecs** (*3xN ndarray*) - (cross, long, up) for platform
     """
 
-    # get increments, assuming initial is 0
-    inc_ct = np.append(ctilt[0], np.diff(ctilt))
-    inc_lt = np.append(ltilt[0], np.diff(ltilt))
-
     # trig functions of tilts
     sc = np.sin(ctilt)
     cc = np.cos(ctilt)
@@ -756,8 +753,6 @@ def grav2d_folding(X, Y, Z, dx, dy, drho=0.6, dz=6000, ifold=True, npower=5):
     # folding frequency:
     mfx = int(nxt/2 + 1)
     mfy = int(nyt/2 + 1)
-    mfx2 = mfx*2
-    mfy2 = mfy*2
 
     # the important part:
     # (1) compute wavenumbers
@@ -1172,8 +1167,6 @@ def crustal_thickness_2D(ur, nx=1000, ny=1, dx=1.3, dy=0, zdown=10, rho=0.4,
     """
     assert wlarge > wsmall, 'wlarge must be larger than wsmall'
 
-    zmin = min(ur)
-    zmax = max(ur)
     ave = np.mean(ur)
 
     # shift to a new reference and convert milligal to gal

shipgrav/io.py

@@ -1,11 +1,12 @@
-import numpy as np
-import pandas as pd
-from datetime import datetime, timezone
-import yaml
 import mmap
-import sys
 import os
 import re
+from datetime import datetime, timezone
+
+import numpy as np
+import pandas as pd
+import yaml
+from tqdm import tqdm
 
 # TODO need a fix for places where we cross the international date line (read_nav)
 # TODO check pandas versions for datetime parsing (eg RGS read)
@@ -16,7 +17,7 @@ import re
 ########################################################################
 
 
-def read_nav(ship, pathlist, sampling=1, talker=None, ship_function=None):
+def read_nav(ship, pathlist, sampling=1, talker=None, ship_function=None, progressbar=True):
     """ Read navigation strings from .GPS (or similar) files.
 
     Ships have different formats and use different talkers for preferred
@@ -37,6 +38,8 @@ def read_nav(ship, pathlist, sampling=1, talker=None, ship_function=None):
         This function should return arrays of lon, lat, and timestamps.
         Look at _navcoords() and navdate_Atlantis() (and similar functions) for examples.
     :type ship_function: function, optional
+    :param progressbar: display progress bar while list of files is read
+    :type progressbar: bool
 
     :returns: (*pd.DataFrame*) time series of geographic coordinates and timestamps
     """
@@ -48,7 +51,7 @@ def read_nav(ship, pathlist, sampling=1, talker=None, ship_function=None):
     nav_str = info['nav-talkers']
 
     # check to make sure we have some way to read nav data for this ship
-    if ship not in nav_str.keys() and ship_function == None:
+    if ship not in nav_str.keys() and ship_function is None:
         print('R/V %s not yet supported for nav read; must supply read function' % ship)
         return -999
 
@@ -70,7 +73,7 @@ def read_nav(ship, pathlist, sampling=1, talker=None, ship_function=None):
     lonlon = np.array([])
     latlat = np.array([])
 
-    for fpath in pathlist:  # loop nav files (may be a lot of them)
+    for fpath in tqdm(pathlist,desc='reading nav',disable=not progressbar):  # loop nav files (may be a lot of them)
         with open(fpath, 'r') as f:
             allnav = np.array(f.readlines())  # read the entire file
 
@@ -95,11 +98,17 @@ def read_nav(ship, pathlist, sampling=1, talker=None, ship_function=None):
             elif ship == 'Langseth':
                 lon, lat = _navcoords(allnav, talker)
                 timest = _navdate_Langseth(allnav, talker)
+            elif ship == 'Sikuliaq' and talker == 'GPGGA':
+                lon, lat = _navcoords(allnav, talker)
+                timest = _navdate_Ride(allnav, talker)
             else:  # in theory we never get to this option, but catch just in case
                 print(
                     'R/V %s not yet supported for nav read; must supply read function' % ship)
                 return -999
 
+        if type(timest) is int and timest == -999:
+            return timest  # eg looking for a talker that is not in a file
+
         # posix, seconds, for interpolation
         sec_time = np.array([d.timestamp() for d in timest])
         _, idx = np.unique(sec_time, return_index=True)
@@ -169,7 +178,7 @@ def _navdate_Atlantis(allnav, talker):
 
     for i in range(N):
         pre = subnav[i].split(talker)[0]
-        date = re.findall('NAV (\d{4})/(\d{2})/(\d{2})', pre)[0]
+        date = re.findall(r'NAV (\d{4})/(\d{2})/(\d{2})', pre)[0]
         year = int(date[0])  # year
         mon = int(date[1])  # month
         day = int(date[2])  # day
@@ -191,7 +200,7 @@ def _navdate_NBP(allnav, talker):
 
     for i in range(N):
         pre = subnav[i].split(talker)[0]
-        date = re.findall('(\d{2})\+(\d{2,3}):.*', pre)[0]
+        date = re.findall(r'(\d{2})\+(\d{2,3}):.*', pre)[0]
         # year (NBP didn't exist before 2000 so this is ok)
         year = '20' + date[0]
         doy = date[1]  # doy
@@ -216,7 +225,7 @@ def _navdate_Thompson(allnav, talker):
 
     for i in range(N):
         pre = subnav[i].split(talker)[0]
-        date = re.findall('(\d{2})/(\d{2})/(\d{4}),*', pre)[0]
+        date = re.findall(r'(\d{2})/(\d{2})/(\d{4}),*', pre)[0]
         year = int(date[2])
         mon = int(date[0])
         day = int(date[1])
@@ -226,7 +235,7 @@ def _navdate_Thompson(allnav, talker):
 
 
 def _navdate_Revelle(allnav, talker):
-    """Extract datetime info from Revelle nav files (mru_seapatah330_rr_navbho-*.txt).
+    """Extract datetime info from Revelle nav files (mru_seapath330_rr_navbho-*.txt).
     """
     hour, mint, sec, msec = _clock_time(allnav, talker)
 
@@ -247,8 +256,8 @@ def _navdate_Revelle(allnav, talker):
         for k in range(inds[i]-1, j, -1):  # step backwards toward the last talker line
             before = allnav[k]
             # date is at the start of this line
-            if re.match('(\d{4})-(\d{2})-(\d{2})T*', before):
-                date = re.findall('(\d{4})-(\d{2})-(\d{2})T*', before)[0]
+            if re.match(r'(\d{4})-(\d{2})-(\d{2})T*', before):
+                date = re.findall(r'(\d{4})-(\d{2})-(\d{2})T*', before)[0]
                 year = int(date[0])
                 mon = int(date[1])
                 day = int(date[2])
@@ -264,17 +273,19 @@ def _navdate_Ride(allnav, talker):
     inav = [talker in s for s in allnav]  # find lines of file with this talker
     subnav = allnav[inav]  # select only those lines
     N = len(subnav)
+    if N == 0:
+        return -999
     # array for timestamps, as datetime objects
     timest = np.empty(N, dtype=datetime)
 
     for i in range(N):
         if talker == 'INGGA':  # on Ride, uses posix timestamps
-            date = re.findall('(\d+(\.\d*)?) \$%s' % talker, subnav[i])[0]
+            date = re.findall(r'(\d+(\.\d*)?) \$%s' % talker, subnav[i])[0]
             timest[i] = datetime.fromtimestamp(
-                float(date[0]), tzinfo=timezone.utc)
+                float(date[0]), timezone.utc)
         elif talker == 'GPGGA':  # includes time only with date, unlike other GPGGAs
             date = re.findall(
-                '(\d{4})\-(\d{2})\-(\d{2})T(\d{2}):(\d{2}):(\d{2})\.(\d.*?)Z', subnav[i])[0]
+                r'(\d{4})\-(\d{2})\-(\d{2})T(\d{2}):(\d{2}):(\d{2})\.(\d.*?)Z', subnav[i])[0]
             year = int(date[0])
             mon = int(date[1])
             day = int(date[2])
@@ -359,7 +370,7 @@ def _clean_180cross(gps_nav):
 ########################################################################
 
 
-def read_bgm_rgs(fp, ship):
+def read_bgm_rgs(fp, ship, progressbar=True):
     """Read BGM gravity from RGS files.
 
     RGS is supposedly a standard format; is consistent between Atlantis 
@@ -369,6 +380,8 @@ def read_bgm_rgs(fp, ship):
     :type fp: string, or list of strings
     :param ship: ship name
     :type ship: string
+    :param progressbar: display progress bar while list of files is read
+    :type progressbar: bool
 
     :returns: (*pd.DataFrame*) timestamps, raw gravity, and geographic coordinates
     """
@@ -377,21 +390,20 @@ def read_bgm_rgs(fp, ship):
         print('R/V %s not supported for RGS read yet' % ship)
         return -999
 
-    if type(fp) == str:
+    if type(fp) is str:
         fp = [fp,]  # make a list if only one path is given
 
     dats = []
-    for path in fp:
+    for path in tqdm(fp, desc='reading RGS files', disable=not progressbar):
         dat = pd.read_csv(path, delimiter=' ', names=['date', 'time', 'grav', 'lat', 'lon'],
-                          usecols=(1, 2, 3, 11, 12), parse_dates=[[0, 1]])
-        ndt = [e.tz_localize(timezone.utc) for e in dat['date_time']]
-        dat['date_time'] = ndt
+                          usecols=(1, 2, 3, 11, 12))
+        dat['date_time'] = pd.to_datetime(dat.pop('date')+' '+dat.pop('time'),utc=True)
         dats.append(dat)
 
     return pd.concat(dats, ignore_index=True)
 
 
-def read_bgm_raw(fp, ship, scale=None, ship_function=None):
+def read_bgm_raw(fp, ship, scale=None, ship_function=None, progressbar=True):
     """Read BGM gravity from raw (serial) files (not RGS).
 
     This function uses scale factors determined for specific BGM meters
@@ -408,6 +420,8 @@ def read_bgm_raw(fp, ship, scale=None, ship_function=None):
         The function should return a pandas.DataFrame with timestamps and counts.
         Look at _bgmserial_Atlantis() and similar functions for examples.
     :type ship_function: function, optional
+    :param progressbar: display progress bar while list of files is read
+    :type progressbar: bool
 
     :return: (*pd.DataFrame*) timestamps and calibrated raw gravity values
     """
@@ -427,10 +441,10 @@ def read_bgm_raw(fp, ship, scale=None, ship_function=None):
         else:
             scale = sc_fac[ship]
 
-    if type(fp) == str:
+    if type(fp) is str:
         fp = [fp,]  # make a list if only one path is given
     dats = []
-    for path in fp:
+    for path in tqdm(fp,desc='reading BGM files',disable=not progressbar):
         if ship_function != None:
             dat = ship_function(path)
         else:
@@ -456,10 +470,8 @@ def _bgmserial_Atlantis(path):
     def count(x): return (
         int(x.split(':')[-1]))  # function to parse counts column
     dat = pd.read_csv(path, delimiter=' ', names=['date', 'time', 'counts'], usecols=(1, 2, 4),
-                      parse_dates=[[0, 1]], converters={'counts': count})
-    ndt = [e.tz_localize(timezone.utc)
-           for e in dat['date_time']]  # timestamps cannot be naive
-    dat['date_time'] = ndt
+                      converters={'counts': count})
+    dat['date_time'] = pd.to_datetime(dat.pop('date')+' '+dat.pop('time'),utc=True)
     return dat
 
 
@@ -468,9 +480,8 @@ def _bgmserial_Thompson(path):
     """
     def count(x): return (int(x.split(' ')[0].split(':')[-1]))
     dat = pd.read_csv(path, delimiter=',', names=['date', 'time', 'counts'],
-                      parse_dates=[[0, 1]], converters={'counts': count})
-    ndt = [e.tz_localize(timezone.utc) for e in dat['date_time']]
-    dat['date_time'] = ndt
+                    converters={'counts': count})
+    dat['date_time'] = pd.to_datetime(dat.pop('date')+' '+dat.pop('time'),utc=True)
     return dat
 
 
@@ -480,8 +491,6 @@ def _bgmserial_Revelle(path):
     def count(x): return (int(x.split(':')[-1]))
     dat = pd.read_csv(path, delimiter=' ', names=['date_time', 'counts'], usecols=(0, 1),
                       parse_dates=[0], converters={'counts': count})
-    ndt = [e.tz_localize(timezone.utc) for e in dat['date_time']]
-    dat['date_time'] = ndt
     return dat
 
 def _bgmserial_Langseth(path):
@@ -523,7 +532,7 @@ def _despike_bgm_serial(dat, thresh=8000):
 ########################################################################
 
 
-def read_dgs_laptop(fp, ship, ship_function=None):
+def read_dgs_laptop(fp, ship, ship_function=None, progressbar=True):
     """Read DGS 'laptop' file(s), usually written as .dat files.
 
     :param fp: filepath(s)
@@ -534,15 +543,17 @@ def read_dgs_laptop(fp, ship, ship_function=None):
         The function should return a pandas.DataFrame. See _dgs_laptop_general()
         for an example.
     :type ship_function: function, optional
+    :param progressbar: display progress bar while list of files is read
+    :type progressbar: bool
 
     :return: *(pd.DataFrame)* DGS output time series
     """
-    if type(fp) == str:
+    if type(fp) is str:
         fp = [fp,]  # listify
 
     dats = []
-    for path in fp:
-        if ship_function != None:
+    for path in tqdm(fp,desc='reading DGS files',disable=not progressbar):
+        if ship_function is not None:
             dat = ship_function(path)
         else:
             if ship in ['Atlantis', 'Revelle', 'NBP', 'Ride', 'DGStest']:
@@ -575,14 +586,12 @@ def _dgs_laptop_Thompson(path):
     """
     dat = pd.read_csv(path, delimiter=',', names=['date', 'time', 'rgrav', 've', 'vcc',
                                                   'al', 'ax', 'lat', 'lon'],
-                      usecols=(0, 1, 3, 12, 13, 14, 15, 16, 17),
-                      parse_dates=[[0, 1]])
-    ndt = [e.tz_localize(timezone.utc) for e in dat['date_time']]
-    dat['date_time'] = ndt
+                      usecols=(0, 1, 3, 12, 13, 14, 15, 16, 17))
+    dat['date_time'] = pd.to_datetime(dat.pop('date')+' '+dat.pop('time'),utc=True)
     return dat
 
 
-def read_dgs_raw(fp, ship, scale_ccp=True):
+def read_dgs_raw(fp, ship, scale_ccp=True, progressbar=True):
     """Read raw (serial) output files from DGS AT1M.
 
     These will be in AD units mostly.
@@ -594,14 +603,16 @@ def read_dgs_raw(fp, ship, scale_ccp=True):
     :type fp: string or list of strings
     :param ship: ship name
     :type ship: string
+    :param progressbar: display progress bar while list of files is read
+    :type progressbar: bool
 
     :return: (*pd.DataFrame*) DGS output time series
     """
 
-    if type(fp) == str:
+    if type(fp) is str:
         fp = [fp,]  # listify
     dats = []
-    for ip, path in enumerate(fp):
+    for path in tqdm(fp,desc='reading DGS files',disable=not progressbar):
         if ship == 'Thompson':  # always with the special file formats
             dat = _dgs_raw_Thompson(path)
         else:  # there might be exceptions besides Thompson but I don't know about them yet
@@ -746,4 +757,13 @@ def read_other_stuff(yaml_file, data_file, tag):
         except KeyError:
             pass
 
-    return df.apply(pd.to_numeric, errors='ignore'), col_info
+    def numeric(col):
+        try:
+            return pd.to_numeric(col)
+        except:
+            return col
+
+    for ckey in df.columns:     # to_numeric by column so errors for datetimes are skipped
+        df[ckey] = df[ckey].apply(numeric)  # slower than applying to whole df, but avoids errors
+
+    return df, col_info

shipgrav/tests/__init__.py

@@ -2,15 +2,16 @@
 Tests for shipgrav
 """
 
-from pkg_resources import resource_filename
+import importlib.resources as importlib_resources
 import sys
 import unittest
 
 
 def run():
     loader = unittest.TestLoader()
-    test_dir = resource_filename('shipgrav', 'tests')
-    suite = loader.discover(test_dir)
+    ref = importlib_resources.files('shipgrav') / 'tests'
+    with importlib_resources.as_file(ref) as path:
+        suite = loader.discover(ref)
     runner = unittest.runner.TextTestRunner()  # verbosity=2)
     ret = not runner.run(suite).wasSuccessful()
     sys.exit(ret)

shipgrav/tests/ex_files/AT01_bgm.BGM

@@ -0,0 +1,4 @@
+BGM 2022/07/01 12:17:33.492 BGM3 04:024963 00
+BGM 2022/07/01 12:17:34.492 BGM3 04:024982 00
+BGM 2022/07/01 12:17:35.492 BGM3 04:024992 00
+BGM 2022/07/01 12:17:36.492 BGM3 04:024983 00

shipgrav/tests/ex_files/AT01_nav.gps

@@ -0,0 +1,8 @@
+NAV 2022/07/01 12:17:33.233 GPS $GPZDA,121733.00,01,07,2022,00,00*67
+NAV 2022/07/01 12:17:33.433 GPS $GPRMC,121733.00,A,4131.403226,N,07040.311159,W,0.01,218.7,010722,,,D*7C
+NAV 2022/07/01 12:17:33.433 GPS $GPVTG,218.7,T,,M,0.01,N,0.02,K,D*07
+NAV 2022/07/01 12:17:33.537 GPS $GPGGA,121733.00,4131.403226,N,07040.311159,W,2,12,0.9,25.169,M,-30.062,M,4.0,0402*46
+NAV 2022/07/01 12:17:34.233 GPS $GPZDA,121734.00,01,07,2022,00,00*60
+NAV 2022/07/01 12:17:34.433 GPS $GPRMC,121734.00,A,4131.403225,N,07040.311162,W,0.00,206.6,010722,,,D*7F
+NAV 2022/07/01 12:17:34.433 GPS $GPVTG,206.6,T,,M,0.00,N,0.01,K,D*0B
+NAV 2022/07/01 12:17:34.537 GPS $GPGGA,121734.00,4131.403225,N,07040.311162,W,2,12,0.9,25.176,M,-30.062,M,5.0,0402*45

shipgrav/tests/ex_files/MGL2003_bgm.y2020d244

@@ -0,0 +1,4 @@
+vc01	2020:244:00:00:00.3244	04:025229 00
+vc01	2020:244:00:00:01.3244	04:025247 00
+vc01	2020:244:00:00:02.3235	04:025404 00
+vc01	2020:244:00:00:03.3248	04:025535 00

shipgrav/tests/ex_files/MGL2003_nav.y2020d244

@@ -0,0 +1,12 @@
+seapath	2020:244:00:00:00.6212	$INGGA,000000.37,5420.890903,N,13237.257959,W,2,09,1.2,1.62,M,-7.75,M,4.2,0291*55
+seapath	2020:244:00:00:00.8547	$INGLL,5420.890903,N,13237.257959,W,000000.37,A,D*63
+seapath	2020:244:00:00:00.8549	$INVTG,77.26,T,59.62,M,9.8,N,18.1,K,D*03
+seapath	2020:244:00:00:00.8550	$INHDT,75.05,T*22
+seapath	2020:244:00:00:00.8551	$PSXN,20,0,0,0,0*3B
+seapath	2020:244:00:00:00.9253	$PSXN,23,1.06,0.15,75.05,0.04*08
+seapath	2020:244:00:00:01.6218	$INGGA,000001.37,5420.891483,N,13237.253415,W,2,09,1.2,1.60,M,-7.75,M,5.2,0291*52
+seapath	2020:244:00:00:01.8561	$INGLL,5420.891483,N,13237.253415,W,000001.37,A,D*67
+seapath	2020:244:00:00:01.8562	$INVTG,77.89,T,60.26,M,9.8,N,18.2,K,D*0F
+seapath	2020:244:00:00:01.8563	$INHDT,74.82,T*2C
+seapath	2020:244:00:00:01.8564	$PSXN,20,0,0,0,0*3B
+seapath	2020:244:00:00:01.9253	$PSXN,23,1.09,0.10,74.82,0.05*0D

shipgrav/tests/ex_files/NBP_2301_nav.d013

@@ -0,0 +1,22 @@
+23+013:00:00:00.113 $GPVTG,5.71,T,,M,5.2,N,9.6,K,A*36
+23+013:00:00:00.115 $GPRMC,000000.81,A,7520.156405,S,17945.635852,W,5.2,5.71,130123,,,A*6C
+23+013:00:00:00.213 $GPHDT,4.62,T*05
+23+013:00:00:00.213 $PSXN,20,0,0,0,0*3B
+23+013:00:00:00.213 $PSXN,22,1.32,0.09*30
+23+013:00:00:00.213 $PSXN,23,-0.67,0.42,4.62,0.10*13
+23+013:00:00:00.996 $GPZDA,000001.81,13,01,2023,,*6E
+23+013:00:00:00.996 $GPGGA,000001.81,7520.154976,S,17945.635106,W,1,12,0.5,-1.36,M,-56.73,M,,*6F
+23+013:00:00:01.113 $GPVTG,6.29,T,,M,5.1,N,9.5,K,A*38
+23+013:00:00:01.113 $GPRMC,000001.81,A,7520.154976,S,17945.635106,W,5.1,6.29,130123,,,A*63
+23+013:00:00:01.213 $GPHDT,4.74,T*02
+23+013:00:00:01.213 $PSXN,20,0,0,0,0*3B
+23+013:00:00:01.213 $PSXN,22,1.32,0.09*30
+23+013:00:00:01.213 $PSXN,23,-0.46,0.32,4.74,0.06*17
+23+013:00:00:02.012 $GPZDA,000002.81,13,01,2023,,*6D
+23+013:00:00:02.012 $GPGGA,000002.81,7520.153540,S,17945.635196,W,1,12,0.5,-1.29,M,-56.73,M,,*65
+23+013:00:00:02.142 $GPVTG,2.95,T,,M,5.1,N,9.5,K,A*3B
+23+013:00:00:02.142 $GPRMC,000002.81,A,7520.153540,S,17945.635196,W,5.1,2.95,130123,,,A*64
+23+013:00:00:02.213 $GPHDT,4.81,T*08
+23+013:00:00:02.213 $PSXN,20,0,0,0,0*3B
+23+013:00:00:02.213 $PSXN,22,1.32,0.09*30
+23+013:00:00:02.214 $PSXN,23,-0.61,0.22,4.81,0.01*1E

shipgrav/tests/ex_files/RR2212_bgm.txt

@@ -0,0 +1,6 @@
+2022-11-07T17:13:45.329611Z 04:024882 00
+2022-11-07T17:13:46.336966Z 04:024881 00
+2022-11-07T17:13:47.329375Z 04:024871 00
+2022-11-07T17:13:48.329063Z 04:024860 00
+2022-11-07T17:13:49.328981Z 04:024850 00
+2022-11-07T17:13:50.328786Z 04:024840 00

shipgrav/tests/ex_files/RR2212_nav.txt

@@ -0,0 +1,19 @@
+2022-11-05T00:00:00.002010Z $GPZDA,000000.00,05,11,2022,,*61
+$GPGGA,000000.00,3345.095825,N,11923.337492,W,2,09,0.9,-3.95,M,-33.51,M,2.0,0135*5F
+$GPGLL,3345.095825,N,11923.337492,W,000000.00,A,D*7C
+$GPVTG,200.16,T,,M,11.0,N,20.4,K,D*3B
+$GPVBW,,,V,10.99,-0.08,A,,V,,V*57
+$GPRMC,000000.00,A,3345.095825,N,11923.337492,W,11.0,200.16,051122,,,D*4B
+$GPHDT,200.59,T*0B
+$PSXN,20,0,0,0,0*3B
+$PSXN,23,-1.34,0.12,200.59,-0.07*34
+
+2022-11-05T00:00:00.202094Z $GPZDA,000000.20,05,11,2022,,*63
+$GPGGA,000000.20,3345.095252,N,11923.337748,W,2,09,0.9,-4.00,M,-33.51,M,2.0,0135*58
+$GPGLL,3345.095252,N,11923.337748,W,000000.20,A,D*70
+$GPVTG,200.60,T,,M,11.0,N,20.4,K,D*3A
+$GPVBW,,,V,11.00,-0.02,A,,V,,V*5C
+$GPRMC,000000.20,A,3345.095252,N,11923.337748,W,11.0,200.60,051122,,,D*46
+$GPHDT,200.70,T*00
+$PSXN,20,0,0,0,0*3B
+$PSXN,23,-1.32,0.06,200.70,-0.03*38

shipgrav/tests/ex_files/SR2302_nav.raw

@@ -0,0 +1,36 @@
+1674228947.582 $INZDA,153547.44,20,01,2023,,*77
+1674228947.583 $INGGA,153547.44,3242.434865,N,11714.203305,W,2,09,0.8,-4.31,M,-33.32,M,2.0,0135*4B
+1674228947.583 $INGLL,3242.434865,N,11714.203305,W,153547.44,A,D*65
+1674228947.583 $INVTG,112.40,T,,M,0.0,N,0.1,K,D*2F
+1674228947.583 $INRMC,153547.44,A,3242.434865,N,11714.203305,W,0.0,112.40,200123,,,D*66
+1674228947.603 $GPGSA,M,3,21,31,09,16,03,01,26,04,22,,,,1.7,0.8,1.5*39
+1674228947.633 $INHDT,179.39,T*10
+1674228947.643 $PSXN,20,0,0,0,0*3B
+1674228947.653 $PSXN,23,-0.33,0.42,179.39,0.01*17
+1674228947.673 $PSXN,26,0,44.5000,0.7800,-0.9000,NRP*6D
+1674228947.693 $PSXN,26,1,44.5000,0.7800,-0.9000,Monitoring Point #1*44
+1674228947.723 $PSXN,26,2,44.5000,0.7800,-0.9000,Monitoring Point #2*1F
+1674228947.754 $PSXN,26,3,44.5000,0.7800,-0.9000,Monitoring Point #3*33
+1674228947.784 $PSXN,26,4,44.5000,0.7800,-0.9000,Monitoring Point #4*68
+1674228947.814 $PSXN,26,5,44.5000,0.7800,-0.9000,Monitoring Point #5*3D
+1674228947.844 $PSXN,26,6,44.5000,0.7800,-0.9000,Monitoring Point #6*11
+1674228947.874 $PSXN,26,7,44.5000,0.7800,-0.9000,Monitoring Point #7*4A
+1674228947.904 $PSXN,26,8,44.5000,0.7800,-0.9000,Monitoring Point #8*64
+1674228948.444 $INZDA,153548.44,20,01,2023,,*78
+1674228948.474 $INGGA,153548.44,3242.434850,N,11714.203293,W,2,09,0.8,-4.32,M,-33.32,M,3.0,0135*4E
+1674228948.514 $INGLL,3242.434850,N,11714.203293,W,153548.44,A,D*62
+1674228948.544 $INVTG,117.77,T,,M,0.0,N,0.1,K,D*2E
+1674228948.564 $INRMC,153548.44,A,3242.434850,N,11714.203293,W,0.0,117.77,200123,,,D*60
+1674228948.604 $GPGSA,M,3,21,31,09,16,03,01,26,04,22,,,,1.7,0.8,1.5*39
+1674228948.634 $INHDT,179.38,T*11
+1674228948.644 $PSXN,20,0,0,0,0*3B
+1674228948.654 $PSXN,23,-0.27,0.41,179.38,0.01*10
+1674228948.674 $PSXN,26,0,44.5000,0.7800,-0.9000,NRP*6D
+1674228948.694 $PSXN,26,1,44.5000,0.7800,-0.9000,Monitoring Point #1*44
+1674228948.724 $PSXN,26,2,44.5000,0.7800,-0.9000,Monitoring Point #2*1F
+1674228948.754 $PSXN,26,3,44.5000,0.7800,-0.9000,Monitoring Point #3*33
+1674228948.784 $PSXN,26,4,44.5000,0.7800,-0.9000,Monitoring Point #4*68
+1674228948.814 $PSXN,26,5,44.5000,0.7800,-0.9000,Monitoring Point #5*3D
+1674228948.844 $PSXN,26,6,44.5000,0.7800,-0.9000,Monitoring Point #6*11
+1674228948.874 $PSXN,26,7,44.5000,0.7800,-0.9000,Monitoring Point #7*4A
+1674228948.904 $PSXN,26,8,44.5000,0.7800,-0.9000,Monitoring Point #8*64

shipgrav/tests/ex_files/TN400_dgs_proc.Raw

@@ -0,0 +1,2 @@
+03/12/2022,16:04:07.033,9984.869108166342,9995.951860660536,  0.031769279452,  0.049233442454, -0.894949104184, 62.394784400000,           16730,         4086876, 30.259833080000, 50.442681000000,  0.000020000000,  0.035710000000,  0.000070000000, -0.000010000000,  0.000000000000,  0.000000000000,  0.000000000000,  0.000000000000,  0.044215194148,2022,03,12,16,04,19.80,5370472
+03/12/2022,16:04:08.017,9984.870433770533,9977.883336291710,  0.030815605022,  0.037431721381, -0.894945527905, 62.394784400000,           16730,         4086005, 30.260701895000, 50.447645000000,  0.000020000000,  0.032780000000,  0.000020000000, -0.000010000000,  0.000000000000,  0.000000000000,  0.000000000000,  0.000000000000,  0.044061782566,2022,03,12,16,04,20.80,5370473

shipgrav/tests/test_grav_data.py

@@ -5,19 +5,25 @@ ccp coeffs are not objectively good because the time series is short but it work
 Not bothering with the leveling correction here because exactly how to calculate it
 is left to the user
 """
+import os
 import unittest
+from glob import glob
+
 import numpy as np
 import shipgrav.grav as sgg
 import shipgrav.io as sgi
 import shipgrav.nav as sgn
-from glob import glob
 
 
 class gravDataTestCase(unittest.TestCase):
+    @property
+    def ex_files(self):
+        return __file__.rsplit(os.sep, 1)[0] + os.sep + "ex_files"
+
     def setUp(self):  # actions to take before running each test: load in some test data
         # test data is part of one of the files supplied by DGS (original file is 30M/86400
         # lines for 24 hr; we cut to 1 hr?)
-        gfiles = glob('ex_files/DGStest*.dat')
+        gfiles = glob(f'{self.ex_files}'+os.sep+'DGStest*.dat')
         data = sgi.read_dgs_laptop(gfiles, 'DGStest')
         data['tsec'] = [e.timestamp()
                         for e in data['date_time']]  # get posix timestamps

shipgrav/tests/test_grav_nodata.py

@@ -2,6 +2,7 @@
 Tests for shipgrav.grav
 """
 import unittest
+
 import numpy as np
 import shipgrav.grav as sgg
 
@@ -35,9 +36,26 @@ class gravNoDataTestCase(unittest.TestCase):
 
     def test_crustalthickness(self):
         rng = np.random.default_rng(123)  # seeded
-        C = sgg.crustal_thickness_2D(10*rng.random(1000))
+        signal = 10*rng.random(1000)
+        C = sgg.crustal_thickness_2D(signal)
         self.assertTrue(np.real(C[0])[0] - 0.04019 < 0.001)
 
+        C2 = sgg.crustal_thickness_2D(signal, back=True)
+        self.assertEqual(np.real(C[0])[0], np.real(C2[0][-1])[0])
+
+    def test_grav2d_folding(self):
+        rng = np.random.default_rng(123)  # seeded
+        X = np.arange(5); Y = np.arange(5)
+        Z = rng.random((5,5))
+        sdat = sgg.grav2d_folding(X, Y, Z, 100, 100,  drho=0.6, dz=6000, ifold=True, npower=5)
+
+        self.assertTrue(sdat[0,0] + 0.0043244755 < 0.001)
+
+    def test_grav2d_layer(self):
+        rng = np.random.default_rng(123)  # seeded
+        rho = 1e4*rng.random((5,5))
+        sdat = sgg.grav2d_layer_variable_density(rho, 100, 100, 3, 5)
+        self.assertTrue(sdat[0,0] - 130.877533 < 0.001)
 
 def suite():
     return unittest.makeSuite(gravNoDataTestCase, 'test')

shipgrav/tests/test_io.py

@@ -4,44 +4,117 @@ This does not test every option for read (because there are many ships)
 but for each overall read function it reads a snippet of an example file
 and checks that the values in key columns are correct
 """
+import os
 import unittest
+
 import shipgrav.io as sgi
 
 
 class ioTestCase(unittest.TestCase):
-    def test_read_nav(self):  # NOTE only tests one of the ship functions
-        # but it does include extracting clock time -> datetime -> posix
-        nav = sgi.read_nav('Thompson', 'ex_files/TN400_nav.Raw')
+    @property
+    def ex_files(self):
+        return __file__.rsplit(os.sep, 1)[0] + os.sep + "ex_files"
+
+    def test_read_nav_Thompson(self):
+        nav = sgi.read_nav('Thompson', f'{self.ex_files}'+os.sep+'TN400_nav.Raw', progressbar=False)
         self.assertEqual(nav.iloc[0].time_sec, 1647129603)
         self.assertTrue(nav.iloc[0].lon + 118.6524 < 0.001)
 
+    def test_read_nav_Atlantis(self):
+        nav = sgi.read_nav('Atlantis', f'{self.ex_files}'+os.sep+'AT01_nav.gps', progressbar=False)
+        self.assertEqual(nav.iloc[0].time_sec, 1656677853)
+        self.assertTrue(nav.iloc[0].lon + 70.67185265 < 0.001)
+
+    def test_read_nav_Langseth(self):
+        nav = sgi.read_nav('Langseth', f'{self.ex_files}'+os.sep+'MGL2003_nav.y2020d244', progressbar=False)
+        self.assertEqual(nav.iloc[0].time_sec, 1598832000.6212)
+        self.assertTrue(nav.iloc[0].lon + 132.620965893 < 0.001)
+
+    def test_read_nav_Revelle(self):
+        nav = sgi.read_nav('Revelle', f'{self.ex_files}'+os.sep+'RR2212_nav.txt', progressbar=False)
+        self.assertEqual(nav.iloc[0].time_sec, 1667606400)
+        self.assertTrue(nav.iloc[0].lon + 119.3889582 < 0.001)
+
+    def test_read_nav_Ride(self):
+        nav = sgi.read_nav('Ride', f'{self.ex_files}'+os.sep+'SR2302_nav.raw',talker='INGGA', progressbar=False)
+        self.assertEqual(nav.iloc[0].time_sec, 1674228947.583)
+        self.assertTrue(nav.iloc[0].lon + 117.23672175 < 0.001)
+
+    def test_read_nav_NBP(self):
+        nav = sgi.read_nav('NBP', f'{self.ex_files}'+os.sep+'NBP_2301_nav.d013', progressbar=False)
+        self.assertEqual(nav.iloc[0].time_sec, 1673568001.81)
+        self.assertTrue(nav.iloc[0].lon + 179.7605851 < 0.001)
+
+    def test_read_nav_nope(self):
+        nav = sgi.read_nav('Titanic', f'{self.ex_files}'+os.sep+'TN400_nav.Raw', progressbar=False)
+        self.assertEqual(nav, -999)
+
     def test_read_bgm_rgs(self):
-        bgm = sgi.read_bgm_rgs('ex_files/AT05_01_bgm.RGS', 'Atlantis')
+        bgm = sgi.read_bgm_rgs(f'{self.ex_files}'+os.sep+'AT05_01_bgm.RGS', 'Atlantis', progressbar=False)
         self.assertEqual(bgm.iloc[0]['date_time'].timestamp(), 1656633600.445)
         self.assertEqual(bgm.iloc[0]['grav'], 980329.272)
 
-    def test_read_bgm_raw(self):
-        bgm = sgi.read_bgm_raw('ex_files/TN400_bgm.Raw', 'Thompson')
+    def test_read_bgm_raw_Atlantis(self):
+        bgm = sgi.read_bgm_raw(f'{self.ex_files}'+os.sep+'AT01_bgm.BGM', 'Atlantis', progressbar=False)
+        self.assertEqual(bgm.iloc[0]['date_time'].timestamp(), 1656677853.492)
+        self.assertEqual(bgm.iloc[0]['counts'], 24963)
+        self.assertEqual(bgm.iloc[0]['rgrav'], 124666.983189576)
+
+    def test_read_bgm_raw_Thompson(self):
+        bgm = sgi.read_bgm_raw(f'{self.ex_files}'+os.sep+'TN400_bgm.Raw', 'Thompson', progressbar=False)
         self.assertEqual(bgm.iloc[0]['date_time'].timestamp(), 1647129602.449)
         self.assertEqual(bgm.iloc[0]['counts'], 25529)
         self.assertEqual(bgm.iloc[0]['rgrav'], 127730.60800402702)
 
-    def test_read_dgs_dat(self):
-        dgs = sgi.read_dgs_laptop('ex_files/DGStest_laptop.dat', 'DGStest')
+    def test_read_bgm_raw_Langseth(self):
+        bgm = sgi.read_bgm_raw(f'{self.ex_files}'+os.sep+'MGL2003_bgm.y2020d244', 'Langseth', progressbar=False)
+        self.assertEqual(bgm.iloc[0]['date_time'].timestamp(), 1598832000.3244)
+        self.assertEqual(bgm.iloc[0]['counts'], 25229)
+        self.assertEqual(bgm.iloc[0]['rgrav'], 126145.0)
+
+    def test_read_bgm_raw_Revelle(self):
+        bgm = sgi.read_bgm_raw(f'{self.ex_files}'+os.sep+'RR2212_bgm.txt', 'Revelle', progressbar=False)
+        self.assertEqual(bgm.iloc[0]['date_time'].timestamp(), 1667841225.329611)
+        self.assertEqual(bgm.iloc[0]['counts'], 24882)
+        self.assertEqual(bgm.iloc[0]['rgrav'], 124405.2114591)
+
+    def test_read_bgm_raw_nope(self):
+        bgm = sgi.read_bgm_raw(f'{self.ex_files}'+os.sep+'TN400_bgm.Raw', 'Boaty McBoatface', progressbar=False)
+        self.assertEqual(bgm, -999)
+
+    def test_read_dgs_dat_general(self):
+        dgs = sgi.read_dgs_laptop(f'{self.ex_files}'+os.sep+'DGStest_laptop.dat', 'DGStest', progressbar=False)
         self.assertEqual(dgs.iloc[0]['date_time'].timestamp(), 1562803200.0)
         self.assertEqual(dgs.iloc[0]['ve'], 0.81098)
         self.assertTrue(dgs.iloc[0]['rgrav'] - 12295.691114 < 0.0001)
 
-    def test_read_dgs_raw(self):
-        dgs = sgi.read_dgs_raw('ex_files/SR2312_dgs_raw.txt', 'Ride')
+    def test_read_dgs_dat_Thompson(self):
+        dgs = sgi.read_dgs_laptop(f'{self.ex_files}'+os.sep+'TN400_dgs_proc.Raw', 'Thompson', progressbar=False)
+        self.assertEqual(dgs.iloc[0]['date_time'].timestamp(), 1647101047.033)
+        self.assertEqual(dgs.iloc[0]['ve'], 2e-5)
+        self.assertTrue(dgs.iloc[0]['rgrav'] - 9995.95186 < 0.0001)
+
+    def test_read_dgs_dat_nope(self):
+        dgs = sgi.read_dgs_laptop(f'{self.ex_files}'+os.sep+'DGStest_laptop.dat', 'Katama', progressbar=False)
+        self.assertEqual(dgs, -999)
+
+    def test_read_dgs_raw_general(self):
+        dgs = sgi.read_dgs_raw(f'{self.ex_files}'+os.sep+'SR2312_dgs_raw.txt', 'Ride', progressbar=False)
         self.assertEqual(
             dgs.iloc[0]['date_time'].timestamp(), 1686873600.857719)
         self.assertEqual(dgs.iloc[0]['Gravity'], -218747)
         self.assertTrue(dgs.iloc[0]['vcc'] - 76.8771 < 0.0001)
 
+    def test_read_dgs_raw_Thompson(self):
+        dgs = sgi.read_dgs_raw(f'{self.ex_files}'+os.sep+'TN400_dgs_raw.Raw', 'Thompson', progressbar=False)
+        self.assertEqual(
+            dgs.iloc[0]['date_time'].timestamp(), 1647101046.634)
+        self.assertEqual(dgs.iloc[0]['Gravity'], -82)
+        self.assertTrue(dgs.iloc[0]['vcc'] - 0.0357100 < 0.0001)
+
     def test_read_mru(self):
         mru, cols = sgi.read_other_stuff(
-            'ex_files/IXBlue.yaml', 'ex_files/SR2312_mru.txt', 'PASHR')
+            f'{self.ex_files}'+os.sep+'IXBlue.yaml', f'{self.ex_files}'+os.sep+'SR2312_mru.txt', 'PASHR')
         self.assertEqual(mru.iloc[0]['Pitch:g'], -0.41)
         self.assertEqual(mru.iloc[0]['Roll:g'], 2.03)
         self.assertEqual(mru.iloc[0]['Heave:g'], -0.6)

shipgrav/tests/test_nav.py

@@ -2,8 +2,9 @@
 Tests for shipgrav.nav
 """
 import unittest
-import shipgrav.nav as sgn
+
 import numpy as np
+import shipgrav.nav as sgn
 
 
 class navTestCase(unittest.TestCase):

shipgrav/tests/test_utils.py

@@ -2,8 +2,9 @@
 Tests for shipgrav.utils
 """
 import unittest
-import shipgrav.utils as sgu
+
 import numpy as np
+import shipgrav.utils as sgu
 
 
 class utilsTestCase(unittest.TestCase):

shipgrav/utils.py

@@ -1,8 +1,8 @@
-import numpy as np
-from datetime import datetime
-from pandas import Series
 import re
 
+import numpy as np
+from pandas import DataFrame
+
 # TODO gaussian filter doesn't *need* time array
 
 
@@ -90,9 +90,9 @@ def decode_dgs_status_bits(stat, key='status'):
     flags = ['clamp status', 'GPSsync', 'reserved', 'feedback', 'R1', 'R2', 'ADlock', 'ack host',
              'NavMod1', 'NavMod2', 'dgs1_trouble', 'dgs2_trouble', 'GPStime', 'ADsat', 'nemo1', 'nemo2']
 
-    assert type(stat) == int or type(
-        stat) == DataFrame, 'bad input type for status bits'
-    if type(stat) == int:
+    assert type(stat) is int or type(
+        stat) is DataFrame, 'bad input type for status bits'
+    if type(stat) is int:
         bt = format(stat, '016b')
         out = {}
         for i, f in enumerate(flags):
@@ -114,6 +114,8 @@ def clean_ini_to_toml(ini_file):
     the extension .ini replaced by .toml
 
     :param ini_file: path to input ini file
+
+    :return: **opath** (*string*) - path to output toml file
     """
 
     with open(ini_file, 'r') as file:
@@ -132,11 +134,12 @@ def clean_ini_to_toml(ini_file):
     text = re.sub('$', '', text)
 
     # fix unquoted strings and write
-    fo = open(ini_file.rstrip('ini')+'toml', 'w')
+    opath = ini_file.rstrip('ini')+'toml'
+    fo = open(opath, 'w')
     lines = text.split('\n')
     for ln in lines:
         if ln.startswith('$'):
-            ln = re.sub('\$', '', ln)  # clean out $ for talkers
+            ln = re.sub(r'\$', '', ln)  # clean out $ for talkers
 
         if ln.startswith('[') or ln.startswith('#') or ln == '':
             fo.write(ln)
@@ -169,7 +172,7 @@ def clean_ini_to_toml(ini_file):
         fo.write('\n')
 
     fo.close()
-    return
+    return opath
 
 
 class _SnappingCursor:

{shipgrav-1.0.2.dist-info → shipgrav-1.0.4.dist-info}/METADATA

@@ -1,10 +1,9 @@
 Metadata-Version: 2.3
 Name: shipgrav
-Version: 1.0.2
+Version: 1.0.4
 Summary: Functions for marine gravity data processing and reduction
 Author-email: "Hannah F. Mark" <hmark@whoi.edu>
 Maintainer-email: "Hannah F. Mark" <hmark@whoi.edu>
-License-File: LICENSE
 Keywords: UNOLS,gravimetry,marine gravity
 Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
 Classifier: Operating System :: OS Independent
@@ -15,12 +14,18 @@ Requires-Dist: pyyaml
 Requires-Dist: scipy
 Requires-Dist: statsmodels
 Requires-Dist: tomli
+Requires-Dist: tqdm
 Provides-Extra: examples
+Requires-Dist: geographiclib; extra == 'examples'
+Requires-Dist: jupyterlab; extra == 'examples'
+Requires-Dist: jupytext; extra == 'examples'
 Requires-Dist: matplotlib; extra == 'examples'
+Requires-Dist: pooch; extra == 'examples'
 Description-Content-Type: text/markdown
 
 # shipgrav
 [![build status](https://github.com/PFPE/shipgrav/workflows/tests/badge.svg)](https://github.com/PFPE/shipgrav/actions)
+[![codecov](https://codecov.io/gh/PFPE/shipgrav/branch/main/graph/badge.svg)](https://codecov.io/gh/PFPE/shipgrav)
 
 shipgrav is a Python package designed for reading, processing, and reducing marine gravity data from UNOLS ships. It is created and maintained by PFPE for the marine gravimetry community. The shipgrav repository also contains scripts with example workflows for gravity data processing and reduction.
 
@@ -31,9 +36,10 @@ scipy\
 pandas 2.0+\
 statsmodels\
 tomli\
-pyyaml
+pyyaml\
+tqdm
 
-To run the example scripts, you will also need matplotlib.
+To run the example scripts, you will also need matplotlib, geographiclib, and pooch. To run the example scripts in jupyter, you will also need jupyterlab and jupytext.
 
 ## Installation
 shipgrav can be installed from PyPI using `pip`. Detailed instructions are in the [documentation](https://shipgrav.readthedocs.io/).
@@ -43,3 +49,7 @@ The shipgrav documentation is available online at [shipgrav.readthedocs.io](http
 
 ## Contributing to shipgrav
 Please do! If you have ideas for how to make shipgrav better, you can raise an issue on github or contact PFPE.
+
+If you raise an issue on github, please include as much detail as possible about any errors you are encountering or any proposed enhancements to the code. Include the text of any error messages, and if the issue is unexpected behavior from the code without any visible error messages, describe both what the code is doing and what you think it *should* be doing instead. PFPE may ask for additional details and/or copies of data files in order to reproduce and diagnose an issue.
+
+Additions or enhancements to the code are also welcome. Contributors are invited to fork the repository and submit pull requests for the maintainers to review.

shipgrav-1.0.4.dist-info/RECORD

@@ -0,0 +1,34 @@
+shipgrav/__init__.py,sha256=Kmg5sCx0ZHOs1bEPZeKLs5kmlrqjvbrcBAkV4G151ws,11874
+shipgrav/database.toml,sha256=8xSnXoAsfBMPDhvzKzw83oDrDT9AxD_QINZ8IWSUmiE,551
+shipgrav/grav.py,sha256=09d5LIczUGys9VerTngumOMLeYOqdVJVtAdBGJZw2R8,43196
+shipgrav/io.py,sha256=N2cqXRHIX_lSyCKJ35h9J4zcvs7tMKvSq0ch90xJncM,30448
+shipgrav/nav.py,sha256=3E2Il-PhTEPVv8_8PrLVe7h6slHq-2HYX_Q8PdpB6X4,2895
+shipgrav/utils.py,sha256=BJSBrKgCOADzdb6gBO3u9n16l22zzFXTAw42e5_QkBY,9582
+shipgrav/tests/__init__.py,sha256=i8hMukDivY8kFp7OBdo1byRJWMBvb8nDDWJcAy2cOOI,427
+shipgrav/tests/__main__.py,sha256=iJ8xSU8ucCmIDcOuv8uasZ9iV5dNR0DDuFRZrG0o2hE,38
+shipgrav/tests/test_grav_data.py,sha256=dmD49rb5p4GQumaldU0d6Ara-2L2-TjS4S7fXMXFFwo,3537
+shipgrav/tests/test_grav_nodata.py,sha256=skVHlimoyohxlDW7UG-sT_ASKOxs7sWakYm3HfkAoIE,2202
+shipgrav/tests/test_io.py,sha256=aiZTB2znmMFYUKgnMBHqjIty2MXlpyo8hqt7gI4_pSA,6141
+shipgrav/tests/test_nav.py,sha256=R2Fd_TIp_lOUjNtUcxvhZ5l-Vm1CmZ0-XzTcFiMFBRY,1195
+shipgrav/tests/test_utils.py,sha256=dz9T_hDBpPhhLj9Xv9U-QpeMC3MsGS3aGnAWIaUVJAc,970
+shipgrav/tests/ex_files/AT01_bgm.BGM,sha256=yaiVxd6vFeuoiKsUhMeTh22U2VOPPOR-UApr165Ua_I,184
+shipgrav/tests/ex_files/AT01_nav.gps,sha256=75q1E_z65N82UW33x1vcaXfIHIwY_7qq1alcD2LbahQ,722
+shipgrav/tests/ex_files/AT05_01_bgm.RGS,sha256=3MnlHJiIyzYe7KGr-bD8dfQ1nS_-Dt4A4g6TiVuNaBw,515
+shipgrav/tests/ex_files/DGStest_laptop.dat,sha256=kUwAMltUXm32Dv9BnaDakA-KtNAuMxgSC504F8_0aws,353351
+shipgrav/tests/ex_files/IXBlue.yaml,sha256=W-GGSMrSSvIXzX-b4rIU0u6Fv8bgNccZzNjTy-K1dNE,5394
+shipgrav/tests/ex_files/MGL2003_bgm.y2020d244,sha256=MdF9duKY138vuVcOzZ26XcpKPSrGp20MbWYkCUtXTIk,164
+shipgrav/tests/ex_files/MGL2003_nav.y2020d244,sha256=HheoWXYaevWkWE5uzNdv1vG5SHzh4AEog7UCu1O6Lw4,866
+shipgrav/tests/ex_files/NBP_2301_nav.d013,sha256=Q-68kTPwGvYJ3u-nED3rem9JMSjaFxz75dYaxxrIb7c,1251
+shipgrav/tests/ex_files/RR2212_bgm.txt,sha256=sB3Si6hhdZevjQrtoEcdztiX7Zf1H6g5GxRpZyXK-po,246
+shipgrav/tests/ex_files/RR2212_nav.txt,sha256=BzazwIVkgILjHffoaWEKrqxN-4oOv_zGWqRvx3cF6I8,857
+shipgrav/tests/ex_files/SR2302_nav.raw,sha256=7LhR1GEG17QDnTIp5NAamqjDteTLD2bTHaNfZrVFFh4,2352
+shipgrav/tests/ex_files/SR2312_dgs_raw.txt,sha256=rTlIn75MBICBjnkjcn9SPCwpC5J3lMs4aq7-y6-dAmU,368
+shipgrav/tests/ex_files/SR2312_mru.txt,sha256=YBM4qy4BS_D0U4ngEyoIFZr4QQbMWQgEO681fVEmgYM,1060
+shipgrav/tests/ex_files/TN400_bgm.Raw,sha256=DL7T2aJednjGLMKJo6K8KcUYLMjQCiv6bsUiczlVwKw,74
+shipgrav/tests/ex_files/TN400_dgs_proc.Raw,sha256=oxTXwDdyoz5yuKe253L1GgOSR5oWGGMuXfSQWrT24Ag,760
+shipgrav/tests/ex_files/TN400_dgs_raw.Raw,sha256=tb-zerwCH6kkbGDbnnJ6I-sAUdCHOvkKgXXrBb6zTj8,311
+shipgrav/tests/ex_files/TN400_nav.Raw,sha256=66dgRQTXjf0JItMWE1ZGUO6NptrGqV6JyN2wdYe5bW8,194
+shipgrav-1.0.4.dist-info/METADATA,sha256=4IrgRSFtaOZcoYB5KwR-Qn0ynZjLKg6rfPt3hPT0K-w,2855
+shipgrav-1.0.4.dist-info/WHEEL,sha256=aO3RJuuiFXItVSnAUEmQ0yRBvv9e1sbJh68PtuQkyAE,105
+shipgrav-1.0.4.dist-info/licenses/LICENSE,sha256=5X8cMguM-HmKfS_4Om-eBqM6A1hfbgZf6pfx2G24QFI,35150
+shipgrav-1.0.4.dist-info/RECORD,,

{shipgrav-1.0.2.dist-info → shipgrav-1.0.4.dist-info}/WHEEL

@@ -1,5 +1,5 @@
 Wheel-Version: 1.0
-Generator: hatchling 1.25.0
+Generator: hatchling 1.26.3
 Root-Is-Purelib: true
 Tag: py2-none-any
 Tag: py3-none-any

shipgrav-1.0.2.dist-info/RECORD

@@ -1,24 +0,0 @@
-shipgrav/__init__.py,sha256=hkdZdfpVRK4AIdSk99WM6tJaMYmYNMQ6TU11qk2thDY,11058
-shipgrav/database.toml,sha256=qRSZBRsoMsbyhjkaYlXsWtVDz3JAhoW-cYfSoo8U3K4,532
-shipgrav/grav.py,sha256=OQK2HNiZtLlycjVwidpD902K08NaWbZXnjZSbjyPF8I,43410
-shipgrav/io.py,sha256=j9uy5z-CqhkUN4T-3iW5o5-cRLgjoGwfNc4o7cjw_pA,29273
-shipgrav/nav.py,sha256=3E2Il-PhTEPVv8_8PrLVe7h6slHq-2HYX_Q8PdpB6X4,2895
-shipgrav/utils.py,sha256=h5gd8cnWlO1pyAqLkJIVrbFRRLbZzHj-xJ9pWchaHtI,9521
-shipgrav/tests/__init__.py,sha256=mejp00TQif0WLU45j40zfBQXN2RujaOP6ZSTgxUm4A8,367
-shipgrav/tests/__main__.py,sha256=iJ8xSU8ucCmIDcOuv8uasZ9iV5dNR0DDuFRZrG0o2hE,38
-shipgrav/tests/test_grav_data.py,sha256=OnPNW8XIG3DvWOr_trQY71XLzA0rff8VfGd4Az3B37Q,3403
-shipgrav/tests/test_grav_nodata.py,sha256=0z5KUTzT9cJPko_bhyDG4EFm4i5xgOdVhJVKU2cPxX8,1493
-shipgrav/tests/test_io.py,sha256=RBRB2GYtesuSv68lJl5yMOLH2DGFz2AAKADyPKq8Dhk,2259
-shipgrav/tests/test_nav.py,sha256=6Ndn1mHN8kGruwk3R5yJFiK0n9Wt3XcK0GQ9PGbRIP8,1194
-shipgrav/tests/test_utils.py,sha256=10brOKgB5ij-oXqQnfo3CkMmZxSQMxSr_KVLLN1g1u0,969
-shipgrav/tests/ex_files/AT05_01_bgm.RGS,sha256=3MnlHJiIyzYe7KGr-bD8dfQ1nS_-Dt4A4g6TiVuNaBw,515
-shipgrav/tests/ex_files/DGStest_laptop.dat,sha256=kUwAMltUXm32Dv9BnaDakA-KtNAuMxgSC504F8_0aws,353351
-shipgrav/tests/ex_files/IXBlue.yaml,sha256=W-GGSMrSSvIXzX-b4rIU0u6Fv8bgNccZzNjTy-K1dNE,5394
-shipgrav/tests/ex_files/SR2312_dgs_raw.txt,sha256=rTlIn75MBICBjnkjcn9SPCwpC5J3lMs4aq7-y6-dAmU,368
-shipgrav/tests/ex_files/SR2312_mru.txt,sha256=YBM4qy4BS_D0U4ngEyoIFZr4QQbMWQgEO681fVEmgYM,1060
-shipgrav/tests/ex_files/TN400_bgm.Raw,sha256=DL7T2aJednjGLMKJo6K8KcUYLMjQCiv6bsUiczlVwKw,74
-shipgrav/tests/ex_files/TN400_nav.Raw,sha256=66dgRQTXjf0JItMWE1ZGUO6NptrGqV6JyN2wdYe5bW8,194
-shipgrav-1.0.2.dist-info/METADATA,sha256=aDE--gWj6U7q4-I37DIoosd4Tw6q2OE3QNqd-A9d8Og,1801
-shipgrav-1.0.2.dist-info/WHEEL,sha256=fl6v0VwpzfGBVsGtkAkhILUlJxROXbA3HvRL6Fe3140,105
-shipgrav-1.0.2.dist-info/licenses/LICENSE,sha256=5X8cMguM-HmKfS_4Om-eBqM6A1hfbgZf6pfx2G24QFI,35150
-shipgrav-1.0.2.dist-info/RECORD,,