shipgrav 1.0.0__tar.gz → 1.0.3__tar.gz

{shipgrav-1.0.0 → shipgrav-1.0.3}/.github/workflows/test.yml

@@ -1,7 +1,7 @@
 name: tests
 on:
-  #push:
-  workflow_dispatch:
+  push:
+  #workflow_dispatch:
   pull_request:
     branches:
       # only branches from forks which have the form 'user:branch-name'
@@ -13,29 +13,10 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        os: [ubuntu-latest]
-        python: [3.9, "3.10"]
+        os: [ubuntu-latest, macos-latest, windows-latest]
+        python: [3.9, "3.10","3.13"]
         add_package: [""]
         continue-on-error: [false]
-        include:
-          - os: macos-latest
-            python: 3.9
-            continue-on-error: false
-          - os: windows-latest
-            python: 3.9
-            continue-on-error: false
-          - os: ubuntu-latest
-            python: 3.9
-            add_package: features
-            continue-on-error: true
-          - os: macos-latest
-            python: 3.9
-            add_package: features
-            continue-on-error: true
-          - os: windows-latest
-            python: 3.9
-            add_package: features
-            continue-on-error: true
     runs-on: ${{ matrix.os }}
     continue-on-error: ${{ matrix.continue-on-error }}
     defaults:
@@ -63,6 +44,6 @@ jobs:
           coverage run --include='*/shipgrav/*' -m shipgrav.tests
           coverage xml -o ../coverage.xml
       - name: upload coverage
-        uses: codecov/codecov-action@v3
+        uses: codecov/codecov-action@v4
         env:
           CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

shipgrav-1.0.3/CHANGELOG

@@ -0,0 +1,25 @@
+2024-10-25  Hannah F Mark  <hmark@whoi.edu>
+
+    * update pkg_resources (deprecated from 3.13) to importlib.resources for test suite
+    * fix download_data.sh curl by adding -L to follow R2R links
+    * fix SyntaxWarning for regular expressions in io.py by making raw strings
+    * fix DeprecationWarning for parse_dates with nested sequences, sub in pd.to_datetime
+    * fix FutureWarning for to_numeric, apply by column instead of whole DataFrame
+    * add geographiclib as dependency for RMBA example script (used for line extension)
+    * set tests to run on push (keeping scheduled ones too)
+    * add Python 3.13 to the test matrix, simplify the os matrix logic
+    * update codecov action to v4 for nodejs version
+    * update CONTRIBUTORS -> to markdown, with info on who did what prior to github upload
+    * fix indexing in one part of RMBA example script for future deprecation of chained assignment
+    * add detail to contributor guidelines
+
+2024-07-15  Hannah F Mark  <hmark@whoi.edu>
+
+    * PEP8 reformat, typos
+    * include some 2D functions for Parker method, rename the 1D one
+    * repo is now in PFPE org instead of WHOIGit
+
+2024-04-23  Hannah F Mark  <hmark@whoi.edu>
+
+    * docs/conf.py, readme.md, shipgrav/__init__.py: updated repo URLS
+    after repo tranfered to WHOIGit

shipgrav-1.0.3/CONTRIBUTORS.md

@@ -0,0 +1,27 @@
+# Contributors
+Contributors to shipgrav are listed and described using the [Contributor Role Taxonomoy (CRediT)](https://credit.niso.org/)
+
+## Hannah F. Mark
+- Conceptualization
+- Data curation
+- Software
+- Visualization
+- Writing -- original draft (JOSS)
+- Writing -- review and editing (JOSS)
+
+## Masako Tominaga
+- Conceptualization
+- Funding acquisition
+- Supervision
+- Validation
+- Writing -- review and editing (JOSS)
+
+## Jasmine Zhu
+- Software
+
+## Maurice Tivey
+- Funding acquisition
+- Writing -- review and editing (JOSS)
+
+## Daniel Aliod
+- Software

{shipgrav-1.0.0 → shipgrav-1.0.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.3
 Name: shipgrav
-Version: 1.0.0
+Version: 1.0.3
 Summary: Functions for marine gravity data processing and reduction
 Author-email: "Hannah F. Mark" <hmark@whoi.edu>
 Maintainer-email: "Hannah F. Mark" <hmark@whoi.edu>
@@ -9,13 +9,15 @@ Keywords: UNOLS,gravimetry,marine gravity
 Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
 Classifier: Operating System :: OS Independent
 Classifier: Programming Language :: Python :: 3
-Requires-Dist: matplotlib
 Requires-Dist: numpy
 Requires-Dist: pandas
 Requires-Dist: pyyaml
 Requires-Dist: scipy
 Requires-Dist: statsmodels
 Requires-Dist: tomli
+Provides-Extra: examples
+Requires-Dist: geographiclib; extra == 'examples'
+Requires-Dist: matplotlib; extra == 'examples'
 Description-Content-Type: text/markdown
 
 # shipgrav
@@ -35,10 +37,14 @@ pyyaml
 To run the example scripts, you will also need matplotlib.
 
 ## Installation
-shipgrav can be installed using `pip`. Detailed instructions are in the [documentation](https://shipgrav.readthedocs.io/).
+shipgrav can be installed from PyPI using `pip`. Detailed instructions are in the [documentation](https://shipgrav.readthedocs.io/).
 
 ## Documentation and usage
 The shipgrav documentation is available online at [shipgrav.readthedocs.io](https://shipgrav.readthedocs.io/). It can also be accessed offline by building the contents of the `docs` folder using sphinx.
 
 ## Contributing to shipgrav
 Please do! If you have ideas for how to make shipgrav better, you can raise an issue on github or contact PFPE.
+
+If you raise an issue on github, please include as much detail as possible about any errors you are encountering or any proposed enhancements to the code. Include the text of any error messages, and if the issue is unexpected behavior from the code without any visible error messages, describe both what the code is doing and what you think it *should* be doing instead. PFPE may ask for additional details and/or copies of data files in order to reproduce and diagnose an issue.
+
+Additions or enhancements to the code are also welcome. Contributors are invited to fork the repository and submit pull requests for the maintainers to review.

{shipgrav-1.0.0 → shipgrav-1.0.3}/example-scripts/RMBA_calc.py

@@ -142,7 +142,7 @@ B = firwin(taps, wn, window='blackman')  # approx equivalent to matlab fir1
 ffaa = filtfilt(B, 1, data['faa'])
 
 # embed our FAA in the tracked line (everything else remains tracked)
-track['faa'].iloc[n_ext+taps:-n_ext-taps] = ffaa[taps:-taps]
+track.loc[n_ext+taps:n_ext+taps+len(ffaa[taps:-taps])-1,'faa'] = ffaa[taps:-taps]
 # NOTE you may want to check your embedded FAA for large jumps at the ends
 # and smooth them out into the gridded data. Not bothering here because
 # the jumps aren't *too* big* and also we don't care that much about the

{shipgrav-1.0.0 → shipgrav-1.0.3}/example-scripts/download_data.sh

@@ -41,9 +41,9 @@ cd temp
 # TN400 data: BGM3 and nav
 # https://doi.org/10.7284/151470 (grav)
 # https://doi.org/10.7284/151457 (nav)
-curl -X GET https://service.rvdata.us/data/cruise/TN400/fileset/151470 --output TN400-grav.tar.gz
+curl -L -X GET https://service.rvdata.us/data/cruise/TN400/fileset/151470 --output TN400-grav.tar.gz
 tar -xvzf TN400-grav.tar.gz
-curl -X GET https://service.rvdata.us/data/cruise/TN400/fileset/151457 --output TN400-nav.tar.gz
+curl -L -X GET https://service.rvdata.us/data/cruise/TN400/fileset/151457 --output TN400-nav.tar.gz
 tar -xvzf TN400-nav.tar.gz
 for e in 20220313 20220314; do
 mv TN400/151470/data/*$e* ../data/Thompson/TN400/gravimeter/BGM3/serial
@@ -58,11 +58,11 @@ rm TN400-nav.tar.gz
 # https://doi.org/10.7284/157179 (grav)
 # https://doi.org/10.7284/157188 (nav)
 # https://doi.org/10.7284/157177 (mru)
-curl -X GET https://service.rvdata.us/data/cruise/SR2312/fileset/157179 --output SR2312-grav.tar.gz
+curl -L -X GET https://service.rvdata.us/data/cruise/SR2312/fileset/157179 --output SR2312-grav.tar.gz
 tar -xvzf SR2312-grav.tar.gz
-curl -X GET https://service.rvdata.us/data/cruise/SR2312/fileset/157188 --output SR2312-nav.tar.gz
+curl -L -X GET https://service.rvdata.us/data/cruise/SR2312/fileset/157188 --output SR2312-nav.tar.gz
 tar -xvzf SR2312-nav.tar.gz
-curl -X GET https://service.rvdata.us/data/cruise/SR2312/fileset/157177 --output SR2312-mru.tar.gz
+curl -L -X GET https://service.rvdata.us/data/cruise/SR2312/fileset/157177 --output SR2312-mru.tar.gz
 tar -xvzf SR2312-mru.tar.gz
 for e in 16 17 18 19 20; do
 mv SR2312/157179/data/*_202306$e*.dat ../data/Ride/SR2312/gravimeter/DGS

{shipgrav-1.0.0 → shipgrav-1.0.3}/joss/paper.md

@@ -38,7 +38,7 @@ Here we present a Python package implementing the calibrations and corrections c
 
 # Statement of need
 
-`shipgrav` is a Python package for marine gravimetry, designed as a tool for the marine geophysical research community. Marine gravity data is routinely collected on UNOLS vessels and archived for public access (after an embargo period) through the [Rolling Deck to Repository](https://www.rvdata.us) program, but researchers who work with this data have traditionally used their own personal processing tools for data analysis. One major goal of `shipgrav` is to make marine gravity data more accessible to the broader community, including researchers who want to incorporate gravity into other geophysical analyses without re-writing implementations of the standard workflow. The package includes a suite of example scripts demonstrating all the steps for marine gravity processing using publicly available data. These examples can be used as teaching tools, and as a starting framework for research applications.
+`shipgrav` is a Python package for marine gravimetry, designed as a tool for the marine geophysical research community. Marine gravity data is routinely collected on UNOLS vessels and archived for public access (after an embargo period) through the [Rolling Deck to Repository](https://www.rvdata.us) program, but researchers who work with this data have traditionally used their own personal processing tools for data analysis. There are some public software tools that handle parts of the processing workflow, such as @jrleeman's [Longman tide correction](https://github.com/jrleeman/LongmanTide) package and a [MATLAB toolkit](https://github.com/MAG-tominaga/DgSGravCode) that does some AT1M data processing, but the authors are not aware of any complete processing workflows for this type of data that are publicly available and entirely open-source. One major goal of `shipgrav` is to make marine gravity data more accessible to the broader community, including researchers who want to incorporate gravity into other geophysical analyses without re-writing implementations of the standard workflow or needing a MATLAB license. The package includes a suite of example scripts demonstrating all the steps for marine gravity processing using publicly available data. These examples can be used as teaching tools, and as a starting framework for research applications.
 
 `shipgrav` includes a set of vessel-specific input functions which take into account the differences in file formats between different instruments and operators. While shipboard gravimeters are commonly configured to output the same or similar sets of data fields in ASCII files, there is not a standard data string format. Known formats for gravimeters on UNOLS vessels are built into `shipgrav` so that users can easily read in data. The framework is designed so that users can plug in their own functions to read and parse files for vessels or formats that are not yet included. There are also functions for reading separately recorded navigation data and synchronizing geographic coordinates with gravimeter records based on timestamps, which is necessary for integrating gravity data with other geophysical analyses.
 
@@ -57,6 +57,6 @@ Figure sizes can be customized by adding an optional second parameter:
 
 # Acknowledgements
 
-The development of shipgrav was supported by the Potential Field Pool Equipment (PFPE) facility, which is funded by the National Science Foundation under grants OCE-2234277 and OCE-1824508; and by the Office of Naval Research under DURIP grant N00014-23-1-2475. Functions for calculating thermal structure were adapted from scripts provided by Mark Behn.
+The development of shipgrav was supported by the Potential Field Pool Equipment (PFPE) facility, which is funded by the National Science Foundation under grants OCE-2234277 and OCE-1824508; and by the Office of Naval Research under DURIP grant N00014-23-1-2475. Functions for calculating thermal structure were adapted from scripts provided by Mark Behn. Nigel Brady at DGS helpfully answered many questions about the AT1M system.
 
 # References

{shipgrav-1.0.0 → shipgrav-1.0.3}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "shipgrav"
-version = "1.0.0"
+version = "1.0.3"
 authors = [
     {name = "Hannah F. Mark", email = "hmark@whoi.edu"},
 ]
@@ -24,7 +24,9 @@ dependencies = [
     "pandas",
     "scipy",
     "tomli",
-    "matplotlib",
     "statsmodels",
     "pyyaml"
 ]
+
+[project.optional-dependencies]
+examples = ["matplotlib", "geographiclib"]

{shipgrav-1.0.0 → shipgrav-1.0.3}/readme.md

@@ -15,10 +15,14 @@ pyyaml
 To run the example scripts, you will also need matplotlib.
 
 ## Installation
-shipgrav can be installed using `pip`. Detailed instructions are in the [documentation](https://shipgrav.readthedocs.io/).
+shipgrav can be installed from PyPI using `pip`. Detailed instructions are in the [documentation](https://shipgrav.readthedocs.io/).
 
 ## Documentation and usage
 The shipgrav documentation is available online at [shipgrav.readthedocs.io](https://shipgrav.readthedocs.io/). It can also be accessed offline by building the contents of the `docs` folder using sphinx.
 
 ## Contributing to shipgrav
 Please do! If you have ideas for how to make shipgrav better, you can raise an issue on github or contact PFPE.
+
+If you raise an issue on github, please include as much detail as possible about any errors you are encountering or any proposed enhancements to the code. Include the text of any error messages, and if the issue is unexpected behavior from the code without any visible error messages, describe both what the code is doing and what you think it *should* be doing instead. PFPE may ask for additional details and/or copies of data files in order to reproduce and diagnose an issue.
+
+Additions or enhancements to the code are also welcome. Contributors are invited to fork the repository and submit pull requests for the maintainers to review.

{shipgrav-1.0.0 → shipgrav-1.0.3}/shipgrav/__init__.py

@@ -9,7 +9,11 @@ DGS gravimeters output two types of files: serial, or 'raw' files; and 'laptop'
 Installation
 ------------
 
-shipgrav can be installed from the source files (`available on github <https://github.com/PFPE/shipgrav>`_) using ``pip``.
+shipgrav can be installed from `PyPI <https://pypi.org/project/shipgrav/>`_ using ``pip``. we recommend using an environment management tool like `conda <https://anaconda.org>`_. An exemplary set of commands to make a conda enviroment with shipgrav would be: ::
+
+    conda create --name shipgrav numpy scipy pandas statsmodels tomli pyyaml matplotlib geographiclib
+    conda activate shipgrav
+    pip install shipgrav
 
 shipgrav's dependencies are
 
@@ -21,15 +25,9 @@ shipgrav's dependencies are
 * tomli
 * pyyaml
 * matplotlib (optional, needed to run some of the example scripts)
+* geographiclib (optional, needed to run one of the example scripts)
 
-To install shipgrav, we recommend using an environment management tool like `conda <https://anaconda.org>`_. An exemplary set of commands to make a conda enviroment with shipgrav would be: ::
-
-    conda create --name shipgrav numpy scipy pandas statsmodels tomli pyyaml matplotlib
-    conda activate shipgrav
-    cd /path/to/put/source/files
-    git clone https://github.com/PFPE/shipgrav.git
-    cd shipgrav/
-    pip install .
+The example scripts are `available on github <https://github.com/PFPE/shipgrav>`_. They are not packaged with the PyPI package and must be downloaded separately.
 
 Modules and files
 -----------------
@@ -144,4 +142,6 @@ Contributing
 
 Do you have ideas for making this software better? Go ahead and `raise an issue <https://github.com/PFPE/shipgrav/issues>`_ on the github page or, if you're a savvy Python programmer, submit a pull request. You can also email PFPE at pfpe-interal(at)whoi.edu.
 
+If you raise an issue on github, please include as much detail as possible, such as the text of error messages. If there are no visible errors but you think the code is behaving oddly, provide a description of what the code is doing and what you think it *should* be doing instead. PFPE may ask for additional details or copies of data files in order to reproduce and diagnose an issue.
+
 """

{shipgrav-1.0.0 → shipgrav-1.0.3}/shipgrav/io.py

@@ -169,7 +169,7 @@ def _navdate_Atlantis(allnav, talker):
 
     for i in range(N):
         pre = subnav[i].split(talker)[0]
-        date = re.findall('NAV (\d{4})/(\d{2})/(\d{2})', pre)[0]
+        date = re.findall(r'NAV (\d{4})/(\d{2})/(\d{2})', pre)[0]
         year = int(date[0])  # year
         mon = int(date[1])  # month
         day = int(date[2])  # day
@@ -191,7 +191,7 @@ def _navdate_NBP(allnav, talker):
 
     for i in range(N):
         pre = subnav[i].split(talker)[0]
-        date = re.findall('(\d{2})\+(\d{2,3}):.*', pre)[0]
+        date = re.findall(r'(\d{2})\+(\d{2,3}):.*', pre)[0]
         # year (NBP didn't exist before 2000 so this is ok)
         year = '20' + date[0]
         doy = date[1]  # doy
@@ -216,7 +216,7 @@ def _navdate_Thompson(allnav, talker):
 
     for i in range(N):
         pre = subnav[i].split(talker)[0]
-        date = re.findall('(\d{2})/(\d{2})/(\d{4}),*', pre)[0]
+        date = re.findall(r'(\d{2})/(\d{2})/(\d{4}),*', pre)[0]
         year = int(date[2])
         mon = int(date[0])
         day = int(date[1])
@@ -247,8 +247,8 @@ def _navdate_Revelle(allnav, talker):
         for k in range(inds[i]-1, j, -1):  # step backwards toward the last talker line
             before = allnav[k]
             # date is at the start of this line
-            if re.match('(\d{4})-(\d{2})-(\d{2})T*', before):
-                date = re.findall('(\d{4})-(\d{2})-(\d{2})T*', before)[0]
+            if re.match(r'(\d{4})-(\d{2})-(\d{2})T*', before):
+                date = re.findall(r'(\d{4})-(\d{2})-(\d{2})T*', before)[0]
                 year = int(date[0])
                 mon = int(date[1])
                 day = int(date[2])
@@ -269,12 +269,12 @@ def _navdate_Ride(allnav, talker):
 
     for i in range(N):
         if talker == 'INGGA':  # on Ride, uses posix timestamps
-            date = re.findall('(\d+(\.\d*)?) \$%s' % talker, subnav[i])[0]
+            date = re.findall(r'(\d+(\.\d*)?) \$%s' % talker, subnav[i])[0]
             timest[i] = datetime.fromtimestamp(
                 float(date[0]), tzinfo=timezone.utc)
         elif talker == 'GPGGA':  # includes time only with date, unlike other GPGGAs
             date = re.findall(
-                '(\d{4})\-(\d{2})\-(\d{2})T(\d{2}):(\d{2}):(\d{2})\.(\d.*?)Z', subnav[i])[0]
+                r'(\d{4})\-(\d{2})\-(\d{2})T(\d{2}):(\d{2}):(\d{2})\.(\d.*?)Z', subnav[i])[0]
             year = int(date[0])
             mon = int(date[1])
             day = int(date[2])
@@ -383,9 +383,8 @@ def read_bgm_rgs(fp, ship):
     dats = []
     for path in fp:
         dat = pd.read_csv(path, delimiter=' ', names=['date', 'time', 'grav', 'lat', 'lon'],
-                          usecols=(1, 2, 3, 11, 12), parse_dates=[[0, 1]])
-        ndt = [e.tz_localize(timezone.utc) for e in dat['date_time']]
-        dat['date_time'] = ndt
+                          usecols=(1, 2, 3, 11, 12))
+        dat['date_time'] = pd.to_datetime(dat.pop('date')+' '+dat.pop('time'),utc=True)
         dats.append(dat)
 
     return pd.concat(dats, ignore_index=True)
@@ -456,10 +455,8 @@ def _bgmserial_Atlantis(path):
     def count(x): return (
         int(x.split(':')[-1]))  # function to parse counts column
     dat = pd.read_csv(path, delimiter=' ', names=['date', 'time', 'counts'], usecols=(1, 2, 4),
-                      parse_dates=[[0, 1]], converters={'counts': count})
-    ndt = [e.tz_localize(timezone.utc)
-           for e in dat['date_time']]  # timestamps cannot be naive
-    dat['date_time'] = ndt
+                      converters={'counts': count})
+    dat['date_time'] = pd.to_datetime(dat.pop('date')+' '+dat.pop('time'),utc=True)
     return dat
 
 
@@ -468,9 +465,8 @@ def _bgmserial_Thompson(path):
     """
     def count(x): return (int(x.split(' ')[0].split(':')[-1]))
     dat = pd.read_csv(path, delimiter=',', names=['date', 'time', 'counts'],
-                      parse_dates=[[0, 1]], converters={'counts': count})
-    ndt = [e.tz_localize(timezone.utc) for e in dat['date_time']]
-    dat['date_time'] = ndt
+                    converters={'counts': count})
+    dat['date_time'] = pd.to_datetime(dat.pop('date')+' '+dat.pop('time'),utc=True)
     return dat
 
 
@@ -575,10 +571,8 @@ def _dgs_laptop_Thompson(path):
     """
     dat = pd.read_csv(path, delimiter=',', names=['date', 'time', 'rgrav', 've', 'vcc',
                                                   'al', 'ax', 'lat', 'lon'],
-                      usecols=(0, 1, 3, 12, 13, 14, 15, 16, 17),
-                      parse_dates=[[0, 1]])
-    ndt = [e.tz_localize(timezone.utc) for e in dat['date_time']]
-    dat['date_time'] = ndt
+                      usecols=(0, 1, 3, 12, 13, 14, 15, 16, 17))
+    dat['date_time'] = pd.to_datetime(dat.pop('date')+' '+dat.pop('time'),utc=True)
     return dat
 
 
@@ -746,4 +740,13 @@ def read_other_stuff(yaml_file, data_file, tag):
         except KeyError:
             pass
 
-    return df.apply(pd.to_numeric, errors='ignore'), col_info
+    def numeric(col):
+        try:
+            return pd.to_numeric(col)
+        except:
+            return col
+
+    for ckey in df.columns:     # to_numeric by column so errors for datetimes are skipped
+        df[ckey] = df[ckey].apply(numeric)  # slower than applying to whole df, but avoids errors
+
+    return df, col_info

{shipgrav-1.0.0 → shipgrav-1.0.3}/shipgrav/tests/__init__.py

@@ -2,15 +2,16 @@
 Tests for shipgrav
 """
 
-from pkg_resources import resource_filename
 import sys
 import unittest
+import importlib.resources as importlib_resources
 
 
 def run():
     loader = unittest.TestLoader()
-    test_dir = resource_filename('shipgrav', 'tests')
-    suite = loader.discover(test_dir)
+    ref = importlib_resources.files('shipgrav') / 'tests'
+    with importlib_resources.as_file(ref) as path:
+        suite = loader.discover(ref)
     runner = unittest.runner.TextTestRunner()  # verbosity=2)
     ret = not runner.run(suite).wasSuccessful()
     sys.exit(ret)

shipgrav-1.0.0/CHANGELOG

@@ -1,10 +0,0 @@
-2024-07-15  Hannah F Mark  <hmark@whoi.edu>
-
-    * PEP8 reformat, typos
-    * include some 2D functions for Parker method, rename the 1D one
-    * repo is now in PFPE org instead of WHOIGit
-
-2024-04-23  Hannah F Mark  <hmark@whoi.edu>
-
-    * docs/conf.py, readme.md, shipgrav/__init__.py: updated repo URLS
-    after repo tranfered to WHOIGit

shipgrav-1.0.0/CONTRIBUTORS

@@ -1 +0,0 @@
-Mark, Hannah F.