GLODAP 0.3__tar.gz → 1.0.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,24 +1,25 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: glodap
3
- Version: 0.3
3
+ Version: 1.0.0
4
4
  Summary: GLODAP downloader and importer in Python
5
- Classifier: Development Status :: 4 - Beta
6
- Classifier: Programming Language :: Python :: 3.10
5
+ Classifier: Development Status :: 5 - Production/Stable
7
6
  Classifier: Programming Language :: Python :: 3.11
8
7
  Classifier: Programming Language :: Python :: 3.12
9
8
  Classifier: Programming Language :: Python :: 3.13
9
+ Classifier: Programming Language :: Python :: 3.14
10
+ Classifier: Programming Language :: Python :: 3.15
10
11
  Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
11
12
  Classifier: Operating System :: OS Independent
12
13
  Classifier: Natural Language :: English
13
14
  Classifier: Intended Audience :: Science/Research
14
15
  Classifier: Topic :: Scientific/Engineering :: Chemistry
15
16
  Classifier: Topic :: Scientific/Engineering :: Oceanography
16
- Requires-Python: >=3.10
17
+ Requires-Python: >=3.11
17
18
  Description-Content-Type: text/markdown
18
19
  License-File: LICENSE
19
- Requires-Dist: pandas
20
- Requires-Dist: requests
21
- Requires-Dist: scipy
20
+ Requires-Dist: pandas<4,>=3
21
+ Requires-Dist: pooch<2,>=1.9.0
22
+ Requires-Dist: scipy<2,>=1.17
22
23
  Dynamic: license-file
23
24
 
24
25
  # glodap
@@ -26,6 +27,7 @@ Dynamic: license-file
26
27
  [![Tests](https://github.com/mvdh7/glodap/actions/workflows/tests.yml/badge.svg?branch=main)](https://github.com/mvdh7/glodap/actions)
27
28
  [![pypi badge](https://img.shields.io/pypi/v/glodap.svg?style=popout)](https://pypi.org/project/glodap/)
28
29
  [![Conda Version](https://img.shields.io/conda/vn/conda-forge/glodap.svg)](https://anaconda.org/conda-forge/glodap)
30
+ [![DOI](https://img.shields.io/badge/DOI-10.5281%2Fzenodo.21295574-informational)](https://doi.org/10.5281/zenodo.21295574)
29
31
  [![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0)
30
32
  [![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
31
33
 
@@ -48,29 +50,48 @@ df_atlantic = glodap.atlantic()
48
50
  Files are saved by default at `"~/.glodap"`, but this can be controlled with the
49
51
  kwarg `gpath`. See the function docstrings for more information.
50
52
 
51
- The columns of the imported DataFrames can be passed
52
- directly into [PyCO2SYS v2](https://mvdh.xyz/PyCO2SYS):
53
+ The columns are the same as in the original GLODAP .mat files available from [glodap.info](https://glodap.info), except:
54
+ * The `"G2"` at the start of each parameter has been removed.
55
+ * Flags end with `"_f"` instead of just `"f"`.
56
+ * There is a `"datetime"` column, which combines the `"year"`, `"month"` and `"day"` but NOT the `"hour"` and `"minute"` (because some of these are missing).
57
+
58
+ The functions `download` and `read` can also be used for finer control, such as
59
+ specifying a particular GLODAP version rather than using the latest one (currently `v3.2026`). See
60
+ their function docstrings for more information.
61
+
62
+ The files from the GEOMAR mirrors are downloaded (v2: .mat files; v3: .csv.zip files), and the SHA256 checksum
63
+ of each downloaded file is checked before the file is written to disk.
64
+
65
+ ### With PyCO2SYS
66
+
67
+ The columns of the imported DataFrames can be passed directly into [PyCO2SYS v2](https://mvdh.xyz/PyCO2SYS):
53
68
 
54
69
  ```python
55
70
  import PyCO2SYS as pyco2
56
- co2s_atlantic = pyco2.sys(data=df_atlantic, nitrite=None)
71
+ co2s_atlantic = pyco2.sys(data=df_atlantic.drop(columns="fco2"), nitrite=0)
57
72
  ```
58
73
 
59
- Note `nitrite=None` - this means PyCO2SYS will ignore the `"nitrite"` column,
74
+ Note `nitrite=0` - this means PyCO2SYS will ignore the `"nitrite"` column,
60
75
  which is necessary because while PyCO2SYS includes the nitrite-nitrous acid
61
76
  equilibrium, its equilibrium constant is valid only under lab conditions.
62
77
 
63
- Because of how the columns are named, when passing the DataFrame directly to
64
- PyCO2SYS as above, the system will be solved from DIC and alkalinity, not pH.
78
+ Because of how the columns are named, and because we dropped the `"fco2"` column, when passing the DataFrame directly to
79
+ PyCO2SYS as above, the system will be solved from DIC and alkalinity, not pH nor fCO<sub>2</sub>.
65
80
 
66
- The columns are the same as in the original GLODAP .mat files available from [glodap.info](https://glodap.info), except:
67
- * The `"G2"` at the start of each parameter has been removed.
68
- * Flags end with `"_f"` instead of just `"f"`.
69
- * There is a `"datetime"` column, which combines the `"year"`, `"month"` and `"day"` but NOT the `"hour"` and `"minute"` (because some of these are missing).
81
+ ## Cite
70
82
 
71
- The functions `download` and `read` can also be used for finer control, such as
72
- specifying a particular GLODAP version rather than using the latest one. See
73
- their function docstrings for more information.
83
+ If you use GLODAP data, please cite the paper corresponding to the version you used:
74
84
 
75
- The .mat files from the GEOMAR mirrors are downloaded, and the SHA256 checksum
76
- of each downloaded file is checked before the file is written to disk.
85
+ * `v3.2026`: TBC (submitted to *Earth System Science Data*)
86
+ * `v2.2023`: https://doi.org/10.5194/essd-16-2047-2024
87
+ * `v2.2022`: https://doi.org/10.5194/essd-14-5543-2022
88
+ * `v2.2021`: https://doi.org/10.5194/essd-13-5565-2021
89
+ * `v2.2020`: https://doi.org/10.5194/essd-12-3653-2020
90
+ * `v2.2019`: https://doi.org/10.5194/essd-11-1437-2019
91
+ * `v2.2016`: https://doi.org/10.5194/essd-8-297-2016
92
+
93
+ If your analysis leans heavily on a specific subset of GLODAP, please consider inviting the PIs of the relevant datasets to be contribute as co-authors.
94
+
95
+ If you would also like to cite this tool directly:
96
+
97
+ > Humphreys, M. P. (2026). GLODAP downloader and importer in Python [Computer software]. *Zenodo.* https://doi.org/10.5281/zenodo.21295574
@@ -12,6 +12,6 @@ glodap.egg-info/SOURCES.txt
12
12
  glodap.egg-info/dependency_links.txt
13
13
  glodap.egg-info/requires.txt
14
14
  glodap.egg-info/top_level.txt
15
- tests/test_import.py
15
+ tests/test_download_and_read.py
16
16
  tests/test_regions.py
17
- tests/test_response.py
17
+ tests/test_versions.py
@@ -0,0 +1,3 @@
1
+ pandas<4,>=3
2
+ pooch<2,>=1.9.0
3
+ scipy<2,>=1.17
@@ -1,24 +1,25 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: glodap
3
- Version: 0.3
3
+ Version: 1.0.0
4
4
  Summary: GLODAP downloader and importer in Python
5
- Classifier: Development Status :: 4 - Beta
6
- Classifier: Programming Language :: Python :: 3.10
5
+ Classifier: Development Status :: 5 - Production/Stable
7
6
  Classifier: Programming Language :: Python :: 3.11
8
7
  Classifier: Programming Language :: Python :: 3.12
9
8
  Classifier: Programming Language :: Python :: 3.13
9
+ Classifier: Programming Language :: Python :: 3.14
10
+ Classifier: Programming Language :: Python :: 3.15
10
11
  Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
11
12
  Classifier: Operating System :: OS Independent
12
13
  Classifier: Natural Language :: English
13
14
  Classifier: Intended Audience :: Science/Research
14
15
  Classifier: Topic :: Scientific/Engineering :: Chemistry
15
16
  Classifier: Topic :: Scientific/Engineering :: Oceanography
16
- Requires-Python: >=3.10
17
+ Requires-Python: >=3.11
17
18
  Description-Content-Type: text/markdown
18
19
  License-File: LICENSE
19
- Requires-Dist: pandas
20
- Requires-Dist: requests
21
- Requires-Dist: scipy
20
+ Requires-Dist: pandas<4,>=3
21
+ Requires-Dist: pooch<2,>=1.9.0
22
+ Requires-Dist: scipy<2,>=1.17
22
23
  Dynamic: license-file
23
24
 
24
25
  # glodap
@@ -26,6 +27,7 @@ Dynamic: license-file
26
27
  [![Tests](https://github.com/mvdh7/glodap/actions/workflows/tests.yml/badge.svg?branch=main)](https://github.com/mvdh7/glodap/actions)
27
28
  [![pypi badge](https://img.shields.io/pypi/v/glodap.svg?style=popout)](https://pypi.org/project/glodap/)
28
29
  [![Conda Version](https://img.shields.io/conda/vn/conda-forge/glodap.svg)](https://anaconda.org/conda-forge/glodap)
30
+ [![DOI](https://img.shields.io/badge/DOI-10.5281%2Fzenodo.21295574-informational)](https://doi.org/10.5281/zenodo.21295574)
29
31
  [![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0)
30
32
  [![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
31
33
 
@@ -48,29 +50,48 @@ df_atlantic = glodap.atlantic()
48
50
  Files are saved by default at `"~/.glodap"`, but this can be controlled with the
49
51
  kwarg `gpath`. See the function docstrings for more information.
50
52
 
51
- The columns of the imported DataFrames can be passed
52
- directly into [PyCO2SYS v2](https://mvdh.xyz/PyCO2SYS):
53
+ The columns are the same as in the original GLODAP .mat files available from [glodap.info](https://glodap.info), except:
54
+ * The `"G2"` at the start of each parameter has been removed.
55
+ * Flags end with `"_f"` instead of just `"f"`.
56
+ * There is a `"datetime"` column, which combines the `"year"`, `"month"` and `"day"` but NOT the `"hour"` and `"minute"` (because some of these are missing).
57
+
58
+ The functions `download` and `read` can also be used for finer control, such as
59
+ specifying a particular GLODAP version rather than using the latest one (currently `v3.2026`). See
60
+ their function docstrings for more information.
61
+
62
+ The files from the GEOMAR mirrors are downloaded (v2: .mat files; v3: .csv.zip files), and the SHA256 checksum
63
+ of each downloaded file is checked before the file is written to disk.
64
+
65
+ ### With PyCO2SYS
66
+
67
+ The columns of the imported DataFrames can be passed directly into [PyCO2SYS v2](https://mvdh.xyz/PyCO2SYS):
53
68
 
54
69
  ```python
55
70
  import PyCO2SYS as pyco2
56
- co2s_atlantic = pyco2.sys(data=df_atlantic, nitrite=None)
71
+ co2s_atlantic = pyco2.sys(data=df_atlantic.drop(columns="fco2"), nitrite=0)
57
72
  ```
58
73
 
59
- Note `nitrite=None` - this means PyCO2SYS will ignore the `"nitrite"` column,
74
+ Note `nitrite=0` - this means PyCO2SYS will ignore the `"nitrite"` column,
60
75
  which is necessary because while PyCO2SYS includes the nitrite-nitrous acid
61
76
  equilibrium, its equilibrium constant is valid only under lab conditions.
62
77
 
63
- Because of how the columns are named, when passing the DataFrame directly to
64
- PyCO2SYS as above, the system will be solved from DIC and alkalinity, not pH.
78
+ Because of how the columns are named, and because we dropped the `"fco2"` column, when passing the DataFrame directly to
79
+ PyCO2SYS as above, the system will be solved from DIC and alkalinity, not pH nor fCO<sub>2</sub>.
65
80
 
66
- The columns are the same as in the original GLODAP .mat files available from [glodap.info](https://glodap.info), except:
67
- * The `"G2"` at the start of each parameter has been removed.
68
- * Flags end with `"_f"` instead of just `"f"`.
69
- * There is a `"datetime"` column, which combines the `"year"`, `"month"` and `"day"` but NOT the `"hour"` and `"minute"` (because some of these are missing).
81
+ ## Cite
70
82
 
71
- The functions `download` and `read` can also be used for finer control, such as
72
- specifying a particular GLODAP version rather than using the latest one. See
73
- their function docstrings for more information.
83
+ If you use GLODAP data, please cite the paper corresponding to the version you used:
74
84
 
75
- The .mat files from the GEOMAR mirrors are downloaded, and the SHA256 checksum
76
- of each downloaded file is checked before the file is written to disk.
85
+ * `v3.2026`: TBC (submitted to *Earth System Science Data*)
86
+ * `v2.2023`: https://doi.org/10.5194/essd-16-2047-2024
87
+ * `v2.2022`: https://doi.org/10.5194/essd-14-5543-2022
88
+ * `v2.2021`: https://doi.org/10.5194/essd-13-5565-2021
89
+ * `v2.2020`: https://doi.org/10.5194/essd-12-3653-2020
90
+ * `v2.2019`: https://doi.org/10.5194/essd-11-1437-2019
91
+ * `v2.2016`: https://doi.org/10.5194/essd-8-297-2016
92
+
93
+ If your analysis leans heavily on a specific subset of GLODAP, please consider inviting the PIs of the relevant datasets to be contribute as co-authors.
94
+
95
+ If you would also like to cite this tool directly:
96
+
97
+ > Humphreys, M. P. (2026). GLODAP downloader and importer in Python [Computer software]. *Zenodo.* https://doi.org/10.5281/zenodo.21295574
@@ -3,6 +3,7 @@
3
3
  [![Tests](https://github.com/mvdh7/glodap/actions/workflows/tests.yml/badge.svg?branch=main)](https://github.com/mvdh7/glodap/actions)
4
4
  [![pypi badge](https://img.shields.io/pypi/v/glodap.svg?style=popout)](https://pypi.org/project/glodap/)
5
5
  [![Conda Version](https://img.shields.io/conda/vn/conda-forge/glodap.svg)](https://anaconda.org/conda-forge/glodap)
6
+ [![DOI](https://img.shields.io/badge/DOI-10.5281%2Fzenodo.21295574-informational)](https://doi.org/10.5281/zenodo.21295574)
6
7
  [![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0)
7
8
  [![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
8
9
 
@@ -25,29 +26,48 @@ df_atlantic = glodap.atlantic()
25
26
  Files are saved by default at `"~/.glodap"`, but this can be controlled with the
26
27
  kwarg `gpath`. See the function docstrings for more information.
27
28
 
28
- The columns of the imported DataFrames can be passed
29
- directly into [PyCO2SYS v2](https://mvdh.xyz/PyCO2SYS):
29
+ The columns are the same as in the original GLODAP .mat files available from [glodap.info](https://glodap.info), except:
30
+ * The `"G2"` at the start of each parameter has been removed.
31
+ * Flags end with `"_f"` instead of just `"f"`.
32
+ * There is a `"datetime"` column, which combines the `"year"`, `"month"` and `"day"` but NOT the `"hour"` and `"minute"` (because some of these are missing).
33
+
34
+ The functions `download` and `read` can also be used for finer control, such as
35
+ specifying a particular GLODAP version rather than using the latest one (currently `v3.2026`). See
36
+ their function docstrings for more information.
37
+
38
+ The files from the GEOMAR mirrors are downloaded (v2: .mat files; v3: .csv.zip files), and the SHA256 checksum
39
+ of each downloaded file is checked before the file is written to disk.
40
+
41
+ ### With PyCO2SYS
42
+
43
+ The columns of the imported DataFrames can be passed directly into [PyCO2SYS v2](https://mvdh.xyz/PyCO2SYS):
30
44
 
31
45
  ```python
32
46
  import PyCO2SYS as pyco2
33
- co2s_atlantic = pyco2.sys(data=df_atlantic, nitrite=None)
47
+ co2s_atlantic = pyco2.sys(data=df_atlantic.drop(columns="fco2"), nitrite=0)
34
48
  ```
35
49
 
36
- Note `nitrite=None` - this means PyCO2SYS will ignore the `"nitrite"` column,
50
+ Note `nitrite=0` - this means PyCO2SYS will ignore the `"nitrite"` column,
37
51
  which is necessary because while PyCO2SYS includes the nitrite-nitrous acid
38
52
  equilibrium, its equilibrium constant is valid only under lab conditions.
39
53
 
40
- Because of how the columns are named, when passing the DataFrame directly to
41
- PyCO2SYS as above, the system will be solved from DIC and alkalinity, not pH.
54
+ Because of how the columns are named, and because we dropped the `"fco2"` column, when passing the DataFrame directly to
55
+ PyCO2SYS as above, the system will be solved from DIC and alkalinity, not pH nor fCO<sub>2</sub>.
42
56
 
43
- The columns are the same as in the original GLODAP .mat files available from [glodap.info](https://glodap.info), except:
44
- * The `"G2"` at the start of each parameter has been removed.
45
- * Flags end with `"_f"` instead of just `"f"`.
46
- * There is a `"datetime"` column, which combines the `"year"`, `"month"` and `"day"` but NOT the `"hour"` and `"minute"` (because some of these are missing).
57
+ ## Cite
47
58
 
48
- The functions `download` and `read` can also be used for finer control, such as
49
- specifying a particular GLODAP version rather than using the latest one. See
50
- their function docstrings for more information.
59
+ If you use GLODAP data, please cite the paper corresponding to the version you used:
51
60
 
52
- The .mat files from the GEOMAR mirrors are downloaded, and the SHA256 checksum
53
- of each downloaded file is checked before the file is written to disk.
61
+ * `v3.2026`: TBC (submitted to *Earth System Science Data*)
62
+ * `v2.2023`: https://doi.org/10.5194/essd-16-2047-2024
63
+ * `v2.2022`: https://doi.org/10.5194/essd-14-5543-2022
64
+ * `v2.2021`: https://doi.org/10.5194/essd-13-5565-2021
65
+ * `v2.2020`: https://doi.org/10.5194/essd-12-3653-2020
66
+ * `v2.2019`: https://doi.org/10.5194/essd-11-1437-2019
67
+ * `v2.2016`: https://doi.org/10.5194/essd-8-297-2016
68
+
69
+ If your analysis leans heavily on a specific subset of GLODAP, please consider inviting the PIs of the relevant datasets to be contribute as co-authors.
70
+
71
+ If you would also like to cite this tool directly:
72
+
73
+ > Humphreys, M. P. (2026). GLODAP downloader and importer in Python [Computer software]. *Zenodo.* https://doi.org/10.5281/zenodo.21295574
@@ -0,0 +1,406 @@
1
+ """
2
+ GLODAP
3
+ ======
4
+ Download Global Ocean Data Analysis Project (https://glodap.info)
5
+ version 2 and 3 datasets and import them as pandas DataFrames.
6
+
7
+ FUNCTIONS
8
+ ---------
9
+ The functions `arctic`, `atlantic`, `indian`, `pacific` and `world`
10
+ import the latest version (by default) of the GLODAP dataset for the
11
+ corresponding region, first downloading the file if it's not already
12
+ saved locally. For example:
13
+
14
+ >>> import glodap
15
+ >>> df_atlantic = glodap.atlantic()
16
+
17
+ Function kwargs
18
+ ---------------
19
+ All functions (`arctic`, `atlantic`, `indian`, `pacific` and `world`)
20
+ have the same kwargs:
21
+
22
+ version : str or None, optional
23
+ Which GLODAP version to download. All of the options are listed in
24
+ `glodap.versions`. If `None`, then the most recent version is used,
25
+ which is given by `glodap.version_latest`.
26
+ gpath : str or pathlib.Path or None, optional
27
+ Where to look for the GLODAP file and to download it to if not
28
+ found there. If `None`, then `"~/.glodap"` is used.
29
+
30
+ When files are missing, the .mat (v2) or zipped CSV (v3) files from the
31
+ GEOMAR mirrors (i.e., from https://glodap.info) are downloaded. After
32
+ download, the SHA256 checksum of each file is verified and the download
33
+ is rejected in case of mismatch.
34
+
35
+ Function return values
36
+ ----------------------
37
+ The requested GLODAP dataset is returned as a pandas DataFrame.
38
+
39
+ The columns are the same as in the original GLODAP files, except:
40
+ - v2 datasets only: the "G2" at the start of each column name has been
41
+ removed.
42
+ - Flags end with `"_f"` instead of just `"f"`.
43
+ - A `"datetime"` column has been added, which combines the `"year"`,
44
+ `"month"` and `"day"`, but NOT the `"hour"` and `"minute"` (because
45
+ some of these are missing).
46
+
47
+ ATTRIBUTES
48
+ ----------
49
+ versions : list[str]
50
+ All possible GLODAP dataset version strings.
51
+ version_latest : str
52
+ The current latest version, which will be used if the `version`
53
+ kwarg is not provided.
54
+ """
55
+
56
+ from pathlib import Path
57
+ from typing import NamedTuple
58
+ from urllib.parse import quote
59
+
60
+ import numpy as np
61
+ import pandas as pd
62
+ import pooch
63
+ from scipy.io import loadmat
64
+
65
+
66
+ # Package metadata
67
+ __author__ = "Matthew P. Humphreys"
68
+ __version__ = "1.0.0"
69
+ __all__ = [
70
+ "arctic",
71
+ "atlantic",
72
+ "indian",
73
+ "pacific",
74
+ "version_latest",
75
+ "versions",
76
+ "world",
77
+ ]
78
+
79
+ # GLODAP metadata
80
+ version_latest = "v3.2026"
81
+ versions = [
82
+ "v3.2026",
83
+ "v2.2023",
84
+ "v2.2022",
85
+ "v2.2021",
86
+ "v2.2020",
87
+ "v2.2019",
88
+ "v2.2016",
89
+ ]
90
+ _regions = {
91
+ "arctic": "Arctic_Ocean",
92
+ "atlantic": "Atlantic_Ocean",
93
+ "indian": "Indian_Ocean",
94
+ "pacific": "Pacific_Ocean",
95
+ "world": "Merged_Master_File",
96
+ }
97
+ _checksums = {
98
+ "v3.2026": {
99
+ "arctic": "03dfcf59525931065247e142439d97c0ed88d8f572e87df919476deebdc57d5c",
100
+ "atlantic": "a4577bbb676c30327b185fc5bf0ddc0e302464d5332b6d630922faf536dbe68c",
101
+ "indian": "79d655bd203e4579007b7224fc53b87ee8a5d70bb733550f0acac260a6baa0d8",
102
+ "pacific": "1a5afbb6be199773e00772ca3b1b84241dfc52a591b3f6b85786180f5711a9b4",
103
+ "world": "214aeb49c618474bf92622a37d2795a10160954ab49a94f9052cff1dd41947fd",
104
+ },
105
+ "v2.2023": {
106
+ "arctic": "a998a8d99db84e9357b21a7e345bbfde62eae77870e651119b4c1929a2f1e2fd",
107
+ "atlantic": "a3ff7bc4f4ff3a1b0f540886a0eb7c9b3ec52aa4eae2dea57d47ae61f56da56f",
108
+ "indian": "8c9c21d4af9db506b09fdec79e2a311c2dbbf41b8fc4e6a7a855d1d50b3ccb51",
109
+ "pacific": "9891160dba6211d2f13923171564a06bbea34cda799f3091148061c32fc6070c",
110
+ "world": "002881fa71923d15c1bd5aca3b99bc220ad5d1c77a90f130ba1ff0d3910d8766",
111
+ },
112
+ "v2.2022": {
113
+ "arctic": "b2cdeeebfb3ff75701ccc15edb22b5c8e1153623aefc6463f9699ae8780edf15",
114
+ "atlantic": "cffdb28aca195e189b16834a4f9711b90a995ea2b99d95bc6fe498049f975631",
115
+ "indian": "00381b0542ed4e8a39c33ee071fcb2388c1e3b3fb13f76c75fc342b26f4c8a13",
116
+ "pacific": "4037f093d89ea422aa62524273ab40ca147934e34637496372e83f2e34fe9a66",
117
+ "world": "3d7ba5bcf59b0c1f513b6fa8140df0618d984fa7b320899e7f188dd7c6cef565",
118
+ },
119
+ "v2.2021": {
120
+ "arctic": "21e511e808b9cc9f5b8532bf3b1c6754bb4ad6c344adb4a498b71967a0b6d217",
121
+ "atlantic": "22731e474a33d10c3efd3cc6748aafca7e1488495acefe1bbbd26b4105973a21",
122
+ "indian": "d3c43da540e606091a981958b4ce674dbe1ef9777ffee4c90de29bcc74f010a2",
123
+ "pacific": "925cf0424f4cd1caad30cfbe57bdc19740c85e715bee06bd298893915cfc5a01",
124
+ "world": "8329acd8722a3c198455642f8fca29b05f460b149e178fef9e1041f2caa19453",
125
+ },
126
+ "v2.2020": {
127
+ "arctic": "de52e1086b61f68d46cb0142e4bcffae18b578f41c2a00bda40a8960959a94b7",
128
+ "atlantic": "e927d87d873e089311ed5b747195aa150880f59559831bab27a54dcc0ff7ff69",
129
+ "indian": "1d8dfbefb820541d9b10f73d3785dae8653e67166b94666dbb433d3ddfa4e255",
130
+ "pacific": "9a75bf66b38341117afaae947af465f8f7e4082a9f773355c1fbadf26484d544",
131
+ "world": "51e1000101aa61712edb9c0074289bb83c105d17ee8cea575f8e18fa05bde953",
132
+ },
133
+ "v2.2019": {
134
+ "arctic": "f972a835601a5b00ac5ffc5093414d5fd2a520bbc7b1c7ff0a7d343852302284",
135
+ "atlantic": "6ac0826bacbaa43a9813fe18bcb84b1786efb44538b7a2d24e8a0c993ff23193",
136
+ "indian": "aa6695b715cce7658f94911d38a344f1e835ff6bbed8f9dc1548b167983a2967",
137
+ "pacific": "88df995e272dd63db55421b911be1f99bfec4cac6f97a73e017c9e275bf29db2",
138
+ "world": "4637ce5eb560fe020e9fccf0370156bfe19498fc74c2e8b66dee190cc44cff63",
139
+ },
140
+ "v2.2016": {
141
+ "arctic": "f2214409e0526052903da2427f3f8c6fd433b1759ba2a1be3a61181b4b0d04b7",
142
+ "atlantic": "fb26a86ca1135a0afc94157ba7963bc1cb8138523bcba36360595641175f8da1",
143
+ "indian": "ba162aa6f1fa7209f586e503581d3cc05cdffbfa6f72bdc535824f7a78d694f9",
144
+ "pacific": "0d1666de27ad6853533651027459d2525f75c6681d540f129df705b018d3c931",
145
+ "world": "aa202e330d5e77b98d0062e40a40665b8397b83ebbfa00424532139c53814f5e",
146
+ },
147
+ }
148
+
149
+ _keys_flags = [
150
+ "salinityf",
151
+ "oxygenf",
152
+ "aouf",
153
+ "nitratef",
154
+ "nitritef",
155
+ "silicatef",
156
+ "phosphatef",
157
+ "tco2f",
158
+ "talkf",
159
+ "fco2f",
160
+ "phts25p0f",
161
+ "phtsinsitutpf",
162
+ "cfc11f",
163
+ "cfc12f",
164
+ "cfc113f",
165
+ "ccl4f",
166
+ "sf6f",
167
+ "c13f",
168
+ "c14f",
169
+ "h3f",
170
+ "he3f",
171
+ "hef",
172
+ "neonf",
173
+ "o18f",
174
+ "tocf",
175
+ "docf",
176
+ "donf",
177
+ "tdnf",
178
+ "chlaf",
179
+ ]
180
+ _renamer_flags = {k: k[:-1] + "_f" for k in _keys_flags}
181
+
182
+
183
+ class ParsedInputs(NamedTuple):
184
+ gpath: Path
185
+ region: str
186
+ fileregion: str
187
+ filename: str
188
+ version: str
189
+
190
+
191
+ def _parse_inputs(
192
+ region: str,
193
+ version: str | None,
194
+ gpath: str | Path | None,
195
+ ) -> ParsedInputs:
196
+ region = region.lower()
197
+ if region not in _regions:
198
+ raise ValueError("`region` not valid!")
199
+ if version is None:
200
+ version = version_latest
201
+ version = version.lower()
202
+ if version not in versions:
203
+ raise ValueError("`version` not valid!")
204
+ if gpath is None:
205
+ gpath = "~/.glodap"
206
+ gpath = Path(gpath).expanduser()
207
+ fileregion = _regions[region]
208
+ if version.startswith("v2"):
209
+ filename = f"{fileregion}_{version}.mat"
210
+ elif version.startswith("v3"):
211
+ filename = f"{fileregion}_{version}.csv.zip"
212
+ return ParsedInputs(gpath, region, fileregion, filename, version)
213
+
214
+
215
+ def _download(
216
+ region: str = "world",
217
+ version: str | None = None,
218
+ gpath: str | Path | None = None,
219
+ ) -> None:
220
+ """Download a GLODAP data file and save it locally, after checking
221
+ that it has the expected checksum (SHA256).
222
+
223
+ Parameters
224
+ ----------
225
+ region : str, optional
226
+ Which GLODAP region to download, by default "world", which is
227
+ the Merged Master File. The options are:
228
+ "arctic" Arctic Ocean
229
+ "atlantic" Atlantic Ocean
230
+ "indian" Indian Ocean
231
+ "pacific" Pacific Ocean
232
+ "world" Merged Master File
233
+ version : str or None, optional
234
+ Which GLODAP version to download, by default `None`, in which
235
+ case the most recent version is downloaded. The options are
236
+ given in `glodap.versions`.
237
+ gpath : str or pathlib.Path or None, optional
238
+ Where to check for or save the downloaded file, by default
239
+ `None`, in which case "~/.glodap" is used.
240
+ """
241
+ gpath, region, fileregion, filename, version = _parse_inputs(
242
+ region, version, gpath
243
+ )
244
+ gpath.mkdir(parents=True, exist_ok=True)
245
+ if version == "v2.2016":
246
+ fr = fileregion.replace("_", " ")
247
+ url = f"{version}/GLODAPv2 {fr}.mat"
248
+ elif version.startswith("v2"):
249
+ url = f"{version}/GLODAP{version}_{fileregion}.mat"
250
+ elif version.startswith("v3"):
251
+ url = f"v3/GLODAPv3_{fileregion}.csv.zip"
252
+ checksum = _checksums[version][region]
253
+ pooch.retrieve(
254
+ "https://glodap.info/glodap_files/" + quote(url, safe="/"),
255
+ known_hash=f"sha256:{checksum}",
256
+ fname=filename,
257
+ path=gpath,
258
+ )
259
+
260
+
261
+ def _read(
262
+ region: str = "world",
263
+ version: str | None = None,
264
+ gpath: str | Path | None = None,
265
+ ) -> pd.DataFrame:
266
+ """Import a GLODAP data file as a pandas DataFrame, downloading it
267
+ first if it's not already available locally.
268
+
269
+ Parameters
270
+ ----------
271
+ region : str, optional
272
+ Which GLODAP region to import, by default "world", which is the
273
+ Merged Master File. The options are:
274
+ "arctic" Arctic Ocean
275
+ "atlantic" Atlantic Ocean
276
+ "indian" Indian Ocean
277
+ "pacific" Pacific Ocean
278
+ "world" Merged Master File
279
+ version : str or None, optional
280
+ Which GLODAP version to import, by default `None`, in which case
281
+ the most recent version is imported. The options are given in
282
+ `glodap.versions`.
283
+ gpath : str or pathlib.Path or None, optional
284
+ Where the downloaded file is or should be saved, by default
285
+ `None`, in which case the file is imported from or saved at
286
+ "~/.glodap".
287
+
288
+ Returns
289
+ -------
290
+ pd.DataFrame
291
+ The GLODAP dataset as a pandas DataFrame.
292
+ """
293
+ gpath, region, _, filename, version = _parse_inputs(region, version, gpath)
294
+ # _download checks if the file is already there before downloading,
295
+ # so it's fine to always run it here. It's also useful to have in
296
+ # because it checks the checksum of already-downloaded files.
297
+ _download(region=region, version=version, gpath=gpath)
298
+ if version.startswith("v2"):
299
+ df = loadmat(gpath / filename)
300
+ df = pd.DataFrame(
301
+ {
302
+ k[2:]: [w[0][0] for w in v] if v.dtype == object else v.ravel()
303
+ for k, v in df.items()
304
+ if k.startswith("G2")
305
+ }
306
+ )
307
+ elif version.startswith("v3"):
308
+ df = pd.read_csv(
309
+ gpath / filename,
310
+ compression="zip",
311
+ dtype={
312
+ "expocode": str,
313
+ "ph_scale": str,
314
+ },
315
+ na_values=[-9999, "-9999"],
316
+ )
317
+ # Correct fatal datetime errors and missing flags in v2.2016
318
+ if version == "v2.2016":
319
+ if region in ["indian", "world"]:
320
+ L = (df.cruise == 696) & (df.station == 15537)
321
+ df.loc[L, "month"] = 12
322
+ df.loc[L, "day"] = 1
323
+ ks = [k for k in _keys_flags if k in df]
324
+ df[ks] = df[ks].fillna(9)
325
+ if region in ["pacific", "world"]:
326
+ L = (df.cruise == 270) & (df.station == 113)
327
+ df.loc[L, "month"] = 4
328
+ df.loc[L, "day"] = 30
329
+ # Convert columns that should be integers into integers
330
+ # Can't convert "cast", "hour" or "minute" to integers because they
331
+ # have missing values
332
+ dtypes = {
333
+ k: np.uint8 # 0 to 255
334
+ for k in _keys_flags + ["region", "month", "day"]
335
+ if k in df
336
+ }
337
+ dtypes["year"] = np.uint16 # 0 to 65535
338
+ dtypes["cruise"] = int
339
+ dtypes["station"] = int
340
+ df = df.astype(dtypes)
341
+ # Rename flag columns and defragment
342
+ df = df.rename(columns=_renamer_flags).copy()
343
+ # Calculate datetime for convenience - but exclude hour and minute,
344
+ # because some are missing
345
+ df["datetime"] = pd.to_datetime(df[["year", "month", "day"]])
346
+ return df
347
+
348
+
349
+ def arctic(
350
+ version: str | None = None,
351
+ gpath: str | Path | None = None,
352
+ ) -> pd.DataFrame:
353
+ """Import the latest version of the GLODAP Arctic Ocean dataset,
354
+ downloading it first if it's not already available locally.
355
+
356
+ See the `glodap` module docstring for info on the optional kwargs.
357
+ """
358
+ return _read(region="arctic", version=version, gpath=gpath)
359
+
360
+
361
+ def atlantic(
362
+ version: str | None = None,
363
+ gpath: str | Path | None = None,
364
+ ) -> pd.DataFrame:
365
+ """Import the latest version of the GLODAP Atlantic Ocean dataset,
366
+ downloading it first if it's not already available locally.
367
+
368
+ See the `glodap` module docstring for info on the optional kwargs.
369
+ """
370
+ return _read(region="atlantic", version=version, gpath=gpath)
371
+
372
+
373
+ def indian(
374
+ version: str | None = None,
375
+ gpath: str | Path | None = None,
376
+ ) -> pd.DataFrame:
377
+ """Import the latest version of the GLODAP Indian Ocean dataset,
378
+ downloading it first if it's not already available locally.
379
+
380
+ See the `glodap` module docstring for info on the optional kwargs.
381
+ """
382
+ return _read(region="indian", version=version, gpath=gpath)
383
+
384
+
385
+ def pacific(
386
+ version: str | None = None,
387
+ gpath: str | Path | None = None,
388
+ ) -> pd.DataFrame:
389
+ """Import the latest version of the GLODAP Pacific Ocean dataset,
390
+ downloading it first if it's not already available locally.
391
+
392
+ See the `glodap` module docstring for info on the optional kwargs.
393
+ """
394
+ return _read(region="pacific", version=version, gpath=gpath)
395
+
396
+
397
+ def world(
398
+ version: str | None = None,
399
+ gpath: str | Path | None = None,
400
+ ) -> pd.DataFrame:
401
+ """Import the latest version of the GLODAP Merged Master File
402
+ dataset, downloading it first if it's not already available locally.
403
+
404
+ See the `glodap` module docstring for info on the optional kwargs.
405
+ """
406
+ return _read(region="world", version=version, gpath=gpath)
@@ -8,18 +8,19 @@ build-backend = "setuptools.build_meta"
8
8
  name = "glodap"
9
9
  description = "GLODAP downloader and importer in Python"
10
10
  readme = "README.md"
11
- requires-python = ">= 3.10"
11
+ requires-python = ">= 3.11"
12
12
  dependencies = [
13
- "pandas",
14
- "requests",
15
- "scipy",
13
+ "pandas>=3,<4",
14
+ "pooch>=1.9.0,<2",
15
+ "scipy>=1.17,<2",
16
16
  ]
17
17
  classifiers = [
18
- "Development Status :: 4 - Beta",
19
- "Programming Language :: Python :: 3.10",
18
+ "Development Status :: 5 - Production/Stable",
20
19
  "Programming Language :: Python :: 3.11",
21
20
  "Programming Language :: Python :: 3.12",
22
21
  "Programming Language :: Python :: 3.13",
22
+ "Programming Language :: Python :: 3.14",
23
+ "Programming Language :: Python :: 3.15",
23
24
  "License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
24
25
  "Operating System :: OS Independent",
25
26
  "Natural Language :: English",
@@ -34,3 +35,18 @@ include = ["glodap*"]
34
35
 
35
36
  [tool.setuptools.dynamic]
36
37
  version = {attr = "glodap.__version__"}
38
+
39
+ [tool.ruff]
40
+ line-length = 79
41
+
42
+ [tool.ruff.lint.isort]
43
+ combine-as-imports = true
44
+ force-single-line = false
45
+ lines-after-imports = 2
46
+
47
+ [dependency-groups]
48
+ dev = [
49
+ "ipykernel>=7.3.0",
50
+ "pytest>=9.2.0",
51
+ "pytest-xdist>=3.8.0",
52
+ ]
@@ -0,0 +1,19 @@
1
+ # %%
2
+ import itertools
3
+ import tempfile
4
+
5
+ import pandas as pd
6
+
7
+ import glodap
8
+
9
+
10
+ def test_download_and_read():
11
+ with tempfile.TemporaryDirectory() as tdir:
12
+ for region, version in itertools.product(
13
+ glodap._regions, glodap.versions
14
+ ):
15
+ df = glodap._read(region=region, version=version, gpath=tdir)
16
+ assert isinstance(df, pd.DataFrame)
17
+
18
+
19
+ # test_download_and_read()
@@ -0,0 +1,20 @@
1
+ # %%
2
+ import pandas as pd
3
+
4
+ import glodap
5
+
6
+
7
+ def test_regions_latest():
8
+ df = glodap.arctic()
9
+ assert isinstance(df, pd.DataFrame)
10
+ df = glodap.atlantic()
11
+ assert isinstance(df, pd.DataFrame)
12
+ df = glodap.indian()
13
+ assert isinstance(df, pd.DataFrame)
14
+ df = glodap.pacific()
15
+ assert isinstance(df, pd.DataFrame)
16
+ df = glodap.world()
17
+ assert isinstance(df, pd.DataFrame)
18
+
19
+
20
+ # test_regions_latest()
@@ -0,0 +1,21 @@
1
+ # %%
2
+ import glodap
3
+
4
+
5
+ def test_version_latest():
6
+ assert glodap.version_latest in glodap.versions
7
+
8
+
9
+ def test_checksum_versions_and_regions():
10
+ for v, rs in glodap._checksums.items():
11
+ assert v in glodap.versions
12
+ for r in rs:
13
+ assert r in glodap._regions
14
+ for r in glodap._regions:
15
+ assert r in rs
16
+ for v in glodap.versions:
17
+ assert v in glodap._checksums
18
+
19
+
20
+ # test_version_latest()
21
+ # test_checksum_versions_and_regions()
@@ -1,3 +0,0 @@
1
- pandas
2
- requests
3
- scipy
@@ -1,402 +0,0 @@
1
- """
2
- GLODAP
3
- ======
4
- Download GLODAP (https://glodap.info) datasets and import them as pandas
5
- DataFrames.
6
-
7
- The functions `arctic`, `atlantic`, `indian`, `pacific` and `world` import the
8
- latest version of the GLODAP dataset for the corresponding region, first
9
- downloading the file if it's not already saved locally. For example:
10
-
11
- >>> import glodap
12
- >>> df_atlantic = glodap.atlantic()
13
-
14
- Files are saved by default at "~/.glodap", but this can be controlled with the
15
- kwarg `gpath`. See the function docstrings for more information.
16
-
17
- The columns of the imported DataFrames can be passed directly into PyCO2SYS v2:
18
-
19
- >>> import PyCO2SYS as pyco2
20
- >>> co2s_atlantic = pyco2.sys(data=df_atlantic, nitrite=None)
21
-
22
- Note `nitrite=None` - this means PyCO2SYS will ignore the "nitrite" column,
23
- which is necessary because while PyCO2SYS includes the nitrite-nitrous acid
24
- equilibrium, its equilibrium constant is valid only under lab conditions.
25
-
26
- Because of how the columns are named, when passing the DataFrame directly to
27
- PyCO2SYS as above, the system will be solved from DIC and alkalinity, not pH.
28
-
29
- The columns are the same as in the original GLODAP .mat files, except:
30
- * The "G2" at the start of each parameter has been removed.
31
- * Flags end with "_f" instead of just "f".
32
- * There is a "datetime" column, which combines the "year", "month" and "day"
33
- but NOT the "hour" and "minute" (because some of these are missing).
34
-
35
- The functions `download` and `read` can also be used for finer control, such as
36
- specifying a particular GLODAP version rather than using the latest one. See
37
- their function docstrings for more information.
38
-
39
- The .mat files from the GEOMAR mirrors are downloaded, and the SHA256 checksum
40
- of each downloaded file is checked before the file is written to disk.
41
- """
42
-
43
- import hashlib
44
- import os
45
- import tempfile
46
- from warnings import warn
47
-
48
- import pandas as pd
49
- import requests
50
- from scipy.io import loadmat
51
-
52
-
53
- # Package metadata
54
- __author__ = "Humphreys, Matthew P."
55
- __version__ = "0.3"
56
-
57
- # GLODAP metadata
58
- version_latest = "v2.2023"
59
- versions = ["v2.2023", "v2.2022", "v2.2021", "v2.2020", "v2.2019"]
60
- regions = {
61
- "arctic": "Arctic_Ocean",
62
- "atlantic": "Atlantic_Ocean",
63
- "indian": "Indian_Ocean",
64
- "pacific": "Pacific_Ocean",
65
- "world": "Merged_Master_File",
66
- }
67
- regions_full = regions.copy()
68
- for k, v in regions.items():
69
- regions_full[k[:3]] = v
70
- checksums = {
71
- "v2.2023": {
72
- "arc": "a998a8d99db84e9357b21a7e345bbfde62eae77870e651119b4c1929a2f1e2fd",
73
- "atl": "a3ff7bc4f4ff3a1b0f540886a0eb7c9b3ec52aa4eae2dea57d47ae61f56da56f",
74
- "ind": "8c9c21d4af9db506b09fdec79e2a311c2dbbf41b8fc4e6a7a855d1d50b3ccb51",
75
- "pac": "9891160dba6211d2f13923171564a06bbea34cda799f3091148061c32fc6070c",
76
- "wor": "002881fa71923d15c1bd5aca3b99bc220ad5d1c77a90f130ba1ff0d3910d8766",
77
- },
78
- "v2.2022": {
79
- "arc": "b2cdeeebfb3ff75701ccc15edb22b5c8e1153623aefc6463f9699ae8780edf15",
80
- "atl": "cffdb28aca195e189b16834a4f9711b90a995ea2b99d95bc6fe498049f975631",
81
- "ind": "00381b0542ed4e8a39c33ee071fcb2388c1e3b3fb13f76c75fc342b26f4c8a13",
82
- "pac": "4037f093d89ea422aa62524273ab40ca147934e34637496372e83f2e34fe9a66",
83
- "wor": "3d7ba5bcf59b0c1f513b6fa8140df0618d984fa7b320899e7f188dd7c6cef565",
84
- },
85
- "v2.2021": {
86
- "arc": "21e511e808b9cc9f5b8532bf3b1c6754bb4ad6c344adb4a498b71967a0b6d217",
87
- "atl": "22731e474a33d10c3efd3cc6748aafca7e1488495acefe1bbbd26b4105973a21",
88
- "ind": "d3c43da540e606091a981958b4ce674dbe1ef9777ffee4c90de29bcc74f010a2",
89
- "pac": "925cf0424f4cd1caad30cfbe57bdc19740c85e715bee06bd298893915cfc5a01",
90
- "wor": "8329acd8722a3c198455642f8fca29b05f460b149e178fef9e1041f2caa19453",
91
- },
92
- "v2.2020": {
93
- "arc": "de52e1086b61f68d46cb0142e4bcffae18b578f41c2a00bda40a8960959a94b7",
94
- "atl": "e927d87d873e089311ed5b747195aa150880f59559831bab27a54dcc0ff7ff69",
95
- "ind": "1d8dfbefb820541d9b10f73d3785dae8653e67166b94666dbb433d3ddfa4e255",
96
- "pac": "9a75bf66b38341117afaae947af465f8f7e4082a9f773355c1fbadf26484d544",
97
- "wor": "51e1000101aa61712edb9c0074289bb83c105d17ee8cea575f8e18fa05bde953",
98
- },
99
- "v2.2019": {
100
- "arc": "f972a835601a5b00ac5ffc5093414d5fd2a520bbc7b1c7ff0a7d343852302284",
101
- "atl": "6ac0826bacbaa43a9813fe18bcb84b1786efb44538b7a2d24e8a0c993ff23193",
102
- "ind": "aa6695b715cce7658f94911d38a344f1e835ff6bbed8f9dc1548b167983a2967",
103
- "pac": "88df995e272dd63db55421b911be1f99bfec4cac6f97a73e017c9e275bf29db2",
104
- "wor": "4637ce5eb560fe020e9fccf0370156bfe19498fc74c2e8b66dee190cc44cff63",
105
- },
106
- }
107
-
108
-
109
- def _get_paths(region, version, gpath):
110
- assert region in regions_full, "`region` not valid!"
111
- if version is None:
112
- version = version_latest
113
- version = version.lower()
114
- assert version in versions, "`version` not valid!"
115
- if gpath is None:
116
- gpath = os.path.join(os.path.expanduser("~"), ".glodap")
117
- fileregion = regions_full[region.lower()]
118
- filename = f"{fileregion}_{version}.mat"
119
- return gpath, fileregion, filename, version
120
-
121
-
122
- def download(region="world", version=None, gpath=None, chunk_size=8192):
123
- """Download a GLODAP data file and save it locally, after checking that it
124
- has the expected checksum (SHA256).
125
-
126
- Parameters
127
- ----------
128
- region : str, optional
129
- Which GLODAP region to download, by default "world", which is the
130
- Merged Master File. The options are:
131
- "arctic" "arc" Arctic Ocean
132
- "atlantic" "atl" Atlantic Ocean
133
- "indian" "ind" Indian Ocean
134
- "pacific" "pac" Pacific Ocean
135
- "world" "wor" Merged Master File
136
- version : str or None, optional
137
- Which GLODAP version to download, by default `None`, in which case the
138
- most recent version is downloaded. The options are:
139
- "v2.2023", "v2.2022", "v2.2021", "v2.2020", "v2.2019"
140
- gpath : str of None, optional
141
- Where to save the downloaded file, by default `None`, in which case the
142
- file is saved at "~/.glodap".
143
- chunk_size : int, optional
144
- Chunk size for streamed file download, by default 8192.
145
- """
146
- gpath, fileregion, filename, version = _get_paths(region, version, gpath)
147
- if not os.path.isdir(gpath):
148
- os.makedirs(gpath)
149
- url = (
150
- f"https://glodap.info/glodap_files/{version}/"
151
- + f"GLODAP{version}_{fileregion}.mat"
152
- )
153
- hasher = hashlib.new("sha256")
154
- checksum = checksums[version][region[:3].lower()]
155
- with tempfile.TemporaryDirectory() as tdir:
156
- temp_path = os.path.join(tdir, filename)
157
- with requests.get(url, stream=True) as r:
158
- r.raise_for_status()
159
- with open(temp_path, "wb") as f:
160
- for chunk in r.iter_content(chunk_size=chunk_size):
161
- f.write(chunk)
162
- hasher.update(chunk)
163
- if hasher.hexdigest() == checksum:
164
- os.rename(temp_path, os.path.join(gpath, filename))
165
- else:
166
- warn("File checksum not as expected - download failed.")
167
-
168
-
169
- def read(region="world", version=None, gpath=None):
170
- """Import a GLODAP data file as a pandas DataFrame, downloading it first
171
- if it's not already available locally.
172
-
173
- Parameters
174
- ----------
175
- region : str, optional
176
- Which GLODAP region to import, by default "world", which is the
177
- Merged Master File. The options are:
178
- "arctic" "arc" Arctic Ocean
179
- "atlantic" "atl" Atlantic Ocean
180
- "indian" "ind" Indian Ocean
181
- "pacific" "pac" Pacific Ocean
182
- "world" "wor" Merged Master File
183
- version : str or None, optional
184
- Which GLODAP version to import, by default `None`, in which case the
185
- most recent version is imported. The options are:
186
- "v2.2023", "v2.2022", "v2.2021", "v2.2020", "v2.2019"
187
- gpath : str of None, optional
188
- Where to the downloaded file is or should be saved, by default `None`,
189
- in which case the file is imported from or saved at "~/.glodap".
190
-
191
- Returns
192
- -------
193
- pd.DataFrame
194
- The GLODAP dataset as a pandas DataFrame.
195
- """
196
- gpath, _, filename, version = _get_paths(region, version, gpath)
197
- try:
198
- df = loadmat(os.path.join(gpath, filename))
199
- except FileNotFoundError:
200
- download(region=region, version=version, gpath=gpath)
201
- df = loadmat(os.path.join(gpath, filename))
202
- df = pd.DataFrame(
203
- {
204
- k[2:]: [w[0][0] for w in v] if v.dtype == "O" else v.ravel()
205
- for k, v in df.items()
206
- if k.startswith("G2")
207
- }
208
- )
209
- # Convert columns that should be integers into integers
210
- # Can't convert cast, hour, minute to integers because they have missing
211
- # values
212
- keys_integers = [
213
- "cruise",
214
- "station",
215
- "region",
216
- # "cast",
217
- "year",
218
- "month",
219
- "day",
220
- # "hour",
221
- # "minute",
222
- ]
223
- keys_flags = [
224
- "salinityf",
225
- "oxygenf",
226
- "aouf",
227
- "nitratef",
228
- "nitritef",
229
- "silicatef",
230
- "phosphatef",
231
- "tco2f",
232
- "talkf",
233
- "fco2f",
234
- "phts25p0f",
235
- "phtsinsitutpf",
236
- "cfc11f",
237
- "cfc12f",
238
- "cfc113f",
239
- "ccl4f",
240
- "sf6f",
241
- "c13f",
242
- "c14f",
243
- "h3f",
244
- "he3f",
245
- "hef",
246
- "neonf",
247
- "o18f",
248
- "tocf",
249
- "docf",
250
- "donf",
251
- "tdnf",
252
- "chlaf",
253
- ]
254
- for k in keys_integers + keys_flags:
255
- df[k] = df[k].astype(int)
256
- # Rename columns for PyCO2SYS, if requested
257
- renamer_flags = {k: k[:-1] + "_f" for k in keys_flags}
258
- df = df.rename(columns=renamer_flags)
259
- # Calculate datetime for convenience - don't include hour and minute
260
- # because some are missing
261
- df["datetime"] = pd.to_datetime(
262
- df[
263
- [
264
- "year",
265
- "month",
266
- "day",
267
- # "hour",
268
- # "minute",
269
- ]
270
- ]
271
- )
272
- return df
273
-
274
-
275
- def arctic(version=None, gpath=None):
276
- """Import the latest version of the GLODAP Arctic Ocean dataset,
277
- downloading it first if it's not already available locally.
278
-
279
- Parameters
280
- ----------
281
- version : str or None, optional
282
- Which GLODAP version to import, by default `None`, in which case the
283
- most recent version is imported. The options are:
284
- "v2.2023", "v2.2022", "v2.2021", "v2.2020", "v2.2019"
285
- gpath : str of None, optional
286
- Where to the downloaded file is or should be saved, by default `None`,
287
- in which case the file is imported from or saved at "~/.glodap".
288
-
289
- Returns
290
- -------
291
- pd.DataFrame
292
- The GLODAP Arctic Ocean dataset as a pandas DataFrame.
293
- """
294
- return read(
295
- region="arctic",
296
- version=version,
297
- gpath=gpath,
298
- )
299
-
300
-
301
- def atlantic(version=None, gpath=None):
302
- """Import the latest version of the GLODAP Atlantic Ocean dataset,
303
- downloading it first if it's not already available locally.
304
-
305
- Parameters
306
- ----------
307
- version : str or None, optional
308
- Which GLODAP version to import, by default `None`, in which case the
309
- most recent version is imported. The options are:
310
- "v2.2023", "v2.2022", "v2.2021", "v2.2020", "v2.2019"
311
- gpath : str of None, optional
312
- Where to the downloaded file is or should be saved, by default `None`,
313
- in which case the file is imported from or saved at "~/.glodap".
314
-
315
- Returns
316
- -------
317
- pd.DataFrame
318
- The GLODAP Atlantic Ocean dataset as a pandas DataFrame.
319
- """
320
- return read(
321
- region="atlantic",
322
- version=version,
323
- gpath=gpath,
324
- )
325
-
326
-
327
- def indian(version=None, gpath=None):
328
- """Import the latest version of the GLODAP Indian Ocean dataset,
329
- downloading it first if it's not already available locally.
330
-
331
- Parameters
332
- ----------
333
- version : str or None, optional
334
- Which GLODAP version to import, by default `None`, in which case the
335
- most recent version is imported. The options are:
336
- "v2.2023", "v2.2022", "v2.2021", "v2.2020", "v2.2019"
337
- gpath : str of None, optional
338
- Where to the downloaded file is or should be saved, by default `None`,
339
- in which case the file is imported from or saved at "~/.glodap".
340
-
341
- Returns
342
- -------
343
- pd.DataFrame
344
- The GLODAP Indian Ocean dataset as a pandas DataFrame.
345
- """
346
- return read(
347
- region="indian",
348
- version=version,
349
- gpath=gpath,
350
- )
351
-
352
-
353
- def pacific(version=None, gpath=None):
354
- """Import the latest version of the GLODAP Pacific Ocean dataset,
355
- downloading it first if it's not already available locally.
356
-
357
- Parameters
358
- ----------
359
- version : str or None, optional
360
- Which GLODAP version to import, by default `None`, in which case the
361
- most recent version is imported. The options are:
362
- "v2.2023", "v2.2022", "v2.2021", "v2.2020", "v2.2019"
363
- gpath : str of None, optional
364
- Where to the downloaded file is or should be saved, by default `None`,
365
- in which case the file is imported from or saved at "~/.glodap".
366
-
367
- Returns
368
- -------
369
- pd.DataFrame
370
- The GLODAP Pacific Ocean dataset as a pandas DataFrame.
371
- """
372
- return read(
373
- region="pacific",
374
- version=version,
375
- gpath=gpath,
376
- )
377
-
378
-
379
- def world(version=None, gpath=None):
380
- """Import the latest version of the GLODAP Merged Master File dataset,
381
- downloading it first if it's not already available locally.
382
-
383
- Parameters
384
- ----------
385
- version : str or None, optional
386
- Which GLODAP version to import, by default `None`, in which case the
387
- most recent version is imported. The options are:
388
- "v2.2023", "v2.2022", "v2.2021", "v2.2020", "v2.2019"
389
- gpath : str of None, optional
390
- Where to the downloaded file is or should be saved, by default `None`,
391
- in which case the file is imported from or saved at "~/.glodap".
392
-
393
- Returns
394
- -------
395
- pd.DataFrame
396
- The GLODAP Merged Master File as a pandas DataFrame dataset.
397
- """
398
- return read(
399
- region="world",
400
- version=version,
401
- gpath=gpath,
402
- )
@@ -1,12 +0,0 @@
1
- # %%
2
- import pandas as pd
3
-
4
- import glodap
5
-
6
-
7
- def test_import():
8
- df = glodap.arctic(gpath="tests/data")
9
- assert isinstance(df, pd.DataFrame)
10
-
11
-
12
- # test_import()
@@ -1,11 +0,0 @@
1
- # %%
2
- import glodap
3
-
4
-
5
- def test_region_names():
6
- # Check that the first 3 letters of each region name are unique
7
- regions_short = {k[:3] for k in glodap.regions}
8
- assert len(regions_short) == len(glodap.regions)
9
-
10
-
11
- # test_region_names()
@@ -1,17 +0,0 @@
1
- # %%
2
- import itertools
3
- import tempfile
4
-
5
- import glodap
6
-
7
-
8
- def test_response():
9
- with tempfile.TemporaryDirectory() as tdir:
10
- for region, version in itertools.product(
11
- glodap.regions, glodap.versions
12
- ):
13
- print(region, version)
14
- glodap.download(region=region, version=version, gpath=tdir)
15
-
16
-
17
- # test_response()
File without changes
File without changes