water-column-sonar-processing 25.1.7__py3-none-any.whl → 25.3.0__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Potentially problematic release.


This version of water-column-sonar-processing might be problematic. Click here for more details.

Files changed (26) hide show
  1. water_column_sonar_processing/aws/dynamodb_manager.py +27 -32
  2. water_column_sonar_processing/aws/s3_manager.py +52 -64
  3. water_column_sonar_processing/aws/s3fs_manager.py +3 -9
  4. water_column_sonar_processing/cruise/create_empty_zarr_store.py +14 -14
  5. water_column_sonar_processing/cruise/datatree_manager.py +3 -6
  6. water_column_sonar_processing/cruise/resample_regrid.py +67 -49
  7. water_column_sonar_processing/geometry/__init__.py +7 -2
  8. water_column_sonar_processing/geometry/elevation_manager.py +16 -17
  9. water_column_sonar_processing/geometry/geometry_manager.py +25 -25
  10. water_column_sonar_processing/geometry/line_simplification.py +150 -0
  11. water_column_sonar_processing/geometry/pmtile_generation.py +99 -64
  12. water_column_sonar_processing/index/index_manager.py +67 -32
  13. water_column_sonar_processing/model/zarr_manager.py +32 -21
  14. water_column_sonar_processing/process.py +15 -13
  15. water_column_sonar_processing/processing/__init__.py +2 -2
  16. water_column_sonar_processing/processing/batch_downloader.py +66 -41
  17. water_column_sonar_processing/processing/raw_to_zarr.py +121 -82
  18. water_column_sonar_processing/utility/constants.py +10 -1
  19. water_column_sonar_processing/utility/pipeline_status.py +11 -15
  20. {water_column_sonar_processing-25.1.7.dist-info → water_column_sonar_processing-25.3.0.dist-info}/METADATA +21 -12
  21. water_column_sonar_processing-25.3.0.dist-info/RECORD +34 -0
  22. {water_column_sonar_processing-25.1.7.dist-info → water_column_sonar_processing-25.3.0.dist-info}/WHEEL +1 -1
  23. water_column_sonar_processing/geometry/geometry_simplification.py +0 -82
  24. water_column_sonar_processing-25.1.7.dist-info/RECORD +0 -34
  25. {water_column_sonar_processing-25.1.7.dist-info → water_column_sonar_processing-25.3.0.dist-info/licenses}/LICENSE +0 -0
  26. {water_column_sonar_processing-25.1.7.dist-info → water_column_sonar_processing-25.3.0.dist-info}/top_level.txt +0 -0
@@ -1,10 +1,12 @@
1
- Metadata-Version: 2.2
1
+ Metadata-Version: 2.4
2
2
  Name: water_column_sonar_processing
3
- Version: 25.1.7
3
+ Version: 25.3.0
4
4
  Summary: Processing tool for water column sonar data.
5
5
  Author-email: Rudy Klucik <rudy.klucik@noaa.gov>
6
+ Maintainer-email: Rudy Klucik <rudy.klucik@noaa.gov>
6
7
  Project-URL: Homepage, https://github.com/CI-CMG/water-column-sonar-processing
7
8
  Project-URL: Issues, https://github.com/CI-CMG/water-column-sonar-processing/issues
9
+ Keywords: ocean,sonar,water column
8
10
  Classifier: Programming Language :: Python :: 3
9
11
  Classifier: License :: OSI Approved :: MIT License
10
12
  Classifier: Operating System :: OS Independent
@@ -20,13 +22,16 @@ Requires-Dist: geopandas==1.0.1
20
22
  Requires-Dist: mock==5.1.0
21
23
  Requires-Dist: moto[all]==5.0.27
22
24
  Requires-Dist: moto[server]==5.0.27
25
+ Requires-Dist: networkx==3.4.2
23
26
  Requires-Dist: numcodecs==0.13.1
24
27
  Requires-Dist: numpy==1.26.4
25
28
  Requires-Dist: pandas==2.2.3
26
29
  Requires-Dist: pyarrow==18.1.0
30
+ Requires-Dist: pykalman==0.10.1
27
31
  Requires-Dist: python-dotenv==1.0.1
28
32
  Requires-Dist: requests==2.32.3
29
33
  Requires-Dist: s3fs==2024.2.0
34
+ Requires-Dist: safety==3.2.14
30
35
  Requires-Dist: scipy==1.14.1
31
36
  Requires-Dist: setuptools
32
37
  Requires-Dist: shapely==2.0.3
@@ -36,22 +41,26 @@ Requires-Dist: xbatcher==0.4.0
36
41
  Requires-Dist: zarr==2.18.3
37
42
  Provides-Extra: dev
38
43
  Requires-Dist: bandit[toml]==1.8.0; extra == "dev"
44
+ Requires-Dist: bandit; extra == "dev"
39
45
  Requires-Dist: build; extra == "dev"
40
- Requires-Dist: pre-commit; extra == "dev"
41
- Requires-Dist: pyinstaller; extra == "dev"
42
- Requires-Dist: twine; extra == "dev"
43
46
  Requires-Dist: flake8==7.1.1; extra == "dev"
47
+ Requires-Dist: isort; extra == "dev"
48
+ Requires-Dist: matplotlib==3.10.1; extra == "dev"
44
49
  Requires-Dist: pooch==1.8.2; extra == "dev"
50
+ Requires-Dist: pre-commit; extra == "dev"
51
+ Requires-Dist: pyinstaller; extra == "dev"
45
52
  Requires-Dist: pytest~=8.3.3; extra == "dev"
53
+ Requires-Dist: ruff; extra == "dev"
46
54
  Requires-Dist: tqdm; extra == "dev"
47
- Requires-Dist: bandit; extra == "dev"
55
+ Requires-Dist: twine; extra == "dev"
48
56
  Provides-Extra: test
49
57
  Requires-Dist: pytest-cov; extra == "test"
58
+ Dynamic: license-file
50
59
 
51
60
  # Water Column Sonar Processing
52
- Processing tool for converting L0 data to L1 and L2 as well as generating geospatial information
61
+ Processing tool for converting Level_0 water column sonar data to Level_1 and Level_2 derived data sets as well as generating geospatial information.
53
62
 
54
- ![GitHub Actions Workflow Status](https://img.shields.io/github/actions/workflow/status/CI-CMG/water-column-sonar-processing/test_action.yaml?color=black) ![PyPI - Implementation](https://img.shields.io/pypi/v/water-column-sonar-processing?color=black) ![GitHub License](https://img.shields.io/github/license/CI-CMG/water-column-sonar-processing?color=black) ![PyPI - Downloads](https://img.shields.io/pypi/dd/water-column-sonar-processing?color=black) ![GitHub code size in bytes](https://img.shields.io/github/languages/code-size/CI-CMG/water-column-sonar-processing?color=black) ![GitHub repo size](https://img.shields.io/github/repo-size/CI-CMG/water-column-sonar-processing?color=black)
63
+ ![GitHub Actions Workflow Status](https://img.shields.io/github/actions/workflow/status/CI-CMG/water-column-sonar-processing/test_action.yaml) ![PyPI - Implementation](https://img.shields.io/pypi/v/water-column-sonar-processing) ![GitHub License](https://img.shields.io/github/license/CI-CMG/water-column-sonar-processing) ![PyPI - Downloads](https://img.shields.io/pypi/dd/water-column-sonar-processing) ![GitHub code size in bytes](https://img.shields.io/github/languages/code-size/CI-CMG/water-column-sonar-processing) ![GitHub repo size](https://img.shields.io/github/repo-size/CI-CMG/water-column-sonar-processing)
55
64
 
56
65
  # Setting up the Python Environment
57
66
  > Python 3.10.12
@@ -89,8 +98,8 @@ Processing tool for converting L0 data to L1 and L2 as well as generating geospa
89
98
  4. Choose ~/.pyenv/versions/mocking_aws/bin/python
90
99
  3. Set up Python Facet (not sure if this is required)
91
100
  1. File -> Project Structure or CMD + ;
92
- 2. Facets -> + -> Python
93
- 3. Set interpreter
101
+ 2. Facets -> + -> Python
102
+ 3. Set interpreter
94
103
 
95
104
  # Installing Dependencies
96
105
  ```
@@ -136,13 +145,13 @@ https://colab.research.google.com/drive/1KiLMueXiz9WVB9o4RuzYeGjNZ6PsZU7a#scroll
136
145
  # Tag a Release
137
146
  Step 1 --> increment the semantic version in the zarr_manager.py "metadata" & the "pyproject.toml"
138
147
  ```commandline
139
- git tag -a v25.1.7 -m "Releasing version v25.1.7"
148
+ git tag -a v25.1.8 -m "Releasing version v25.1.8"
140
149
  git push origin --tags
141
150
  ```
142
151
 
143
152
  # To Publish To PROD
144
153
  ```commandline
145
- python -m build
154
+ uv build
146
155
  python -m twine upload --repository pypi dist/*
147
156
  ```
148
157
 
@@ -0,0 +1,34 @@
1
+ water_column_sonar_processing/__init__.py,sha256=fvRK4uFo_A0l7w_T4yckvDqJ3wMUq4JB3VVPXqWfewE,226
2
+ water_column_sonar_processing/process.py,sha256=eatdHHc7_mPapTjxsOyD24OlEWeqj56MmVTMPjxF2QM,5290
3
+ water_column_sonar_processing/aws/__init__.py,sha256=KJqK8oYMn-u8n8i-Jp_lG5BvCOTjwWSjWP8yAyDlWVo,297
4
+ water_column_sonar_processing/aws/dynamodb_manager.py,sha256=zD2VkI1tTt6HgxXPjCp5im3_jl12wANaP_vz37FmHNo,13957
5
+ water_column_sonar_processing/aws/s3_manager.py,sha256=JT9sCqlE1vm0rKo7bb_-T_uWd5ob21UzU58EcmC8454,16146
6
+ water_column_sonar_processing/aws/s3fs_manager.py,sha256=U5AO9-V2VdBpVZtQyIfMORz7hSyJMiATuevfe5z6VdA,2497
7
+ water_column_sonar_processing/aws/sns_manager.py,sha256=Dp9avG5VSugSWPR1dZ-askuAw1fCZkNUHbOUP65iR-k,1867
8
+ water_column_sonar_processing/aws/sqs_manager.py,sha256=NSUrWmnSC8h8Gf7gT0U8zFaQQ-yX89h0Q0mDLKGqp2Y,1597
9
+ water_column_sonar_processing/cruise/__init__.py,sha256=H5hW0JMORuaFvQk_R31B4VL8RnRyKeanOOiWmqEMZJk,156
10
+ water_column_sonar_processing/cruise/create_empty_zarr_store.py,sha256=UcUN5COMvHsxI8cNL1htUivnkI8zF9pQEnGxgVQhcDk,7855
11
+ water_column_sonar_processing/cruise/datatree_manager.py,sha256=V0gVlqIN7qZh1X8e9DSH745j6eX6rmCM3OZAxCDpiXo,620
12
+ water_column_sonar_processing/cruise/resample_regrid.py,sha256=fGJX2_ts4K_ZLcEGxo6x-nBQQeltp-FrWRCh30928Ac,14860
13
+ water_column_sonar_processing/geometry/__init__.py,sha256=KNhgfKimxuMtyZwMQlIc_OTtEkYXNXl1WyvfNJThlWY,306
14
+ water_column_sonar_processing/geometry/elevation_manager.py,sha256=Wi7xl6LFsxqUPGbrquqtPEKnOMpl3dMLJtEb7DR5o1c,4249
15
+ water_column_sonar_processing/geometry/geometry_manager.py,sha256=noPx6Es87jeE5ApQJ5ZhVUHhpJAaiFTlh5zJSxdCyys,10988
16
+ water_column_sonar_processing/geometry/line_simplification.py,sha256=b-rqPnb3zl0LNFaFoEiNCrZ-wyagIHWKWUDBDBImKwk,8353
17
+ water_column_sonar_processing/geometry/pmtile_generation.py,sha256=jgCZgmG-K-hi0O0MRUlnbx50_XM5UTP9BewE9tDpY4M,13016
18
+ water_column_sonar_processing/index/__init__.py,sha256=izEObsKiOoIJ0kZCFhvaYsBd6Ga71XJxnogjrNInw68,68
19
+ water_column_sonar_processing/index/index_manager.py,sha256=wvG0rzyR-osqpkF_ufuz2JheZAnpE_eYr-Z088rJPAc,16073
20
+ water_column_sonar_processing/model/__init__.py,sha256=FXaCdbPqxp0ogmZm9NplRirqpgMiYs1iRYgJbFbbX2Y,65
21
+ water_column_sonar_processing/model/zarr_manager.py,sha256=yv36DljPdFJn91rjtl-An9TbDa_FgN1fmN0o3_b1Seg,16026
22
+ water_column_sonar_processing/processing/__init__.py,sha256=9zLaJZlI9ylGwCWderusidz9d9wPQoadMstwyKfnao8,169
23
+ water_column_sonar_processing/processing/batch_downloader.py,sha256=PJvm8NKF0bygqnL9yYwoK5ydntUNrCEC4xPyjSrj_ZE,6650
24
+ water_column_sonar_processing/processing/raw_to_zarr.py,sha256=9Kl8Tl4pZTJ-nVTGAk5ALunPjF--jlVAtc-_dE874jI,18280
25
+ water_column_sonar_processing/utility/__init__.py,sha256=yDObMOL0_OxKWet5wffK2-XVJgoE9iwiY2q04GZrtBQ,234
26
+ water_column_sonar_processing/utility/cleaner.py,sha256=bNbs-hopWxtKAFBK0Eu18xdRErZCGZvtla3j-1bTwQw,619
27
+ water_column_sonar_processing/utility/constants.py,sha256=Xj1DeQD1YKlqSHlLs_mhxY-dyWgpx8cR6Lz0StiHvWM,2483
28
+ water_column_sonar_processing/utility/pipeline_status.py,sha256=xbl6-4ePq1Krfo18Mgr0jzWz9C_g2Kbey_QZNJuwdkI,4406
29
+ water_column_sonar_processing/utility/timestamp.py,sha256=bO0oir7KxxoEHPGRkz9FCBfOligkocUyRiWRzAq8fnU,361
30
+ water_column_sonar_processing-25.3.0.dist-info/licenses/LICENSE,sha256=lz4IpJ5_adG3S0ali-WaIpQFVTnEAOucMDQPECUVEYw,1110
31
+ water_column_sonar_processing-25.3.0.dist-info/METADATA,sha256=s2FUIanPfkXjnzg-yN4uGcBLrdAxmgWAp69qVC6_Nw0,6257
32
+ water_column_sonar_processing-25.3.0.dist-info/WHEEL,sha256=1tXe9gY0PYatrMPMDd6jXqjfpz_B-Wqm32CPfRC58XU,91
33
+ water_column_sonar_processing-25.3.0.dist-info/top_level.txt,sha256=aRYU4A7RNBlNrL4vzjytFAir3BNnmOgsvIGKKA36tg4,30
34
+ water_column_sonar_processing-25.3.0.dist-info/RECORD,,
@@ -1,5 +1,5 @@
1
1
  Wheel-Version: 1.0
2
- Generator: setuptools (75.8.0)
2
+ Generator: setuptools (77.0.3)
3
3
  Root-Is-Purelib: true
4
4
  Tag: py3-none-any
5
5
 
@@ -1,82 +0,0 @@
1
- # import json
2
-
3
-
4
- # lambda for timestamp in form "yyyy-MM-ddTHH:mm:ssZ"
5
- # dt = lambda: datetime.now().isoformat(timespec="seconds") + "Z"
6
-
7
- # https://shapely.readthedocs.io/en/stable/reference/shapely.MultiLineString.html#shapely.MultiLineString
8
- """
9
- // [Decimal / Places / Degrees / Object that can be recognized at scale / N/S or E/W at equator, E/W at 23N/S, E/W at 45N/S, E/W at 67N/S]
10
- // 0 1.0 1° 00′ 0″ country or large region 111.32 km 102.47 km 78.71 km 43.496 km
11
- // 1 0.1 0° 06′ 0″ large city or district 11.132 km 10.247 km 7.871 km 4.3496 km
12
- // 2 0.01 0° 00′ 36″ town or village 1.1132 km 1.0247 km 787.1 m 434.96 m
13
- // 3 0.001 0° 00′ 3.6″ neighborhood, street 111.32 m 102.47 m 78.71 m 43.496 m
14
- // 4 0.0001 0° 00′ 0.36″ individual street, land parcel 11.132 m 10.247 m 7.871 m 4.3496 m
15
- // 5 0.00001 0° 00′ 0.036″ individual trees, door entrance 1.1132 m 1.0247 m 787.1 mm 434.96 mm
16
- // 6 0.000001 0° 00′ 0.0036″ individual humans 111.32 mm 102.47 mm 78.71 mm 43.496 mm
17
- // 7 0.0000001 0° 00′ 0.00036″ practical limit of commercial surveying 11.132 mm 10.247 mm 7.871 mm 4.3496 mm
18
- """
19
-
20
- """
21
- private static final int SRID = 8307;
22
- private static final double simplificationTolerance = 0.0001;
23
- private static final long splitGeometryMs = 900000L;
24
- private static final int batchSize = 10000;
25
- private static final int geoJsonPrecision = 5;
26
- final int geoJsonPrecision = 5;
27
- final double simplificationTolerance = 0.0001;
28
- final int simplifierBatchSize = 3000;
29
- final long maxCount = 0;
30
- private static final double maxAllowedSpeedKnts = 60D;
31
-
32
-
33
- """
34
-
35
-
36
- class GeometrySimplification:
37
- # TODO: in the future move to standalone library
38
- #######################################################
39
- def __init__(
40
- self,
41
- ):
42
- pass
43
-
44
- #######################################################
45
- def speed_check(
46
- self,
47
- speed_knots=50,
48
- ) -> None:
49
- print(speed_knots)
50
- pass
51
-
52
- def remove_null_island_values(
53
- self,
54
- epsilon=1e-5,
55
- ) -> None:
56
- print(epsilon)
57
- pass
58
-
59
- def stream_geometry(
60
- self,
61
- ) -> None:
62
- pass
63
-
64
- def break_linestring_into_multi_linestring(
65
- self,
66
- ) -> None:
67
- # For any line-strings across the antimeridian, break into multilinestring
68
- pass
69
-
70
- def simplify(
71
- self,
72
- ) -> None:
73
- pass
74
-
75
- def kalman_filter(self):
76
- # for cruises with bad signal, filter so that
77
- pass
78
-
79
- #######################################################
80
-
81
-
82
- ###########################################################
@@ -1,34 +0,0 @@
1
- water_column_sonar_processing/__init__.py,sha256=fvRK4uFo_A0l7w_T4yckvDqJ3wMUq4JB3VVPXqWfewE,226
2
- water_column_sonar_processing/process.py,sha256=-yQtK3rnZq6lGAr3q02zLDe1NuMH9c0PiUOxKzG_r18,5386
3
- water_column_sonar_processing/aws/__init__.py,sha256=KJqK8oYMn-u8n8i-Jp_lG5BvCOTjwWSjWP8yAyDlWVo,297
4
- water_column_sonar_processing/aws/dynamodb_manager.py,sha256=htP4Y2rmOSFtdzUFrgK14Bn-UXAFG22Ow-dDrR2alSw,13949
5
- water_column_sonar_processing/aws/s3_manager.py,sha256=-PCiW7YF31nGIPa1oVOVTzjTSExAAkT_IyNNnvWv2HU,16214
6
- water_column_sonar_processing/aws/s3fs_manager.py,sha256=oouzV9DZLplPC6vzbouWPzyfyNPABx_LGxRGJGc1vWg,2563
7
- water_column_sonar_processing/aws/sns_manager.py,sha256=Dp9avG5VSugSWPR1dZ-askuAw1fCZkNUHbOUP65iR-k,1867
8
- water_column_sonar_processing/aws/sqs_manager.py,sha256=NSUrWmnSC8h8Gf7gT0U8zFaQQ-yX89h0Q0mDLKGqp2Y,1597
9
- water_column_sonar_processing/cruise/__init__.py,sha256=H5hW0JMORuaFvQk_R31B4VL8RnRyKeanOOiWmqEMZJk,156
10
- water_column_sonar_processing/cruise/create_empty_zarr_store.py,sha256=ZsFQTDA0gXfQHlxDsXBGD1qQ0ipmx4kS81DcY6ml5Ew,7767
11
- water_column_sonar_processing/cruise/datatree_manager.py,sha256=Qy4dZCW8_q31lbjxbMsx3JtBS4BvQT17_2P0QD1RQcY,639
12
- water_column_sonar_processing/cruise/resample_regrid.py,sha256=lEFBXVxh0l-LvLKbvM8XNrS7cPBpPr4g7SNikWMU16o,14049
13
- water_column_sonar_processing/geometry/__init__.py,sha256=GIzzc-_7pwEwbOkGpc4i_fmjWI5ymllXqzdHq_d3Rio,299
14
- water_column_sonar_processing/geometry/elevation_manager.py,sha256=eq9w691WJknPwWYkvO3giKTPleIxCVc2tMGR0e8ZRxQ,4267
15
- water_column_sonar_processing/geometry/geometry_manager.py,sha256=nz5T1vCDWHYIfQ853EqKYHDetTul7jRWS3y8Evep8QU,10855
16
- water_column_sonar_processing/geometry/geometry_simplification.py,sha256=im1HG9nfYIerQv3w-PUHzphw2B7aGgnsA3Zcdy2oTmA,3016
17
- water_column_sonar_processing/geometry/pmtile_generation.py,sha256=7Lm08Jr6YaM4nYmexClxbIMOqSV1teo9wMm6dfjFuNA,12384
18
- water_column_sonar_processing/index/__init__.py,sha256=izEObsKiOoIJ0kZCFhvaYsBd6Ga71XJxnogjrNInw68,68
19
- water_column_sonar_processing/index/index_manager.py,sha256=qsS6rKObJlFXKyzRuT1bk2_qW1YagW-Fg_AkQ1U_KRs,14213
20
- water_column_sonar_processing/model/__init__.py,sha256=FXaCdbPqxp0ogmZm9NplRirqpgMiYs1iRYgJbFbbX2Y,65
21
- water_column_sonar_processing/model/zarr_manager.py,sha256=YOoUBaSo_63lPY0BbTRodAjBQ33r1UY81RAZ98hBE5E,15652
22
- water_column_sonar_processing/processing/__init__.py,sha256=tdpSfwnY6lbAS_yBTu4aG0SjPgCKqh6LAFvIj_t3j3U,168
23
- water_column_sonar_processing/processing/batch_downloader.py,sha256=qXoruHdbgzAolmroK6eRn9bWgeHFgaVQLwhJ6X5oHRE,6299
24
- water_column_sonar_processing/processing/raw_to_zarr.py,sha256=Sn0_zBT7yYP6abbSTlQBPA6iZSBxeVqPYYSgoroiBEU,17599
25
- water_column_sonar_processing/utility/__init__.py,sha256=yDObMOL0_OxKWet5wffK2-XVJgoE9iwiY2q04GZrtBQ,234
26
- water_column_sonar_processing/utility/cleaner.py,sha256=bNbs-hopWxtKAFBK0Eu18xdRErZCGZvtla3j-1bTwQw,619
27
- water_column_sonar_processing/utility/constants.py,sha256=FZ2X4qrxTzy2cCj7oh6-eRm40gH-qh9N27J1Iq0IUWk,2238
28
- water_column_sonar_processing/utility/pipeline_status.py,sha256=O-0SySqdRGJ6bs3zQe1NV9vkOpmsRM7zj5QoHgzYioY,4395
29
- water_column_sonar_processing/utility/timestamp.py,sha256=bO0oir7KxxoEHPGRkz9FCBfOligkocUyRiWRzAq8fnU,361
30
- water_column_sonar_processing-25.1.7.dist-info/LICENSE,sha256=lz4IpJ5_adG3S0ali-WaIpQFVTnEAOucMDQPECUVEYw,1110
31
- water_column_sonar_processing-25.1.7.dist-info/METADATA,sha256=PA4RDK_heU1fkxr6CP8qfEu5mFjMa5RuH4vVr5yN-WQ,5959
32
- water_column_sonar_processing-25.1.7.dist-info/WHEEL,sha256=In9FTNxeP60KnTkGw7wk6mJPYd_dQSjEZmXdBdMCI-8,91
33
- water_column_sonar_processing-25.1.7.dist-info/top_level.txt,sha256=aRYU4A7RNBlNrL4vzjytFAir3BNnmOgsvIGKKA36tg4,30
34
- water_column_sonar_processing-25.1.7.dist-info/RECORD,,