dist-s1-enumerator 1.0.7__tar.gz → 1.0.9__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/.github/workflows/bump-tag-for-release.yml +1 -1
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/.github/workflows/changelog-updated.yml +1 -1
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/.github/workflows/github-release.yml +1 -1
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/.github/workflows/labeled-pr.yml +1 -1
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/.github/workflows/nb_tests.yml +1 -1
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/.github/workflows/pypi-release.yml +1 -1
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/.github/workflows/static_analysis.yml +2 -2
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/.github/workflows/tests.yml +1 -1
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/CHANGELOG.md +14 -0
- dist_s1_enumerator-1.0.9/PKG-INFO +295 -0
- dist_s1_enumerator-1.0.9/README.md +272 -0
- dist_s1_enumerator-1.0.9/src/dist_s1_enumerator/constants.py +50 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/src/dist_s1_enumerator/data/jpl_burst_geo.parquet +0 -0
- dist_s1_enumerator-1.0.9/src/dist_s1_enumerator/data/mgrs.parquet +0 -0
- dist_s1_enumerator-1.0.9/src/dist_s1_enumerator/data/mgrs_burst_lookup_table.parquet +0 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/src/dist_s1_enumerator/dist_enum.py +4 -4
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/src/dist_s1_enumerator/dist_enum_inputs.py +1 -1
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/src/dist_s1_enumerator/mgrs_burst_data.py +0 -23
- dist_s1_enumerator-1.0.9/src/dist_s1_enumerator.egg-info/PKG-INFO +295 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/src/dist_s1_enumerator.egg-info/SOURCES.txt +2 -0
- dist_s1_enumerator-1.0.9/tests/data/dist_hls_tiles.txt +19043 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/tests/test_dist_enum.py +12 -3
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/tests/test_dist_enum_inputs.py +1 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/tests/test_mgrs_burst_data.py +12 -8
- dist_s1_enumerator-1.0.7/PKG-INFO +0 -158
- dist_s1_enumerator-1.0.7/README.md +0 -135
- dist_s1_enumerator-1.0.7/src/dist_s1_enumerator/data/mgrs.parquet +0 -0
- dist_s1_enumerator-1.0.7/src/dist_s1_enumerator/data/mgrs_burst_lookup_table.parquet +0 -0
- dist_s1_enumerator-1.0.7/src/dist_s1_enumerator.egg-info/PKG-INFO +0 -158
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/.github/dependabot.yml +0 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/.gitignore +0 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/LICENSE +0 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/environment.yml +0 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/notebooks/A__Staging_Inputs_for_One_MGRS_Tile.ipynb +0 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/notebooks/B__Enumerate_MGRS_tile.ipynb +0 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/pyproject.toml +0 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/setup.cfg +0 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/src/dist_s1_enumerator/__init__.py +0 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/src/dist_s1_enumerator/asf.py +0 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/src/dist_s1_enumerator/exceptions.py +0 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/src/dist_s1_enumerator/param_models.py +0 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/src/dist_s1_enumerator/py.typed +0 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/src/dist_s1_enumerator/rtc_s1_io.py +0 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/src/dist_s1_enumerator/tabular_models.py +0 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/src/dist_s1_enumerator.egg-info/dependency_links.txt +0 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/src/dist_s1_enumerator.egg-info/not-zip-safe +0 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/src/dist_s1_enumerator.egg-info/requires.txt +0 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/src/dist_s1_enumerator.egg-info/top_level.txt +0 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/tests/conftest.py +0 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/tests/data/mock_ts_data_for_enum.ipynb +0 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/tests/data/rtc_s1_ts_metadata/chile_19HBD.parquet +0 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/tests/data/rtc_s1_ts_metadata/mgrs01UBT.parquet +0 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/tests/data/rtc_s1_ts_metadata/mgrs11SLT_11SLU_11SMT.parquet +0 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/tests/data/rtc_s1_ts_metadata/mgrs15RXN__track63.parquet +0 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/tests/data/rtc_s1_ts_metadata/mgrs22WFD.parquet +0 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/tests/data/ts_data_for_workflow_inputs.ipynb +0 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/tests/test_asf.py +0 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/tests/test_notebooks.py +0 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/tests/test_package.py +0 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/tests/test_param_model.py +0 -0
- {dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/tests/test_rtc_s1_io.py +0 -0
{dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/.github/workflows/bump-tag-for-release.yml
RENAMED
|
@@ -7,7 +7,7 @@ on:
|
|
|
7
7
|
|
|
8
8
|
jobs:
|
|
9
9
|
call-bump-version-workflow:
|
|
10
|
-
uses: ASFHyP3/actions/.github/workflows/reusable-bump-version.yml@v0.
|
|
10
|
+
uses: ASFHyP3/actions/.github/workflows/reusable-bump-version.yml@v0.21.0
|
|
11
11
|
with:
|
|
12
12
|
user: opera-pst-dev
|
|
13
13
|
email: opera-pst-dev@jpl.nasa.gov
|
{dist_s1_enumerator-1.0.7 → dist_s1_enumerator-1.0.9}/.github/workflows/changelog-updated.yml
RENAMED
|
@@ -13,6 +13,6 @@ on:
|
|
|
13
13
|
|
|
14
14
|
jobs:
|
|
15
15
|
call-changelog-check-workflow:
|
|
16
|
-
uses: ASFHyP3/actions/.github/workflows/reusable-changelog-check.yml@v0.
|
|
16
|
+
uses: ASFHyP3/actions/.github/workflows/reusable-changelog-check.yml@v0.21.0
|
|
17
17
|
secrets:
|
|
18
18
|
USER_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
@@ -5,7 +5,7 @@ on: push
|
|
|
5
5
|
jobs:
|
|
6
6
|
|
|
7
7
|
call-secrets-analysis-workflow:
|
|
8
|
-
uses: ASFHyP3/actions/.github/workflows/reusable-secrets-analysis.yml@v0.
|
|
8
|
+
uses: ASFHyP3/actions/.github/workflows/reusable-secrets-analysis.yml@v0.21.0
|
|
9
9
|
|
|
10
10
|
call-ruff-workflow:
|
|
11
|
-
uses: ASFHyP3/actions/.github/workflows/reusable-ruff.yml@v0.
|
|
11
|
+
uses: ASFHyP3/actions/.github/workflows/reusable-ruff.yml@v0.21.0
|
|
@@ -6,6 +6,20 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
|
|
6
6
|
and this project adheres to [PEP 440](https://www.python.org/dev/peps/pep-0440/)
|
|
7
7
|
and uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
8
8
|
|
|
9
|
+
## [1.0.9] - 2026-01-07
|
|
10
|
+
|
|
11
|
+
### Changed
|
|
12
|
+
* Update the delta_window_days from 365 to 60 (i.e. from a year to 2 months). This enforces a baseline to account for some seasonality via its construction.
|
|
13
|
+
|
|
14
|
+
## [1.0.8] - 2025-11-10
|
|
15
|
+
|
|
16
|
+
### Changed
|
|
17
|
+
* Updated all tables to use
|
|
18
|
+
- Water mask generated by UMD water mask utilized by DIST-S1
|
|
19
|
+
- Burst to use the correct geometries see v1.0.6
|
|
20
|
+
- 100 km^2 minimum overlap of burst data and MGRS tile.
|
|
21
|
+
|
|
22
|
+
|
|
9
23
|
## [1.0.7] - 2025-10-24
|
|
10
24
|
|
|
11
25
|
### Added
|
|
@@ -0,0 +1,295 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: dist-s1-enumerator
|
|
3
|
+
Version: 1.0.9
|
|
4
|
+
Summary: Enumeration and ops library for the OPERA DIST-S1 project
|
|
5
|
+
Author-email: "Richard West, Charlie Marshak, Talib Oliver-Cabrera, and Jungkyo Jung" <charlie.z.marshak@jpl.nasa.gov>
|
|
6
|
+
License: Apache-2.0
|
|
7
|
+
Project-URL: Homepage, https://github.com/opera-adt/dist-s1-enumerator
|
|
8
|
+
Project-URL: Bug Tracker, https://github.com/opera-adt/dist-s1-enumerator/issues
|
|
9
|
+
Project-URL: Discussions, https://github.com/opera-adt/dist-s1-enumerator/discussions
|
|
10
|
+
Project-URL: Changelog, https://github.com/opera-adt/dist-s1-enumerator/releases
|
|
11
|
+
Classifier: Intended Audience :: Developers
|
|
12
|
+
Classifier: Intended Audience :: Science/Research
|
|
13
|
+
Classifier: Programming Language :: Python :: 3
|
|
14
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
16
|
+
Classifier: Programming Language :: Python
|
|
17
|
+
Classifier: Topic :: Scientific/Engineering
|
|
18
|
+
Requires-Python: >=3.12
|
|
19
|
+
Description-Content-Type: text/markdown
|
|
20
|
+
License-File: LICENSE
|
|
21
|
+
Requires-Dist: numpy
|
|
22
|
+
Dynamic: license-file
|
|
23
|
+
|
|
24
|
+
# dist-s1-enumerator
|
|
25
|
+
|
|
26
|
+
[](https://pypi.python.org/pypi/dist-s1-enumerator/)
|
|
27
|
+
[](https://pypi.python.org/pypi/dist-s1-enumerator/)
|
|
28
|
+
[](https://pypi.python.org/pypi/dist-s1-enumerator/)
|
|
29
|
+
[](https://anaconda.org/conda-forge/dist_s1_enumerator)
|
|
30
|
+
[](https://anaconda.org/conda-forge/dist_s1_enumerator)
|
|
31
|
+
|
|
32
|
+
This is a Python library for enumerating OPERA RTC-S1 inputs necessary for the creation of OPERA DIST-S1 products.
|
|
33
|
+
The library can enumerate inputs for the creation of a single DIST-S1 product or a time-series of DIST-S1 products over a large area spanning multiple passes.
|
|
34
|
+
The DIST-S1 measures disturbance comparing a baseline of RTC-S1 images (pre-images) to a current set of acquisition images (post-images).
|
|
35
|
+
This library also provides functionality for downloading the OPERA RTC-S1 data from ASF DAAC.
|
|
36
|
+
We use "enumeration" to describe the "curation of required DIST-S1 inputs."
|
|
37
|
+
|
|
38
|
+
|
|
39
|
+
## Installation/Setup
|
|
40
|
+
|
|
41
|
+
We recommend managing dependencies and virutal environments using [mamba/conda](https://mamba.readthedocs.io/en/latest/user_guide/installation.html).
|
|
42
|
+
|
|
43
|
+
```bash
|
|
44
|
+
mamba update -f environment.yml # creates a new environment dist-s1-enumerator
|
|
45
|
+
conda activate dist-s1-enumerator
|
|
46
|
+
pip install dist-s1-enumerator
|
|
47
|
+
python -m ipykernel install --user --name dist-s1-enumerator
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
### Downloading data
|
|
51
|
+
|
|
52
|
+
For searching through the metadata of OPERA RTC-S1, you will not need any earthdata credentials.
|
|
53
|
+
For downloading data from the ASF DAAC, you will need to make sure you have a Earthdata credentials (see: https://urs.earthdata.nasa.gov/) and successfully accepted the ASF terms of use (this can be checked by downloading any product at the ASF DAAC using your Earthdata credentials: https://search.asf.alaska.edu/).
|
|
54
|
+
You will need to create or append to `~/.netrc` file with these credentials:
|
|
55
|
+
```
|
|
56
|
+
machine urs.earthdata.nasa.gov
|
|
57
|
+
login <your_username>
|
|
58
|
+
password <your_password>
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
### Development installation
|
|
62
|
+
|
|
63
|
+
Same as above replacing `pip install dist-s1-enumerator` with `pip install -e .`.
|
|
64
|
+
|
|
65
|
+
## Usage
|
|
66
|
+
|
|
67
|
+
### Motivation
|
|
68
|
+
|
|
69
|
+
We want to generate a DIST-S1 product using [dist-s1](https://github.com/opera-adt/dist-s1). We successfully installed the software, but don't know how to call the CLI:
|
|
70
|
+
|
|
71
|
+
```
|
|
72
|
+
dist-s1 run \
|
|
73
|
+
--mgrs_tile_id '19HBD' \
|
|
74
|
+
--post_date '2024-03-28' \
|
|
75
|
+
--track_number 91
|
|
76
|
+
```
|
|
77
|
+
Where do these inputs come from? Can we get them without looking up RTC-S1 products manually? Of course! That's the point of this library.
|
|
78
|
+
|
|
79
|
+
### Triggering the DIST-S1 Workflow
|
|
80
|
+
|
|
81
|
+
Each DIST-S1 product is uniquely identified in space and time by:
|
|
82
|
+
|
|
83
|
+
1. an MGRS Tile ID
|
|
84
|
+
2. a Track Number of Sentinel-1
|
|
85
|
+
3. the post-image acquisition time (within 1 day)
|
|
86
|
+
|
|
87
|
+
These pieces of information are required to generate any given DIST-S1 product.
|
|
88
|
+
Identifying all such products over time (acceptable times of the post-image) and space (MGRS tiles) allows us to enumerate all DIST-S1 products.
|
|
89
|
+
We can enumerate DIST-S1 products with this library as follows:
|
|
90
|
+
```
|
|
91
|
+
from dist_s1_enumerator import enumerate_dist_s1_workflow_inputs
|
|
92
|
+
|
|
93
|
+
workflow_inputs = enumerate_dist_s1_workflow_inputs(mgrs_tile_ids='19HBD',
|
|
94
|
+
track_numbers=None,
|
|
95
|
+
start_acq_dt='2023-11-01',
|
|
96
|
+
stop_acq_dt='2024-04-01')
|
|
97
|
+
```
|
|
98
|
+
Yields:
|
|
99
|
+
<details>
|
|
100
|
+
<summary>Output</summary>
|
|
101
|
+
|
|
102
|
+
```[{'mgrs_tile_id': '19HBD', 'post_acq_date': '2023-11-05', 'track_number': 91},
|
|
103
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2023-11-10', 'track_number': 156},
|
|
104
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2023-11-12', 'track_number': 18},
|
|
105
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2023-11-17', 'track_number': 91},
|
|
106
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2023-11-22', 'track_number': 156},
|
|
107
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2023-11-24', 'track_number': 18},
|
|
108
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2023-12-04', 'track_number': 156},
|
|
109
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2023-12-06', 'track_number': 18},
|
|
110
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2023-12-11', 'track_number': 91},
|
|
111
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2023-12-16', 'track_number': 156},
|
|
112
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2023-12-18', 'track_number': 18},
|
|
113
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2023-12-23', 'track_number': 91},
|
|
114
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2023-12-28', 'track_number': 156},
|
|
115
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2023-12-30', 'track_number': 18},
|
|
116
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2024-01-04', 'track_number': 91},
|
|
117
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2024-01-09', 'track_number': 156},
|
|
118
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2024-01-11', 'track_number': 18},
|
|
119
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2024-01-16', 'track_number': 91},
|
|
120
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2024-01-21', 'track_number': 156},
|
|
121
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2024-01-23', 'track_number': 18},
|
|
122
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2024-01-28', 'track_number': 91},
|
|
123
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2024-02-02', 'track_number': 156},
|
|
124
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2024-02-04', 'track_number': 18},
|
|
125
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2024-02-09', 'track_number': 91},
|
|
126
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2024-02-14', 'track_number': 156},
|
|
127
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2024-02-16', 'track_number': 18},
|
|
128
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2024-02-21', 'track_number': 91},
|
|
129
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2024-02-26', 'track_number': 156},
|
|
130
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2024-02-28', 'track_number': 18},
|
|
131
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2024-03-04', 'track_number': 91},
|
|
132
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2024-03-09', 'track_number': 156},
|
|
133
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2024-03-11', 'track_number': 18},
|
|
134
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2024-03-16', 'track_number': 91},
|
|
135
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2024-03-21', 'track_number': 156},
|
|
136
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2024-03-23', 'track_number': 18},
|
|
137
|
+
{'mgrs_tile_id': '19HBD', 'post_acq_date': '2024-03-28', 'track_number': 91}]
|
|
138
|
+
```
|
|
139
|
+
</details>
|
|
140
|
+
Each dictionary uniquely determines a DIST-S1 product.
|
|
141
|
+
In fact, the list above is a complete account of all DIST-S1 products over this MGRS tile and during this time period.
|
|
142
|
+
We can use any of the dictionaries in the list to trigger the DIST-S1 workflow e.g. using the last dictionary in the list above:
|
|
143
|
+
|
|
144
|
+
```
|
|
145
|
+
dist-s1 run \
|
|
146
|
+
--mgrs_tile_id '19HBD' \
|
|
147
|
+
--post_date '2024-03-28' \
|
|
148
|
+
--track_number 91
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
See the [dist-s1](https://github.com/opera-adt/dist-s1) repository for more details on the `dist-s1` usage and workflow.
|
|
152
|
+
|
|
153
|
+
### Obtaining RTC-S1 Inputs for a given DIST-S1 product
|
|
154
|
+
|
|
155
|
+
In addition to figuring out the relevant information to trigger the DIST-S1 workflow, we can query NASA's Common Metadata Repository to identify all RTC-S1 products required to create this DIST-S1 product that are used in the workflow.
|
|
156
|
+
This is done above, except we only save information required to trigger the actual DIST-S1 worklow.
|
|
157
|
+
Here is an example to get the full account of the necessary RTC-S1 input products for a given set of DIST-S1 workflow inputs:
|
|
158
|
+
```
|
|
159
|
+
from dist_s1_enumerator import enumerate_one_dist_s1_product
|
|
160
|
+
|
|
161
|
+
df_product_t91 = enumerate_one_dist_s1_product('20TLP', track_number=[91], post_date='2025-09-25')
|
|
162
|
+
df_product_t91.head()
|
|
163
|
+
```
|
|
164
|
+
<details>
|
|
165
|
+
<summary>Output</summary>
|
|
166
|
+
|
|
167
|
+
```opera_id jpl_burst_id \
|
|
168
|
+
0 OPERA_L2_RTC-S1_T091-193570-IW3_20240807T22192... T091-193570-IW3
|
|
169
|
+
1 OPERA_L2_RTC-S1_T091-193570-IW3_20240819T22192... T091-193570-IW3
|
|
170
|
+
2 OPERA_L2_RTC-S1_T091-193570-IW3_20240831T22192... T091-193570-IW3
|
|
171
|
+
3 OPERA_L2_RTC-S1_T091-193570-IW3_20240912T22192... T091-193570-IW3
|
|
172
|
+
4 OPERA_L2_RTC-S1_T091-193570-IW3_20240924T22192... T091-193570-IW3
|
|
173
|
+
|
|
174
|
+
acq_dt acq_date_for_mgrs_pass polarizations \
|
|
175
|
+
0 2024-08-07 22:19:28+00:00 2024-08-07 VV+VH
|
|
176
|
+
1 2024-08-19 22:19:28+00:00 2024-08-19 VV+VH
|
|
177
|
+
2 2024-08-31 22:19:28+00:00 2024-08-31 VV+VH
|
|
178
|
+
3 2024-09-12 22:19:29+00:00 2024-09-12 VV+VH
|
|
179
|
+
4 2024-09-24 22:19:29+00:00 2024-09-24 VV+VH
|
|
180
|
+
|
|
181
|
+
track_number pass_id url_crosspol \
|
|
182
|
+
0 91 645 https://cumulus.asf.earthdatacloud.nasa.gov/OP...
|
|
183
|
+
1 91 647 https://cumulus.asf.earthdatacloud.nasa.gov/OP...
|
|
184
|
+
2 91 649 https://cumulus.asf.earthdatacloud.nasa.gov/OP...
|
|
185
|
+
3 91 651 https://cumulus.asf.earthdatacloud.nasa.gov/OP...
|
|
186
|
+
4 91 653 https://cumulus.asf.earthdatacloud.nasa.gov/OP...
|
|
187
|
+
|
|
188
|
+
url_copol \
|
|
189
|
+
0 https://cumulus.asf.earthdatacloud.nasa.gov/OP...
|
|
190
|
+
1 https://cumulus.asf.earthdatacloud.nasa.gov/OP...
|
|
191
|
+
2 https://cumulus.asf.earthdatacloud.nasa.gov/OP...
|
|
192
|
+
3 https://cumulus.asf.earthdatacloud.nasa.gov/OP...
|
|
193
|
+
4 https://cumulus.asf.earthdatacloud.nasa.gov/OP...
|
|
194
|
+
|
|
195
|
+
geometry mgrs_tile_id \
|
|
196
|
+
0 POLYGON ((-65.58616 43.67944, -65.07523 43.740... 20TLP
|
|
197
|
+
1 POLYGON ((-65.58746 43.68056, -65.07652 43.741... 20TLP
|
|
198
|
+
2 POLYGON ((-65.58803 43.68023, -65.07706 43.741... 20TLP
|
|
199
|
+
3 POLYGON ((-65.58995 43.68007, -65.07902 43.740... 20TLP
|
|
200
|
+
4 POLYGON ((-65.5893 43.67982, -65.07838 43.7406... 20TLP
|
|
201
|
+
|
|
202
|
+
acq_group_id_within_mgrs_tile track_token input_category
|
|
203
|
+
0 2 91 pre
|
|
204
|
+
1 2 91 pre
|
|
205
|
+
2 2 91 pre
|
|
206
|
+
3 2 91 pre
|
|
207
|
+
4 2 91 pre
|
|
208
|
+
```
|
|
209
|
+
</details>
|
|
210
|
+
The output is a pandas dataframe that can be serialized using the pandas API:
|
|
211
|
+
|
|
212
|
+
```
|
|
213
|
+
df_product_t91.to_csv("df_product.csv", index=False)
|
|
214
|
+
```
|
|
215
|
+
|
|
216
|
+
For more details see the [Jupyter notebooks](./notebooks):
|
|
217
|
+
|
|
218
|
+
- [Enumerating inputs for a single DIST-S1 product](./notebooks/A__Staging_Inputs_for_One_MGRS_Tile.ipynb)
|
|
219
|
+
- [Enumerating inputs for a time-series of DIST-S1 products](./notebooks/B__Enumerate_MGRS_tile.ipynb)
|
|
220
|
+
|
|
221
|
+
### Identifiers for DIST-S1 products
|
|
222
|
+
|
|
223
|
+
As noted above, each DIST-S1 product is uniquely identified by:
|
|
224
|
+
|
|
225
|
+
1. MGRS Tile ID
|
|
226
|
+
2. Track Number
|
|
227
|
+
3. Post-image acquisition time (within 1 day)
|
|
228
|
+
|
|
229
|
+
We briefly explain why these fields uniquely identify DIST-S1 products.
|
|
230
|
+
These pieces information uniquely describe the space (MGRS tile and track) and time (post-image acquisition) that a Sentinel-1 makes a pass over a fixed area.
|
|
231
|
+
Each DIST-S1 product is resampled to an MGRS tile, so we need that.
|
|
232
|
+
While the post-image acquisition time is a lot - there are particular instances when Sentinel-1 constellation passes over the same area in a single day and so fixing the track number differentiates between the two different sets of acquired imagery occurring in the same 24 hour period.
|
|
233
|
+
In theory, we could specify the exact time of acquisition, but we have elected to use track numbers to differentiate when there Sentinel-1 constellation collects data over the same area in a single day.
|
|
234
|
+
It is also important to note that we are assuming the selection of pre-images (once a post-image set is selected) is fixed.
|
|
235
|
+
Although varying a baseline of pre-images to measure disturbance will alter the final DIST-S1 product, we assume with a fixed strategy to construct this baseline, the above 3 fields uniquely identify a DIST-S1 product.
|
|
236
|
+
|
|
237
|
+
# About the Data Tables in this Library
|
|
238
|
+
|
|
239
|
+
One of the purposes of this data is to provide easy access via standard lookups to a variety of tables associated with enumerating DIST-S1 products.
|
|
240
|
+
There are three data tables:
|
|
241
|
+
|
|
242
|
+
1. [Burst Geometry Table](src/dist_s1_enumerator/data/jpl_burst_geo.parquet) - the JPL spatially fixed bursts within 2 km of land as identified via the UMD Ocean Mask ([link](https://console.cloud.google.com/storage/browser/earthenginepartners-hansen/OceanMask;tab=objects?prefix=&forceOnObjectsSortingFiltering=false))
|
|
243
|
+
2. [MGRS Table](src/dist_s1_enumerator/data/mgrs.parquet) - the MGRS tiles that are (1) used in DIST-HLS processing (see this [list](tests/data/dist_hls_tiles.txt)) and (2) have overlapping bursts from 1.
|
|
244
|
+
3. [MGRS/Burst Lookup Table](src/dist_s1_enumerator/data/mgrs_burst_lookup_table.parquet) - this is effectively a spatial join of burst geometries and MGRS tiles to allow us to get all relevant bursts from a pass. A pass is defined to be all the data collected over an MGRS tile from Sentinel-1, i.e. all the RTC-S1 products coming from the Sentinel-1.
|
|
245
|
+
|
|
246
|
+
How these tables were created be found in this [notebook](https://github.com/OPERA-Cal-Val/dist-s1-research/blob/dev/marshak/Zc_check_bursts_without_mgrs_tile/1__Lookup%20Tables%20for%20MGRS%20and%20Bursts.ipynb).
|
|
247
|
+
It's worth noting there is some care taken to do the accounting of track numbers within a Sentinel-1 acquisition to properly identify a single data take.
|
|
248
|
+
Sentinel-1 track numbers of products increment near the equator even though they are still within the same pass.
|
|
249
|
+
Thus, we include the column `acq_group_id_within_mgrs_tile` to identify different data takes within a single MGRS tile.
|
|
250
|
+
We also filter out burst/mgrs pairs if the a Sentinel-1 pass that is smaller than 250 km^2 within the intersection. The MGRS tiles are 3660 x 3660 pixels at 30 meter resolution and so have total area of 12,056 km^2. Thus, this minimum overlap means if a data acquisition over an MGRS tile has less than about 2 percent of total possible data, then we do not need to create a DIST-S1 product for it. Because there is at least 10 km of overlap<sup>*</sup> between adjacent tiles (more at higher latitudes), this minimum coverage requirement means such excluded products will likely be better represented in adjacent MGRS tiles.
|
|
251
|
+
|
|
252
|
+
<sup>*</sup>Although there is [documentation](https://hls.gsfc.nasa.gov/products-description/tiling-system/) saying there is 4.9 overlap between tiles, looking at the MGRS tile table above, we see that overlap is closer to 10 km, or 9% of overlap of the area (since the MGRS tiles are about 109 km x 109 km).
|
|
253
|
+
|
|
254
|
+
# Testing
|
|
255
|
+
|
|
256
|
+
For the test suite:
|
|
257
|
+
|
|
258
|
+
1. Install `papermill` via `conda-forge` (currently not supported by 3.13)
|
|
259
|
+
2. Run `pytest tests`
|
|
260
|
+
|
|
261
|
+
There are two category of tests: unit tests and integration tests. The former can be run using `pytest tests -m 'not integration'` and similarly the latter with `pytest tests -m 'integration'`. The intgeration tests are those that can be integrated into the DAAC data access workflows and thus require internet access with earthdata credentials setup correctly (as described above). The unit tests mock the necessary data inputs.
|
|
262
|
+
The integration tests that are the most time consuming are represented by the notebooks and are run only upon a release PR.
|
|
263
|
+
These notebook tests are tagged with `notebooks` and can be excluded from the other tests with `pytest tests -m 'not notebooks'`.
|
|
264
|
+
|
|
265
|
+
# Remarks about the Dateline/Dateline and Geometry
|
|
266
|
+
|
|
267
|
+
The antimeridian (or dateline) is the line at the -180 longitude mark that global CRS tiles are wrapped by standard global reference systems.
|
|
268
|
+
The geometries of the bursts and the MGRS tiles in this package are all in `epsg:4326` (standard lon/lat).
|
|
269
|
+
The geometries are all between -180 and 180 so those geometries that cross the antimeridian/dateline are generally wrapped.
|
|
270
|
+
For MGRS tiles, the statement that a geometry overlaps the antimeridian occurs if and only if the geometry is a Polygon.
|
|
271
|
+
The same is true for burst geometries.
|
|
272
|
+
See `test_antimeridian_crossing` in [`tests/test_mgrs_burst_data.py`](tests/test_mgrs_burst_data.py).
|
|
273
|
+
|
|
274
|
+
# Contributing
|
|
275
|
+
|
|
276
|
+
We welcome contributions to this open-source package. To do so:
|
|
277
|
+
|
|
278
|
+
1. Create an GitHub issue ticket desrcribing what changes you need (e.g. issue-1)
|
|
279
|
+
2. Fork this repo
|
|
280
|
+
3. Make your modifications in your own fork
|
|
281
|
+
4. Make a pull-request (PR) in this repo with the code in your fork and tag the repo owner or a relevant contributor.
|
|
282
|
+
|
|
283
|
+
We use `ruff` and associated linting packages to ensure some basic code quality (see the `environment.yml`). These will be checked for each commit in a PR. Try to write tests wherever possible.
|
|
284
|
+
|
|
285
|
+
# Support
|
|
286
|
+
|
|
287
|
+
1. Create an GitHub issue ticket desrcribing what changes you would like to see or to report a bug.
|
|
288
|
+
2. We will work on solving this issue (hopefully with you).
|
|
289
|
+
|
|
290
|
+
# Acknowledgements
|
|
291
|
+
|
|
292
|
+
See the [LICENSE](LICENSE.txt) file for copyright information.
|
|
293
|
+
|
|
294
|
+
This package was developed as part of the Observational Products for End-Users from Remote Sensing Analysis ([OPERA](https://www.jpl.nasa.gov/go/opera/)) project. This work was originally carried out at the Jet Propulsion Laboratory, California Institute of Technology, under a contract with the National Aeronautics and Space Administration (80NM0018D0004).
|
|
295
|
+
Copyright 2024 by the California Institute of Technology. United States Government Sponsorship acknowledged.
|