opencode-skills-collection 4.0.72 → 4.0.73

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,159 @@
1
+ ---
2
+ name: weather-data-lifecycle-management
3
+ description: "Manage ownership, retention, and cleanup of downloaded weather data across one-shot jobs, interactive viewers, caches, failures, and cancellation."
4
+ category: data
5
+ risk: safe
6
+ source: self
7
+ source_type: self
8
+ date_added: "2026-09-24"
9
+ author: ShianMike
10
+ tags: [weather, data-lifecycle, cleanup, caching, desktop-apps]
11
+ tools: [claude, cursor, gemini, codex]
12
+ ---
13
+
14
+ # Weather Data Lifecycle Management
15
+
16
+ ## Overview
17
+
18
+ Keep downloaded and derived weather data for exactly as long as its consumer
19
+ needs it. Make ownership explicit so one-shot jobs clean promptly, interactive
20
+ views retain their inputs until close, and shared caches are never deleted as
21
+ request cleanup.
22
+
23
+ This skill governs lifetime and ownership. It does not choose a provider,
24
+ decode scientific data, or define a cache eviction policy by itself.
25
+
26
+ ## When to Use This Skill
27
+
28
+ - A completed render leaves large temporary files behind.
29
+ - An interactive view fails because its backing data was deleted too early.
30
+ - Cancellation or exceptions leak partial downloads and derived files.
31
+ - Request-owned files and a shared user cache are mixed together.
32
+ - Background workers transfer ownership of data to a UI or later processing
33
+ stage.
34
+
35
+ ## Classify Every Path
36
+
37
+ Assign each file or directory one owner and one lifetime:
38
+
39
+ | Class | Owner | Delete when |
40
+ | --- | --- | --- |
41
+ | Request workspace | One operation | Operation finishes or fails after durable outputs are saved |
42
+ | Interactive-session data | Viewer or session | Final consumer closes or display creation fails |
43
+ | Shared cache | Cache manager | Explicit eviction policy permits it |
44
+ | User export | User | Never as automatic request cleanup |
45
+ | Partial file | Active writer | Failure, cancellation, or successful atomic promotion |
46
+
47
+ If ownership cannot be stated, do not delete the path. Resolve ownership first.
48
+
49
+ ## One-Shot Workflow
50
+
51
+ Use a request-specific temporary directory, keep all transient downloads and
52
+ intermediate products inside it, and copy or atomically move only the requested
53
+ artifact to its durable destination.
54
+
55
+ ```python
56
+ from pathlib import Path
57
+ from tempfile import TemporaryDirectory
58
+
59
+
60
+ def render_once(destination):
61
+ destination = Path(destination)
62
+ destination.parent.mkdir(parents=True, exist_ok=True)
63
+
64
+ with TemporaryDirectory(prefix="weather-job-") as workspace:
65
+ workspace = Path(workspace)
66
+ source = fetch_into(workspace)
67
+ temporary_output = render(source, workspace / "result.png")
68
+ temporary_output.replace(destination)
69
+
70
+ return destination
71
+ ```
72
+
73
+ The final artifact must live outside the temporary directory. If replacement
74
+ across filesystems is not atomic, copy to a temporary file beside the final
75
+ destination, verify it, then replace locally.
76
+
77
+ ## Interactive Workflow
78
+
79
+ An interactive consumer outlives the worker that created its data. Transfer
80
+ ownership only after the viewer or session is successfully created:
81
+
82
+ 1. The worker owns the request workspace during acquisition.
83
+ 2. If acquisition or display creation fails, the worker cleans it immediately.
84
+ 3. On successful display, attach one idempotent cleanup callback to the real
85
+ consumer close or destroyed event.
86
+ 4. The consumer becomes the owner; the worker must not also delete the data in
87
+ its ordinary `finally` path.
88
+ 5. If multiple consumers share the same data, release it only after the final
89
+ lease closes.
90
+
91
+ Do not tie cleanup to a temporary dialog, local variable, or signal that can
92
+ fire before the actual consumer is finished.
93
+
94
+ ## Failure and Cancellation Rules
95
+
96
+ - Keep incomplete downloads under a distinct suffix or directory so they cannot
97
+ be mistaken for valid data.
98
+ - Close datasets, file handles, and memory maps before removing their paths.
99
+ - Put request-owned cleanup in `finally`, but exclude resources whose ownership
100
+ was successfully transferred.
101
+ - Make cleanup idempotent; a cancellation signal and a close event may race.
102
+ - Preserve a durable partial-result manifest when the user can retry remaining
103
+ work.
104
+ - On cleanup failure, report the exact owned path without deleting broader
105
+ parent directories.
106
+
107
+ ## Shared Cache Boundary
108
+
109
+ - A request may read or populate a shared cache but does not own the cache root.
110
+ - Promote validated entries atomically; never expose a partial file as a hit.
111
+ - Use cache identity and eviction metadata rather than deleting files by age or
112
+ filename guesswork inside request code.
113
+ - Do not place user exports inside a directory that normal cache cleanup owns.
114
+ - Test that two concurrent consumers do not remove data still leased by the
115
+ other.
116
+
117
+ ## Verification Checklist
118
+
119
+ - Every created path has a named owner and deletion event.
120
+ - One-shot operations retain only requested durable outputs.
121
+ - Interactive data remains available until the final consumer closes.
122
+ - Display-creation failure cleans immediately.
123
+ - Exceptions and cancellation remove partial request-owned files.
124
+ - Cleanup is idempotent and closes open handles first.
125
+ - Shared caches and user exports are outside request cleanup scope.
126
+ - Tests exercise success, failure, cancellation, and close-event paths.
127
+
128
+ ## Security & Safety Notes
129
+
130
+ - Resolve and verify cleanup targets before recursive deletion.
131
+ - Never delete a home directory, workspace root, shared cache root, or user-
132
+ selected directory as a computed fallback.
133
+ - Avoid globs and unresolved environment variables for destructive cleanup.
134
+ - Do not follow untrusted symlinks or junctions outside the owned workspace.
135
+ - Keep sensitive temporary data in an access-controlled location and remove it
136
+ when its approved lifetime ends.
137
+ - Report material cleanup failures instead of silently leaving private data.
138
+
139
+ ## Common Pitfalls
140
+
141
+ - **The image renders but source data leaks:** Cleanup is absent from the
142
+ one-shot `finally` path or data was written outside the request workspace.
143
+ - **The viewer opens and then breaks:** The worker deleted data immediately
144
+ after emitting the viewer. Transfer ownership to the viewer close event.
145
+ - **Cancellation deletes useful completed work:** Durable artifacts and
146
+ request-owned intermediates share a directory. Separate them.
147
+ - **A cache disappears:** Request cleanup treated a shared cache as owned data.
148
+ Delegate eviction to the cache manager.
149
+ - **Windows refuses deletion:** A dataset or mapped file is still open. Close
150
+ every consumer before cleanup.
151
+
152
+ ## Limitations
153
+
154
+ - Multi-process leases require coordination beyond an in-memory reference
155
+ count.
156
+ - Some decoders create files outside their requested directory; configure or
157
+ inventory those outputs before promising complete cleanup.
158
+ - Lifecycle correctness prevents leaks and premature deletion but does not
159
+ validate the scientific contents of retained data.
@@ -0,0 +1,292 @@
1
+ ---
2
+ name: weather-data-reproducibility
3
+ description: "Record and verify provenance manifests for weather-data inputs and derived artifacts, including object identity, selections, software versions, transformations, and hashes."
4
+ category: data
5
+ risk: safe
6
+ source: self
7
+ source_type: self
8
+ date_added: "2026-09-24"
9
+ author: ShianMike
10
+ tags: [weather, reproducibility, provenance, checksums, grib2, netcdf]
11
+ tools: [claude, cursor, gemini, codex]
12
+ ---
13
+
14
+ # Weather Data Reproducibility
15
+
16
+ ## Overview
17
+
18
+ Create a compact provenance manifest that identifies the exact weather inputs,
19
+ subsets, software, transformations, and output artifacts used by a run. Verify
20
+ the manifest before claiming that a result can be replayed or reproduced.
21
+
22
+ This skill records and checks lineage. It does not discover or download data by
23
+ itself; pair it with the appropriate model, observation, radar, or satellite
24
+ fetching skill.
25
+
26
+ ## When to Use This Skill
27
+
28
+ - A map, sounding, time series, dataset, or benchmark must be auditable.
29
+ - Weather inputs came from mutable URLs, mirrors, byte-range subsets, or caches.
30
+ - A result must distinguish model initialization, forecast lead, and valid time.
31
+ - Temporary GRIB2, NetCDF, radar, satellite, or observation files will be
32
+ deleted after processing.
33
+ - Two outputs differ and their inputs or processing environment must be compared.
34
+ - A publication or release needs machine-readable data and software provenance.
35
+
36
+ Do not claim bit-for-bit reproducibility merely because a source URL and run
37
+ time were logged.
38
+
39
+ ## Reproducibility Contract
40
+
41
+ Decide which claim the workflow supports:
42
+
43
+ - **Traceable:** The source and processing history can be audited.
44
+ - **Replayable:** The same identified inputs can still be retrieved and the
45
+ recorded procedure can be rerun.
46
+ - **Bitwise reproducible:** The rerun is expected to produce identical bytes in
47
+ a pinned environment.
48
+ - **Scientifically reproducible:** Equivalent inputs and methods produce the
49
+ same scientific conclusion within a stated tolerance.
50
+
51
+ Record the intended claim and its tolerance. Mutable remote objects, floating
52
+ software versions, lossy images, parallel reductions, and platform-dependent
53
+ code can make the stronger claims impossible.
54
+
55
+ ## Minimum Manifest
56
+
57
+ Use a versioned JSON document. Keep request intent separate from the source that
58
+ was actually resolved.
59
+
60
+ ```json
61
+ {
62
+ "schema_version": 1,
63
+ "created_utc": "2026-09-18T09:00:00Z",
64
+ "claim": {"level": "replayable", "tolerance": null},
65
+ "request": {
66
+ "dataset": "hrrr",
67
+ "product": "prs",
68
+ "requested_valid_time": "2026-09-18T12:00:00Z",
69
+ "variables": ["temperature", "relative_humidity"]
70
+ },
71
+ "resolved": {
72
+ "initialization": "2026-09-18T06:00:00Z",
73
+ "forecast_hour": 6,
74
+ "valid_time": "2026-09-18T12:00:00Z",
75
+ "provider": "aws"
76
+ },
77
+ "inputs": [
78
+ {
79
+ "role": "pressure_fields",
80
+ "uri": "s3://bucket/exact-object-key",
81
+ "object_identity": {
82
+ "etag": "opaque-identity",
83
+ "last_modified": "2026-09-18T07:01:02Z",
84
+ "size_bytes": 987654321
85
+ },
86
+ "selection": {
87
+ "inventory_uri": "s3://bucket/exact-object-key.idx",
88
+ "inventory_sha256": "HEX_DIGEST",
89
+ "byte_ranges": [[1000, 1999], [5000, 6999]]
90
+ },
91
+ "materialized_sha256": "HEX_DIGEST"
92
+ }
93
+ ],
94
+ "processing": {
95
+ "software": {"application": "1.2.3", "python": "3.13.7"},
96
+ "parameters": {"point": [35.22, -97.44], "nearest_method": "model-aware"},
97
+ "transformations": ["decode GRIB2", "convert K to degC"]
98
+ },
99
+ "artifacts": [
100
+ {
101
+ "path": "sounding.png",
102
+ "media_type": "image/png",
103
+ "size_bytes": 123456,
104
+ "sha256": "HEX_DIGEST"
105
+ }
106
+ ]
107
+ }
108
+ ```
109
+
110
+ Use `null` for an intentionally absent value and omit fields whose meaning is
111
+ unknown. Never fill a required-looking field with a guess.
112
+
113
+ ## Identify Every Input
114
+
115
+ For each remote object or API response, record as available:
116
+
117
+ - provider, dataset, endpoint, bucket, and exact key or immutable URL;
118
+ - version ID, generation number, release, or dataset revision;
119
+ - ETag as opaque object identity, `Last-Modified`, and content length;
120
+ - provider-supplied checksum and checksum algorithm;
121
+ - a locally computed SHA-256 for downloaded or materialized bytes;
122
+ - response media type and content encoding;
123
+ - retrieval time and successful fallback provider;
124
+ - license or attribution identifier when required.
125
+
126
+ An S3 ETag is not always an MD5 digest, particularly for multipart or encrypted
127
+ objects. Store it for identity and compute a cryptographic content hash when
128
+ the actual bytes must be verified.
129
+
130
+ ## Record the Selection
131
+
132
+ The source object alone is insufficient when only part of it was used. Record:
133
+
134
+ - GRIB2 inventory URI and inventory content hash;
135
+ - exact inventory rows, search expression, and inclusive byte ranges;
136
+ - Zarr group, array names, chunk keys, and coordinate slices;
137
+ - API query parameters and a hash or retained copy of the raw response;
138
+ - station identifier scheme and observation time interval;
139
+ - radar site/product and scan interval;
140
+ - satellite platform, product, sector, channel, and scan interval;
141
+ - requested coordinates plus the actual selected grid point and distance.
142
+
143
+ Preserve the order in which selected binary ranges were assembled. Hash the
144
+ materialized subset separately from the full remote object's identity.
145
+
146
+ ## Record Weather Time Semantics
147
+
148
+ Use explicit UTC timestamps and name their meaning:
149
+
150
+ - models: initialization, forecast lead, and valid time;
151
+ - station observations: observation, correction, and ingestion time;
152
+ - radar: volume or product scan start and end;
153
+ - satellite: scan start, scan end, and file creation time;
154
+ - retrieval: when the client fetched the metadata and bytes.
155
+
156
+ Do not replace these with one ambiguous `timestamp` field. Record the calendar
157
+ and leap-second handling if the source or application requires it.
158
+
159
+ ## Record Processing and Environment
160
+
161
+ Capture only details that can change the result:
162
+
163
+ - application version and source commit, including dirty-tree status;
164
+ - decoder and scientific library versions;
165
+ - runtime and operating-system or architecture details when relevant;
166
+ - command arguments or structured parameters;
167
+ - unit conversions, QC decisions, interpolation, coordinate selection, and
168
+ aggregation rules;
169
+ - deterministic random seed when randomness is present;
170
+ - container image digest or environment lock-file hash when available.
171
+
172
+ Do not dump an entire environment full of unrelated packages merely because it
173
+ is easy. Prefer a lock file plus the versions of software that actually touched
174
+ the data.
175
+
176
+ ## Hash and Write Atomically
177
+
178
+ The Python standard library is sufficient for local artifact hashes and a
179
+ canonical, atomically replaced JSON manifest:
180
+
181
+ ```python
182
+ import hashlib
183
+ import json
184
+ import os
185
+ from pathlib import Path
186
+
187
+
188
+ def sha256_file(path, chunk_size=1024 * 1024):
189
+ digest = hashlib.sha256()
190
+ with Path(path).open("rb") as handle:
191
+ for chunk in iter(lambda: handle.read(chunk_size), b""):
192
+ digest.update(chunk)
193
+ return digest.hexdigest()
194
+
195
+
196
+ def write_manifest(path, manifest):
197
+ path = Path(path)
198
+ temporary = path.with_suffix(path.suffix + ".tmp")
199
+ payload = json.dumps(
200
+ manifest, ensure_ascii=False, sort_keys=True, separators=(",", ":")
201
+ ) + "\n"
202
+ temporary.write_text(payload, encoding="utf-8", newline="\n")
203
+ os.replace(temporary, path)
204
+ ```
205
+
206
+ Hash artifacts only after their writers are closed. If the manifest itself must
207
+ be signed, sign the canonical bytes using the project's established signing
208
+ workflow rather than inventing a custom scheme.
209
+
210
+ ## Verify and Replay
211
+
212
+ Before replay:
213
+
214
+ 1. Validate the manifest schema version and required fields.
215
+ 2. Re-resolve every immutable object identity. Fail if an object changed unless
216
+ the reproducibility contract explicitly permits an equivalent replacement.
217
+ 3. Download or locate inputs and compare size plus a real checksum.
218
+ 4. Reapply the recorded selection and confirm the materialized subset hash.
219
+ 5. Recreate the pinned processing environment and transformations.
220
+ 6. Produce artifacts in a new output directory.
221
+ 7. Compare artifact hashes for bitwise claims or documented scientific metrics
222
+ and tolerances for scientific claims.
223
+ 8. Write a new run manifest that references, but does not overwrite, the
224
+ original.
225
+
226
+ Report the first mismatch with its role, expected value, and actual value.
227
+
228
+ ## Temporary Data Lifecycle
229
+
230
+ - Write the final artifact and its manifest before deleting request-owned input
231
+ files.
232
+ - Keep cleanup in `finally` so failures and cancellation do not leak large
233
+ GRIB2, NetCDF, radar, or observation files.
234
+ - Never delete a shared user cache as request cleanup.
235
+ - If raw inputs are deleted and the remote source is mutable or short-lived,
236
+ label the run traceable rather than replayable.
237
+ - A manifest is small and should normally outlive transient downloads.
238
+
239
+ ## Verification Checklist
240
+
241
+ - Request intent and resolved data identity are separate.
242
+ - Every timestamp is UTC and has explicit semantics.
243
+ - Every input has an exact locator, identity metadata, and content hash when
244
+ bytes were materialized.
245
+ - Subset rules, inventory identity, byte ranges, and selected coordinates are
246
+ recorded.
247
+ - Software versions and result-changing transformations are explicit.
248
+ - Final artifacts have size, media type, and SHA-256.
249
+ - The claimed reproducibility level matches what can actually be replayed.
250
+ - The manifest contains no secrets or machine-specific private information.
251
+
252
+ ## Security & Safety Notes
253
+
254
+ - Never record credentials, authorization headers, cookies, signed-query
255
+ strings, private bucket names, or secret environment variables.
256
+ - Redact user names and unnecessary absolute local paths before sharing a
257
+ manifest.
258
+ - Treat a hash as an integrity value, not proof that an untrusted input is safe.
259
+ - Validate manifest paths before opening files; do not allow `..` traversal or
260
+ writes outside the intended output directory.
261
+ - Keep TLS verification enabled when re-fetching inputs.
262
+ - Preserve license, attribution, redistribution, and access restrictions.
263
+
264
+ ## Common Pitfalls
265
+
266
+ - **ETag was labeled SHA-256 or MD5:** ETag semantics depend on the provider and
267
+ upload method. Store it as identity and compute a real local hash.
268
+ - **A model result cannot be recreated:** Initialization and forecast lead were
269
+ collapsed into valid time. Record all three.
270
+ - **The same URI returns different bytes:** The remote object was mutable and no
271
+ version or checksum was pinned. Downgrade the claim or retain the input.
272
+ - **A GRIB subset differs:** The inventory version, selected rows, or inclusive
273
+ byte ranges were omitted.
274
+ - **The artifact hash changes across systems:** The process is scientifically,
275
+ but not bitwise, reproducible. Define and test a numerical tolerance.
276
+
277
+ ## Limitations
278
+
279
+ - A manifest cannot restore data that was deleted from a mutable source unless
280
+ the input bytes or an immutable archive were retained.
281
+ - Exact replay may require licensed software, hardware, or provider access not
282
+ captured in a public manifest.
283
+ - Provenance establishes lineage and integrity; it does not establish forecast
284
+ skill, observational truth, or scientific validity.
285
+
286
+ ## Additional Resources
287
+
288
+ - [W3C PROV overview](https://www.w3.org/TR/prov-overview/)
289
+ - [Amazon S3 object metadata](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingMetadata.html)
290
+ - [Amazon S3 checksum guidance](https://docs.aws.amazon.com/AmazonS3/latest/userguide/checking-object-integrity.html)
291
+ - [CF metadata conventions](https://cfconventions.org/)
292
+ - [NetCDF Climate and Forecast conventions](https://www.ncei.noaa.gov/products/netcdf)
@@ -0,0 +1,241 @@
1
+ ---
2
+ name: weather-model-run-discovery
3
+ description: "Resolve the newest complete numerical weather prediction cycle and forecast objects across provider mirrors without downloading full payloads."
4
+ category: data
5
+ risk: safe
6
+ source: self
7
+ source_type: self
8
+ date_added: "2026-09-24"
9
+ author: ShianMike
10
+ tags: [weather, nwp, model-cycle, aws-s3, nomads, herbie]
11
+ tools: [claude, cursor, gemini, codex]
12
+ ---
13
+
14
+ # Weather Model Run Discovery
15
+
16
+ ## Overview
17
+
18
+ Resolve a numerical weather prediction run before downloading model fields.
19
+ Probe exact objects or narrow prefixes, verify a request-specific completion
20
+ contract, and return the newest qualifying initialization with its source
21
+ metadata.
22
+
23
+ This skill decides *which run and objects exist*. Use
24
+ `weather-model-data-fetching` afterward to transfer or subset the data.
25
+
26
+ ## When to Use This Skill
27
+
28
+ - An application needs the newest usable GFS, GEFS, HRRR, RAP, NAM, or similar
29
+ model cycle.
30
+ - The nominal latest cycle returns `404`, has only early forecast hours, or is
31
+ still changing.
32
+ - Equivalent AWS, NOMADS, Google, Azure, or other mirrors must be compared
33
+ without silently mixing runs.
34
+ - A scheduled job needs a bounded fallback to an older complete cycle.
35
+ - The resolved run and provider must be recorded for later reproduction.
36
+
37
+ Do not activate this skill when the initialization is already fixed or when the
38
+ task is to download, decode, or interpret model variables.
39
+
40
+ ## Define Complete Before Probing
41
+
42
+ Collect the following request contract:
43
+
44
+ - model, domain, product, and ensemble member;
45
+ - legal UTC cycle hours and cycle interval;
46
+ - required forecast hours;
47
+ - required companion objects, such as pressure and surface products;
48
+ - required inventory or index sidecars;
49
+ - provider priority and whether cross-provider fallback is allowed;
50
+ - maximum lookback, maximum probes, and freshness requirement.
51
+
52
+ A run is complete only when every sentinel required by the consumer exists.
53
+ Examples include:
54
+
55
+ - one product at one forecast hour for a single-image job;
56
+ - both pressure-level and surface files for a sounding;
57
+ - the terminal forecast hour needed by a time series;
58
+ - every required member for an ensemble statistic.
59
+
60
+ The presence of forecast hour zero does not prove that a run is complete.
61
+
62
+ ## Discovery Workflow
63
+
64
+ 1. Normalize the current time and all candidate initialization times to UTC.
65
+ 2. Enumerate only legal cycles, newest first, within the lookback bound.
66
+ 3. Build documented object keys for the exact model, product, member, and
67
+ forecast hour. Never recursively scan an entire bucket.
68
+ 4. Probe exact objects with `HeadObject`, HTTP `HEAD`, or a narrow paginated
69
+ prefix listing. A probe should retrieve metadata, not the model payload.
70
+ 5. Require every sentinel and sidecar in the completion contract. Reject zero-
71
+ length or obviously placeholder objects.
72
+ 6. For a live mirror that may expose objects before publication finishes,
73
+ repeat the metadata probe after a short bounded interval and require stable
74
+ size and object identity.
75
+ 7. Select the first complete candidate. A fallback provider must refer to the
76
+ same initialization, product, member, and forecast hour.
77
+ 8. Emit a resolution record before starting the download.
78
+
79
+ Use a provider's documented cycle status page as supporting evidence, not as a
80
+ substitute for probing the exact objects the consumer requires.
81
+
82
+ ## Minimal Resolver Example
83
+
84
+ Keep provider-specific key construction separate from the selection rule. This
85
+ example accepts a metadata-only `probe` callback so it can be used with S3,
86
+ HTTP, or a test fixture.
87
+
88
+ ```python
89
+ from datetime import datetime, timedelta, timezone
90
+
91
+
92
+ def candidate_cycles(now, cycle_hours, lookback):
93
+ now = now.astimezone(timezone.utc).replace(minute=0, second=0, microsecond=0)
94
+ allowed = sorted(set(cycle_hours), reverse=True)
95
+ candidates = []
96
+ for days_back in range((lookback // 24) + 2):
97
+ day = (now - timedelta(days=days_back)).date()
98
+ for hour in allowed:
99
+ cycle = datetime(day.year, day.month, day.day, hour, tzinfo=timezone.utc)
100
+ if cycle <= now and now - cycle <= timedelta(hours=lookback):
101
+ candidates.append(cycle)
102
+ return sorted(set(candidates), reverse=True)
103
+
104
+
105
+ def resolve_latest(now, cycle_hours, lookback, required_objects, probe):
106
+ """Return one complete run; probe(uri) returns metadata or None."""
107
+ attempts = []
108
+ for cycle in candidate_cycles(now, cycle_hours, lookback):
109
+ requested = required_objects(cycle)
110
+ found = {name: probe(uri) for name, uri in requested.items()}
111
+ missing = [name for name, metadata in found.items() if not metadata]
112
+ attempts.append({"cycle": cycle.isoformat(), "missing": missing})
113
+ if not missing:
114
+ return {
115
+ "initialization": cycle.isoformat(),
116
+ "objects": [
117
+ {"role": name, "uri": requested[name], **found[name]}
118
+ for name in requested
119
+ ],
120
+ "attempts": attempts,
121
+ }
122
+ raise LookupError(f"no complete run within {lookback} hours: {attempts}")
123
+ ```
124
+
125
+ The caller must make `required_objects` express the real completion contract;
126
+ checking a single convenient file defeats the purpose of discovery.
127
+
128
+ ## Public S3 Diagnostics
129
+
130
+ For public NOAA Open Data buckets, inspect exact objects without loading AWS
131
+ credentials:
132
+
133
+ ```bash
134
+ aws s3api head-object --no-sign-request \
135
+ --bucket BUCKET \
136
+ --key "EXACT/DOCUMENTED/OBJECT"
137
+
138
+ aws s3api list-objects-v2 --no-sign-request \
139
+ --bucket BUCKET \
140
+ --prefix "MODEL.DATE/DOMAIN/PRECISE-RUN-PREFIX" \
141
+ --max-items 100
142
+ ```
143
+
144
+ Paginate narrow listings. S3 returns at most one page at a time, and a
145
+ successful listing still needs defensive response parsing. `403` and `404`
146
+ from `HeadObject` can be intentionally nonspecific; distinguish a missing
147
+ public object from a permission or endpoint error before falling back.
148
+
149
+ ## Resolution Record
150
+
151
+ Return enough information for the fetcher and provenance layer to use the exact
152
+ same objects:
153
+
154
+ ```json
155
+ {
156
+ "requested_at_utc": "2026-09-18T08:45:00Z",
157
+ "model": "hrrr",
158
+ "product": "prs",
159
+ "initialization": "2026-09-18T06:00:00Z",
160
+ "required_forecast_hours": [0, 6, 18],
161
+ "provider": "aws",
162
+ "objects": [
163
+ {
164
+ "role": "pressure_f018",
165
+ "uri": "s3://bucket/exact-key",
166
+ "size_bytes": 123456,
167
+ "etag": "opaque-object-identity",
168
+ "last_modified": "2026-09-18T07:12:34Z"
169
+ }
170
+ ],
171
+ "fallback_cycles_rejected": []
172
+ }
173
+ ```
174
+
175
+ Keep ETags as opaque identity values. They are not guaranteed to be full-file
176
+ MD5 checksums.
177
+
178
+ ## Reliability and Caching
179
+
180
+ - Cache positive results only briefly enough for the model cadence and
181
+ publication latency. Cache negative probes for a shorter interval.
182
+ - Retry timeouts, `408`, `429`, and transient `5xx` responses with bounded
183
+ exponential backoff and jitter; honor `Retry-After`.
184
+ - Bound lookback and total probes so a provider outage cannot create an
185
+ unbounded bucket walk.
186
+ - Do not treat one mirror's lag as proof that the run is absent everywhere.
187
+ - Do not mix objects from different providers unless their run identity and
188
+ product semantics have been verified equivalent.
189
+ - Record rejected cycles and missing sentinels so fallback is visible.
190
+
191
+ ## Verification Checklist
192
+
193
+ - Candidate cycles are legal for the requested model and expressed in UTC.
194
+ - Every required product, member, forecast hour, and sidecar is present.
195
+ - Object sizes are nonzero and stable when a stabilization check is required.
196
+ - The resolved provider and exact object identities are recorded.
197
+ - The selected run satisfies the user's freshness and lookback bounds.
198
+ - No model payload was downloaded merely to discover availability.
199
+ - A failure reports which cycles were checked and which sentinels were absent.
200
+
201
+ ## Security & Safety Notes
202
+
203
+ - Probe only public datasets or resources the user is authorized to access.
204
+ - Never embed cloud credentials, signed URLs, session tokens, or private
205
+ endpoint details in a resolution record.
206
+ - Keep TLS verification enabled. Unsigned public S3 access is not the same as
207
+ disabling certificate verification.
208
+ - Bound prefix width, pagination, retries, and request rate before probing.
209
+ - Treat provider object names and metadata as untrusted input when writing logs
210
+ or constructing local paths.
211
+
212
+ ## Common Pitfalls
213
+
214
+ - **Newest cycle has `f000`:** Later required forecast hours are still
215
+ publishing. Test the actual terminal sentinels.
216
+ - **A provider fallback changes the forecast:** The fallback changed a product,
217
+ member, or cycle as well as the mirror. Reject it.
218
+ - **A listing found an object but the download changes:** The publisher was
219
+ still updating it. Require stable metadata or conditional reads.
220
+ - **Discovery is slow and expensive:** The prefix is too broad or pagination is
221
+ unbounded. Construct exact keys wherever possible.
222
+ - **Local time selected the wrong day:** Perform cycle arithmetic in UTC and
223
+ convert only for display.
224
+
225
+ ## Limitations
226
+
227
+ - Publication schedules, provider retention, key layouts, and mirror status can
228
+ change; verify them against current provider documentation.
229
+ - Metadata stability reduces the chance of a partial publication but does not
230
+ prove that the model contents are scientifically valid.
231
+ - This skill does not fetch GRIB2 fields, decode model data, or assess forecast
232
+ quality.
233
+
234
+ ## Additional Resources
235
+
236
+ - [NCEP NOMADS data availability](https://nomads.ncep.noaa.gov/)
237
+ - [NCEP NOMADS status](https://nomads.ncep.noaa.gov/status.php)
238
+ - [Herbie latest model run guidance](https://herbie.readthedocs.io/en/latest/user_guide/tutorial/latest.html)
239
+ - [Amazon S3 `HeadObject`](https://docs.aws.amazon.com/AmazonS3/latest/API/API_HeadObject.html)
240
+ - [Amazon S3 `ListObjectsV2`](https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html)
241
+ - [Amazon S3 conditional requests](https://docs.aws.amazon.com/AmazonS3/latest/userguide/conditional-requests.html)