opencode-skills-collection 4.0.71 → 4.0.73
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bundled-skills/.antigravity-install-manifest.json +13 -1
- package/bundled-skills/agent-memory-discipline/SKILL.md +204 -0
- package/bundled-skills/agent-memory-discipline/references/entry-format.md +37 -0
- package/bundled-skills/antigravity-maintainer-batch-release/SKILL.md +2 -2
- package/bundled-skills/chatexport-need-miner/SKILL.md +196 -0
- package/bundled-skills/dali-short-address-commissioner/SKILL.md +185 -0
- package/bundled-skills/eol-resistor-calculator/SKILL.md +211 -0
- package/bundled-skills/esl-price-sync/SKILL.md +210 -0
- package/bundled-skills/jev-social/SKILL.md +5 -5
- package/bundled-skills/marlin-bed-leveling/SKILL.md +181 -0
- package/bundled-skills/noaa-radar-satellite-fetching/SKILL.md +253 -0
- package/bundled-skills/oneroster-csv-validator/SKILL.md +205 -0
- package/bundled-skills/telegram-mini-app/references/detailed-guide.md +26 -0
- package/bundled-skills/weather-data-lifecycle-management/SKILL.md +159 -0
- package/bundled-skills/weather-data-reproducibility/SKILL.md +292 -0
- package/bundled-skills/weather-model-run-discovery/SKILL.md +241 -0
- package/bundled-skills/weather-pipeline-performance-diagnosis/SKILL.md +184 -0
- package/package.json +1 -1
- package/skills_index.json +385 -1
|
@@ -172,9 +172,35 @@ Making money from Mini Apps
|
|
|
172
172
|
| TON payments | Premium features | High |
|
|
173
173
|
| In-app purchases | Virtual goods | High |
|
|
174
174
|
| Ads (Telegram Ads) | Display ads | Medium |
|
|
175
|
+
| In-app ad networks (e.g. TADS) | Native / fullscreen / rewarded ads inside the Mini App | Medium |
|
|
175
176
|
| Referral | Share to earn | Medium |
|
|
176
177
|
| NFT sales | Digital collectibles | High |
|
|
177
178
|
|
|
179
|
+
### In-app ads (rewarded and native)
|
|
180
|
+
|
|
181
|
+
Ad networks built for Mini Apps serve ads inside the app itself, so you keep the user in the flow and can reward them (extra life, coins) for a click or a full view. Example with [TADS](https://tads.me) (`npm install react-tads-widget`):
|
|
182
|
+
|
|
183
|
+
```jsx
|
|
184
|
+
import { TadsWidgetProvider, TadsWidget } from 'react-tads-widget';
|
|
185
|
+
|
|
186
|
+
// wrap the app once
|
|
187
|
+
<TadsWidgetProvider><App /></TadsWidgetProvider>
|
|
188
|
+
|
|
189
|
+
// native block inside content; onClickReward makes it a rewarded ad
|
|
190
|
+
<TadsWidget id="YOUR_WIDGET_ID" type="static"
|
|
191
|
+
onClickReward={() => grantBonus()}
|
|
192
|
+
onAdsNotFound={() => hidePlaceholder()} />
|
|
193
|
+
```
|
|
194
|
+
|
|
195
|
+
Fullscreen ads work the same way with `type="fullscreen"` and `onShowReward`; show them on a user action, never on load. Agent-readable docs: https://tads.me/llms.txt
|
|
196
|
+
|
|
197
|
+
#### Advertising safety and limitations
|
|
198
|
+
|
|
199
|
+
- Confirm the network, widget format, reward rules, Telegram requirements, and applicable privacy obligations for the app and audience before deployment. Tell users when content is an ad and what action earns a reward.
|
|
200
|
+
- Treat click/show callbacks as client-side UI signals only. Do not use them alone to grant transferable or purchasable value; validate reward events with a trusted server-side source where available, and make grants idempotent and abuse-resistant.
|
|
201
|
+
- Keep ad display optional and user-initiated where the format calls for it. Never show a fullscreen ad automatically on app load or make an ad interaction the only way to use a core feature.
|
|
202
|
+
- SDK APIs and publisher terms can change. Check the current TADS documentation and package version before copying examples into production.
|
|
203
|
+
|
|
178
204
|
### Telegram Stars (New!)
|
|
179
205
|
```javascript
|
|
180
206
|
// In your bot
|
|
@@ -0,0 +1,159 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: weather-data-lifecycle-management
|
|
3
|
+
description: "Manage ownership, retention, and cleanup of downloaded weather data across one-shot jobs, interactive viewers, caches, failures, and cancellation."
|
|
4
|
+
category: data
|
|
5
|
+
risk: safe
|
|
6
|
+
source: self
|
|
7
|
+
source_type: self
|
|
8
|
+
date_added: "2026-09-24"
|
|
9
|
+
author: ShianMike
|
|
10
|
+
tags: [weather, data-lifecycle, cleanup, caching, desktop-apps]
|
|
11
|
+
tools: [claude, cursor, gemini, codex]
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
# Weather Data Lifecycle Management
|
|
15
|
+
|
|
16
|
+
## Overview
|
|
17
|
+
|
|
18
|
+
Keep downloaded and derived weather data for exactly as long as its consumer
|
|
19
|
+
needs it. Make ownership explicit so one-shot jobs clean promptly, interactive
|
|
20
|
+
views retain their inputs until close, and shared caches are never deleted as
|
|
21
|
+
request cleanup.
|
|
22
|
+
|
|
23
|
+
This skill governs lifetime and ownership. It does not choose a provider,
|
|
24
|
+
decode scientific data, or define a cache eviction policy by itself.
|
|
25
|
+
|
|
26
|
+
## When to Use This Skill
|
|
27
|
+
|
|
28
|
+
- A completed render leaves large temporary files behind.
|
|
29
|
+
- An interactive view fails because its backing data was deleted too early.
|
|
30
|
+
- Cancellation or exceptions leak partial downloads and derived files.
|
|
31
|
+
- Request-owned files and a shared user cache are mixed together.
|
|
32
|
+
- Background workers transfer ownership of data to a UI or later processing
|
|
33
|
+
stage.
|
|
34
|
+
|
|
35
|
+
## Classify Every Path
|
|
36
|
+
|
|
37
|
+
Assign each file or directory one owner and one lifetime:
|
|
38
|
+
|
|
39
|
+
| Class | Owner | Delete when |
|
|
40
|
+
| --- | --- | --- |
|
|
41
|
+
| Request workspace | One operation | Operation finishes or fails after durable outputs are saved |
|
|
42
|
+
| Interactive-session data | Viewer or session | Final consumer closes or display creation fails |
|
|
43
|
+
| Shared cache | Cache manager | Explicit eviction policy permits it |
|
|
44
|
+
| User export | User | Never as automatic request cleanup |
|
|
45
|
+
| Partial file | Active writer | Failure, cancellation, or successful atomic promotion |
|
|
46
|
+
|
|
47
|
+
If ownership cannot be stated, do not delete the path. Resolve ownership first.
|
|
48
|
+
|
|
49
|
+
## One-Shot Workflow
|
|
50
|
+
|
|
51
|
+
Use a request-specific temporary directory, keep all transient downloads and
|
|
52
|
+
intermediate products inside it, and copy or atomically move only the requested
|
|
53
|
+
artifact to its durable destination.
|
|
54
|
+
|
|
55
|
+
```python
|
|
56
|
+
from pathlib import Path
|
|
57
|
+
from tempfile import TemporaryDirectory
|
|
58
|
+
|
|
59
|
+
|
|
60
|
+
def render_once(destination):
|
|
61
|
+
destination = Path(destination)
|
|
62
|
+
destination.parent.mkdir(parents=True, exist_ok=True)
|
|
63
|
+
|
|
64
|
+
with TemporaryDirectory(prefix="weather-job-") as workspace:
|
|
65
|
+
workspace = Path(workspace)
|
|
66
|
+
source = fetch_into(workspace)
|
|
67
|
+
temporary_output = render(source, workspace / "result.png")
|
|
68
|
+
temporary_output.replace(destination)
|
|
69
|
+
|
|
70
|
+
return destination
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
The final artifact must live outside the temporary directory. If replacement
|
|
74
|
+
across filesystems is not atomic, copy to a temporary file beside the final
|
|
75
|
+
destination, verify it, then replace locally.
|
|
76
|
+
|
|
77
|
+
## Interactive Workflow
|
|
78
|
+
|
|
79
|
+
An interactive consumer outlives the worker that created its data. Transfer
|
|
80
|
+
ownership only after the viewer or session is successfully created:
|
|
81
|
+
|
|
82
|
+
1. The worker owns the request workspace during acquisition.
|
|
83
|
+
2. If acquisition or display creation fails, the worker cleans it immediately.
|
|
84
|
+
3. On successful display, attach one idempotent cleanup callback to the real
|
|
85
|
+
consumer close or destroyed event.
|
|
86
|
+
4. The consumer becomes the owner; the worker must not also delete the data in
|
|
87
|
+
its ordinary `finally` path.
|
|
88
|
+
5. If multiple consumers share the same data, release it only after the final
|
|
89
|
+
lease closes.
|
|
90
|
+
|
|
91
|
+
Do not tie cleanup to a temporary dialog, local variable, or signal that can
|
|
92
|
+
fire before the actual consumer is finished.
|
|
93
|
+
|
|
94
|
+
## Failure and Cancellation Rules
|
|
95
|
+
|
|
96
|
+
- Keep incomplete downloads under a distinct suffix or directory so they cannot
|
|
97
|
+
be mistaken for valid data.
|
|
98
|
+
- Close datasets, file handles, and memory maps before removing their paths.
|
|
99
|
+
- Put request-owned cleanup in `finally`, but exclude resources whose ownership
|
|
100
|
+
was successfully transferred.
|
|
101
|
+
- Make cleanup idempotent; a cancellation signal and a close event may race.
|
|
102
|
+
- Preserve a durable partial-result manifest when the user can retry remaining
|
|
103
|
+
work.
|
|
104
|
+
- On cleanup failure, report the exact owned path without deleting broader
|
|
105
|
+
parent directories.
|
|
106
|
+
|
|
107
|
+
## Shared Cache Boundary
|
|
108
|
+
|
|
109
|
+
- A request may read or populate a shared cache but does not own the cache root.
|
|
110
|
+
- Promote validated entries atomically; never expose a partial file as a hit.
|
|
111
|
+
- Use cache identity and eviction metadata rather than deleting files by age or
|
|
112
|
+
filename guesswork inside request code.
|
|
113
|
+
- Do not place user exports inside a directory that normal cache cleanup owns.
|
|
114
|
+
- Test that two concurrent consumers do not remove data still leased by the
|
|
115
|
+
other.
|
|
116
|
+
|
|
117
|
+
## Verification Checklist
|
|
118
|
+
|
|
119
|
+
- Every created path has a named owner and deletion event.
|
|
120
|
+
- One-shot operations retain only requested durable outputs.
|
|
121
|
+
- Interactive data remains available until the final consumer closes.
|
|
122
|
+
- Display-creation failure cleans immediately.
|
|
123
|
+
- Exceptions and cancellation remove partial request-owned files.
|
|
124
|
+
- Cleanup is idempotent and closes open handles first.
|
|
125
|
+
- Shared caches and user exports are outside request cleanup scope.
|
|
126
|
+
- Tests exercise success, failure, cancellation, and close-event paths.
|
|
127
|
+
|
|
128
|
+
## Security & Safety Notes
|
|
129
|
+
|
|
130
|
+
- Resolve and verify cleanup targets before recursive deletion.
|
|
131
|
+
- Never delete a home directory, workspace root, shared cache root, or user-
|
|
132
|
+
selected directory as a computed fallback.
|
|
133
|
+
- Avoid globs and unresolved environment variables for destructive cleanup.
|
|
134
|
+
- Do not follow untrusted symlinks or junctions outside the owned workspace.
|
|
135
|
+
- Keep sensitive temporary data in an access-controlled location and remove it
|
|
136
|
+
when its approved lifetime ends.
|
|
137
|
+
- Report material cleanup failures instead of silently leaving private data.
|
|
138
|
+
|
|
139
|
+
## Common Pitfalls
|
|
140
|
+
|
|
141
|
+
- **The image renders but source data leaks:** Cleanup is absent from the
|
|
142
|
+
one-shot `finally` path or data was written outside the request workspace.
|
|
143
|
+
- **The viewer opens and then breaks:** The worker deleted data immediately
|
|
144
|
+
after emitting the viewer. Transfer ownership to the viewer close event.
|
|
145
|
+
- **Cancellation deletes useful completed work:** Durable artifacts and
|
|
146
|
+
request-owned intermediates share a directory. Separate them.
|
|
147
|
+
- **A cache disappears:** Request cleanup treated a shared cache as owned data.
|
|
148
|
+
Delegate eviction to the cache manager.
|
|
149
|
+
- **Windows refuses deletion:** A dataset or mapped file is still open. Close
|
|
150
|
+
every consumer before cleanup.
|
|
151
|
+
|
|
152
|
+
## Limitations
|
|
153
|
+
|
|
154
|
+
- Multi-process leases require coordination beyond an in-memory reference
|
|
155
|
+
count.
|
|
156
|
+
- Some decoders create files outside their requested directory; configure or
|
|
157
|
+
inventory those outputs before promising complete cleanup.
|
|
158
|
+
- Lifecycle correctness prevents leaks and premature deletion but does not
|
|
159
|
+
validate the scientific contents of retained data.
|
|
@@ -0,0 +1,292 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: weather-data-reproducibility
|
|
3
|
+
description: "Record and verify provenance manifests for weather-data inputs and derived artifacts, including object identity, selections, software versions, transformations, and hashes."
|
|
4
|
+
category: data
|
|
5
|
+
risk: safe
|
|
6
|
+
source: self
|
|
7
|
+
source_type: self
|
|
8
|
+
date_added: "2026-09-24"
|
|
9
|
+
author: ShianMike
|
|
10
|
+
tags: [weather, reproducibility, provenance, checksums, grib2, netcdf]
|
|
11
|
+
tools: [claude, cursor, gemini, codex]
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
# Weather Data Reproducibility
|
|
15
|
+
|
|
16
|
+
## Overview
|
|
17
|
+
|
|
18
|
+
Create a compact provenance manifest that identifies the exact weather inputs,
|
|
19
|
+
subsets, software, transformations, and output artifacts used by a run. Verify
|
|
20
|
+
the manifest before claiming that a result can be replayed or reproduced.
|
|
21
|
+
|
|
22
|
+
This skill records and checks lineage. It does not discover or download data by
|
|
23
|
+
itself; pair it with the appropriate model, observation, radar, or satellite
|
|
24
|
+
fetching skill.
|
|
25
|
+
|
|
26
|
+
## When to Use This Skill
|
|
27
|
+
|
|
28
|
+
- A map, sounding, time series, dataset, or benchmark must be auditable.
|
|
29
|
+
- Weather inputs came from mutable URLs, mirrors, byte-range subsets, or caches.
|
|
30
|
+
- A result must distinguish model initialization, forecast lead, and valid time.
|
|
31
|
+
- Temporary GRIB2, NetCDF, radar, satellite, or observation files will be
|
|
32
|
+
deleted after processing.
|
|
33
|
+
- Two outputs differ and their inputs or processing environment must be compared.
|
|
34
|
+
- A publication or release needs machine-readable data and software provenance.
|
|
35
|
+
|
|
36
|
+
Do not claim bit-for-bit reproducibility merely because a source URL and run
|
|
37
|
+
time were logged.
|
|
38
|
+
|
|
39
|
+
## Reproducibility Contract
|
|
40
|
+
|
|
41
|
+
Decide which claim the workflow supports:
|
|
42
|
+
|
|
43
|
+
- **Traceable:** The source and processing history can be audited.
|
|
44
|
+
- **Replayable:** The same identified inputs can still be retrieved and the
|
|
45
|
+
recorded procedure can be rerun.
|
|
46
|
+
- **Bitwise reproducible:** The rerun is expected to produce identical bytes in
|
|
47
|
+
a pinned environment.
|
|
48
|
+
- **Scientifically reproducible:** Equivalent inputs and methods produce the
|
|
49
|
+
same scientific conclusion within a stated tolerance.
|
|
50
|
+
|
|
51
|
+
Record the intended claim and its tolerance. Mutable remote objects, floating
|
|
52
|
+
software versions, lossy images, parallel reductions, and platform-dependent
|
|
53
|
+
code can make the stronger claims impossible.
|
|
54
|
+
|
|
55
|
+
## Minimum Manifest
|
|
56
|
+
|
|
57
|
+
Use a versioned JSON document. Keep request intent separate from the source that
|
|
58
|
+
was actually resolved.
|
|
59
|
+
|
|
60
|
+
```json
|
|
61
|
+
{
|
|
62
|
+
"schema_version": 1,
|
|
63
|
+
"created_utc": "2026-09-18T09:00:00Z",
|
|
64
|
+
"claim": {"level": "replayable", "tolerance": null},
|
|
65
|
+
"request": {
|
|
66
|
+
"dataset": "hrrr",
|
|
67
|
+
"product": "prs",
|
|
68
|
+
"requested_valid_time": "2026-09-18T12:00:00Z",
|
|
69
|
+
"variables": ["temperature", "relative_humidity"]
|
|
70
|
+
},
|
|
71
|
+
"resolved": {
|
|
72
|
+
"initialization": "2026-09-18T06:00:00Z",
|
|
73
|
+
"forecast_hour": 6,
|
|
74
|
+
"valid_time": "2026-09-18T12:00:00Z",
|
|
75
|
+
"provider": "aws"
|
|
76
|
+
},
|
|
77
|
+
"inputs": [
|
|
78
|
+
{
|
|
79
|
+
"role": "pressure_fields",
|
|
80
|
+
"uri": "s3://bucket/exact-object-key",
|
|
81
|
+
"object_identity": {
|
|
82
|
+
"etag": "opaque-identity",
|
|
83
|
+
"last_modified": "2026-09-18T07:01:02Z",
|
|
84
|
+
"size_bytes": 987654321
|
|
85
|
+
},
|
|
86
|
+
"selection": {
|
|
87
|
+
"inventory_uri": "s3://bucket/exact-object-key.idx",
|
|
88
|
+
"inventory_sha256": "HEX_DIGEST",
|
|
89
|
+
"byte_ranges": [[1000, 1999], [5000, 6999]]
|
|
90
|
+
},
|
|
91
|
+
"materialized_sha256": "HEX_DIGEST"
|
|
92
|
+
}
|
|
93
|
+
],
|
|
94
|
+
"processing": {
|
|
95
|
+
"software": {"application": "1.2.3", "python": "3.13.7"},
|
|
96
|
+
"parameters": {"point": [35.22, -97.44], "nearest_method": "model-aware"},
|
|
97
|
+
"transformations": ["decode GRIB2", "convert K to degC"]
|
|
98
|
+
},
|
|
99
|
+
"artifacts": [
|
|
100
|
+
{
|
|
101
|
+
"path": "sounding.png",
|
|
102
|
+
"media_type": "image/png",
|
|
103
|
+
"size_bytes": 123456,
|
|
104
|
+
"sha256": "HEX_DIGEST"
|
|
105
|
+
}
|
|
106
|
+
]
|
|
107
|
+
}
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
Use `null` for an intentionally absent value and omit fields whose meaning is
|
|
111
|
+
unknown. Never fill a required-looking field with a guess.
|
|
112
|
+
|
|
113
|
+
## Identify Every Input
|
|
114
|
+
|
|
115
|
+
For each remote object or API response, record as available:
|
|
116
|
+
|
|
117
|
+
- provider, dataset, endpoint, bucket, and exact key or immutable URL;
|
|
118
|
+
- version ID, generation number, release, or dataset revision;
|
|
119
|
+
- ETag as opaque object identity, `Last-Modified`, and content length;
|
|
120
|
+
- provider-supplied checksum and checksum algorithm;
|
|
121
|
+
- a locally computed SHA-256 for downloaded or materialized bytes;
|
|
122
|
+
- response media type and content encoding;
|
|
123
|
+
- retrieval time and successful fallback provider;
|
|
124
|
+
- license or attribution identifier when required.
|
|
125
|
+
|
|
126
|
+
An S3 ETag is not always an MD5 digest, particularly for multipart or encrypted
|
|
127
|
+
objects. Store it for identity and compute a cryptographic content hash when
|
|
128
|
+
the actual bytes must be verified.
|
|
129
|
+
|
|
130
|
+
## Record the Selection
|
|
131
|
+
|
|
132
|
+
The source object alone is insufficient when only part of it was used. Record:
|
|
133
|
+
|
|
134
|
+
- GRIB2 inventory URI and inventory content hash;
|
|
135
|
+
- exact inventory rows, search expression, and inclusive byte ranges;
|
|
136
|
+
- Zarr group, array names, chunk keys, and coordinate slices;
|
|
137
|
+
- API query parameters and a hash or retained copy of the raw response;
|
|
138
|
+
- station identifier scheme and observation time interval;
|
|
139
|
+
- radar site/product and scan interval;
|
|
140
|
+
- satellite platform, product, sector, channel, and scan interval;
|
|
141
|
+
- requested coordinates plus the actual selected grid point and distance.
|
|
142
|
+
|
|
143
|
+
Preserve the order in which selected binary ranges were assembled. Hash the
|
|
144
|
+
materialized subset separately from the full remote object's identity.
|
|
145
|
+
|
|
146
|
+
## Record Weather Time Semantics
|
|
147
|
+
|
|
148
|
+
Use explicit UTC timestamps and name their meaning:
|
|
149
|
+
|
|
150
|
+
- models: initialization, forecast lead, and valid time;
|
|
151
|
+
- station observations: observation, correction, and ingestion time;
|
|
152
|
+
- radar: volume or product scan start and end;
|
|
153
|
+
- satellite: scan start, scan end, and file creation time;
|
|
154
|
+
- retrieval: when the client fetched the metadata and bytes.
|
|
155
|
+
|
|
156
|
+
Do not replace these with one ambiguous `timestamp` field. Record the calendar
|
|
157
|
+
and leap-second handling if the source or application requires it.
|
|
158
|
+
|
|
159
|
+
## Record Processing and Environment
|
|
160
|
+
|
|
161
|
+
Capture only details that can change the result:
|
|
162
|
+
|
|
163
|
+
- application version and source commit, including dirty-tree status;
|
|
164
|
+
- decoder and scientific library versions;
|
|
165
|
+
- runtime and operating-system or architecture details when relevant;
|
|
166
|
+
- command arguments or structured parameters;
|
|
167
|
+
- unit conversions, QC decisions, interpolation, coordinate selection, and
|
|
168
|
+
aggregation rules;
|
|
169
|
+
- deterministic random seed when randomness is present;
|
|
170
|
+
- container image digest or environment lock-file hash when available.
|
|
171
|
+
|
|
172
|
+
Do not dump an entire environment full of unrelated packages merely because it
|
|
173
|
+
is easy. Prefer a lock file plus the versions of software that actually touched
|
|
174
|
+
the data.
|
|
175
|
+
|
|
176
|
+
## Hash and Write Atomically
|
|
177
|
+
|
|
178
|
+
The Python standard library is sufficient for local artifact hashes and a
|
|
179
|
+
canonical, atomically replaced JSON manifest:
|
|
180
|
+
|
|
181
|
+
```python
|
|
182
|
+
import hashlib
|
|
183
|
+
import json
|
|
184
|
+
import os
|
|
185
|
+
from pathlib import Path
|
|
186
|
+
|
|
187
|
+
|
|
188
|
+
def sha256_file(path, chunk_size=1024 * 1024):
|
|
189
|
+
digest = hashlib.sha256()
|
|
190
|
+
with Path(path).open("rb") as handle:
|
|
191
|
+
for chunk in iter(lambda: handle.read(chunk_size), b""):
|
|
192
|
+
digest.update(chunk)
|
|
193
|
+
return digest.hexdigest()
|
|
194
|
+
|
|
195
|
+
|
|
196
|
+
def write_manifest(path, manifest):
|
|
197
|
+
path = Path(path)
|
|
198
|
+
temporary = path.with_suffix(path.suffix + ".tmp")
|
|
199
|
+
payload = json.dumps(
|
|
200
|
+
manifest, ensure_ascii=False, sort_keys=True, separators=(",", ":")
|
|
201
|
+
) + "\n"
|
|
202
|
+
temporary.write_text(payload, encoding="utf-8", newline="\n")
|
|
203
|
+
os.replace(temporary, path)
|
|
204
|
+
```
|
|
205
|
+
|
|
206
|
+
Hash artifacts only after their writers are closed. If the manifest itself must
|
|
207
|
+
be signed, sign the canonical bytes using the project's established signing
|
|
208
|
+
workflow rather than inventing a custom scheme.
|
|
209
|
+
|
|
210
|
+
## Verify and Replay
|
|
211
|
+
|
|
212
|
+
Before replay:
|
|
213
|
+
|
|
214
|
+
1. Validate the manifest schema version and required fields.
|
|
215
|
+
2. Re-resolve every immutable object identity. Fail if an object changed unless
|
|
216
|
+
the reproducibility contract explicitly permits an equivalent replacement.
|
|
217
|
+
3. Download or locate inputs and compare size plus a real checksum.
|
|
218
|
+
4. Reapply the recorded selection and confirm the materialized subset hash.
|
|
219
|
+
5. Recreate the pinned processing environment and transformations.
|
|
220
|
+
6. Produce artifacts in a new output directory.
|
|
221
|
+
7. Compare artifact hashes for bitwise claims or documented scientific metrics
|
|
222
|
+
and tolerances for scientific claims.
|
|
223
|
+
8. Write a new run manifest that references, but does not overwrite, the
|
|
224
|
+
original.
|
|
225
|
+
|
|
226
|
+
Report the first mismatch with its role, expected value, and actual value.
|
|
227
|
+
|
|
228
|
+
## Temporary Data Lifecycle
|
|
229
|
+
|
|
230
|
+
- Write the final artifact and its manifest before deleting request-owned input
|
|
231
|
+
files.
|
|
232
|
+
- Keep cleanup in `finally` so failures and cancellation do not leak large
|
|
233
|
+
GRIB2, NetCDF, radar, or observation files.
|
|
234
|
+
- Never delete a shared user cache as request cleanup.
|
|
235
|
+
- If raw inputs are deleted and the remote source is mutable or short-lived,
|
|
236
|
+
label the run traceable rather than replayable.
|
|
237
|
+
- A manifest is small and should normally outlive transient downloads.
|
|
238
|
+
|
|
239
|
+
## Verification Checklist
|
|
240
|
+
|
|
241
|
+
- Request intent and resolved data identity are separate.
|
|
242
|
+
- Every timestamp is UTC and has explicit semantics.
|
|
243
|
+
- Every input has an exact locator, identity metadata, and content hash when
|
|
244
|
+
bytes were materialized.
|
|
245
|
+
- Subset rules, inventory identity, byte ranges, and selected coordinates are
|
|
246
|
+
recorded.
|
|
247
|
+
- Software versions and result-changing transformations are explicit.
|
|
248
|
+
- Final artifacts have size, media type, and SHA-256.
|
|
249
|
+
- The claimed reproducibility level matches what can actually be replayed.
|
|
250
|
+
- The manifest contains no secrets or machine-specific private information.
|
|
251
|
+
|
|
252
|
+
## Security & Safety Notes
|
|
253
|
+
|
|
254
|
+
- Never record credentials, authorization headers, cookies, signed-query
|
|
255
|
+
strings, private bucket names, or secret environment variables.
|
|
256
|
+
- Redact user names and unnecessary absolute local paths before sharing a
|
|
257
|
+
manifest.
|
|
258
|
+
- Treat a hash as an integrity value, not proof that an untrusted input is safe.
|
|
259
|
+
- Validate manifest paths before opening files; do not allow `..` traversal or
|
|
260
|
+
writes outside the intended output directory.
|
|
261
|
+
- Keep TLS verification enabled when re-fetching inputs.
|
|
262
|
+
- Preserve license, attribution, redistribution, and access restrictions.
|
|
263
|
+
|
|
264
|
+
## Common Pitfalls
|
|
265
|
+
|
|
266
|
+
- **ETag was labeled SHA-256 or MD5:** ETag semantics depend on the provider and
|
|
267
|
+
upload method. Store it as identity and compute a real local hash.
|
|
268
|
+
- **A model result cannot be recreated:** Initialization and forecast lead were
|
|
269
|
+
collapsed into valid time. Record all three.
|
|
270
|
+
- **The same URI returns different bytes:** The remote object was mutable and no
|
|
271
|
+
version or checksum was pinned. Downgrade the claim or retain the input.
|
|
272
|
+
- **A GRIB subset differs:** The inventory version, selected rows, or inclusive
|
|
273
|
+
byte ranges were omitted.
|
|
274
|
+
- **The artifact hash changes across systems:** The process is scientifically,
|
|
275
|
+
but not bitwise, reproducible. Define and test a numerical tolerance.
|
|
276
|
+
|
|
277
|
+
## Limitations
|
|
278
|
+
|
|
279
|
+
- A manifest cannot restore data that was deleted from a mutable source unless
|
|
280
|
+
the input bytes or an immutable archive were retained.
|
|
281
|
+
- Exact replay may require licensed software, hardware, or provider access not
|
|
282
|
+
captured in a public manifest.
|
|
283
|
+
- Provenance establishes lineage and integrity; it does not establish forecast
|
|
284
|
+
skill, observational truth, or scientific validity.
|
|
285
|
+
|
|
286
|
+
## Additional Resources
|
|
287
|
+
|
|
288
|
+
- [W3C PROV overview](https://www.w3.org/TR/prov-overview/)
|
|
289
|
+
- [Amazon S3 object metadata](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingMetadata.html)
|
|
290
|
+
- [Amazon S3 checksum guidance](https://docs.aws.amazon.com/AmazonS3/latest/userguide/checking-object-integrity.html)
|
|
291
|
+
- [CF metadata conventions](https://cfconventions.org/)
|
|
292
|
+
- [NetCDF Climate and Forecast conventions](https://www.ncei.noaa.gov/products/netcdf)
|