opencode-skills-collection 4.0.66 → 4.0.68
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bundled-skills/.antigravity-install-manifest.json +18 -1
- package/bundled-skills/anti-slop-design/SKILL.md +393 -0
- package/bundled-skills/antigravity-maintainer-batch-release/SKILL.md +1 -0
- package/bundled-skills/artifact-yylo/SKILL.md +122 -0
- package/bundled-skills/beatra-ai-video-studio/SKILL.md +272 -0
- package/bundled-skills/google-no-code/SKILL.md +136 -0
- package/bundled-skills/idea-evaluator/SKILL.md +75 -0
- package/bundled-skills/idea-evaluator/idea-evaluator-con/SKILL.md +64 -0
- package/bundled-skills/idea-evaluator/idea-evaluator-pro/SKILL.md +64 -0
- package/bundled-skills/ledger-tasks-yylo/SKILL.md +219 -0
- package/bundled-skills/loki-mode/examples/todo-app-generated/backend/package-lock.json +4 -4
- package/bundled-skills/loki-mode/examples/todo-app-generated/backend/package.json +1 -1
- package/bundled-skills/meteora-dlmm-pool-screening/SKILL.md +166 -0
- package/bundled-skills/meteora-dlmm-pool-screening/references/meteora-apis.md +74 -0
- package/bundled-skills/meteora-dlmm-pool-screening/references/meteora-screener.md +352 -0
- package/bundled-skills/plan-ledger-tasks-yylo/SKILL.md +52 -0
- package/bundled-skills/ralph-loop-yylo/SKILL.md +55 -0
- package/bundled-skills/ralph-loop-yylo/references/first_check.md +18 -0
- package/bundled-skills/ralph-loop-yylo/references/implement.md +60 -0
- package/bundled-skills/resumable-implementation-contracts/SKILL.md +254 -0
- package/bundled-skills/understand-project-yylo/SKILL.md +62 -0
- package/bundled-skills/weather-model-data-fetching/SKILL.md +277 -0
- package/bundled-skills/weather-observation-fetching/SKILL.md +246 -0
- package/bundled-skills/wiki-yylo/SKILL.md +114 -0
- package/bundled-skills/workflow-yylo/SKILL.md +107 -0
- package/package.json +1 -1
- package/skills_index.json +422 -0
|
@@ -0,0 +1,254 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: resumable-implementation-contracts
|
|
3
|
+
description: "Create repository-based execution contracts for multi-session implementation work, with stable task IDs, evidence, checkpoints, and exact resume state."
|
|
4
|
+
category: project-management
|
|
5
|
+
risk: safe
|
|
6
|
+
source: self
|
|
7
|
+
source_type: self
|
|
8
|
+
date_added: "2026-09-19"
|
|
9
|
+
author: ShianMike
|
|
10
|
+
tags: [project-management, execution-contracts, checkpoints, verification, agent-workflows]
|
|
11
|
+
tools: [claude, cursor, gemini, codex, antigravity]
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
# Resumable Implementation Contracts
|
|
15
|
+
|
|
16
|
+
## Overview
|
|
17
|
+
|
|
18
|
+
Turn a large, mostly defined implementation request into a repository contract
|
|
19
|
+
that a fresh agent can read, update, verify, and resume without reconstructing
|
|
20
|
+
the project from chat history. Keep stable intent separate from mutable
|
|
21
|
+
execution state, and require evidence before any task is called complete.
|
|
22
|
+
|
|
23
|
+
This is a tool-neutral document pattern. It does not depend on, replace, or
|
|
24
|
+
configure any agent platform's built-in goal, task, plan, or project feature.
|
|
25
|
+
|
|
26
|
+
This skill fills the layer between planning and execution. It does not replace
|
|
27
|
+
product discovery, detailed technical design, project-wide state governance,
|
|
28
|
+
or retrospective auditing.
|
|
29
|
+
|
|
30
|
+
## When to Use This Skill
|
|
31
|
+
|
|
32
|
+
Use it when work:
|
|
33
|
+
|
|
34
|
+
- spans multiple sessions, agents, branches, or context windows;
|
|
35
|
+
- has dependencies or acceptance criteria that must survive interruption;
|
|
36
|
+
- needs an exact resume point rather than a narrative handoff;
|
|
37
|
+
- can appear complete before runtime, artifact, UI, or test evidence exists;
|
|
38
|
+
- must preserve partial work and unrelated repository changes.
|
|
39
|
+
|
|
40
|
+
Skip it for a small task that can be completed and verified in one session.
|
|
41
|
+
If requirements are still unsettled, resolve them before freezing the
|
|
42
|
+
implementation contract.
|
|
43
|
+
|
|
44
|
+
## Reuse Existing Project Documents
|
|
45
|
+
|
|
46
|
+
Inspect the repository before creating files. Reuse equivalent documents and
|
|
47
|
+
the project's established names when their ownership is clear. For substantial
|
|
48
|
+
multi-session work, the minimum logical document set is:
|
|
49
|
+
|
|
50
|
+
| Document | Owns | Must not become |
|
|
51
|
+
| --- | --- | --- |
|
|
52
|
+
| `IMPLEMENTATION_CONTRACT.md` | stable intent, scope, tasks, acceptance, definition of done | a live activity log |
|
|
53
|
+
| `TASKS.md` | current task/subtask status, dependencies, evidence links | a second copy of the contract |
|
|
54
|
+
| `CHECKPOINT.md` | authoritative current position and exact next action | a vague progress summary |
|
|
55
|
+
| `DECISIONS.md` | material decisions, alternatives, and reasons | a transcript |
|
|
56
|
+
| `VALIDATION.md` | checks actually run, results, evidence, and unresolved gates | a list of planned tests |
|
|
57
|
+
|
|
58
|
+
These may live together under `docs/<contract-slug>/` or follow an existing
|
|
59
|
+
repo layout. An existing `GOAL.md`, specification, or execution brief may own
|
|
60
|
+
the contract role; do not rename it or create parallel files solely to match
|
|
61
|
+
this skill.
|
|
62
|
+
|
|
63
|
+
## Define Document Authority
|
|
64
|
+
|
|
65
|
+
- Current owner instructions and applicable repository instructions govern
|
|
66
|
+
authorization and scope.
|
|
67
|
+
- `IMPLEMENTATION_CONTRACT.md`, or its existing repository equivalent, owns
|
|
68
|
+
the implementation contract. Do not silently change it to fit the current
|
|
69
|
+
code.
|
|
70
|
+
- The working tree, branch, commit, produced artifacts, and executed checks are
|
|
71
|
+
truth for implementation state.
|
|
72
|
+
- The tracker files summarize that state; they do not override contrary
|
|
73
|
+
evidence on disk.
|
|
74
|
+
|
|
75
|
+
When documents and reality disagree, reconcile status from evidence while
|
|
76
|
+
preserving the contract's intent. Escalate any conflict that would materially
|
|
77
|
+
change scope, behavior, or acceptance.
|
|
78
|
+
|
|
79
|
+
## Write a Self-Contained Contract
|
|
80
|
+
|
|
81
|
+
A new agent should be able to understand the work from the contract without
|
|
82
|
+
the original conversation. Include only what is needed to execute correctly:
|
|
83
|
+
|
|
84
|
+
1. objective and observable outcomes;
|
|
85
|
+
2. current baseline and important constraints;
|
|
86
|
+
3. in-scope work, exclusions, and authorization boundaries;
|
|
87
|
+
4. operating rules, including interruption and validation policy;
|
|
88
|
+
5. tasks with stable IDs, dependencies, acceptance, and breakpoints;
|
|
89
|
+
6. final definition of done and handoff requirements.
|
|
90
|
+
|
|
91
|
+
Use stable task and subtask IDs such as `T03` and `T03.2`. Never renumber them
|
|
92
|
+
after execution starts; add new IDs or mark obsolete work explicitly.
|
|
93
|
+
|
|
94
|
+
Each task should use this compact form:
|
|
95
|
+
|
|
96
|
+
```markdown
|
|
97
|
+
### T03 - <observable task outcome>
|
|
98
|
+
|
|
99
|
+
- [ ] T03.1 <first implementation slice>
|
|
100
|
+
- [ ] T03.2 <second implementation slice>
|
|
101
|
+
|
|
102
|
+
Dependencies: T01
|
|
103
|
+
|
|
104
|
+
Acceptance:
|
|
105
|
+
- <observable behavior or artifact>
|
|
106
|
+
- <required focused check and evidence>
|
|
107
|
+
|
|
108
|
+
Breakpoint:
|
|
109
|
+
- Update TASKS.md, VALIDATION.md, and CHECKPOINT.md after the accepted slice.
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
Write acceptance in pass/fail terms. File creation, code presence, or an
|
|
113
|
+
agent's completion claim is not acceptance unless that is genuinely the whole
|
|
114
|
+
requirement.
|
|
115
|
+
|
|
116
|
+
## Read Before Acting
|
|
117
|
+
|
|
118
|
+
At the start of a session or after interruption, read in this order:
|
|
119
|
+
|
|
120
|
+
1. applicable repository instructions;
|
|
121
|
+
2. the implementation contract;
|
|
122
|
+
3. `TASKS.md`, `CHECKPOINT.md`, `DECISIONS.md`, and `VALIDATION.md`;
|
|
123
|
+
4. current branch, `HEAD`, status, and relevant diff;
|
|
124
|
+
5. only the code, tests, and artifacts needed for the active task.
|
|
125
|
+
|
|
126
|
+
Reconcile the checkpoint with the working tree before editing. Preserve partial
|
|
127
|
+
and unrelated changes. Resume the exact unfinished subtask when it is still
|
|
128
|
+
valid; otherwise record why the next action changed.
|
|
129
|
+
|
|
130
|
+
## Keep Writes Narrow and Durable
|
|
131
|
+
|
|
132
|
+
- Change the implementation contract only when the owner changes intent or an
|
|
133
|
+
ambiguity is deliberately resolved.
|
|
134
|
+
- Update `TASKS.md` when work starts, blocks, or becomes evidence-backed done.
|
|
135
|
+
- Replace `CHECKPOINT.md` with the latest authoritative resume state; Git owns
|
|
136
|
+
its detailed history.
|
|
137
|
+
- Add to `DECISIONS.md` only for choices that constrain later work.
|
|
138
|
+
- Add to `VALIDATION.md` only after a check is actually run or explicitly
|
|
139
|
+
recorded as not run.
|
|
140
|
+
- Update the checkpoint after a meaningful increment and before stopping.
|
|
141
|
+
|
|
142
|
+
Do not duplicate the same mutable status across every document. Link to the
|
|
143
|
+
owning record instead.
|
|
144
|
+
|
|
145
|
+
## Use Evidence-Gated Status
|
|
146
|
+
|
|
147
|
+
Use a small status vocabulary: `pending`, `in_progress`, `blocked`, and `done`.
|
|
148
|
+
A task may become `done` only when its acceptance criteria have supporting
|
|
149
|
+
evidence.
|
|
150
|
+
|
|
151
|
+
For each validation record, capture:
|
|
152
|
+
|
|
153
|
+
- timestamp and relevant commit or working-tree state;
|
|
154
|
+
- exact command or manual procedure;
|
|
155
|
+
- environment when it affects the result;
|
|
156
|
+
- result, including failures and skipped checks;
|
|
157
|
+
- artifact, log, screenshot, route, or report path when applicable;
|
|
158
|
+
- unresolved caveats or gates.
|
|
159
|
+
|
|
160
|
+
Run the smallest check that proves the current slice during implementation.
|
|
161
|
+
Run broader integration or release suites at defined milestones or when the
|
|
162
|
+
change's risk requires them. Never present a planned, mocked, or nominally
|
|
163
|
+
successful check as observed behavior.
|
|
164
|
+
|
|
165
|
+
## Make the Checkpoint Executable
|
|
166
|
+
|
|
167
|
+
`CHECKPOINT.md` must let another agent continue immediately. Record:
|
|
168
|
+
|
|
169
|
+
```markdown
|
|
170
|
+
# Checkpoint
|
|
171
|
+
|
|
172
|
+
Updated: <UTC timestamp>
|
|
173
|
+
Branch / HEAD: <branch> / <commit>
|
|
174
|
+
Active task: T03
|
|
175
|
+
Active subtask: T03.2
|
|
176
|
+
Status: in_progress
|
|
177
|
+
|
|
178
|
+
Completed behavior:
|
|
179
|
+
- <verified result and evidence link>
|
|
180
|
+
|
|
181
|
+
Work in progress:
|
|
182
|
+
- <files and partial state that must be preserved>
|
|
183
|
+
|
|
184
|
+
Validation performed:
|
|
185
|
+
- `<exact command>` -> <result>
|
|
186
|
+
|
|
187
|
+
Blockers or uncertainties:
|
|
188
|
+
- <blocker, owner, and condition for clearing it>
|
|
189
|
+
|
|
190
|
+
Pre-existing or unrelated changes:
|
|
191
|
+
- <paths or explicit none observed>
|
|
192
|
+
|
|
193
|
+
Next exact action:
|
|
194
|
+
- <one concrete edit, inspection, or command>
|
|
195
|
+
|
|
196
|
+
Next verification:
|
|
197
|
+
- <focused check that should follow that action>
|
|
198
|
+
```
|
|
199
|
+
|
|
200
|
+
Avoid next steps such as "continue implementation" or "finish tests." If the
|
|
201
|
+
next agent must decide what those words mean, the checkpoint is incomplete.
|
|
202
|
+
|
|
203
|
+
## Execution Loop
|
|
204
|
+
|
|
205
|
+
1. Select the smallest dependency-ready task.
|
|
206
|
+
2. Mark it `in_progress` and state the intended slice.
|
|
207
|
+
3. Inspect the real path that owns the behavior.
|
|
208
|
+
4. Make the minimum scoped change.
|
|
209
|
+
5. Run focused verification and record the actual outcome.
|
|
210
|
+
6. Update task status only from that evidence.
|
|
211
|
+
7. Refresh the checkpoint with one exact next action.
|
|
212
|
+
8. Repeat until every definition-of-done item is verified or explicitly
|
|
213
|
+
blocked.
|
|
214
|
+
|
|
215
|
+
At final handoff, report completed outcomes, evidence, unresolved blockers,
|
|
216
|
+
branch and commit state, and the exact next action if anything remains. Do not
|
|
217
|
+
promote partial task completion into overall contract completion.
|
|
218
|
+
|
|
219
|
+
## Limitations
|
|
220
|
+
|
|
221
|
+
- This structure preserves execution state but cannot resolve unclear product
|
|
222
|
+
intent or choose among materially different outcomes for the owner.
|
|
223
|
+
- Evidence quality depends on running checks that exercise the real behavior;
|
|
224
|
+
complete-looking documents do not prove implementation correctness.
|
|
225
|
+
- Concurrent writers still need repository-level coordination to avoid
|
|
226
|
+
conflicting checkpoints and status updates.
|
|
227
|
+
- The five-document layout is unnecessary overhead for small, single-session
|
|
228
|
+
tasks; reuse fewer existing documents when they provide the same ownership.
|
|
229
|
+
|
|
230
|
+
## Common Failure Modes
|
|
231
|
+
|
|
232
|
+
- Rewriting or renumbering task IDs after work has begun.
|
|
233
|
+
- Treating checked boxes as evidence instead of linking the proof.
|
|
234
|
+
- Recording only happy-path checks and losing failed attempts that constrain
|
|
235
|
+
the next decision.
|
|
236
|
+
- Replaying all history instead of loading the authoritative current state.
|
|
237
|
+
- Allowing implementation discoveries to silently expand the contract.
|
|
238
|
+
- Running an expensive full suite after every small edit while skipping the
|
|
239
|
+
focused check that would isolate the defect.
|
|
240
|
+
- Persisting credentials, tokens, private data, or sensitive command output in
|
|
241
|
+
tracking documents.
|
|
242
|
+
|
|
243
|
+
## Relationship to Other Skills
|
|
244
|
+
|
|
245
|
+
- Use `spec-driven-loop` when product requirements and technical design still
|
|
246
|
+
need structured discovery, freezing, and approval.
|
|
247
|
+
- Use `planning-and-task-breakdown` when only an executable plan is needed.
|
|
248
|
+
- Use `project-state-governor` when the need is canonical state across the
|
|
249
|
+
whole project rather than one scoped implementation effort.
|
|
250
|
+
- Use `audit-agent-run-evidence` for a read-only retrospective audit of an
|
|
251
|
+
already completed run.
|
|
252
|
+
|
|
253
|
+
This skill owns the compact execution contract that connects those concerns:
|
|
254
|
+
stable intent, mutable progress, verification evidence, and exact resume state.
|
|
@@ -0,0 +1,62 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: understand-project-yylo
|
|
3
|
+
description: Inspect the current product architecture, dependencies, and validation
|
|
4
|
+
loops before planning or implementing a requested change.
|
|
5
|
+
category: project-management
|
|
6
|
+
risk: safe
|
|
7
|
+
source: https://github.com/yylo-dev/yylo-skills
|
|
8
|
+
source_repo: yylo-dev/yylo-skills
|
|
9
|
+
source_type: community
|
|
10
|
+
date_added: '2026-09-19'
|
|
11
|
+
license: MIT
|
|
12
|
+
license_source: https://github.com/yylo-dev/yylo-skills/blob/main/LICENSE
|
|
13
|
+
compatibility: Requires read access to the product worktree and the `yy` CLI for task/spec
|
|
14
|
+
reads through the canonical controller. Read-only inspection; no mutations.
|
|
15
|
+
argument-hint: '[Main Task] [Constraints] [Ultimate Goal]'
|
|
16
|
+
enable-shell-directives: true
|
|
17
|
+
---
|
|
18
|
+
|
|
19
|
+
# Understand the project
|
|
20
|
+
|
|
21
|
+
1. Read `AGENTS.md`/`CLAUDE.md`, repository status, relevant source, tests, and existing product documentation in the integration or assigned feature worktree.
|
|
22
|
+
2. Read related Kanban tasks and durable specs through the canonical metadata controller. Do not assume `.juno_task/plan.md` exists and do not materialize controller-private metadata in a product worktree.
|
|
23
|
+
3. Trace only the dependencies and runtime paths needed for the requested goal. Use bounded parallel investigation when independent questions justify it.
|
|
24
|
+
4. Report current behavior, sources of truth, affected components, risks, unknowns, and the smallest useful validation loop.
|
|
25
|
+
5. If the user requested planning, hand the findings to `plan-ledger-tasks-yylo`. If implementation was requested, work only in the task worktree returned by `yy task start TASK_ID`.
|
|
26
|
+
6. Write a durable operational spec only when requested or materially useful. Draft it externally, preflight the installed `yy ledger artifact` API, capture it as an immutable `report` Artifact Record with provenance/retention, and verify retrieval, digest, and history. If that API is unavailable, stop with the external draft intact; never fall back to product `docs/`, task bodies/responses, new `.juno_task/specs`, or direct controller-store edits. Product `docs/` remains reserved for documentation shipped with the product. Do not update root instructions with transient status.
|
|
27
|
+
|
|
28
|
+
## Invocation contract
|
|
29
|
+
|
|
30
|
+
The structured views below intentionally alias parts of the complete request; they are not additional user arguments.
|
|
31
|
+
|
|
32
|
+
### Main task
|
|
33
|
+
|
|
34
|
+
$1
|
|
35
|
+
|
|
36
|
+
### Constraints and context
|
|
37
|
+
|
|
38
|
+
$2
|
|
39
|
+
|
|
40
|
+
### Complete raw request
|
|
41
|
+
|
|
42
|
+
$ARGUMENTS
|
|
43
|
+
|
|
44
|
+
## When to Use
|
|
45
|
+
|
|
46
|
+
- Before planning or implementing a requested change, when you need current architecture, dependencies, and validation loops.
|
|
47
|
+
- To hand grounded findings to `plan-ledger-tasks-yylo` (planning) or a task worktree (implementation).
|
|
48
|
+
|
|
49
|
+
## Limitations
|
|
50
|
+
|
|
51
|
+
- Read-only: reports current behavior, sources of truth, affected components, risks, unknowns, and the smallest validation loop - does not implement.
|
|
52
|
+
- Bounded investigation only; do not materialize controller-private metadata in a product worktree.
|
|
53
|
+
- Durable specs are captured as immutable Artifact Records only when the installed `yy ledger artifact` API supports it; otherwise stop with the external draft intact.
|
|
54
|
+
|
|
55
|
+
### Example
|
|
56
|
+
|
|
57
|
+
```bash
|
|
58
|
+
yy ledger search --status todo --limit 5
|
|
59
|
+
yy ledger get TASK_ID
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
> Adapted from [yylo-dev/yylo-skills](https://github.com/yylo-dev/yylo-skills) (MIT) - v2.0.1; frontmatter, When to Use/Limitations, and safety boundaries added for upstream compliance.
|
|
@@ -0,0 +1,277 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: weather-model-data-fetching
|
|
3
|
+
description: "Retrieve numerical weather prediction data from public AWS S3 and HTTP archives using GRIB2 inventories, byte ranges, Herbie, provider fallbacks, and verified caching."
|
|
4
|
+
category: data
|
|
5
|
+
risk: safe
|
|
6
|
+
source: self
|
|
7
|
+
source_type: self
|
|
8
|
+
date_added: "2026-09-18"
|
|
9
|
+
author: ShianMike
|
|
10
|
+
tags: [weather, grib2, aws-s3, herbie, noaa, nwp]
|
|
11
|
+
tools: [claude, cursor, gemini, codex]
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
# Weather Model Data Fetching
|
|
15
|
+
|
|
16
|
+
## Overview
|
|
17
|
+
|
|
18
|
+
Fetch numerical weather prediction data without treating a multi-gigabyte GRIB2
|
|
19
|
+
file as one indivisible download. Prefer an existing project adapter or Herbie;
|
|
20
|
+
use direct object-store byte ranges only when the supported path cannot express
|
|
21
|
+
the request.
|
|
22
|
+
|
|
23
|
+
This skill covers transport, inventory selection, caching, and verification. It
|
|
24
|
+
does not interpret the forecast or decide whether a model is meteorologically
|
|
25
|
+
appropriate.
|
|
26
|
+
|
|
27
|
+
## When to Use This Skill
|
|
28
|
+
|
|
29
|
+
- A task needs GFS, GEFS, HRRR, RAP, NAM, IFS, or similar model output.
|
|
30
|
+
- Data lives in a public AWS S3 bucket, NOMADS, or a public cloud mirror.
|
|
31
|
+
- The input is GRIB2 and only selected variables or levels are needed.
|
|
32
|
+
- A point, sounding, time series, map, or batch job needs a reliable fetch path.
|
|
33
|
+
- A download is missing, partial, unexpectedly large, slow, or hard to resume.
|
|
34
|
+
|
|
35
|
+
Do not activate this skill for ordinary weather-forecast questions that do not
|
|
36
|
+
require model files.
|
|
37
|
+
|
|
38
|
+
## Define the Request First
|
|
39
|
+
|
|
40
|
+
Resolve these values before downloading:
|
|
41
|
+
|
|
42
|
+
- model and product;
|
|
43
|
+
- initialization cycle in UTC;
|
|
44
|
+
- forecast hour and therefore valid time (`valid = initialization + lead`);
|
|
45
|
+
- ensemble member when applicable;
|
|
46
|
+
- variables, vertical levels, and surface fields;
|
|
47
|
+
- point, region, or full-grid output;
|
|
48
|
+
- cache location and when the downloaded data may be deleted.
|
|
49
|
+
|
|
50
|
+
Confirm that the cycle is complete, the forecast hour exists for that cycle,
|
|
51
|
+
and the requested location is inside the model domain. A recent `404` often
|
|
52
|
+
means the cycle is not published yet; step back to a completed cycle instead of
|
|
53
|
+
retrying indefinitely.
|
|
54
|
+
|
|
55
|
+
## Choose the Smallest Retrieval Route
|
|
56
|
+
|
|
57
|
+
1. Reuse the project's existing fetch/cache abstraction when it already handles
|
|
58
|
+
the model.
|
|
59
|
+
2. Use Herbie for a supported GRIB2 model. It discovers AWS, NOMADS, Google,
|
|
60
|
+
Azure, and other configured sources and understands their key layouts.
|
|
61
|
+
3. Use a provider-native point or Zarr endpoint when the task needs a tiny
|
|
62
|
+
spatial slice from many times or members.
|
|
63
|
+
4. Use direct S3 or HTTPS object access when the key is known and no suitable
|
|
64
|
+
adapter exists.
|
|
65
|
+
|
|
66
|
+
Do not recursively list a large public bucket to discover one run. Build the
|
|
67
|
+
documented prefix for the model, cycle, product, forecast hour, and member, then
|
|
68
|
+
probe that exact object and its inventory.
|
|
69
|
+
|
|
70
|
+
Use an explicit provider priority and record the provider that succeeded. A
|
|
71
|
+
fallback must refer to the same model run, product, member, and forecast hour;
|
|
72
|
+
never silently substitute a different forecast.
|
|
73
|
+
|
|
74
|
+
## Subset GRIB2 by Inventory
|
|
75
|
+
|
|
76
|
+
GRIB2 files contain consecutive messages. A companion inventory such as
|
|
77
|
+
`.idx`, `.grib2.idx`, or `.grb2.inv` records each message's starting byte.
|
|
78
|
+
|
|
79
|
+
1. Fetch the small inventory first.
|
|
80
|
+
2. Inspect its actual rows before writing a regex.
|
|
81
|
+
3. Select exact variables, levels, and forecast-step records.
|
|
82
|
+
4. Set each selected message's end byte to one less than the next message's
|
|
83
|
+
start; request the final selected message through EOF when no end is known.
|
|
84
|
+
5. Coalesce adjacent selected messages into one range.
|
|
85
|
+
6. Issue one `Range: bytes=START-END` request per range. S3 does not support
|
|
86
|
+
multiple ranges in one `GetObject` request.
|
|
87
|
+
7. Require `206 Partial Content` and a matching `Content-Range`. If a server
|
|
88
|
+
answers `200`, do not append the whole object as though it were a fragment.
|
|
89
|
+
8. Pin the object's length and identity (`ETag` and/or `Last-Modified`) while
|
|
90
|
+
downloading. Discard fragments if the object changes.
|
|
91
|
+
9. Assemble into a temporary file, verify it with a GRIB decoder, then rename
|
|
92
|
+
atomically into the cache.
|
|
93
|
+
|
|
94
|
+
A GRIB message contains one field over its grid. Message-range subsetting saves
|
|
95
|
+
variables and levels, not geography. A point request still downloads the full
|
|
96
|
+
grid for every selected message unless the provider offers a point, regional,
|
|
97
|
+
Zarr, or other chunked endpoint.
|
|
98
|
+
|
|
99
|
+
## Herbie Example
|
|
100
|
+
|
|
101
|
+
Use the current `search` argument; `searchString` is deprecated. Start from the
|
|
102
|
+
inventory, fail on an empty match, and keep the download directory explicit.
|
|
103
|
+
|
|
104
|
+
```python
|
|
105
|
+
from pathlib import Path
|
|
106
|
+
|
|
107
|
+
from herbie import Herbie
|
|
108
|
+
|
|
109
|
+
PRESSURE_FIELDS = (
|
|
110
|
+
r":(?:HGT|TMP|RH|SPFH|UGRD|VGRD):\d+(?:\.\d+)? mb:"
|
|
111
|
+
)
|
|
112
|
+
|
|
113
|
+
|
|
114
|
+
def fetch_hrrr_pressure_run(initialization, forecast_hour, cache_dir):
|
|
115
|
+
cache_dir = Path(cache_dir)
|
|
116
|
+
h = Herbie(
|
|
117
|
+
initialization,
|
|
118
|
+
model="hrrr",
|
|
119
|
+
product="prs",
|
|
120
|
+
fxx=forecast_hour,
|
|
121
|
+
priority=["aws", "nomads", "google", "azure"],
|
|
122
|
+
save_dir=cache_dir,
|
|
123
|
+
verbose=False,
|
|
124
|
+
)
|
|
125
|
+
|
|
126
|
+
selected = h.inventory(PRESSURE_FIELDS)
|
|
127
|
+
if selected.empty:
|
|
128
|
+
raise RuntimeError("inventory matched no pressure-level fields")
|
|
129
|
+
|
|
130
|
+
downloaded = h.download(PRESSURE_FIELDS, errors="raise")
|
|
131
|
+
path = Path(downloaded) if downloaded is not None else None
|
|
132
|
+
if path is None or not path.is_file() or path.stat().st_size == 0:
|
|
133
|
+
raise RuntimeError("GRIB2 subset was not materialized")
|
|
134
|
+
|
|
135
|
+
return path, {
|
|
136
|
+
"model": h.model,
|
|
137
|
+
"product": h.product,
|
|
138
|
+
"initialization": h.date.isoformat(),
|
|
139
|
+
"forecast_hour": h.fxx,
|
|
140
|
+
"valid_time": h.valid_date.isoformat(),
|
|
141
|
+
"provider": h.grib_source,
|
|
142
|
+
"remote_object": str(h.grib),
|
|
143
|
+
"messages": len(selected),
|
|
144
|
+
}
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
For xarray output, call `h.xarray(search, ...)` and handle either one
|
|
148
|
+
`xarray.Dataset` or a list of incompatible GRIB hypercubes. Merge only groups
|
|
149
|
+
whose coordinates and dimensions are compatible, and close every dataset when
|
|
150
|
+
finished.
|
|
151
|
+
|
|
152
|
+
## Public AWS S3 Diagnostics
|
|
153
|
+
|
|
154
|
+
NOAA Open Data buckets allow unsigned reads. `--no-sign-request` prevents the
|
|
155
|
+
AWS CLI from loading credentials; it does not disable TLS verification.
|
|
156
|
+
|
|
157
|
+
```bash
|
|
158
|
+
aws s3 ls --no-sign-request s3://noaa-hrrr-bdp-pds/hrrr.YYYYMMDD/conus/
|
|
159
|
+
|
|
160
|
+
aws s3api get-object --no-sign-request \
|
|
161
|
+
--bucket noaa-hrrr-bdp-pds \
|
|
162
|
+
--key "hrrr.YYYYMMDD/conus/hrrr.tHHz.wrfprsfFF.grib2" \
|
|
163
|
+
--range "bytes=START-END" fragment.grib2
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
Use these commands to inspect a documented public object or reproduce one
|
|
167
|
+
known range. For normal multi-message assembly, reuse Herbie or the project's
|
|
168
|
+
tested downloader instead of scripting binary concatenation in shell.
|
|
169
|
+
|
|
170
|
+
## Complete Sounding Contract
|
|
171
|
+
|
|
172
|
+
A pressure-level file alone may not contain a usable surface row. Before
|
|
173
|
+
building a vertical profile, require:
|
|
174
|
+
|
|
175
|
+
- all published isobaric levels for geopotential height, temperature, a
|
|
176
|
+
moisture variable (dew point, relative humidity, or specific humidity), and
|
|
177
|
+
U/V wind;
|
|
178
|
+
- surface pressure and terrain or surface height;
|
|
179
|
+
- 2 m temperature and moisture;
|
|
180
|
+
- 10 m U/V wind.
|
|
181
|
+
|
|
182
|
+
Some providers split pressure and surface fields into separate products. Fetch
|
|
183
|
+
and join the companion product from the same run, or reject the request with a
|
|
184
|
+
list of missing fields. Do not fabricate a ground row or silently reduce the
|
|
185
|
+
profile to a short mandatory-level list.
|
|
186
|
+
|
|
187
|
+
After decoding, sort pressure monotonically, remove duplicate levels, normalize
|
|
188
|
+
units and longitude conventions, and run the consuming project's profile QC.
|
|
189
|
+
|
|
190
|
+
## Point and Batch Extraction
|
|
191
|
+
|
|
192
|
+
- On one-dimensional latitude/longitude grids, labeled nearest selection may be
|
|
193
|
+
sufficient.
|
|
194
|
+
- On projected or curvilinear grids with two-dimensional coordinates, use the
|
|
195
|
+
project's model-aware nearest-cell routine; verify the selected latitude,
|
|
196
|
+
longitude, and distance.
|
|
197
|
+
- For many points from one model hour, fetch and decode once, then reuse it.
|
|
198
|
+
- For many hours, members, or regional slices, compare the GRIB route with a
|
|
199
|
+
chunked Zarr or provider-native endpoint before scaling up.
|
|
200
|
+
|
|
201
|
+
## Reliability, Cache, and Cleanup
|
|
202
|
+
|
|
203
|
+
- Cache by provider, object key, object identity, and field selection. A
|
|
204
|
+
filename alone is not enough provenance.
|
|
205
|
+
- Retry timeouts, `408`, `429`, and transient `5xx` responses with bounded
|
|
206
|
+
exponential backoff and jitter; honor `Retry-After`.
|
|
207
|
+
- Do not retry permission errors, malformed inventories, or impossible model
|
|
208
|
+
coordinates as transient failures.
|
|
209
|
+
- Bound concurrency. More range workers can increase throttling and make
|
|
210
|
+
cancellation slower.
|
|
211
|
+
- Keep partial files separate from valid cache entries and resume only when the
|
|
212
|
+
remote object identity still matches.
|
|
213
|
+
- For a one-shot render or export, isolate data in a request-specific temporary
|
|
214
|
+
directory and remove it in `finally` after the derived artifact is durable.
|
|
215
|
+
- For an interactive viewer, retain data until the final consumer closes. Never
|
|
216
|
+
delete a shared user cache as request cleanup.
|
|
217
|
+
|
|
218
|
+
Measure discovery, inventory, transfer, decode, point extraction, and rendering
|
|
219
|
+
separately. A slow end-to-end request is not evidence that GRIB decoding is the
|
|
220
|
+
bottleneck.
|
|
221
|
+
|
|
222
|
+
## Verification Checklist
|
|
223
|
+
|
|
224
|
+
- The resolved initialization time, forecast hour, valid time, product, and
|
|
225
|
+
member match the request.
|
|
226
|
+
- The selected inventory is nonempty and contains every required field/level.
|
|
227
|
+
- The response status, byte ranges, lengths, and object identity are consistent.
|
|
228
|
+
- The final file is nonempty and opens with the intended GRIB decoder.
|
|
229
|
+
- Decoded variables, units, level count, grid coordinates, and valid time are
|
|
230
|
+
plausible and explicit.
|
|
231
|
+
- A point result reports the actual selected grid coordinate.
|
|
232
|
+
- Cancellation leaves no file that can be mistaken for a complete cache hit.
|
|
233
|
+
- Cleanup preserves the requested final artifact and removes only data owned by
|
|
234
|
+
that request.
|
|
235
|
+
|
|
236
|
+
## Security & Safety Notes
|
|
237
|
+
|
|
238
|
+
- Fetch only public datasets or resources the user is authorized to access.
|
|
239
|
+
- Do not put cloud credentials in code, URLs, logs, examples, or skill files.
|
|
240
|
+
- Keep certificate verification enabled; never solve TLS errors with
|
|
241
|
+
`--no-verify-ssl`.
|
|
242
|
+
- Validate inventory-derived ranges against the remote object length before
|
|
243
|
+
allocating buffers or writing files.
|
|
244
|
+
- Bound requested cycles, members, forecast hours, concurrency, disk usage, and
|
|
245
|
+
retries before a large batch.
|
|
246
|
+
- Follow provider usage policies and preserve required dataset attribution.
|
|
247
|
+
|
|
248
|
+
## Common Pitfalls
|
|
249
|
+
|
|
250
|
+
- **No data for the newest run:** The cycle is still publishing. Use the newest
|
|
251
|
+
completed cycle and report the fallback.
|
|
252
|
+
- **Subset is as large as the full file:** The inventory was missing, the regex
|
|
253
|
+
was too broad, or the server ignored `Range`.
|
|
254
|
+
- **xarray returns a list:** The selected messages form multiple incompatible
|
|
255
|
+
hypercubes. Process them separately or merge only compatible groups.
|
|
256
|
+
- **Point extraction is still expensive:** GRIB message ranges are not spatial
|
|
257
|
+
chunks. Use a point/regional service or Zarr when available.
|
|
258
|
+
- **A cached file opens but has missing fields:** Validate the inventory contract
|
|
259
|
+
and object identity before accepting a cache hit.
|
|
260
|
+
|
|
261
|
+
## Limitations
|
|
262
|
+
|
|
263
|
+
- Provider key layouts, retention windows, model schedules, and Herbie templates
|
|
264
|
+
can change; verify them against current provider documentation.
|
|
265
|
+
- Variable subsetting requires a usable remote inventory. Without one, download
|
|
266
|
+
the full object or use a different provider.
|
|
267
|
+
- This skill does not validate forecast skill, scientific suitability, or
|
|
268
|
+
proprietary-provider credentials and quotas.
|
|
269
|
+
|
|
270
|
+
## Additional Resources
|
|
271
|
+
|
|
272
|
+
- [Herbie documentation](https://herbie.readthedocs.io/)
|
|
273
|
+
- [Herbie source and model templates](https://github.com/blaylockbk/Herbie)
|
|
274
|
+
- [NOAA fast GRIB2 downloads with inventories](https://nomads.ncep.noaa.gov/info.php?page=fastdownload)
|
|
275
|
+
- [Amazon S3 `GetObject` byte ranges](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html)
|
|
276
|
+
- [NOAA HRRR on the AWS Registry of Open Data](https://registry.opendata.aws/noaa-hrrr-pds/)
|
|
277
|
+
- [NOAA GFS on the AWS Registry of Open Data](https://registry.opendata.aws/noaa-gfs-bdp-pds/)
|