@zalom/plastic 1.0.0-alpha.10 → 1.0.0-alpha.12
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/PLASTIC.md +106 -475
- package/README.md +43 -60
- package/deprecations.yml +2 -2
- package/package.json +1 -1
- package/scripts/install.rb +19 -8
- package/skills/auto/SKILL.md +4 -0
- package/skills/auto/references/agent-architecture.md +60 -0
- package/skills/continuing/SKILL.md +4 -0
- package/skills/continuing/references/context-management.md +32 -0
- package/skills/creating-intent/SKILL.md +5 -0
- package/skills/creating-intent/references/lifecycle.md +74 -0
- package/skills/creating-intent/references/wikilinks.md +8 -0
- package/skills/creating-project/SKILL.md +4 -0
- package/skills/creating-project/references/hubs-projects.md +55 -0
- package/skills/doctor/SKILL.md +5 -1
- package/skills/doctor/references/gates-stuck-detection.md +38 -0
- package/skills/evaluating-skills/SKILL.md +140 -0
- package/skills/evaluating-skills/assets/eval-template.json +12 -0
- package/skills/evaluating-skills/evals/evals.json +75 -0
- package/skills/evaluating-skills/references/convention-checks.md +76 -0
- package/skills/evaluating-skills/references/eval-methodology.md +154 -0
- package/skills/install/SKILL.md +25 -1
- package/skills/linking-intents/SKILL.md +4 -0
- package/skills/linking-intents/references/zettelkasten.md +33 -0
- package/skills/managing-index/SKILL.md +4 -0
- package/skills/releasing/SKILL.md +33 -3
- package/skills/releasing/references/deprecations.md +44 -0
- package/skills/savepoint/SKILL.md +4 -0
- package/skills/savepoint/references/context-management.md +32 -0
- package/skills/update/SKILL.md +79 -26
- package/skills/writing-instructions/SKILL.md +159 -0
- package/skills/writing-instructions/references/agentskills-spec.md +135 -0
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
# Context Management (Start-Save-Continue)
|
|
2
|
+
|
|
3
|
+
## Save Point
|
|
4
|
+
Triggered by PreCompact hook or manually:
|
|
5
|
+
1. Find active intent(s) from `~/.plastic/INDEX.md`
|
|
6
|
+
2. Update active intent's `checklist.md` (check off completed items)
|
|
7
|
+
3. Update active intent's `savepoint.md` (in-progress, next steps, blockers, discoveries)
|
|
8
|
+
4. Add observations to `## Insights`
|
|
9
|
+
5. Update INDEX.md
|
|
10
|
+
6. Commit: `cd ~/.plastic && git add . && git commit -m "chore: savepoint — [intent name]"`
|
|
11
|
+
7. Notify user to `/clear`
|
|
12
|
+
|
|
13
|
+
## Continue
|
|
14
|
+
Triggered by UserPromptSubmit hook when user says "continue". Priority order:
|
|
15
|
+
|
|
16
|
+
**1. Active intents first (resume work):**
|
|
17
|
+
1. Read INDEX.md → find active intent(s)
|
|
18
|
+
2. Read active intent's `intent.md` → what and why
|
|
19
|
+
3. Read active intent's `savepoint.md` → where we left off
|
|
20
|
+
4. Read active intent's `checklist.md` → what's next
|
|
21
|
+
5. Announce: intent name, current state, next step, blockers
|
|
22
|
+
6. Resume
|
|
23
|
+
|
|
24
|
+
**2. No active intents → offer future intents:**
|
|
25
|
+
1. List all future intents from INDEX.md
|
|
26
|
+
2. Present them as options
|
|
27
|
+
3. When user picks one, move to Active in INDEX.md
|
|
28
|
+
|
|
29
|
+
**3. Stale future intents (untouched 3+ days) → triage:**
|
|
30
|
+
- **activate** — start working on it now
|
|
31
|
+
- **abandon** — mark as abandoned
|
|
32
|
+
- **defer to agent** — implement, research, or ideate
|
package/skills/update/SKILL.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
|
-
name: update
|
|
3
|
-
description: Use when updating Plastic
|
|
2
|
+
name: plastic:update
|
|
3
|
+
description: Use when updating Plastic. Queries npm for available versions across all channels, presents options interactively, or accepts --alpha/--beta/--latest flags for direct update.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Update Plastic
|
|
@@ -10,43 +10,98 @@ description: Use when updating Plastic after a plugin update, or when the user s
|
|
|
10
10
|
- Statusline shows "Plastic update available"
|
|
11
11
|
- After a version bump notification
|
|
12
12
|
|
|
13
|
+
## Flags
|
|
14
|
+
|
|
15
|
+
| Flag | Behavior |
|
|
16
|
+
|------|----------|
|
|
17
|
+
| `--alpha` | Update to latest alpha, skip interactive prompt |
|
|
18
|
+
| `--beta` | Update to latest beta, skip interactive prompt |
|
|
19
|
+
| `--latest` | Update to latest stable, skip interactive prompt |
|
|
20
|
+
|
|
21
|
+
No flag = interactive mode (show all available versions).
|
|
22
|
+
|
|
13
23
|
## Prerequisites
|
|
14
24
|
|
|
15
25
|
Global install must exist (`~/.plastic/INDEX.md` present). If not, tell the
|
|
16
|
-
user to run `npx @zalom/plastic@
|
|
26
|
+
user to run `npx @zalom/plastic@alpha --claude` first.
|
|
17
27
|
|
|
18
28
|
## Procedure
|
|
19
29
|
|
|
20
|
-
### Step 1:
|
|
30
|
+
### Step 1: Read current version
|
|
31
|
+
|
|
32
|
+
```bash
|
|
33
|
+
cat ~/.plastic/VERSION
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
Parse the version string to determine the current channel:
|
|
37
|
+
- Contains `-alpha` → alpha channel
|
|
38
|
+
- Contains `-beta` → beta channel
|
|
39
|
+
- No pre-release suffix → stable/latest channel
|
|
40
|
+
|
|
41
|
+
### Step 2: Query npm for available versions
|
|
21
42
|
|
|
22
43
|
```bash
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
44
|
+
npm view @zalom/plastic dist-tags --json
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
This returns a JSON object like:
|
|
48
|
+
```json
|
|
49
|
+
{
|
|
50
|
+
"latest": "0.0.1",
|
|
51
|
+
"alpha": "1.0.0-alpha.14",
|
|
52
|
+
"beta": "1.0.0-beta.2"
|
|
53
|
+
}
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
A missing dist-tag means no release exists on that channel.
|
|
57
|
+
|
|
58
|
+
### Step 3: Present options or act on flag
|
|
59
|
+
|
|
60
|
+
**If a channel flag was provided** (`--alpha`, `--beta`, `--latest`):
|
|
61
|
+
|
|
62
|
+
Skip the interactive prompt. Install the flagged channel directly. Go to Step 4.
|
|
63
|
+
|
|
64
|
+
**If no flag (interactive mode):**
|
|
65
|
+
|
|
66
|
+
Present the available updates to the user:
|
|
67
|
+
|
|
27
68
|
```
|
|
69
|
+
Currently installed: 1.0.0-alpha.11 (alpha channel)
|
|
28
70
|
|
|
29
|
-
|
|
71
|
+
Available updates:
|
|
72
|
+
alpha: 1.0.0-alpha.14 ← your channel
|
|
73
|
+
beta: 1.0.0-beta.2
|
|
74
|
+
stable: (no stable release yet)
|
|
75
|
+
|
|
76
|
+
Which channel do you want to install?
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
Use AskUserQuestion with the available channels as options. Mark the user's
|
|
80
|
+
current channel with "← your channel". If a dist-tag points to the same
|
|
81
|
+
version as currently installed, show "(up to date)" instead of the version.
|
|
82
|
+
If a dist-tag doesn't exist, show "(no release yet)".
|
|
83
|
+
|
|
84
|
+
Wait for user selection.
|
|
85
|
+
|
|
86
|
+
### Step 4: Run the installer
|
|
30
87
|
|
|
31
88
|
```bash
|
|
32
|
-
npx @zalom/plastic@
|
|
89
|
+
npx @zalom/plastic@{selected-tag} --claude
|
|
33
90
|
```
|
|
34
91
|
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
- Preserves all user data (INDEX.md, config.yml, projects.yml, store/)
|
|
92
|
+
Replace `{selected-tag}` with `alpha`, `beta`, or `latest` based on the
|
|
93
|
+
user's selection or the flag provided. Replace `--claude` with the appropriate
|
|
94
|
+
agent flag(s) — detect current agents from `~/.claude/`, `~/.agents/`,
|
|
95
|
+
`~/.hermes/` directories.
|
|
40
96
|
|
|
41
|
-
### Step
|
|
97
|
+
### Step 5: Announce key changes
|
|
42
98
|
|
|
43
99
|
After the installer completes, read `~/.plastic/PLASTIC.md` and announce any
|
|
44
|
-
convention changes that affect the current session.
|
|
45
|
-
in-context understanding without needing /clear.
|
|
100
|
+
convention changes that affect the current session.
|
|
46
101
|
|
|
47
102
|
Format:
|
|
48
103
|
```
|
|
49
|
-
Plastic updated to vX.Y.Z.
|
|
104
|
+
Plastic updated to vX.Y.Z (channel).
|
|
50
105
|
|
|
51
106
|
Key changes in this version:
|
|
52
107
|
- [list notable convention changes if any]
|
|
@@ -54,24 +109,22 @@ Key changes in this version:
|
|
|
54
109
|
Recommendation: run /clear for a clean session with all new conventions loaded.
|
|
55
110
|
```
|
|
56
111
|
|
|
57
|
-
### Step
|
|
112
|
+
### Step 6: Run health check
|
|
58
113
|
|
|
59
114
|
Invoke `plastic:doctor` to verify the installation is healthy after the update.
|
|
60
115
|
|
|
61
116
|
If all checks pass, show: **"Health check: all clear."**
|
|
62
117
|
|
|
63
|
-
If issues are found, show the full doctor report and offer to fix
|
|
118
|
+
If issues are found, show the full doctor report and offer to fix.
|
|
64
119
|
|
|
65
|
-
### Step
|
|
120
|
+
### Step 7: Commit
|
|
66
121
|
|
|
67
122
|
```bash
|
|
68
|
-
cd ~/.plastic && git add PLASTIC.md scripts/ AGENTS.md VERSION 2>/dev/null && git commit -m "chore: update Plastic
|
|
123
|
+
cd ~/.plastic && git add PLASTIC.md scripts/ AGENTS.md VERSION 2>/dev/null && git commit -m "chore: update Plastic to $(cat ~/.plastic/VERSION)" --allow-empty
|
|
69
124
|
```
|
|
70
125
|
|
|
71
|
-
### Step
|
|
126
|
+
### Step 8: Clear update cache
|
|
72
127
|
|
|
73
128
|
```bash
|
|
74
129
|
rm -f ~/.plastic/.cache/update-check.json
|
|
75
130
|
```
|
|
76
|
-
|
|
77
|
-
This removes the statusline warning since the update is now applied.
|
|
@@ -0,0 +1,159 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: writing-instructions
|
|
3
|
+
description: >
|
|
4
|
+
Write or restructure agent instructions, conventions files, and SKILL.md
|
|
5
|
+
content using progressive disclosure and the agentskills.io specification.
|
|
6
|
+
Use when creating PLASTIC.md, rewriting convention docs, authoring new
|
|
7
|
+
skills, restructuring large instruction files to fit context budgets,
|
|
8
|
+
or when instructions feel too long, too vague, or agents aren't following
|
|
9
|
+
them. Also use when the user says "progressive disclosure", "restructure
|
|
10
|
+
instructions", or "the instructions are too big".
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
# Writing Agent Instructions
|
|
14
|
+
|
|
15
|
+
Based on the [agentskills.io specification](https://agentskills.io).
|
|
16
|
+
|
|
17
|
+
## Progressive Disclosure Architecture
|
|
18
|
+
|
|
19
|
+
All agent instructions follow three loading stages. Design for this — it's the
|
|
20
|
+
architecture, not a nice-to-have.
|
|
21
|
+
|
|
22
|
+
| Stage | What loads | Budget | Design for |
|
|
23
|
+
|-------|-----------|--------|------------|
|
|
24
|
+
| **Discovery** | name + description | ~100 tokens | Trigger accuracy |
|
|
25
|
+
| **Activation** | Full instruction body | <5000 tokens / <500 lines | Core procedures |
|
|
26
|
+
| **Execution** | references/, scripts/, assets/ | As needed | Deep detail |
|
|
27
|
+
|
|
28
|
+
**The description carries the entire burden of triggering.** If agents aren't
|
|
29
|
+
activating your skill, the description is the problem.
|
|
30
|
+
|
|
31
|
+
## Procedure
|
|
32
|
+
|
|
33
|
+
### Step 1: Audit the content
|
|
34
|
+
|
|
35
|
+
Before writing or restructuring, classify every piece of content:
|
|
36
|
+
|
|
37
|
+
| Classification | Where it goes | Example |
|
|
38
|
+
|---------------|---------------|---------|
|
|
39
|
+
| **Trigger context** | description field | "Use when...", keywords |
|
|
40
|
+
| **Core procedure** | SKILL.md body | Step-by-step workflows |
|
|
41
|
+
| **Gotchas** | SKILL.md body (early) | Facts that defy assumptions |
|
|
42
|
+
| **Deep reference** | references/ | API details, full schemas |
|
|
43
|
+
| **Templates** | assets/ or inline | Output format examples |
|
|
44
|
+
| **Executable logic** | scripts/ | Validation, data processing |
|
|
45
|
+
|
|
46
|
+
Ask for each piece: "Would the agent get this wrong without this?"
|
|
47
|
+
If no — cut it. If unsure — test it.
|
|
48
|
+
|
|
49
|
+
### Step 2: Write the description
|
|
50
|
+
|
|
51
|
+
The description must convey WHEN to use, not just WHAT it does.
|
|
52
|
+
|
|
53
|
+
**Rules:**
|
|
54
|
+
- Imperative phrasing: "Use this skill when..." not "This skill does..."
|
|
55
|
+
- Focus on user intent, not implementation
|
|
56
|
+
- Err on the side of being pushy — list contexts explicitly
|
|
57
|
+
- Include cases where user doesn't name the domain directly
|
|
58
|
+
- Under 1024 characters (hard limit)
|
|
59
|
+
- Include specific trigger keywords
|
|
60
|
+
|
|
61
|
+
**Template:**
|
|
62
|
+
```yaml
|
|
63
|
+
description: >
|
|
64
|
+
[One sentence: what it does]. Use when [primary trigger context],
|
|
65
|
+
[secondary trigger], or when [indirect trigger where user doesn't
|
|
66
|
+
name the domain]. Also use when [edge case trigger].
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
### Step 3: Write the body
|
|
70
|
+
|
|
71
|
+
**Principles (in priority order):**
|
|
72
|
+
|
|
73
|
+
1. **Procedures over declarations** — Teach HOW to approach a class of problems,
|
|
74
|
+
not WHAT to produce for a specific instance
|
|
75
|
+
2. **Defaults, not menus** — Pick one approach, mention alternatives briefly.
|
|
76
|
+
"Use X. For Y cases, use Z instead." Never present options as equals.
|
|
77
|
+
3. **Calibrate control to fragility** — Prescriptive for fragile/sequential tasks,
|
|
78
|
+
flexible when multiple approaches are valid. Most skills have a mix.
|
|
79
|
+
4. **Reasoning over rigid directives** — "Do X because Y tends to cause Z" beats
|
|
80
|
+
"ALWAYS do X, NEVER do Y"
|
|
81
|
+
5. **Add what the agent lacks, omit what it knows** — No explaining HTTP, PDFs,
|
|
82
|
+
or what a migration is. Jump to what's non-obvious.
|
|
83
|
+
|
|
84
|
+
**Structure:**
|
|
85
|
+
```markdown
|
|
86
|
+
# Title
|
|
87
|
+
|
|
88
|
+
[1-2 sentence purpose statement]
|
|
89
|
+
|
|
90
|
+
## Gotchas
|
|
91
|
+
- [Highest-value content first — facts that defy assumptions]
|
|
92
|
+
- [Each gotcha is a concrete correction, not general advice]
|
|
93
|
+
|
|
94
|
+
## Procedure
|
|
95
|
+
### Step 1: ...
|
|
96
|
+
### Step 2: ...
|
|
97
|
+
|
|
98
|
+
## Patterns
|
|
99
|
+
[Only if the skill covers multiple approaches to similar problems]
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
### Step 4: Extract to references/
|
|
103
|
+
|
|
104
|
+
Anything over 500 lines or 5000 tokens goes to `references/`. But tell the
|
|
105
|
+
agent WHEN to load each file — conditional references, not a generic "see
|
|
106
|
+
references/ for details."
|
|
107
|
+
|
|
108
|
+
**Good:** "Read `references/api-errors.md` if the API returns a non-200 status"
|
|
109
|
+
**Bad:** "See references/ for more information"
|
|
110
|
+
|
|
111
|
+
### Step 5: Validate
|
|
112
|
+
|
|
113
|
+
Run through this checklist:
|
|
114
|
+
|
|
115
|
+
- [ ] Description under 1024 chars
|
|
116
|
+
- [ ] Body under 500 lines / 5000 tokens
|
|
117
|
+
- [ ] Every instruction passes "would the agent get this wrong without it?"
|
|
118
|
+
- [ ] Gotchas are concrete corrections, not general advice
|
|
119
|
+
- [ ] Defaults chosen, not menus presented
|
|
120
|
+
- [ ] Control calibrated: prescriptive where fragile, flexible where tolerant
|
|
121
|
+
- [ ] References have conditional load triggers
|
|
122
|
+
- [ ] No explaining what the agent already knows
|
|
123
|
+
|
|
124
|
+
For structured evaluation beyond this checklist (paired evals, pass rate
|
|
125
|
+
tracking, regression testing), use `plastic:evaluating-skills`.
|
|
126
|
+
|
|
127
|
+
## Gotchas
|
|
128
|
+
|
|
129
|
+
- Hook `additionalContext` truncates at **10,000 characters**. If instructions
|
|
130
|
+
must load via hooks (not skills), they must fit this budget.
|
|
131
|
+
- `~/.claude/rules/*.md` files load fully with no truncation — use for
|
|
132
|
+
always-loaded conventions that don't fit in a skill.
|
|
133
|
+
- CLAUDE.md supports `@path/to/file` imports (max depth 4) — another way
|
|
134
|
+
to load large instruction sets without truncation.
|
|
135
|
+
- Agents only consult skills for tasks beyond what they can handle alone.
|
|
136
|
+
Simple one-step requests may not trigger even with a perfect description.
|
|
137
|
+
- Content from real domain expertise (runbooks, incident reports, code review)
|
|
138
|
+
dramatically outperforms LLM-generated instructions without project context.
|
|
139
|
+
- The most common cause of agents not following instructions: the instruction
|
|
140
|
+
was too vague, didn't apply to the current task, or presented too many
|
|
141
|
+
options without a clear default. Read execution traces to diagnose.
|
|
142
|
+
|
|
143
|
+
## For Convention Files (PLASTIC.md, CLAUDE.md)
|
|
144
|
+
|
|
145
|
+
Convention files aren't skills — they load differently. Apply progressive
|
|
146
|
+
disclosure by splitting:
|
|
147
|
+
|
|
148
|
+
| Layer | Mechanism | Budget | Content |
|
|
149
|
+
|-------|-----------|--------|---------|
|
|
150
|
+
| **Always-on** | `~/.claude/rules/` or hook | <10K chars | Core identity, gotchas, critical procedures |
|
|
151
|
+
| **On-demand** | Skills (SKILL.md) | <5K tokens each | Specific workflows activated by task |
|
|
152
|
+
| **Deep reference** | `references/` in skills | Unlimited | Full specs, schemas, examples |
|
|
153
|
+
|
|
154
|
+
**The always-on layer should answer: "What does this agent need to know about
|
|
155
|
+
every single task?"** Everything else activates on demand.
|
|
156
|
+
|
|
157
|
+
## References
|
|
158
|
+
|
|
159
|
+
- Read `references/agentskills-spec.md` for the complete agentskills.io specification including frontmatter fields, description optimization methodology, evaluation framework, and script design requirements
|
|
@@ -0,0 +1,135 @@
|
|
|
1
|
+
# agentskills.io Full Reference
|
|
2
|
+
|
|
3
|
+
Source: https://agentskills.io (all sections, verified June 2026)
|
|
4
|
+
|
|
5
|
+
## Specification Details
|
|
6
|
+
|
|
7
|
+
### Frontmatter Fields
|
|
8
|
+
|
|
9
|
+
| Field | Required | Constraints |
|
|
10
|
+
|-------|----------|-------------|
|
|
11
|
+
| name | Yes | 1-64 chars. Lowercase alphanumeric + hyphens. No leading/trailing/consecutive hyphens. Must match directory name. |
|
|
12
|
+
| description | Yes | 1-1024 chars. Non-empty. What + when. |
|
|
13
|
+
| license | No | Short — name or filename reference |
|
|
14
|
+
| compatibility | No | 1-500 chars. Environment requirements only when needed. |
|
|
15
|
+
| metadata | No | String→string map. Use unique key names. |
|
|
16
|
+
| allowed-tools | No | Space-separated. Experimental. |
|
|
17
|
+
|
|
18
|
+
### Progressive Disclosure Token Budgets
|
|
19
|
+
|
|
20
|
+
- Discovery: ~100 tokens per skill (name + description only)
|
|
21
|
+
- Activation: <5000 tokens / <500 lines recommended for SKILL.md body
|
|
22
|
+
- Execution: Unbounded — files in scripts/, references/, assets/ load as needed
|
|
23
|
+
|
|
24
|
+
### File References
|
|
25
|
+
|
|
26
|
+
- Use relative paths from skill root
|
|
27
|
+
- Keep one level deep from SKILL.md
|
|
28
|
+
- Agent resolves paths automatically
|
|
29
|
+
|
|
30
|
+
## Description Optimization
|
|
31
|
+
|
|
32
|
+
### Evaluation Methodology
|
|
33
|
+
|
|
34
|
+
1. Create ~20 eval queries (8-10 should-trigger, 8-10 should-not)
|
|
35
|
+
2. Split 60/40 train/validation (proportional mix in each)
|
|
36
|
+
3. Run each query 3 times, compute trigger rate
|
|
37
|
+
4. Pass threshold: 0.5
|
|
38
|
+
5. Near-miss negatives are most valuable (share keywords, need different thing)
|
|
39
|
+
6. Iterate on train set only, validate on held-out set
|
|
40
|
+
7. Select best by validation pass rate, not last iteration
|
|
41
|
+
8. 5 iterations usually sufficient
|
|
42
|
+
|
|
43
|
+
### Description Anti-patterns
|
|
44
|
+
|
|
45
|
+
- "Helps with PDFs" — too vague, no trigger context
|
|
46
|
+
- "Process CSV files" — no when/why, no user-intent focus
|
|
47
|
+
- Implementation details instead of user intent
|
|
48
|
+
- Missing edge case triggers (user doesn't name the domain)
|
|
49
|
+
|
|
50
|
+
## Instruction Best Practices
|
|
51
|
+
|
|
52
|
+
### Gotchas — Highest-Value Content
|
|
53
|
+
|
|
54
|
+
Concrete corrections, not general advice:
|
|
55
|
+
|
|
56
|
+
```markdown
|
|
57
|
+
## Gotchas
|
|
58
|
+
- The `users` table uses soft deletes. Queries must include
|
|
59
|
+
`WHERE deleted_at IS NULL`.
|
|
60
|
+
- User ID is `user_id` in DB, `uid` in auth, `accountId` in billing.
|
|
61
|
+
All three refer to the same value.
|
|
62
|
+
- The `/health` endpoint returns 200 even if DB is down. Use `/ready`.
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
### Calibrating Control
|
|
66
|
+
|
|
67
|
+
Prescriptive when:
|
|
68
|
+
- Operations are fragile
|
|
69
|
+
- Consistency matters
|
|
70
|
+
- Specific sequence must be followed
|
|
71
|
+
|
|
72
|
+
Flexible when:
|
|
73
|
+
- Multiple approaches are valid
|
|
74
|
+
- Task tolerates variation
|
|
75
|
+
- Explaining WHY is more effective than rigid rules
|
|
76
|
+
|
|
77
|
+
### Instruction Patterns
|
|
78
|
+
|
|
79
|
+
1. **Validation loops**: Do work → validate → fix → repeat
|
|
80
|
+
2. **Plan-validate-execute**: Create plan → validate vs source of truth → execute
|
|
81
|
+
3. **Checklists**: Track progress, enforce dependencies, validation gates
|
|
82
|
+
4. **Bundled scripts**: If agent reinvents same logic each run, bundle it
|
|
83
|
+
5. **Templates**: Concrete output structures > prose descriptions
|
|
84
|
+
|
|
85
|
+
## Script Design
|
|
86
|
+
|
|
87
|
+
### Hard Requirements
|
|
88
|
+
- No interactive prompts (hard requirement — agents hang indefinitely)
|
|
89
|
+
- All input via flags, env vars, or stdin
|
|
90
|
+
|
|
91
|
+
### Agent-Friendly Design
|
|
92
|
+
- --help as primary interface documentation
|
|
93
|
+
- Helpful error messages: what wrong + what expected + what to try
|
|
94
|
+
- Structured output (JSON/CSV/TSV), data on stdout, diagnostics on stderr
|
|
95
|
+
- Idempotent operations (agents may retry)
|
|
96
|
+
- Dry-run for destructive operations
|
|
97
|
+
- Meaningful exit codes documented in --help
|
|
98
|
+
- Output size control: default to summaries, support --offset pagination
|
|
99
|
+
- Agent harnesses truncate at 10-30K characters
|
|
100
|
+
|
|
101
|
+
## Evaluation Framework
|
|
102
|
+
|
|
103
|
+
### Test Case Structure
|
|
104
|
+
```json
|
|
105
|
+
{
|
|
106
|
+
"skill_name": "name",
|
|
107
|
+
"evals": [{
|
|
108
|
+
"id": 1,
|
|
109
|
+
"prompt": "realistic user message",
|
|
110
|
+
"expected_output": "what success looks like",
|
|
111
|
+
"files": ["evals/files/input.csv"],
|
|
112
|
+
"assertions": ["specific, verifiable checks"]
|
|
113
|
+
}]
|
|
114
|
+
}
|
|
115
|
+
```
|
|
116
|
+
|
|
117
|
+
### Running Evals
|
|
118
|
+
- With-skill vs without-skill (or previous version) comparison
|
|
119
|
+
- Clean context per run (subagents or separate sessions)
|
|
120
|
+
- Capture timing: total_tokens, duration_ms
|
|
121
|
+
- Start with 2-3 test cases, expand after first results
|
|
122
|
+
|
|
123
|
+
### Assertion Quality
|
|
124
|
+
Good: Programmatically verifiable, specific, countable
|
|
125
|
+
Weak: Vague ("the output is good")
|
|
126
|
+
Brittle: Exact phrase matching
|
|
127
|
+
|
|
128
|
+
Principle: Require concrete evidence for PASS. No benefit of the doubt.
|
|
129
|
+
|
|
130
|
+
### Iteration Loop
|
|
131
|
+
1. Run evals → grade assertions → aggregate benchmarks
|
|
132
|
+
2. Identify failures (assertions, human feedback, execution transcripts)
|
|
133
|
+
3. Feed all three + SKILL.md to LLM for proposed changes
|
|
134
|
+
4. Apply changes → re-run → compare
|
|
135
|
+
5. Stop when consistently empty feedback or no meaningful improvement
|