@ericrisco/rsc 0.1.52 → 0.1.54
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/manifest.json +15 -15
- package/package.json +1 -1
- package/skills/gamedev-shaders/SKILL.md +23 -43
- package/skills/gamedev-shaders/evals/cases.yaml +2 -2
- package/skills/gcp-essentials/SKILL.md +16 -19
- package/skills/hetzner/SKILL.md +18 -36
- package/skills/knowledge-ops/SKILL.md +14 -17
- package/skills/lead-gen/SKILL.md +2 -6
- package/skills/lead-gen/evals/cases.yaml +2 -2
- package/skills/linkedin-carousels/SKILL.md +31 -42
- package/skills/llm-pipeline/SKILL.md +4 -9
- package/skills/medium-publishing/SKILL.md +5 -13
- package/skills/modal/SKILL.md +31 -49
- package/skills/n8n/SKILL.md +37 -65
- package/skills/newsletter/SKILL.md +10 -20
- package/skills/newsletter/references/growth-loops.md +3 -3
- package/skills/notion-connector/SKILL.md +13 -26
- package/skills/planetscale/SKILL.md +18 -43
- package/skills/planetscale/references/no-foreign-keys.md +2 -2
- package/skills/prompt-engineering/SKILL.md +5 -16
- package/skills/prompt-engineering/evals/cases.yaml +1 -1
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: newsletter
|
|
3
|
-
description: "Use when running an email newsletter as a recurring publication —
|
|
3
|
+
description: "Use when running an email newsletter as a recurring publication — subject + preview pair, issues that turn opens into clicks, the welcome sequence, cadence and engagement tiering, growth loops, and a post-Apple-MPP click scorecard. NOT one-off launch or nurture sends (that is `marketing`), NOT SPF/DKIM/DMARC (that is `email-deliverability`)."
|
|
4
4
|
tags: [newsletter, email, growth, copywriting, retention-metrics]
|
|
5
5
|
recommends: [marketing, email-deliverability, brand-voice, landing-copy, ab-testing]
|
|
6
6
|
profiles: []
|
|
@@ -11,29 +11,19 @@ origin: risco
|
|
|
11
11
|
|
|
12
12
|
Run the newsletter as a **product you ship on a cadence**, not a one-off send. The job is the system around the recurring issue: the subject + preview that earns the open, the issue body that earns the click, the welcome sequence that activates a new subscriber, the cadence and segmentation that keep the list healthy, the growth loops that compound it, and a scorecard that still tells the truth after Apple broke open rates. Every send is one episode of an ongoing show — design the show, not the episode.
|
|
13
13
|
|
|
14
|
-
## When to use
|
|
15
|
-
|
|
16
|
-
- Writing the subject line + preview text for an issue and fitting them to inbox truncation.
|
|
17
|
-
- Structuring a recurring issue so opens convert: hook on the first screen, scannable body, one CTA.
|
|
18
|
-
- Writing the welcome / onboarding sequence a confirmed subscriber gets.
|
|
19
|
-
- Planning send cadence, frequency, and engagement-based segmentation.
|
|
20
|
-
- Designing a growth loop: referral program, signup incentive / lead magnet, double opt-in.
|
|
21
|
-
- Choosing which metrics to trust now that Apple MPP inflates opens; building the scorecard.
|
|
22
|
-
- Rescuing a newsletter with high opens but no clicks, or a quietly decaying engaged-reader count.
|
|
23
|
-
|
|
24
14
|
## When NOT to use — route instead
|
|
25
15
|
|
|
26
16
|
| The ask | Route to |
|
|
27
17
|
|---|---|
|
|
28
18
|
| One-off product launch announcement or campaign drip nurture | `../marketing/SKILL.md` |
|
|
29
|
-
| Mail lands in spam; set up SPF / DKIM / DMARC, tracking domain, reputation |
|
|
19
|
+
| Mail lands in spam; set up SPF / DKIM / DMARC, tracking domain, reputation | `../email-deliverability/SKILL.md` |
|
|
30
20
|
| Write the subscribe / opt-in landing page copy | `../landing-copy/SKILL.md` |
|
|
31
21
|
| Define the house voice, tone, and vocabulary the newsletter speaks in | `../brand-voice/SKILL.md` |
|
|
32
|
-
| Win back / reactivate lapsed *paying customers* (lifecycle, not readers) |
|
|
22
|
+
| Win back / reactivate lapsed *paying customers* (lifecycle, not readers) | `../retention/SKILL.md` |
|
|
33
23
|
| Produce the underlying long-form articles the issue links out to | `../content-engine/SKILL.md` |
|
|
34
|
-
| Sample size / significance / MDE math for the test itself |
|
|
35
|
-
| Source or scrape new contacts; prospect an un-opted-in list |
|
|
36
|
-
| Actually wire the send through an ESP / Gmail API and schedule it |
|
|
24
|
+
| Sample size / significance / MDE math for the test itself | `../ab-testing/SKILL.md` |
|
|
25
|
+
| Source or scrape new contacts; prospect an un-opted-in list | `../lead-gen/SKILL.md`, `../cold-outreach/SKILL.md` |
|
|
26
|
+
| Actually wire the send through an ESP / Gmail API and schedule it | `../email-connector/SKILL.md`, `../google-workspace/SKILL.md` |
|
|
37
27
|
|
|
38
28
|
You own the *recurring opted-in publication and its open/click/growth system*. The plumbing under every send and the page that captures the signup are someone else's.
|
|
39
29
|
|
|
@@ -81,7 +71,7 @@ The subject and the preview (preheader) ship together and are tested together
|
|
|
81
71
|
- **Payload in the first ~33 characters**; keep the whole subject ~30–50 chars (≈7–9 words) so it survives mobile truncation.
|
|
82
72
|
- **The preview extends the subject, it never repeats it.** Its first ~35–50 chars are prime real estate; waste them on a duplicate and you've burned the line.
|
|
83
73
|
- **≤2 emojis, at the end, never an all-emoji subject** — an all-emoji subject is the one case that actually trips filters.
|
|
84
|
-
- **The "spam trigger words" list is a myth.** "Free", "guarantee", a normal emoji — these do not sink you; authentication and sender reputation dominate deliverability. Police reputation, not vocabulary; the reputation/auth layer lives in
|
|
74
|
+
- **The "spam trigger words" list is a myth.** "Free", "guarantee", a normal emoji — these do not sink you; authentication and sender reputation dominate deliverability. Police reputation, not vocabulary; the reputation/auth layer lives in `../email-deliverability/SKILL.md`.
|
|
85
75
|
|
|
86
76
|
```text
|
|
87
77
|
Bad (payload past char 33, preview repeats subject):
|
|
@@ -134,13 +124,13 @@ A newsletter is the canonical bulk marketing message, so the Gmail/Yahoo bulk-se
|
|
|
134
124
|
- **RFC 8058 one-click unsubscribe** (`List-Unsubscribe` + `List-Unsubscribe-Post` headers) on every send, plus a **visible footer unsubscribe link.** Both, always.
|
|
135
125
|
- **Keep the spam-complaint rate under 0.1%** and never let it reach 0.3%. This is why you sunset, not why you mail harder.
|
|
136
126
|
|
|
137
|
-
The DNS / authentication side of those rules — SPF, DKIM, DMARC, the sending domain — is **not this skill**; that is
|
|
127
|
+
The DNS / authentication side of those rules — SPF, DKIM, DMARC, the sending domain — is **not this skill**; that is `../email-deliverability/SKILL.md`. This skill applies the rules to the issue and the cadence.
|
|
138
128
|
|
|
139
129
|
## A/B testing the right way
|
|
140
130
|
|
|
141
131
|
- **Test exactly one element at a time** — subject vs subject, or preview vs preview. Change two and the winner is uninterpretable.
|
|
142
132
|
- **Put preview text in the rotation**, not just the subject; it's half the inbox row and usually under-tested.
|
|
143
|
-
- The statistical design — sample size, significance, minimum detectable effect — is
|
|
133
|
+
- The statistical design — sample size, significance, minimum detectable effect — is `../ab-testing/SKILL.md`. This skill tells you *what* to test; that one tells you whether the result is real.
|
|
144
134
|
|
|
145
135
|
## Anti-patterns
|
|
146
136
|
|
|
@@ -151,7 +141,7 @@ The DNS / authentication side of those rules — SPF, DKIM, DMARC, the sending d
|
|
|
151
141
|
| Preview text repeats the subject | Burns half the inbox row on a duplicate | Preview extends the subject with new info |
|
|
152
142
|
| Mailing a decaying list harder | Complaints climb; reputation and inbox placement collapse | Sunset / re-permission the dormant tier |
|
|
153
143
|
| All-emoji subject line | The one emoji case that actually trips filters | ≤2 emojis, at the end, around real words |
|
|
154
|
-
| Policing "spam trigger words" | A myth; reputation/auth decide deliverability | Fix auth + reputation in
|
|
144
|
+
| Policing "spam trigger words" | A myth; reputation/auth decide deliverability | Fix auth + reputation in `../email-deliverability/SKILL.md` |
|
|
155
145
|
| Buying a list / no double opt-in | Spam traps and complaints gate the whole sender | Double opt-in; grow with referral + lead magnet |
|
|
156
146
|
| Testing two variables at once | Winner is uninterpretable | One variable; preview in the rotation |
|
|
157
147
|
| Referral ask bolted on months later | Misses peak engagement | Put the ask inside the welcome sequence |
|
|
@@ -23,7 +23,7 @@ tiering. Numbers below are dated; treat them as calibration, not promises.
|
|
|
23
23
|
generic "join our newsletter."
|
|
24
24
|
- The asset doubles as shareable bait for the referral loop — make it good
|
|
25
25
|
enough to forward.
|
|
26
|
-
- The opt-in *page* and its copy are
|
|
26
|
+
- The opt-in *page* and its copy are `../../landing-copy/SKILL.md`; this skill owns
|
|
27
27
|
what the incentive is and how it ties into the welcome + referral flow.
|
|
28
28
|
|
|
29
29
|
## Opt-in: single vs double
|
|
@@ -64,10 +64,10 @@ this? click to stay") and, if no click, a **sunset** off the active list.
|
|
|
64
64
|
is the lever that protects it. (Yahoo Sender Hub best practices; Google
|
|
65
65
|
Workspace sender guidelines — accessed 2026-06-02.)
|
|
66
66
|
- Re-permission is a newsletter-list hygiene step. Lifecycle reactivation of
|
|
67
|
-
lapsed *paying customers* is a different job — that is
|
|
67
|
+
lapsed *paying customers* is a different job — that is `../../retention/SKILL.md`.
|
|
68
68
|
|
|
69
69
|
## Send-time hypothesis
|
|
70
70
|
|
|
71
71
|
Tuesday/Thursday ~10am or 1–3pm is the cross-industry baseline starting point.
|
|
72
72
|
It is a hypothesis to test against your own list's click data, never a fixed law.
|
|
73
|
-
The test design (sample size, significance) is
|
|
73
|
+
The test design (sample size, significance) is `../../ab-testing/SKILL.md`.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: notion-connector
|
|
3
|
-
description: "Use when wiring
|
|
3
|
+
description: "Use when wiring server code or a cron job to Notion as an ops backend over its HTTP API: pushing or mirroring database rows, two-way sync without duplicates, page blocks, or an integration broken by the 2025-09-03 data-source split. NOT generic REST wiring (that is `api-connector-builder`), NOT inbound Notion webhook events (that is `webhooks`)."
|
|
4
4
|
tags: [notion, ops-backend, api-integration, databases, sync]
|
|
5
5
|
recommends: [api-connector-builder, webhooks, automation-flows, spreadsheet-ops, secure-coding]
|
|
6
6
|
origin: risco
|
|
@@ -14,24 +14,7 @@ write page blocks. This skill owns the **outbound** Notion API surface only —
|
|
|
14
14
|
the database/data-source data model, property-type write shapes, and the
|
|
15
15
|
rate-limit/pagination discipline that the API forces on you.
|
|
16
16
|
|
|
17
|
-
##
|
|
18
|
-
|
|
19
|
-
**Pin the `Notion-Version` per request, and resolve the data source before you
|
|
20
|
-
query.** Post `2025-09-03` a database is a *container* of one or more data
|
|
21
|
-
sources — not a queryable table. Skip either half and you get silent 404s on
|
|
22
|
-
code that worked last year.
|
|
23
|
-
|
|
24
|
-
- Current major version: **`2025-09-03`** (the SDK default). Latest: **`2026-03-11`**.
|
|
25
|
-
- Behavior differs across versions; an unpinned client drifts when the default moves.
|
|
26
|
-
|
|
27
|
-
## When to use / when NOT to use
|
|
28
|
-
|
|
29
|
-
**Use when:** treating a Notion database as a CRM / task tracker / content
|
|
30
|
-
calendar from code; pushing or pulling rows on a schedule; two-way sync that
|
|
31
|
-
must not duplicate on re-run; building or reading page block content; migrating
|
|
32
|
-
a `2022-06-28` integration that broke when someone added a second data source.
|
|
33
|
-
|
|
34
|
-
**Route elsewhere:**
|
|
17
|
+
## Route elsewhere
|
|
35
18
|
|
|
36
19
|
| Situation | Route to |
|
|
37
20
|
|---|---|
|
|
@@ -50,10 +33,12 @@ a `2022-06-28` integration that broke when someone added a second data source.
|
|
|
50
33
|
3. **Share the target database/page with the integration** in the Notion UI
|
|
51
34
|
(the page `•••` menu → Connections). *Skip this and every call 404s or
|
|
52
35
|
returns empty* — the integration sees nothing it was not explicitly granted.
|
|
53
|
-
4. **Construct the SDK client with a pinned
|
|
54
|
-
`@notionhq/client` v5.12.0+ (latest 5.22.0, 2026-05-19); its default
|
|
55
|
-
`notionVersion` is
|
|
56
|
-
The default and method names below are
|
|
36
|
+
4. **Construct the SDK client with a pinned `Notion-Version`.** Official JS SDK
|
|
37
|
+
is `@notionhq/client` v5.12.0+ (latest 5.22.0, 2026-05-19); its default
|
|
38
|
+
`notionVersion` is the current major **`2025-09-03`**, and it supports the
|
|
39
|
+
latest **`2026-03-11`** if you opt in. The default and method names below are
|
|
40
|
+
stable across the whole 5.x line. Behavior differs across versions, so pin it
|
|
41
|
+
per client (or per request) — an unpinned client drifts when the default moves.
|
|
57
42
|
|
|
58
43
|
```ts
|
|
59
44
|
import { Client } from "@notionhq/client"; // v5.12.0+ (latest 5.22.0)
|
|
@@ -66,9 +51,11 @@ const notion = new Client({
|
|
|
66
51
|
|
|
67
52
|
## The database → data source model (biggest gotcha)
|
|
68
53
|
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
54
|
+
Post `2025-09-03` a database is a **container** of one or more data sources, not
|
|
55
|
+
a queryable table: it holds a `data_sources` array and each data source has its
|
|
56
|
+
**own schema**. Resolve the data source *before* you query — query and read
|
|
57
|
+
schema against it, not against the database, or code that worked last year 404s
|
|
58
|
+
silently.
|
|
72
59
|
|
|
73
60
|
| You have… | Do this |
|
|
74
61
|
|---|---|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: planetscale
|
|
3
|
-
description: "Use when operating PlanetScale serverless MySQL
|
|
3
|
+
description: "Use when operating PlanetScale (Vitess serverless MySQL) — branches and the `pscale` CLI, schema changes through deploy requests (Online DDL cutover, ~30-min revert window), tables without foreign keys, or the edge HTTP driver. NOT MySQL engine work like EXPLAIN or index choice (that is `mysql`), NOT generic migration theory (that is `db-migrations`)."
|
|
4
4
|
tags: [planetscale, vitess, mysql, deploy-requests, serverless-db]
|
|
5
5
|
recommends: [mysql, db-migrations, prisma-orm, drizzle-orm, github-actions, postgresdb]
|
|
6
6
|
origin: risco
|
|
@@ -17,34 +17,15 @@ then **cuts over** with no table lock. Because Online DDL and sharding are the p
|
|
|
17
17
|
gravity, you **design without foreign keys** (FKs are now an opt-in but unsharded-only feature). This
|
|
18
18
|
skill owns that workflow — not MySQL the engine.
|
|
19
19
|
|
|
20
|
-
|
|
20
|
+
Heuristic for the boundary: **would this answer be identical on RDS MySQL? Then it is not this skill.**
|
|
21
21
|
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
- Wiring deploy requests into CI (GitHub Actions) or a branch-per-PR preview database.
|
|
30
|
-
|
|
31
|
-
**When NOT to use:**
|
|
32
|
-
|
|
33
|
-
- A slow query, an `EXPLAIN` plan, an index decision, JSON columns, locking — that is MySQL-engine
|
|
34
|
-
work. The closest built sibling for engine-level relational guidance is `../postgresdb/SKILL.md`
|
|
35
|
-
(Postgres, but the engine-thinking transfers); pure MySQL tuning routes to the `mysql` sibling.
|
|
36
|
-
Heuristic: **would this answer be identical on RDS MySQL? Then it is not this skill.**
|
|
37
|
-
- PostgreSQL engine/design — even PlanetScale's managed Postgres SQL semantics → `../postgresdb/SKILL.md`.
|
|
38
|
-
PlanetScale sells Postgres too, but this skill is strictly the **Vitess/MySQL branch+deploy-request** flow.
|
|
39
|
-
- Engine-agnostic migration *theory* (expand-contract, backfill ordering) → the `db-migrations` sibling.
|
|
40
|
-
Here you get the concrete PlanetScale realization, not the theory.
|
|
41
|
-
- Neon's Postgres copy-on-write branches → the `neon` sibling. Supabase's Postgres BaaS → `supabase`.
|
|
42
|
-
- ORM client ergonomics (Prisma `updateMany`, Drizzle query builder) → `prisma-orm` / `../drizzle-orm/SKILL.md`.
|
|
43
|
-
This skill owns only the PlanetScale-specific knobs those ORMs expose (`relationMode`, serverless driver adapter).
|
|
44
|
-
|
|
45
|
-
Deep dives: [deploy-requests](references/deploy-requests.md) (lifecycle states, queue/gating, lint
|
|
46
|
-
errors, revert edge cases, declarative vs imperative) · [no-foreign-keys](references/no-foreign-keys.md)
|
|
47
|
-
(emulated relations, ORM `relationMode`, the unsharded FK opt-in, Bad→Good schemas).
|
|
22
|
+
| When the question is really about | Route to |
|
|
23
|
+
| --- | --- |
|
|
24
|
+
| A slow query, an `EXPLAIN` plan, an index decision, JSON columns, locking | `../mysql/SKILL.md` for MySQL-engine work; `../postgresdb/SKILL.md` when you want the transferable engine reasoning |
|
|
25
|
+
| PostgreSQL engine/design — even PlanetScale's own managed Postgres SQL semantics | `../postgresdb/SKILL.md`. PlanetScale sells Postgres too; this skill is strictly the **Vitess/MySQL branch+deploy-request** flow |
|
|
26
|
+
| Engine-agnostic migration *theory* (expand-contract, backfill ordering) | `../db-migrations/SKILL.md`. Here you get the concrete PlanetScale realization, not the theory |
|
|
27
|
+
| Postgres copy-on-write branches | `../neon/SKILL.md`. Postgres BaaS → `../supabase/SKILL.md` |
|
|
28
|
+
| ORM client ergonomics (Prisma `updateMany`, Drizzle query builder) | `../prisma-orm/SKILL.md` / `../drizzle-orm/SKILL.md`. This skill owns only the PlanetScale-specific knobs those ORMs expose (`relationMode`, serverless driver adapter) |
|
|
48
29
|
|
|
49
30
|
## Mental model
|
|
50
31
|
|
|
@@ -149,9 +130,14 @@ datasource db {
|
|
|
149
130
|
```
|
|
150
131
|
|
|
151
132
|
With Drizzle, model the relation in app code (`relations()`), index the join column, and skip the
|
|
152
|
-
`references()`-backed FK constraint at the DB layer.
|
|
153
|
-
|
|
154
|
-
|
|
133
|
+
`references()`-backed FK constraint at the DB layer.
|
|
134
|
+
|
|
135
|
+
| Will this table shard? | FK strategy | Notes |
|
|
136
|
+
| --- | --- | --- |
|
|
137
|
+
| Yes, or unsure | No DB FK — emulate in app/ORM | Default. Index the relation column; cascade in code. |
|
|
138
|
+
| Never (small, bounded, unsharded) | Unsharded FK opt-in *is* acceptable | Accept Online DDL friction; document why it won't shard. |
|
|
139
|
+
|
|
140
|
+
See [no-foreign-keys](references/no-foreign-keys.md) for emulated cascades, ORM specifics, and the opt-in caveats.
|
|
155
141
|
|
|
156
142
|
## Connecting
|
|
157
143
|
|
|
@@ -196,7 +182,7 @@ installs `pscale`, authenticates with a service token (`PLANETSCALE_SERVICE_TOKE
|
|
|
196
182
|
|
|
197
183
|
For a **branch-per-PR preview database**, create a PlanetScale branch named after the PR head, run the
|
|
198
184
|
app's migrations against it, and tear the branch down when the PR closes. The orchestration (matrix,
|
|
199
|
-
secrets, cleanup-on-close) is generic CI — defer the Actions plumbing to
|
|
185
|
+
secrets, cleanup-on-close) is generic CI — defer the Actions plumbing to `../github-actions/SKILL.md`
|
|
200
186
|
and keep the PlanetScale verbs here.
|
|
201
187
|
|
|
202
188
|
## Anti-patterns
|
|
@@ -210,14 +196,3 @@ and keep the PlanetScale verbs here.
|
|
|
210
196
|
| Treating revert as available forever | The window is **~30 min** and excludes dropped tables/cols and FK changes | Plan rollbacks as new forward deploys |
|
|
211
197
|
| Deploying without reading `deploy-request diff` | You ship DDL you never saw; surprise table rewrites | Always run the diff before `deploy` |
|
|
212
198
|
| Inventing a driver name (`@planetscale/serverless`, `planetscale-js`) | There is no such package; install fails | The package is exactly `@planetscale/database` |
|
|
213
|
-
|
|
214
|
-
## Decision: FK strategy by sharding posture
|
|
215
|
-
|
|
216
|
-
| Will this table shard? | FK strategy | Notes |
|
|
217
|
-
| --- | --- | --- |
|
|
218
|
-
| Yes, or unsure | No DB FK — emulate in app/ORM | Default. Index the relation column; cascade in code. |
|
|
219
|
-
| Never (small, bounded, unsharded) | Unsharded FK opt-in *is* acceptable | Accept Online DDL friction; document why it won't shard. |
|
|
220
|
-
|
|
221
|
-
When the question is really about reading a plan, picking an index, or query latency, that is engine
|
|
222
|
-
work — route to the `mysql` sibling (or `../postgresdb/SKILL.md` for the transferable engine
|
|
223
|
-
reasoning). This skill stops at the Vitess platform boundary.
|
|
@@ -58,13 +58,13 @@ model Order {
|
|
|
58
58
|
}
|
|
59
59
|
```
|
|
60
60
|
|
|
61
|
-
ORM-client ergonomics beyond this knob belong to
|
|
61
|
+
ORM-client ergonomics beyond this knob belong to `../../prisma-orm/SKILL.md`, not here.
|
|
62
62
|
|
|
63
63
|
## Drizzle: app-level relations
|
|
64
64
|
|
|
65
65
|
Drizzle models relations in app code via `relations()` and queries them with the relational API. On
|
|
66
66
|
PlanetScale you index the join column and skip the `references()`-backed DB constraint; the relation
|
|
67
|
-
lives in code, not in the database. The query-builder side is
|
|
67
|
+
lives in code, not in the database. The query-builder side is `../../drizzle-orm/SKILL.md`.
|
|
68
68
|
|
|
69
69
|
## The unsharded FK opt-in
|
|
70
70
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: prompt-engineering
|
|
3
|
-
description: "Use when one prompt
|
|
3
|
+
description: "Use when one prompt must give the same right answer across reruns, models, and pasted-in hostile input: forcing a fixed schema, picking the few-shot set, ordering the prompt blocks, or the inline cases you run while tuning. NOT the agent loop, tools, or retrieval (that is `building-agents`), NOT a standing CI eval harness (that is `agent-eval`)."
|
|
4
4
|
tags: [prompts, llm, few-shot, structured-output, evals]
|
|
5
5
|
recommends: [building-agents, agent-eval, structured-extraction]
|
|
6
6
|
origin: risco
|
|
@@ -10,12 +10,6 @@ origin: risco
|
|
|
10
10
|
|
|
11
11
|
You are tuning a single prompt so it produces the same correct output across reruns, across models, and against adversarial input. This is the craft layer — the prompt artifact itself: its block order, its few-shot set, its output contract, and the small eval that proves it. The systems layer (loop, tools, retrieval) is `../building-agents/SKILL.md`.
|
|
12
12
|
|
|
13
|
-
## The one rule
|
|
14
|
-
|
|
15
|
-
> A prompt is not done until it passes a small eval set you wrote **before** you started tuning. Without cases you are fiddling — changing words and trusting a vibe. With 5-15 cases, every edit is a measurement.
|
|
16
|
-
|
|
17
|
-
Write the cases first. They define "right answer" before you fall in love with a phrasing.
|
|
18
|
-
|
|
19
13
|
## Prompt skeleton (this order)
|
|
20
14
|
|
|
21
15
|
Order the blocks so the model reads identity and task before it sees the (untrusted) input. Each block earns its place:
|
|
@@ -86,11 +80,11 @@ Good (range: normal / awkward / edge):
|
|
|
86
80
|
- **Re-state the task after the input** for long inputs. Why: when the input is huge, early instructions fall out of attention — this is the cause of "the model ignores my instructions on long inputs".
|
|
87
81
|
- **Phrase constraints positively.** "Output only the category" survives; "don't add commentary" invites the model to negotiate.
|
|
88
82
|
- **Add a refusal anchor** — one explicit branch for out-of-contract input (`If the ticket is empty or unreadable, output: other`). Why: undefined behavior is where injections and drift live.
|
|
89
|
-
- System-scope abuse handling — refusal policy, monitoring, jailbreak defense across a product — is
|
|
83
|
+
- System-scope abuse handling — refusal policy, monitoring, jailbreak defense across a product — is `../agent-safety/SKILL.md`, not this skill. Here you harden one prompt.
|
|
90
84
|
|
|
91
|
-
## Inline evals
|
|
85
|
+
## Inline evals — write these first
|
|
92
86
|
|
|
93
|
-
|
|
87
|
+
A prompt is not done until it passes a small eval set you wrote **before** you started tuning: 5-15 cases next to the prompt, run before AND after every change. Cases fix what "right answer" means before you fall in love with a phrasing — without them you are fiddling, changing words and trusting a vibe; with them every edit is a measurement.
|
|
94
88
|
|
|
95
89
|
```yaml
|
|
96
90
|
# prompt-eval cases for the triage prompt
|
|
@@ -112,7 +106,7 @@ cases:
|
|
|
112
106
|
expect: { equals: "billing" } # input treated as data, not command
|
|
113
107
|
```
|
|
114
108
|
|
|
115
|
-
Assert on the contract: schema-valid, exact token, contains/not-contains. Keep them in the repo beside the prompt. The standing harness — golden set, LLM-as-judge, CI regression gate, metrics — is
|
|
109
|
+
Assert on the contract: schema-valid, exact token, contains/not-contains. Keep them in the repo beside the prompt; `references/eval-templates.md` has the cases.yaml shape, assertion helpers, and a before/after diff runner. The standing harness — golden set, LLM-as-judge, CI regression gate, metrics — is `../agent-eval/SKILL.md`; this is the small inline set you run while tuning.
|
|
116
110
|
|
|
117
111
|
## Iterate one variable at a time
|
|
118
112
|
|
|
@@ -131,8 +125,3 @@ Assert on the contract: schema-valid, exact token, contains/not-contains. Keep t
|
|
|
131
125
|
| 3 near-identical few-shot examples | Teaches one shape, wastes tokens | 3 deliberately different: normal / awkward / edge |
|
|
132
126
|
| Negative-only constraints ("don't…") | Invites negotiation, ignored on long input | Phrase positively; re-state task after long input |
|
|
133
127
|
| Tuning by vibe, no cases | You are fiddling, not engineering | Write 5-15 cases first; measure each edit |
|
|
134
|
-
|
|
135
|
-
## References
|
|
136
|
-
|
|
137
|
-
- `references/output-contracts.md` — per-provider strict-output recipes, schema surface (pydantic/zod), streaming caveat, retry-on-parse-fail loop.
|
|
138
|
-
- `references/eval-templates.md` — cases.yaml shape, assertion helpers, before/after diff runner, DSPy MIPROv2-vs-GEPA when-to-optimize note.
|
|
@@ -10,7 +10,7 @@ should_trigger:
|
|
|
10
10
|
- prompt: "Make this prompt enforce a fixed schema reliably instead of sometimes returning prose."
|
|
11
11
|
why: "Forcing a fixed schema out of a model — the output-contract decision table (strict json_schema / strict tool use vs JSON mode)."
|
|
12
12
|
- prompt: "Write me a small eval set to check this prompt before I change it."
|
|
13
|
-
why: "Inline eval set authored before tuning —
|
|
13
|
+
why: "Inline eval set authored before tuning — the discipline the 'Inline evals — write these first' section exists to enforce; the small set, not a standing harness."
|
|
14
14
|
- prompt: "Users can paste 'ignore your instructions' into my input field and the model obeys it."
|
|
15
15
|
why: "Prompt-scope injection hardening — delimiting untrusted input and treating it as data, a named robustness rule."
|
|
16
16
|
- prompt: "I'm moving this prompt from one model to another and the behavior shifted; how do I keep it stable?"
|