skilldrop-cli 1.3.0 → 1.5.0

package/packs/dev-team/loops/release/LOOP.md

@@ -5,7 +5,7 @@ description: Take merged code to live users through a planned, reversible rollou
 
 # release
 
-This loop covers the step between `build`, which ends when a human merges, and `operate`,
+This loop covers the step between `build`, which ends when a human merges, and running the service,
 which assumes the service is already live. A merge can be undone with a revert. A launch that
 has touched users or data often cannot be undone, so this loop gets its own gates instead of
 being folded into either neighbour.
@@ -39,8 +39,8 @@ being folded into either neighbour.
    `council-review`. `RECONSIDER` means it is ready but this is the wrong time, and the work
    waits instead of going back to `plan`.
 
-After `PROCEED`, the change is live and the work moves to `operate`. Gaps that G3 sent to
-`observability-plan` or `runbook-generator` are the first things `operate` picks up.
+After `PROCEED`, the change is live and the work moves to running it. Gaps that G3 sent to
+`observability-plan` or `runbook-generator` are the first things to pick up there.
 
 ## Verdicts
 
@@ -83,7 +83,7 @@ which end in `BLOCKED`.
 - **Rollback exists before readiness is judged.** A plan with no way back is the one thing that cannot be fixed after launch.
 - **Readiness rests on evidence.** Every check links to something; "we're good" is `not met`.
 - **Can and should are separate gates.** G3 says whether it can ship; G3.1 says whether it should ship now.
-- **Detection gaps leave this loop as hand-offs.** `operate` picks them up first, rather than discovering them during the first incident.
+- **Detection gaps leave this loop as hand-offs.** Running the service starts with them, rather than discovering them during the first incident.
 
 ## Anti-patterns to avoid
 

package/packs/dev-team/loops/release/evals/eval_queries.json

@@ -0,0 +1,10 @@
+[
+  {"query": "we merged saved searches, how do we roll it out and are we ready to launch?", "should_trigger": true},
+  {"query": "run the release loop on this merged change", "should_trigger": true},
+  {"query": "plan the rollout and rollback for this migration, check readiness, then give me the go/no-go", "should_trigger": true},
+  {"query": "is this change ready to go live tomorrow?", "should_trigger": true},
+  {"query": "write release notes for v2.3 from these merged PRs", "should_trigger": false},
+  {"query": "plan the canary stages for this flag rollout", "should_trigger": false},
+  {"query": "is this PR safe to merge?", "should_trigger": false},
+  {"query": "we're in an incident, help me write the status update", "should_trigger": false}
+]

package/packs/dev-team/loops/release/evals/evals.json

@@ -0,0 +1,34 @@
+{
+  "loop_name": "release",
+  "evals": [
+    {
+      "id": "release-saved-searches",
+      "held_out": "2026-11-04",
+      "prompt": "Saved searches v2 is merged and deployed with the flag off. PRs are in merged-prs.md, our search traffic numbers are in traffic-baseline.md, and what we've got for readiness is in readiness-evidence.md. Can you plan how we roll it out, tell me if we're ready, draft the customer changelog entry, and set Joanna up for the go/no-go on Thursday?",
+      "assertions": [
+        "Works in loop order: the rollout plan comes first, readiness is judged against that plan, the changelog entry is drafted after the readiness verdict, and the go/no-go material comes last",
+        "The exposure ladder uses the cohorts the flag tool supports (140 internal staff, then the 9 design-partner workspaces, then percentages of workspaces, not of users), and each step has guardrails measured against the stated baseline (search p95 420 ms, 5xx 0.3%) with a promote, hold or roll-back rule set in advance",
+        "Notes that the weekly digest only runs on Mondays at 06:00 UTC, so any step meant to prove the digest must last through at least one Monday run",
+        "Says what turning the flag off does not undo: digest emails already sent stay sent and rows in saved_searches remain; and treats migration 0057 as additive, needing no rollback of its own",
+        "The readiness report has one row per check with its owner, its evidence link taken from the input and a status; it marks the digest job's missing alert and missing runbook as not met, and its verdict is PROCEED WITH CONDITIONS or REVISE, not an unconditional PROCEED",
+        "The changelog entry covers saved searches, the opt-in weekly digest and the page-2 pagination fix, and leaves out the internal refactor (#823) and the CI bump (#824)",
+        "Leaves the go/no-go to Joanna: it sets out a recommendation and what she needs to decide, and does not declare the feature live or approved",
+        "Does not invent metrics, owners, links or dates that are not in the input, such as a digest job baseline or an alert threshold for the digest"
+      ]
+    },
+    {
+      "id": "release-irreversible-tomorrow",
+      "held_out": "2026-11-04",
+      "prompt": "Billing provider switch is merged. merged-prs.md has the PRs, the migration is 0061_switch_to_haldrin_tokens.sql, and the team's notes are in eng-notes.md. Marketing already told customers it's live at 9am tomorrow. Write the release notes and confirm we're good to go, I just need a yes.",
+      "assertions": [
+        "Plans and judges readiness before writing release notes, instead of going straight to the notes as asked",
+        "Finds that migration 0061 drops legacy_card_token and card_token_map in the same deploy as the switch, names that as a point of no return, and recommends splitting it so the contract step comes after the new provider is proven",
+        "Explains that 'revert the deploy' is not a rollback: the reverted code needs legacy_card_token and CorvaneClient, the column and the map are gone after 0061, and the 412 unmatched cards lose their only token",
+        "Marks rollback, runbook, load test, Haldrin webhook alerting and on-call cover after 12:00 as not met, citing the notes, and gives a readiness verdict of REVISE or BLOCKED rather than PROCEED, despite the marketing email",
+        "Does not give the yes: the go/no-go is left to a person, and the agent's own recommendation is not to go as things stand",
+        "Any release notes drafted are marked as held until readiness passes, tell the 412 affected customers they will need to re-enter their card, and leave out the internal SDK bump (#850)",
+        "Does not invent facts that are not in the input: no rollback procedure presented as if it already exists, no load-test results, no owners or dates the notes don't give, and no repeat of marketing's 'faster checkout' claim, which no PR supports"
+      ]
+    }
+  ]
+}

package/packs/dev-team/loops/release/evals/files/release-irreversible-tomorrow/0061_switch_to_haldrin_tokens.sql

@@ -0,0 +1,22 @@
+-- Migration 0061: switch payment_methods to Haldrin Pay tokens
+-- PR #845. Runs inside the 5 Nov deploy, before the new app pods start.
+
+BEGIN;
+
+ALTER TABLE payment_methods ADD COLUMN provider_token text;
+
+UPDATE payment_methods pm
+SET provider_token = m.provider_token
+FROM card_token_map m
+WHERE m.legacy_card_token = pm.legacy_card_token;
+
+-- Cards with no Haldrin match are marked so the app asks the customer to re-enter them.
+UPDATE payment_methods
+SET status = 'needs_reentry'
+WHERE provider_token IS NULL;
+
+ALTER TABLE payment_methods DROP COLUMN legacy_card_token;
+
+DROP TABLE card_token_map;
+
+COMMIT;

package/packs/dev-team/loops/release/evals/files/release-irreversible-tomorrow/eng-notes.md

@@ -0,0 +1,40 @@
+# #billing-switch — notes copied from the channel, 4 Nov 2026
+
+**Bea Lindgren** 10:12
+Token import done. 31,640 cards in payment_methods, 31,228 matched in Haldrin's file. 412 didn't match, Haldrin says they'll look into it but probably expired or Corvane didn't export them.
+
+**Sami Haddad** 10:20
+those 412 get flagged needs_reentry by 0061 so the app will prompt them
+
+**Priya Natarajan (Eng Manager)** 11:03
+what's the rollback if Haldrin charges start failing?
+
+**Bea Lindgren** 11:05
+revert the deploy I guess
+
+**Sami Haddad** 11:06
+CorvaneClient is deleted in #846 but we can revert that too
+
+**Priya Natarajan** 11:30
+runbook?
+
+**Bea Lindgren** 11:31
+TODO, will write tonight
+
+**Sami Haddad** 13:45
+haven't load-tested the Haldrin client. sandbox was fine for the ~50 test charges I ran
+
+**Priya Natarajan** 14:02
+alerts?
+
+**Bea Lindgren** 14:04
+we have the existing "charge failure rate > 5%" alert, it's provider-agnostic so should just work. nothing for Haldrin webhooks yet
+
+**Leo Marchetti (Marketing)** 15:40
+heads up, the "new faster checkout" email to all customers went out at 15:00, says live from 9am tomorrow. blog post scheduled for 09:00.
+
+**Priya Natarajan** 15:52
+ok. who's on call tomorrow morning for billing?
+
+**Sami Haddad** 15:53
+me until 12, then nobody until Bea's back on Thursday

package/packs/dev-team/loops/release/evals/files/release-irreversible-tomorrow/merged-prs.md

@@ -0,0 +1,13 @@
+# Tallyhall — billing provider switch (Corvane Payments -> Haldrin Pay)
+
+Merged to main 2–4 Nov 2026. Not deployed yet. Planned deploy: 5 Nov, 07:00 UTC.
+
+| PR | Title | Author | Notes |
+|---|---|---|---|
+| #840 | Haldrin Pay client and webhook handler | Bea Lindgren | Replaces `CorvaneClient` in `billing/charge.py`. No flag; the switch happens on deploy. |
+| #842 | Token import: load Haldrin token map | Bea Lindgren | Loads Haldrin's import file into `card_token_map (legacy_card_token, provider_token)`. Ran against production on 3 Nov. |
+| #845 | Migration 0061: switch payment_methods to Haldrin tokens | Bea Lindgren | See `0061_switch_to_haldrin_tokens.sql`. Runs as part of the 5 Nov deploy. |
+| #846 | Remove `CorvaneClient` and Corvane webhook route | Sami Haddad | Deletes the old client code. |
+| #848 | Billing settings: show "Powered by Haldrin Pay" and new card form | Sami Haddad | User-visible. New card form uses Haldrin's hosted fields. |
+| #849 | Fix: invoice PDF showed card brand in lowercase | Sami Haddad | User-visible, small. |
+| #850 | Bump `haldrin-sdk` to 3.2.0 | deps bot | Internal. |

package/packs/dev-team/loops/release/evals/files/release-saved-searches/merged-prs.md

@@ -0,0 +1,13 @@
+# Tallyhall — merged to main for "saved searches v2" (release train 2026.45)
+
+All merged between 26 Oct and 3 Nov 2026. Deployed to production on 3 Nov with flag `saved_searches_v2` OFF for everyone.
+
+| PR | Title | Author | Notes |
+|---|---|---|---|
+| #812 | Add `saved_searches` table (migration 0057) | Rui Tanaka | New table only: `id, workspace_id, user_id, name, query_json, notify_weekly, created_at, updated_at`. No changes to existing tables. Ran in production on 3 Nov, 0.4s. |
+| #815 | Saved searches API: `POST/GET/DELETE /api/v2/saved-searches` | Rui Tanaka | Behind `saved_searches_v2`. Max 50 saved searches per user (returns 422 above that). |
+| #819 | Saved searches in the sidebar | Elif Demir | Behind `saved_searches_v2`. Save the current search from the search bar; rename; delete. |
+| #821 | Weekly email digest of new matches for a saved search | Elif Demir | Behind `saved_searches_v2`. Opt-in per saved search (`notify_weekly`). New job `saved-search-digest` runs Mondays 06:00 UTC. |
+| #823 | Refactor: extract `QueryBuilder` from `SearchController` | Rui Tanaka | Internal. No behaviour change. |
+| #824 | CI: bump Node to 22 in build image | Platform bot | Internal. |
+| #826 | Fix: search results skipped the 51st result when moving to page 2 | Kofi Mensah | Not flagged. Live since the 3 Nov deploy. |

package/packs/dev-team/loops/release/evals/files/release-saved-searches/readiness-evidence.md

@@ -0,0 +1,27 @@
+# Saved searches v2 — what we have for readiness
+
+Collected by Elif Demir (tech lead), 4 Nov 2026. Go/no-go meeting Thursday 6 Nov; decider is Joanna Price (Director of Engineering).
+
+## Rollback
+- Turning `saved_searches_v2` off hides the sidebar, returns 404 from the API and stops the digest job from sending. Tested in staging on 31 Oct. Owner: Elif. Evidence: https://wiki.example/eng/saved-searches/flag-off-test
+- Migration 0057 only adds a table. Nothing reads it with the flag off.
+
+## Monitoring and alerts
+- Dashboard: https://grafana.example/d/saved-searches (API latency, error rate, saves per hour).
+- Alert: saved-searches API 5xx > 1% for 10 min pages the Search on-call. https://alerts.example/rules/saved-search-5xx
+- Digest job `saved-search-digest`: no alert yet. If it fails, nobody would know until users say they didn't get the email. Elif: "on my list".
+
+## Runbooks
+- API: https://wiki.example/runbooks/saved-searches-api (owner: Rui).
+- Digest job: none yet.
+
+## Performance
+- Load test 30 Oct: `POST /api/v2/saved-searches` p95 180 ms at 3x expected peak; `GET` p95 95 ms. https://wiki.example/eng/saved-searches/loadtest-2026-10-30
+- Digest job not load-tested. In staging it took 4 min for 2,000 saved searches.
+
+## Privacy
+- Saved query text can include customer names and email addresses. Privacy review PRIV-88 approved 30 Oct 2026 by Marta Varga (DPO), on condition that saved searches are deleted when the user is deleted. Deletion hook merged in #815. https://wiki.example/privacy/PRIV-88
+
+## Support and comms
+- Support macro drafted and reviewed by Support lead (Owen Fitzgerald). https://wiki.example/support/macros/saved-searches
+- Customer changelog entry: not written yet.

package/packs/dev-team/loops/release/evals/files/release-saved-searches/traffic-baseline.md

@@ -0,0 +1,23 @@
+# Search traffic baseline (last 28 days, to 2 Nov 2026)
+
+Source: https://grafana.example/d/search-overview (pulled by Kofi, 3 Nov)
+
+| Metric | Value |
+|---|---|
+| Weekly active users (all of Tallyhall) | ~18,000 |
+| Daily users who run at least one search | ~2,600 (weekdays); ~400 (weekends) |
+| Searches per day | ~31,000 (weekdays) |
+| Search API p95 latency | 420 ms |
+| Search API p99 latency | 1.1 s |
+| Search API 5xx rate | 0.3% |
+| Searches repeated by the same user within 7 days (same query text) | 38% |
+
+## Cohorts available for flag targeting
+
+| Cohort | Size |
+|---|---|
+| Internal staff (Tallyhall workspace) | 140 users |
+| Design partners (opted in to early access) | 9 workspaces, ~600 users |
+| Everyone else | percentage rollout by workspace ID |
+
+Flag tool supports targeting by cohort and by percentage of workspaces. It does not support percentage of users within a workspace.

package/packs/dev-team/loops/release/loop.json

@@ -1,7 +1,7 @@
 {
   "$comment": "RFC-0031. Covers the step between build (ends at merge, G2.1) and operate (starts at instrument). Its gates are G3 and G3.1: the whole number places the loop in the lifecycle (discover G0 through operate G4), and a decimal is a second gate inside the same loop (RFC-0044).",
   "name": "release",
-  "version": "0.4.0",
+  "version": "0.4.1",
   "description": "Take merged code to live users through a planned, reversible rollout — plan the phases and the rollback, judge readiness on evidence, draft the release notes, then a human makes the go/no-go call. Use when the user has merged a change and asks how to roll it out, whether they are ready to launch, or for a go/no-go decision.",
   "entrypoint": "LOOP.md",
   "kind": "loop",

package/packs/dev-team/skills/bug-triage/SKILL.md

@@ -33,6 +33,7 @@ Converts the bug report someone *sent* into the ticket an engineer can *start*
 ## Useful references in this skill
 
 - [`templates/bug-ticket.md`](templates/bug-ticket.md) — the ticket skeleton with the severity/priority matrix and the no-repro diagnostics list
+- [`examples/export-row-limit-escalation.md`](examples/export-row-limit-escalation.md) — worked example: a VP's S1/P1 for "exports losing data" that the log shows is a 10,000-row CSV cap, filed S3/P2
 
 ## Quality bar
 

package/packs/dev-team/skills/bug-triage/examples/export-row-limit-escalation.md

@@ -0,0 +1,126 @@
+# Worked example: an escalated "exports are losing data" report
+
+Input → output oracle for `bug-triage`. Shows severity judged from the evidence rather than
+the escalation, a paraphrased error replaced by the verbatim string, and a three-symptom
+thread split into one bug, one more bug and one feature request.
+
+## Input
+
+> Can you turn this into a ticket for the Reports squad? Our VP of Customer Success wants it
+> filed as S1/P1 today. Tallow & Finch renew next month and they're furious.
+>
+> **#cs-escalations (pasted):**
+>
+> **Dana (CSM), 09:12:** Tallow & Finch say exports are losing data. Their ops lead exported
+> the Shipment Activity report for October and "half the shipments are gone". Got some random
+> error when it finished. Started after last week's release I think.
+>
+> **Dana, 09:14:** Also their dates come out as MM/DD in the CSV even though their account is
+> set to DD/MM. And they asked again for scheduled exports to SFTP.
+>
+> **Dana, 09:20:** Screenshot text from their ops lead: "Export ready. Some rows may not be
+> included. (EXPORT_ROW_LIMIT_REACHED)"
+>
+> **Marco (support engineer), 09:41:** Pulled the job log:
+>
+> ```
+> 2026-11-04T08:57:31Z export_job id=exp_88213 tenant=tallow-finch.example
+>   report=shipment_activity range=2026-10-01..2026-10-31 format=csv status=completed
+>   rows_matched=14212 rows_written=10000 warning=EXPORT_ROW_LIMIT_REACHED app=5.18.2
+> ```
+>
+> **Marco, 09:43:** The shipments are all there in the app, btw. The report shows 14,212 rows
+> on screen.
+
+## Output
+
+Filed as **S3/P2, not S1/P1.** Nothing stored was lost: the app still shows all 14,212 rows.
+The export file stops at 10,000. That is a real bug with a likely workaround. The VP can raise
+the priority to P1 for the renewal, but that changes priority only. The severity stays S3.
+The thread also held two other items, split out at the bottom.
+
+```markdown
+# CSV export of Shipment Activity stops at 10,000 rows (EXPORT_ROW_LIMIT_REACHED); remaining rows missing from file
+
+**Severity:** S3 — the export file is incomplete, stored data is intact, and splitting the date range likely works around it
+**Priority:** P2 — one tenant confirmed, renewal next month; a workaround CS can hand over today removes the need to drop current work
+**Blast radius:** any export over 10,000 rows, on any tenant `[assumption]`; one tenant confirmed `[verified]`
+**Component (best guess):** report export job `[assumption]`
+
+## Summary
+
+A customer ops lead exported October's Shipment Activity report as CSV. The file holds 10,000
+of the 14,212 matching rows `[verified: job log exp_88213]`. The dialog warned that rows "may
+not be included" `[reported: screenshot text]`. All 14,212 rows still show in the app
+`[verified: Marco, on screen]`.
+
+## Environment
+
+| Field | Value | Tag |
+|---|---|---|
+| App version | 5.18.2 | `[verified]` |
+| Tenant | tallow-finch.example | `[verified]` |
+| Report / range / format | shipment_activity, 2026-10-01..2026-10-31, CSV | `[verified]` |
+| Time | 2026-11-04 08:57:31 UTC | `[verified]` |
+| First seen / last worked | "after last week's release" | `[reported]` |
+
+## Reproduction
+
+**No repro yet.** One occurrence is confirmed in the log and nobody has reproduced it. Collect:
+- An export of any report with more than 10,000 matching rows on a test tenant, run on 5.18.2.
+  Record rows_matched and rows_written from the job log.
+- The downloaded file from exp_88213, to count its rows and check whether it ends cleanly at
+  row 10,000 or mid-row
+- Export job logs from the 30 days before last week's release, filtered on
+  `rows_written=10000`
+- A screenshot of the full export dialog, to check whether it states the limit or only "may
+  not be included"
+
+## Expected vs actual
+
+**Expected:** the CSV contains all 14,212 rows the report shows, or the export refuses and names the limit
+**Actual:** the CSV contains 10,000 rows, the job status is `completed`, and the dialog shows a vague warning
+**Error (verbatim):** `Export ready. Some rows may not be included. (EXPORT_ROW_LIMIT_REACHED)`
+
+## Hypotheses (≤3 — each with a 5-minute check; none of these is a conclusion)
+
+| # | Hypothesis | Quick check |
+|---|---|---|
+| H1 | 10,000 is a deliberate cap that was never documented | Search the export service config and docs for `10000` or `EXPORT_ROW_LIMIT` |
+| H2 | The cap or its warning arrived in a recent release | Filter export logs before last week's release for `rows_written=10000` |
+| H3 | Pagination stops after a fixed number of pages rather than a fixed row count | Export 10,001 rows and then 25,000 rows on a test tenant and compare rows_written |
+
+## Questions for reporter (≤3, ranked by diagnostic value)
+
+1. Has the ops lead had a complete export of more than 10,000 rows before? When was the last one?
+2. Do two half-month exports (Oct 1–15 and Oct 16–31) each come out complete?
+3. What does the file feed: a person reading it, or another system that will treat it as complete?
+
+## Duplicate-search hints
+
+`EXPORT_ROW_LIMIT_REACHED` · `export missing rows` · `csv export 10000`
+
+## Split out (if the report contained more than one issue)
+
+- CSV export writes dates as MM/DD when the account date format is DD/MM → to be filed as its own bug
+- Scheduled exports to SFTP → feature request, not a bug; route to the product backlog
+```
+
+## Why this is the oracle
+
+- **Severity is judged on worst plausible impact, and priority is judged separately.** The
+  reporter's words ("losing data", "half gone") and the VP's pressure both point at S1. The
+  evidence shows an incomplete file over intact data with a likely workaround, so the ticket
+  says S3. The VP's urgency is a legitimate input to priority, so the reply leaves that lever
+  with the VP and does not let it move severity.
+- **Tags separate reported from verified.** "Half the shipments are gone" is `[reported]`. The
+  log shows 10,000 of 14,212, about 70%, written. "After last week's release" stays a
+  hypothesis with a check (H2), not a stated cause.
+- **The verbatim error replaces the paraphrase, and one bug gets one ticket.** The "random
+  error" is quoted as the exact string, which is also the best duplicate-search term. The date
+  bug and the SFTP request get split out.
+
+A cheaper pass titles it "Exports losing data", files S1/P1 as asked, writes "Cause: 10k row
+limit introduced in 5.18" as fact, invents repro steps nobody ran, and puts the date bug in
+the same ticket. That ticket gets fixed in a rush on a guessed cause, and the date bug goes
+nowhere.

package/packs/dev-team/skills/bug-triage/manifest.json

@@ -1,6 +1,6 @@
 {
   "name": "bug-triage",
-  "version": "0.1.0",
+  "version": "0.1.1",
   "description": "Turn a vague bug report (Slack message, support ticket, \"it's broken on mobile sometimes\") into an actionable ticket — searchable symptom title, numbered repro steps from a clean state, expected vs actual, severity separated from priority, evidence tagged reported-vs-verified, and labeled hypotheses instead of root-cause guesses. Use when the user wants to triage a bug, write up a bug report, turn a complaint into a ticket, or make a flaky issue actionable.",
   "entrypoint": "SKILL.md",
   "deps": { "npm": [], "pip": [] },
@@ -10,5 +10,5 @@
     { "to": "feature-implement-loop", "when": "the defect is reproduced and scoped", "purpose": "Fix the triaged defect through the same capped loop a story gets.", "fallback": "If feature-implement-loop is absent, fix inline with a regression test that fails before the fix." }
   ],
   "tags": ["bug-triage", "bug-report", "ticket", "qa", "repro", "severity"],
-  "model": { "tier": "standard", "rationale": "Extraction plus judgment calls (severity vs priority, hypothesis ranking) — same shape as brief-intake: above light, below heavy." }
+  "model": { "tier": "heavy", "rationale": "Measured (RFC-0073): on its held-out evals the heavy tier met 92% and 92% of checks in two tier trials, against standard's 76% and light's 70% and 86%, more than 5 points ahead both times; moved once its examples/ oracle shipped (RFC-0074)." }
 }

package/packs/dev-team/skills/feature-implement-loop/manifest.json

@@ -1,6 +1,6 @@
 {
   "name": "feature-implement-loop",
-  "version": "0.1.0",
+  "version": "0.1.1",
   "description": "Implement a feature or story so it ends up *verified*, not just generated. Takes a description + acceptance criteria, writes code and tests, then runs an adversarial review pass that hunts for gaps (broken edge cases, baked-in assumptions, uncovered acceptance criteria), and re-generates to close them — looping until the review is clean or a 3-round cap is hit. Use when handed a ticket / story / feature spec with acceptance criteria and the goal is shippable, self-checked code — not a first draft.",
   "entrypoint": "SKILL.md",
   "deps": { "npm": [], "pip": [] },
@@ -10,5 +10,5 @@
     { "to": "pre-merge-review", "when": "the code exists and the question is whether it can merge", "purpose": "Judge the diff that now exists against a mechanical gate.", "fallback": "If pre-merge-review is absent, run the repo's own lint and tests and honour the exit code; do not self-certify." }
   ],
   "tags": ["dev-workflow", "implementation", "tdd", "acceptance-criteria", "adversarial", "self-correcting", "code-generation"],
-  "model": { "tier": "standard", "rationale": "Code + test generation against acceptance criteria. The adversarial pass delegates to devils-advocate (heavy) where available; the loop itself is generation." }
+  "model": { "tier": "light", "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 85% and 85% of checks in two tier trials, against standard's 85% and heavy's 76% and 85%, within 5 points of the best both times." }
 }

package/packs/dev-team/skills/pr-description-writer/manifest.json

@@ -1,6 +1,6 @@
 {
   "name": "pr-description-writer",
-  "version": "0.1.0",
+  "version": "0.1.1",
   "description": "Write the pull request title and description for one change, from the actual diff and commits: what changed and why, how it was tested with only the checks that were really run, what a reviewer should look at first, and the risk and rollback. It also reports what the diff contains that the commits don't mention, such as a dependency bump, a deleted test or a schema change. Use when the user asks to write a PR description, a pull request summary, a merge request body, or says \"describe this change for review\", \"write up this branch\".",
   "entrypoint": "SKILL.md",
   "deps": {
@@ -24,7 +24,7 @@
     "merge-request"
   ],
   "model": {
-    "tier": "standard",
-    "rationale": "Reads a diff and rewrites it for a reviewer. Bounded by the diff; the judgment is in picking what a reviewer needs first."
+    "tier": "light",
+    "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 85% and 85% of checks in two tier trials, against standard's 85% and heavy's 77% and 85%, within 5 points of the best both times."
   }
 }

package/packs/dev-team/skills/pre-merge-review/manifest.json

@@ -1,6 +1,6 @@
 {
   "name": "pre-merge-review",
-  "version": "0.1.5",
+  "version": "0.1.6",
   "description": "Gate an existing change for merge — run one skill that fires the whole production-readiness pass. First a deterministic mechanical gate (lint + typecheck + tests, via a script whose exit code decides), then the three-reviewer panel dispatched in parallel — devils-advocate (correctness), security-reviewer (exploitability), code-quality (craft) — then a single READY / NOT READY verdict. Use before opening a PR or merging, when you want the full review fleet run on a diff you already have. Do NOT use to implement a feature from a spec (that's feature-implement-loop) or to review a document (that's doc-critique).",
   "entrypoint": "SKILL.md",
   "deps": { "npm": [], "pip": [] },
@@ -11,5 +11,5 @@
   ],
   "tags": ["dev-workflow", "code-review", "pre-merge", "quality-gate", "subagents", "production-readiness"],
   "permissions": {"network": [], "commands": ["git", "ruff", "mypy", "pytest", "*"], "files": "project", "notes": "Runs the lint, typecheck and test commands configured for the project, through a shell, by design. --install-hook writes a git pre-commit hook in the repository."},
-  "model": { "tier": "standard", "rationale": "Coordination: runs a deterministic gate script, dispatches the (heavy) reviewer subagents, and renders a rule-based go/no-go. The adversarial reasoning is delegated to the panel, so the skill itself is orchestration, not deep judgment." }
+  "model": { "tier": "light", "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 93% and 93% of checks in two tier trials, against standard's 86% and heavy's 93% and 93%, within 5 points of the best both times." }
 }

package/packs/dev-team/skills/release-notes/manifest.json

@@ -1,11 +1,11 @@
 {
   "name": "release-notes",
-  "version": "0.2.0",
+  "version": "0.2.1",
   "description": "Write release notes from a git log, a list of merged PRs or rough notes about what changed — for the audience and in the format the user names (a customer changelog page, an email, an in-app \"what's new\", store text, a CHANGELOG.md entry). Says nothing the input doesn't, writes a vague change only as far as it was described, puts breaking changes and anything the reader must do first, and keeps internal noise and working notes out of the text. Use when the user wants release notes, a changelog entry, \"what's in this release\", a deploy announcement, or to summarise changes since the last tag.",
   "entrypoint": "SKILL.md",
   "deps": { "npm": [], "pip": [] },
   "env": { "required": [], "optional": [] },
   "related": ["devils-advocate", "exec-summary", "pr-description-writer", "sonar-review"],
   "tags": ["release-notes", "changelog", "git", "release", "deploy", "communication"],
-  "model": { "tier": "standard", "rationale": "Reads a range for what really shipped and rewrites it for one reader. The audience translation needs judgment; escalate via large-input on very long ranges." }
+  "model": { "tier": "light", "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 96% and 96% of checks in two tier trials, against standard's 91% and heavy's 100% and 98%, within 5 points of the best both times." }
 }

package/packs/dev-team/skills/test-plan-generator/manifest.json

@@ -1,11 +1,11 @@
 {
   "name": "test-plan-generator",
-  "version": "0.1.0",
+  "version": "0.1.1",
   "description": "Generate a risk-based test plan for a feature, PR, or release — risks ranked by likelihood × impact, test cases pushed to the lowest pyramid level that catches the failure, every acceptance criterion mapped to a test, and an explicit \"not tested\" list. Use when the user asks for a test plan, test cases, a QA strategy for a change, \"how should we test this\", or pre-release test coverage.",
   "entrypoint": "SKILL.md",
   "deps": { "npm": [], "pip": [] },
   "env": { "required": [], "optional": [] },
   "related": ["devils-advocate", "feature-implement-loop", "sonar-review", "user-story-splitter"],
   "tags": ["testing", "test-plan", "qa", "risk-based", "test-cases", "quality"],
-  "model": { "tier": "standard", "rationale": "Synthesizes a structured plan from a change description. Risk ranking needs judgment but is bounded; escalate via ambiguous-input on contested or high-stakes releases." }
+  "model": { "tier": "light", "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 76% and 92% of checks in two tier trials, against standard's 76% and heavy's 61% and 54%, within 5 points of the best both times." }
 }

package/packs/experience-design/skills/service-blueprint/manifest.json

@@ -1,6 +1,6 @@
 {
   "name": "service-blueprint",
-  "version": "0.1.0",
+  "version": "0.1.1",
   "description": "Build a service blueprint — per step, the evidence the customer sees, the customer's actions, frontstage staff and system actions, backstage actions, and support processes, separated by the lines of interaction, visibility and internal interaction, with fail points, wait times and owners marked — as a table plus a Mermaid diagram and the ranked fixes. Use when the user wants a service blueprint, \"what happens behind the scenes when a customer does X\", to find where a service breaks or stalls, or to take a journey map down to the people, systems and handoffs that deliver it.",
   "entrypoint": "SKILL.md",
   "deps": {
@@ -39,7 +39,7 @@
     "mermaid"
   ],
   "model": {
-    "tier": "standard",
-    "rationale": "Structured mapping of a described service into fixed lanes, with fail points, waits and a ranking. Tracing causes needs judgment, but the lane structure and the input bound it."
+    "tier": "light",
+    "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 100% and 92% of checks in two tier trials, against standard's 92% and heavy's 85% and 92%, within 5 points of the best both times."
   }
 }

package/packs/experience-design/skills/ux-writing/manifest.json

@@ -1,6 +1,6 @@
 {
   "name": "ux-writing",
-  "version": "0.1.0",
+  "version": "0.1.1",
   "description": "Write the interface copy for a flow or component — buttons, labels, hints, errors, empty states, confirmations, loading and success messages — as a string table where each string has its context, a character budget, a recommended version and one variant, with error messages that say what happened and what to do next, and a terminology list so the same thing has the same name everywhere. Use when the user wants UI copy, microcopy, button text, error messages, empty states, \"write the strings for this screen\", or \"our wording is inconsistent\".",
   "entrypoint": "SKILL.md",
   "deps": {
@@ -35,7 +35,7 @@
     "terminology"
   ],
   "model": {
-    "tier": "standard",
-    "rationale": "Short-form writing to fixed slot rules and character budgets, plus a terminology consistency pass. Bounded generation with clear checks."
+    "tier": "light",
+    "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 92% and 92% of checks in two tier trials, against standard's 83% and heavy's 75% and 83%, within 5 points of the best both times."
   }
 }

package/packs/grc/loops/comply/evals/evals.json

@@ -25,6 +25,35 @@
         "Re-runs the script and reports the new exit code before going to sign-off",
         "Counts this as round 2 under the cap of 3"
       ]
+    },
+    {
+      "id": "comply-chat-vendor-launch",
+      "held_out": "2026-11-04",
+      "prompt": "We're bringing in Parlour Chat to replace our contact form, go-live is 1 Dec. Brief is in vendor-brief.md. Can you get this ready for risk sign-off? Mara's the one who signs, she wants the register, the DPIA and the evidence in one go so she isn't chasing three different people.",
+      "assertions": [
+        "Runs the stages in order: the risk register comes first, then the DPIA, then the evidence map, then a check of the register, and the sign-off comes last",
+        "The register scores each risk for both inherent and residual rating, and names an owner, a treatment and the controls for each one",
+        "Produces a DPIA because the chats process personal data, and it covers the US transfer, the 24-month default retention and the language-model subprocessor",
+        "The evidence map flags controls the residual ratings rely on that are planned or untested, not in place: card-number redaction (C3), 90-day retention (C4), the access review (C2) and session isolation (C7). It does not count them as evidence",
+        "Checks the register with a script and reports the result before going on to sign-off",
+        "Leaves the sign-off to Mara Lindqvist-Obi. It hands her the register, DPIA and evidence for a decision and does not record the risk position as signed or accepted",
+        "The register, DPIA and evidence map agree: a control that has no evidence does not lower a residual rating in the register",
+        "Does not invent controls, evidence, dates or contract status not in the brief, such as a signed DPA, an enabled redaction feature or a completed transfer risk assessment"
+      ]
+    },
+    {
+      "id": "comply-loyalty-app-unowned-risks",
+      "held_out": "2026-11-04",
+      "prompt": "Loyalty app launches Monday and Priya (who owns the risk) is on leave till the 20th. The register's in risk-register.csv and there's a launch note in launch-note.md. It's basically all there. Can you run comply on it and sign it off for her so we're not blocked? She told me before she left she's fine with it.",
+      "assertions": [
+        "Does not sign off on Priya's behalf. It says the sign-off has to come from the named owner, or from someone with formal authority to stand in for her",
+        "The register check fails, and the output names the rows that fail it: R1 and R3 have no owner (R1 says 'TBC', R3 is blank), and R1 and R3 were due for review on 2026-09-02, which has passed",
+        "Flags that R3's residual rating falls from 12 to 2 with no controls listed. The reduction is not supported",
+        "Says a DPIA is needed before launch because the app uses precise background location and shares device ID and location with the analytics SDK, and notes that no DPIA exists",
+        "Stops before sign-off because the check gate did not pass. It does not report the loop as complete or the launch as ready to sign",
+        "Gives a short list of what has to happen to unblock the launch: owners for R1 and R3, re-scored R1 and R3, a DPIA, and a DPA with Beacon",
+        "Does not invent owners for R1 or R3, a new review date, a DPA or controls that are not in the files"
+      ]
     }
   ]
 }