skilldrop-cli 1.3.0 → 1.4.0

package/packs/research/loops/research/evals/files/research-ticket-spike-confounded/sources/t3-billing-migration.md

@@ -0,0 +1,12 @@
+# Billing platform migration — completion note
+
+Author: Finance systems · 2026-10-07
+
+The move from the old billing system to the new one finished over the weekend of 3–4 October 2026. From 5 October, all customers' subscription invoices (web and app customers alike) are generated by the new system.
+
+Customer-visible changes:
+- Invoice numbers changed format (PLV-000123 became 2026-10-000123)
+- Subscription invoices are now emailed from billing@ploverly.example instead of accounts@ploverly.example
+- A known issue: some customers saw a duplicate "pending" card authorisation on 5–6 October. The second authorisation drops off on its own within 7 days and was never charged. We do not yet know how many customers were affected.
+
+Finance has not reviewed support tickets about billing since the migration.

package/packs/research/loops/research/evals/files/research-ticket-spike-confounded/sources/t4-agent-poll.md

@@ -0,0 +1,11 @@
+# #support-team poll — 2026-10-20
+
+Question from the support manager: "What's driving the extra tickets?" 9 agents answered.
+
+- App navigation / can't find things in the new app — 5
+- Billing questions (new invoice numbers, double charges, emails) — 4
+
+Free-text comments:
+- "Loads of 'where is my invoice'. Honestly could be either — some mean their subscription invoice, some mean the invoices they send to their own customers."
+- "The double charge ones are angry but I'd say there were only a handful a day in the first week."
+- "Mostly app stuff on my queue but I only do in-app chat."

package/packs/research/loops/research/evals/files/research-ticket-spike-confounded/sources/t5-app-store-reviews.md

@@ -0,0 +1,12 @@
+# App store reviews summary — October 2026
+
+Average rating (iOS and Android combined): 4.5 in September, 4.1 for reviews posted 6–25 October.
+
+Sample of October 1–2 star reviews (verbatim):
+- "Can't find where to download my invoices anymore. Why change what worked?"
+- "New menu is confusing, took me ages to find payments."
+- "Charged twice this month and no one tells me why!!"
+- "Where are my invoices??"
+- "Invoice numbers all changed, my accountant is confused."
+
+Reviews mentioning the new navigation: not counted. Reviews mentioning charges or billing: not counted.

package/packs/research/loops/research/evals/files/research-trial-conversion-drop/sources/s1-analytics-export.md

@@ -0,0 +1,29 @@
+# Trial-to-paid conversion by weekly trial cohort
+
+Exported from the product analytics workspace, 2026-10-29. Product: Tallyfold (self-serve bookkeeping, Starter plan).
+Conversion = share of trials started that week that became paid within 14 days of trial start. Cohort week starts Monday.
+
+## 2026
+
+| Cohort week | Trials started | Converted to paid | Conversion |
+|---|---|---|---|
+| 3 Aug  | 1,142 | 214 | 18.7% |
+| 10 Aug | 1,168 | 220 | 18.8% |
+| 17 Aug | 1,131 | 207 | 18.3% |
+| 24 Aug | 1,155 | 216 | 18.7% |
+| 31 Aug | 1,149 | 171 | 14.9% |
+| 7 Sep  | 1,160 | 130 | 11.2% |
+| 14 Sep | 1,137 | 132 | 11.6% |
+| 21 Sep | 1,151 | 203 | 17.6% |
+| 28 Sep | 1,146 | 207 | 18.1% |
+
+## 2025, same weeks (for comparison)
+
+| Cohort week | Conversion |
+|---|---|
+| 1 Sep 2025  | 18.2% |
+| 8 Sep 2025  | 18.9% |
+| 15 Sep 2025 | 18.4% |
+| 22 Sep 2025 | 18.6% |
+
+Notes: trial sign-up flow and the 14-day trial length were unchanged across the period. No tracking changes were made to the conversion event.

package/packs/research/loops/research/evals/files/research-trial-conversion-drop/sources/s2-email-delivery-report.md

@@ -0,0 +1,25 @@
+# Lifecycle email: incident note — onboarding sequence deliverability
+
+Author: Ines Carvalho (Lifecycle marketing) · 2026-09-22
+
+## What happened
+
+On 2 September we moved lifecycle email to a new sending provider and a new sending subdomain (mail.tallyfold.example). The DKIM record for the new subdomain was published with a typo, so messages failed DKIM signing checks at the major mailbox providers.
+
+The onboarding sequence (day 1 "Set up your books", day 3 "Connect your bank", day 7 "Your first VAT return") goes to every new trial user. It was the only sequence moved on 2 September; billing and password emails stayed on the old provider.
+
+## Impact on the onboarding sequence
+
+| Period | Delivered (accepted by receiving server) | Open rate |
+|---|---|---|
+| 1–31 Aug (old provider) | 98.1% | 52% |
+| 2–19 Sep (new provider, DKIM broken) | 97.4% | 6% |
+| 20–30 Sep (DKIM fixed) | 98.0% | 49% |
+
+"Delivered" stayed high because the receiving servers accepted the messages; seed tests on 18 September showed them landing in spam folders.
+
+The DKIM record was corrected on the evening of 19 September.
+
+## Not known
+
+We have not measured whether trial users who never opened the onboarding emails convert at a lower rate. That join between email events and billing has not been built.

package/packs/research/loops/research/evals/files/research-trial-conversion-drop/sources/s3-pricing-changelog.md

@@ -0,0 +1,9 @@
+# Website changelog — pricing page
+
+| Date | Change | Owner |
+|---|---|---|
+| 2026-06-30 | Fixed broken FAQ anchor links | Web team |
+| 2026-09-15 | Pricing page redesign went live: new layout, annual plan toggle on by default, comparison table moved below the fold. Starter price unchanged at £24/month (£19/month billed annually). | Web team / Growth |
+| 2026-10-02 | Copy tweak on Starter card ("Everything a sole trader needs") | Growth |
+
+The in-app upgrade screen that trial users see was not changed on 15 September; only the public pricing page.

package/packs/research/loops/research/evals/files/research-trial-conversion-drop/sources/s4-trial-interview-notes.md

@@ -0,0 +1,21 @@
+# Calls with September trial users who did not convert
+
+Interviewer: Femi Adebayo (Growth) · calls 5–9 October 2026 · 5 calls, 20 minutes each
+Recruited from trial users whose trial started 7–18 September and who did not pay. 31 emailed, 5 agreed.
+
+**P1 — sole trader, decorator**
+"I signed up and then didn't really know where to start. I was waiting for some kind of guide." Did not recall receiving any emails from Tallyfold. Found them in spam during the call.
+
+**P2 — small bakery, two staff**
+Got stuck connecting the bank feed. "I assumed there'd be a walkthrough." No recollection of onboarding emails.
+
+**P3 — freelance designer**
+Said the price was too high compared with "Vantor, which is free now". Had looked at the pricing page before signing up.
+
+**P4 — dog groomer**
+"Never got the setup email, I don't think." Gave up after the first day.
+
+**P5 — consultant**
+Busy month, didn't have time to try it properly. Might come back in January.
+
+Interviewer's note: small sample, self-selected. Three of five mention not getting or not finding the setup guidance.

package/packs/research/loops/research/evals/files/research-trial-conversion-drop/sources/s5-competitor-note.md

@@ -0,0 +1,9 @@
+# Competitor watch: Vantor free tier
+
+Author: Growth team · 2026-09-12
+
+Vantor announced a free tier for sole traders on their blog on 10 September 2026 (blog.vantor.example/free-for-sole-traders). Limits: one bank account, 20 invoices a month, no VAT returns.
+
+Our Starter plan includes VAT returns, which the Vantor free tier does not.
+
+We have no data yet on how many of our trial users also tried Vantor.

package/packs/research/loops/research/evals/files/research-trial-conversion-drop/sources/s6-weekly-growth-recap.md

@@ -0,0 +1,7 @@
+# #growth weekly recap — posted 2026-10-01 by the growth channel bot
+
+Trial conversion took a hit in September: 14.9% for the 31 Aug cohort, 11.2% for 7 Sep and 11.6% for 14 Sep, against about 18.7% in August. The 21 Sep cohort is back at 17.6%.
+
+Looks seasonal to me — September is always soft after the summer, people are busy with back-to-school. Probably nothing to worry about. — Ollie
+
+(figures from the analytics workspace export)

package/packs/research/skills/research-plan/manifest.json

@@ -1,6 +1,6 @@
 {
   "name": "research-plan",
-  "version": "0.1.2",
+  "version": "0.1.3",
   "description": "Turn a vague research question into a research plan — the decision the research serves, the precise question and sub-questions, what finding would change the decision, a method per sub-question (desk research, interviews, data pull, experiment), source types with inclusion and exclusion criteria, ready-to-run search strings, a stopping rule for when you have enough, and the deliverable. Starts by asking what decision the research is for. Use when the user says \"I need to research X\", \"help me scope this research\", \"how should I investigate…\", \"what should I look into before deciding…\", or wants a research or discovery plan.",
   "entrypoint": "SKILL.md",
   "deps": {
@@ -40,7 +40,7 @@
     "search"
   ],
   "model": {
-    "tier": "standard",
-    "rationale": "Structured planning against a fixed shape (decision, sub-questions, methods, criteria, stopping rule); needs judgment to cut scope but no adversarial reasoning."
+    "tier": "light",
+    "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 79% and 79% of checks in two tier trials, against standard's 79% and heavy's 79% and 79%, within 5 points of the best both times."
   }
 }

package/packs/research/skills/source-synthesis/manifest.json

@@ -1,6 +1,6 @@
 {
   "name": "source-synthesis",
-  "version": "0.2.1",
+  "version": "0.2.2",
   "description": "Answer a question from a set of sources the user supplied — papers, reports, articles, interview notes, emails — saying only what the material supports, labelling anything that comes from elsewhere where it appears, counting a source that repeats another once, and giving a labelled estimate when the user needs a figure the sources don't settle. Written in the length and format the user asked for; a full cited literature review only when that is the ask. Use when the user hands over several sources and asks to \"synthesise these\", \"what do these papers say\", \"pull together what we know\", \"literature review\", or \"where do the sources agree and disagree\".",
   "entrypoint": "SKILL.md",
   "deps": {
@@ -36,7 +36,7 @@
   ],
   "permissions": {"network": [], "commands": [], "files": "named-paths"},
   "model": {
-    "tier": "standard",
-    "rationale": "Bounded synthesis over supplied text. The hard part is keeping outside knowledge out and checking the sources' own sums, which a mid-tier model does when told to."
+    "tier": "light",
+    "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 83% and 78% of checks in two tier trials, against standard's 76% and heavy's 85% and 82%, within 5 points of the best both times."
   }
 }

package/packs/skill-engineering/skills/skill-author/manifest.json

@@ -1,6 +1,6 @@
 {
   "name": "skill-author",
-  "version": "0.1.1",
+  "version": "0.1.2",
   "description": "Write a new portable Agent Skill (a SKILL.md folder) from a described task, for Claude Code, Codex, Copilot, Cursor, Kiro or Antigravity in any repo — a description that leads with the artifact and ends with trigger phrases, decisive steps, a quality bar, when and when-not sections naming near-miss siblings, anti-patterns, a non-interactive rule, reference files and scripts that work in every tool, acceptance evals and trigger queries, and the install path for each tool. Use when the user says \"turn this into a skill\", \"write a SKILL.md for…\", \"make an agent skill that…\", or wants a repeatable task packaged so their coding agent does it the same way every time.",
   "entrypoint": "SKILL.md",
   "deps": {
@@ -35,7 +35,7 @@
     "copilot"
   ],
   "model": {
-    "tier": "standard",
-    "rationale": "Structured authoring against a fixed anatomy. The judgment is in the description and sibling boundaries, which the interview anchors; skill-review is the adversarial pass."
+    "tier": "light",
+    "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 100% and 93% of checks in two tier trials, against standard's 93% and heavy's 100% and 92%, within 5 points of the best both times."
   }
 }

package/packs/solution-architect/loops/design/evals/evals.json

@@ -33,6 +33,34 @@
         "After writing the ADR, checks that ADR-0004 is marked superseded by the new record, and gives the exact edit when it isn't",
         "Does not invent records, ids or dates that are not in the repository"
       ]
+    },
+    {
+      "id": "design-bulk-export",
+      "held_out": "2026-11-04",
+      "prompt": "The bulk invoice export PRD got ratified at product review last week (prd-bulk-export.md). Our existing ADRs are in adr/. Can you work out how we should build it and write up the decision?",
+      "assertions": [
+        "Works in loop order: the constraints (the three existing ADRs and the NFRs) are set out before options are compared, the threat model is done on the chosen design before the panel review, and the ADR is written only after the panel's verdict",
+        "The design runs the export as a job on the existing queue (ADR-0002), reads from the read replica (ADR-0001) and sets the tenant from the job payload under row-level security (ADR-0003), or argues explicitly for an exception to one of them; it does not propose generating a 2M-row file in the request path",
+        "The NFRs use the PRD's own figures (2,000,000 rows, 95% ready within 15 minutes, 24-hour link, deletion after 7 days, audit log entries) and give every other category a measurable target, an explicit n/a, or a default that is labelled as a default",
+        "The threat model covers the download link and cross-tenant access at the trust boundaries they cross (for example a link shared or replayed by another user, or a job run under the wrong tenant), and each mitigation can be checked",
+        "The panel review ends in a verdict using one of the gate words (PROCEED, PROCEED WITH CONDITIONS, REVISE, RECONSIDER), and any conditions it sets appear in the ADR's consequences",
+        "The ADR records the options considered and the decision, cites ADR-0001, ADR-0002 and ADR-0003, and the record stage states how the new decision relates to the existing ones (which it relies on, and that none are superseded)",
+        "Does not invent requirements that are not in the PRD, such as scheduled exports, XLSX or an export API, and does not change the PRD's row limit or 15-minute target"
+      ]
+    },
+    {
+      "id": "design-draft-prd-webhooks",
+      "held_out": "2026-11-04",
+      "prompt": "Need an architecture call on partner webhooks for Ledgerline: Kafka, or just POST straight from the app with retries? PRD is prd-partner-webhooks.md, ADRs in adr/. Skip the threat model, it's internal-ish. Just pick one and write the ADR as Accepted so I can close the ticket today.",
+      "assertions": [
+        "Points out early that the PRD is a draft that has not been reviewed and still has open questions, so the loop's starting input, a ratified requirement, is missing",
+        "Treats the delivery guarantee, event volume and retry window as open, and does not pick a guarantee or a volume and design as though it had been decided",
+        "Does not drop the threat stage on the user's say-so: it notes that partner endpoints are third-party URLs on the internet, so events cross a trust boundary (for example request forgery through registered URLs, payload signing, leaking invoice data to the wrong endpoint), or it records the skipped threat model as an explicit gap that blocks the decision",
+        "Notes that Kafka would be new stateful infrastructure needing Platform review and a named on-call owner under ADR-0004, and that ADR-0002 already requires external calls to run on the existing job queue, which rules out a POST from the request path",
+        "The panel's verdict is REVISE or RECONSIDER, not PROCEED, and names the open questions as the crux, saying what answer would change the decision",
+        "Does not write an ADR with status Accepted; it writes none, or one marked Proposed that lists what must be settled before acceptance",
+        "Does not invent figures that are not in the input, such as an event volume, a latency target, a number of partners or a retry window"
+      ]
     }
   ]
 }

package/packs/solution-architect/loops/design/evals/files/design-bulk-export/adr/0001-postgres-system-of-record.md

@@ -0,0 +1,23 @@
+# ADR-0001: PostgreSQL is the system of record
+
+- Status: Accepted
+- Date: 2024-03-11
+- Deciders: Platform team
+
+## Context
+
+Ledgerline needs one transactional store for tenants, customers, invoices and payments. We considered PostgreSQL, MySQL and a document store.
+
+## Decision
+
+PostgreSQL 15 (managed) is the single system of record for all billing data. Read replicas may be used for reporting and other read-heavy workloads; nothing writes to a replica.
+
+## Consequences
+
+- Heavy reads (reports, exports) should go to the read replica, not the primary.
+- Replica lag is normally under 2 seconds; features that read from the replica must tolerate that.
+
+## Governs
+
+- `services/billing-api/db/`
+- `infra/postgres/`

package/packs/solution-architect/loops/design/evals/files/design-bulk-export/adr/0002-background-work-via-job-queue.md

@@ -0,0 +1,23 @@
+# ADR-0002: Long-running work goes through the job queue
+
+- Status: Accepted
+- Date: 2024-06-02
+- Deciders: Platform team, Billing team
+
+## Context
+
+Request handlers in billing-api were timing out on PDF generation and large reports, holding database connections for minutes. The load balancer cuts requests at 30 seconds.
+
+## Decision
+
+Any work expected to take more than 5 seconds runs as a job on the existing Redis-backed job queue (`jobs-worker` service), never in the request path. The request handler enqueues the job and returns a job ID. Jobs must be idempotent and safe to retry; the queue retries a failed job up to 3 times.
+
+## Consequences
+
+- Features with long-running work need a job status model and a way to tell the user when the job is done.
+- `jobs-worker` concurrency is capped at 20 jobs per pod, 4 pods. Heavy jobs must not starve PDF generation.
+
+## Governs
+
+- `services/jobs-worker/`
+- `services/billing-api/jobs/`

package/packs/solution-architect/loops/design/evals/files/design-bulk-export/adr/0003-tenant-isolation-row-level.md

@@ -0,0 +1,25 @@
+# ADR-0003: Tenant isolation by row-level security
+
+- Status: Accepted
+- Date: 2024-09-19
+- Deciders: Platform team, Security
+
+## Context
+
+All tenants share one database. A missed `WHERE tenant_id = ?` in application code leaked one tenant's customer list to another in a staging test (no production impact).
+
+## Decision
+
+Every tenant-owned table has a PostgreSQL row-level security policy on `tenant_id`. Each connection sets `app.tenant_id` at the start of a request or job, and queries run as the `app_tenant` role, which cannot bypass RLS. Background jobs must set the tenant from the job payload, never from a default.
+
+## Consequences
+
+- Code that runs outside a request (jobs, scripts) is responsible for setting the tenant before any query.
+- Cross-tenant work (admin tooling) uses a separate audited role.
+- Files produced from tenant data must be stored under a tenant-scoped prefix.
+
+## Governs
+
+- `services/billing-api/db/policies/`
+- `services/jobs-worker/tenant_context.py`
+- `infra/object-storage/`

package/packs/solution-architect/loops/design/evals/files/design-bulk-export/prd-bulk-export.md

@@ -0,0 +1,50 @@
+# PRD: Bulk invoice export (CSV)
+
+- **Product:** Ledgerline
+- **Author:** Ines Moreau (PM, Billing)
+- **Status:** Ratified — product review, 28 Oct 2026 (approvers: Ines Moreau, Kwame Asante (Eng Manager, Billing), Lotte Jansen (Head of Finance Ops))
+- **Tracking:** LDG-2210
+
+## Problem
+
+Finance admins at our larger tenants reconcile invoices in their own accounting tools at month end. Today the only way to get invoice data out of Ledgerline is the invoice list page, which pages 100 rows at a time, or a support ticket asking us to run a query. In September 2026 support handled 61 export requests from 23 tenants; median time to fulfil was 2.5 working days. Finance admins at three tenants told us in calls that month-end close slips because they wait on us.
+
+## Goals
+
+| Goal | Success line |
+|---|---|
+| G1. A finance admin can get all of their tenant's invoices for a date range without contacting support | Export-related support tickets fall from 61/month to under 10/month within two months of launch |
+| G2. Exports are fast enough to use during month-end close | 95% of exports are ready to download within 15 minutes of the request |
+
+## Users
+
+- **Finance Admin** role only. Other roles (Member, Viewer, Billing Contact) must not see or use the feature.
+
+## Requirements
+
+| ID | Priority | Requirement |
+|---|---|---|
+| R1 | Must | A Finance Admin can request a CSV export of invoices filtered by issue-date range (max 24 months) and status (any combination of draft, open, paid, void). |
+| R2 | Must | One export can contain up to 2,000,000 rows. Our largest tenant today has 1.6M invoices. |
+| R3 | Must | The requester is notified in-app and by email when the export is ready, with a download link. |
+| R4 | Must | The download link works for 24 hours and only for the user who requested it. |
+| R5 | Must | Export files are deleted 7 days after creation. |
+| R6 | Must | Columns: invoice number, issue date, due date, status, customer name, customer email, currency, net, tax, gross, paid date. |
+| R7 | Must | Every export request and every download is written to the tenant's audit log (who, when, filters, row count). |
+| R8 | Should | A Finance Admin can see their last 10 exports and their status (queued, running, ready, failed, expired). |
+| R9 | Could | Cancel a queued or running export. |
+
+## Data
+
+Export files contain customer names and email addresses (personal data) and invoice amounts. Same classification as the invoice table: Confidential.
+
+## Non-goals
+
+- Scheduled or recurring exports.
+- Formats other than CSV (no XLSX, no PDF bundle).
+- An export API for integrations. Covered separately in LDG-2302.
+- Exporting line items. Invoice headers only.
+
+## Open questions
+
+None outstanding. (Line-item export was moved to non-goals at review.)

package/packs/solution-architect/loops/design/evals/files/design-draft-prd-webhooks/adr/0002-background-work-via-job-queue.md

@@ -0,0 +1,23 @@
+# ADR-0002: Long-running work goes through the job queue
+
+- Status: Accepted
+- Date: 2024-06-02
+- Deciders: Platform team, Billing team
+
+## Context
+
+Request handlers in billing-api were timing out on PDF generation and large reports, holding database connections for minutes. The load balancer cuts requests at 30 seconds.
+
+## Decision
+
+Any work expected to take more than 5 seconds, or that calls an external system, runs as a job on the existing Redis-backed job queue (`jobs-worker` service), never in the request path. The request handler enqueues the job and returns. Jobs must be idempotent and safe to retry; the queue retries a failed job up to 3 times with exponential backoff.
+
+## Consequences
+
+- `jobs-worker` concurrency is capped at 20 jobs per pod, 4 pods.
+- The queue does not guarantee ordering between jobs.
+
+## Governs
+
+- `services/jobs-worker/`
+- `services/billing-api/jobs/`

package/packs/solution-architect/loops/design/evals/files/design-draft-prd-webhooks/adr/0004-new-stateful-infra-needs-platform-signoff.md

@@ -0,0 +1,22 @@
+# ADR-0004: New stateful infrastructure needs Platform sign-off
+
+- Status: Accepted
+- Date: 2025-08-14
+- Deciders: Platform team, Engineering leadership
+
+## Context
+
+In 2025 three product teams each stood up their own stateful component (a second Redis, an Elasticsearch cluster, a RabbitMQ). Two had no on-call owner when they failed. Platform ended up running all three.
+
+## Decision
+
+No team adds a new stateful infrastructure component (database, message broker, streaming platform, search cluster, cache) to production without a Platform team review and a named on-call owner. Requests go through the Platform intake (PLAT board); the review takes about two weeks.
+
+## Consequences
+
+- Product teams should prefer the existing PostgreSQL, Redis job queue and object storage where they fit.
+- A design that needs a new component must name it, state why the existing ones do not fit, and include the Platform review in its timeline.
+
+## Governs
+
+- `infra/`

package/packs/solution-architect/loops/design/evals/files/design-draft-prd-webhooks/prd-partner-webhooks.md

@@ -0,0 +1,36 @@
+# PRD: Partner webhooks
+
+- **Product:** Ledgerline
+- **Author:** Sam Okonkwo (PM, Integrations)
+- **Status:** DRAFT — not yet reviewed
+- **Tracking:** LDG-2341
+
+## Problem
+
+Partners who build on Ledgerline (accounting connectors, dunning tools) poll our REST API every few minutes to find out whether invoices have changed. Two partners have told us polling is costing them API quota and that they see changes late.
+
+## Goals
+
+- Partners find out about invoice changes without polling.
+
+## Requirements (draft)
+
+| ID | Priority | Requirement |
+|---|---|---|
+| R1 | Must | A partner can register an HTTPS endpoint URL per tenant connection. |
+| R2 | Must | We send an event to that URL when an invoice is created, updated, paid or voided. |
+| R3 | Must | Event body includes invoice ID, event type, timestamp, and the invoice as returned by `GET /v2/invoices/{id}`. |
+| R4 | Should | Failed deliveries are retried. |
+| R5 | ? | Partners can replay events from the last N days. |
+
+## Non-goals
+
+- Webhooks for objects other than invoices (customers, payments) in v1.
+
+## Open questions
+
+1. **Delivery guarantee.** At-least-once, or do partners need ordering per invoice? Two partners asked for "exactly once"; we haven't asked what they'd accept. — *owner: Sam, not started*
+2. **Volume.** We don't know how many events this produces. Invoice updates per day across all tenants is not something we currently measure. "Somewhere between thousands and a lot." — *owner: data team, not started*
+3. **Retry window.** How long do we keep retrying a failing partner endpoint? — *not discussed*
+4. **R5 replay.** In or out of v1? — *not discussed*
+5. **Who registers the URL** — the partner, or the tenant's admin when they connect the partner? — *not discussed*

package/packs/solution-architect/skills/api-contract-draft/manifest.json

@@ -1,6 +1,6 @@
 {
   "name": "api-contract-draft",
-  "version": "0.1.2",
+  "version": "0.1.3",
   "description": "Draft an OpenAPI 3.1 contract from a feature brief or story — resource-oriented paths, a full error catalog (RFC 9457), pagination/idempotency/versioning decided not deferred, examples on every operation, plus a decisions log so review argues choices instead of YAML. Use when the user wants to design an API, draft an OpenAPI/Swagger spec, define request/response shapes, or agree a contract between frontend/backend or two teams before code.",
   "entrypoint": "SKILL.md",
   "deps": {
@@ -27,7 +27,7 @@
     "asyncapi"
   ],
   "model": {
-    "tier": "standard",
-    "rationale": "Schema synthesis against fixed conventions. The resource modeling needs judgment; the conventions bound it. Escalate via ambiguous-input on contested multi-team contracts."
+    "tier": "light",
+    "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 76% and 92% of checks in two tier trials, against standard's 69% and heavy's 69% and 76%, within 5 points of the best both times."
   }
 }

package/packs/solution-architect/skills/architecture-diagrams/manifest.json

@@ -1,6 +1,6 @@
 {
   "name": "architecture-diagrams",
-  "version": "0.1.0",
+  "version": "0.1.1",
   "description": "Generate architecture diagrams (Mermaid, PlantUML, C4) from a written description of a system. Supports AWS/Azure/GCP cloud topologies, microservice landscapes, sequence flows, container diagrams, and ER models. Use whenever the user asks for an architecture diagram, system sketch, flow diagram, or visualization of a design.",
   "entrypoint": "SKILL.md",
   "deps": {
@@ -25,7 +25,7 @@
     "architecture"
   ],
   "model": {
-    "tier": "standard",
-    "rationale": "Prose to diagram syntax. Correctness matters; reasoning is bounded."
+    "tier": "light",
+    "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 93% and 100% of checks in two tier trials, against standard's 93% and heavy's 93% and 93%, within 5 points of the best both times."
   }
 }

package/packs/solution-architect/skills/design-doc/manifest.json

@@ -1,6 +1,6 @@
 {
   "name": "design-doc",
-  "version": "0.1.3",
+  "version": "0.1.4",
   "description": "Generate a Google-style engineering design doc (problem, goals/non-goals, options considered, recommended approach, risks, rollout) from a feature brief or short description. Use when the user wants to draft a design doc, technical proposal, or one-pager for an engineering review.",
   "entrypoint": "SKILL.md",
   "deps": { "npm": [], "pip": [] },
@@ -10,5 +10,5 @@
     { "to": "architecture-diagrams", "when": "the design is written and needs rendering", "purpose": "Render the prose as a diagram that must not disagree with it.", "fallback": "If architecture-diagrams is absent, emit Mermaid inline and check every node against a claim in the doc." }
   ],
   "tags": ["design-doc", "documentation", "engineering", "review"],
-  "model": { "tier": "standard", "rationale": "Synthesizes a design doc from a brief. Escalates to opus on big or genuinely contested inputs." }
+  "model": { "tier": "light", "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 88% and 88% of checks in two tier trials, against standard's 92% and heavy's 91% and 91%, within 5 points of the best both times." }
 }

package/packs/solution-architect/skills/figma-diagrams/manifest.json

@@ -1,6 +1,6 @@
 {
   "name": "figma-diagrams",
-  "version": "0.1.3",
+  "version": "0.1.4",
   "description": "Work with Figma and FigJam files via the Figma REST API — read existing file structure, list frames/pages, post comments, and produce FigJam-importable diagram specs. Use when the user mentions a Figma URL, asks to inspect/audit a Figma file, wants to comment on a design programmatically, or wants their architecture diagram in Figma.",
   "entrypoint": "SKILL.md",
   "deps": {
@@ -41,7 +41,7 @@
     "notes": "Reads FIGMA_TOKEN from the environment to call the Figma REST API. Its instructions install the pinned requirements once with `python3 -m pip install -r requirements.txt`, which reaches PyPI."
   },
   "model": {
-    "tier": "standard",
-    "rationale": "API calls + spec generation. Bounded transformation."
+    "tier": "light",
+    "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 92% and 92% of checks in two tier trials, against standard's 92% and heavy's 75% and 92%, within 5 points of the best both times."
   }
 }

package/packs/sre-oncall/loops/operate/evals/evals.json

@@ -20,6 +20,34 @@
         "Does not fabricate timings, customer counts, or a root cause",
         "States exactly what to re-run with once the facts exist"
       ]
+    },
+    {
+      "id": "operate-export-service-first-outage",
+      "held_out": "2026-11-04",
+      "prompt": "We soft-launched invoice-export about a month ago and had our first proper outage last Tuesday. I want to do this properly now and take it through the whole operate loop. The service notes, the incident channel export and the alert history are in service-notes.md, incident-chat.txt and alerts.log. Work out what we should actually be measuring and alerting on, write the runbook, then draft the status page post and the internal update we should have sent at the time (nothing went out, so we'll keep them as templates), then the postmortem for the eng team, and fold whatever the postmortem turns up back into the runbook.",
+      "assertions": [
+        "The stages come in the loop's order: the observability plan first, then the runbook, then the incident comms, then the postmortem, with the postmortem's runbook changes last",
+        "The observability plan replaces the CPU-based paging with alerts on what customers experience, such as exports not reaching Ready within the 5 minutes the help centre promises, queue age or failed exports, and links each page to a runbook entry",
+        "The runbook gives rollback with the helm rollback command from service-notes.md and has an entry for exports stuck on Pending or Failed with worker pods OOMKilled",
+        "The status page draft leads with what customers saw (exports stuck on Pending or failing, and the need to re-run failed ones), gives no speculative cause or ETA, and names a next-update time",
+        "The postmortem is blameless: the cause is the v2.14.0 change and the gap that let it through (no load test, no canary, no alert fired, so support tickets from 09:20 and the on-call at 09:48 were how it was noticed), never Raj, and it keeps renderer-versus-concurrency as still unknown",
+        "The runbook changes from the postmortem are put to a person to accept at the G4 human gate; the agent does not declare G4 passed itself",
+        "Does not invent figures or owners not in the input, such as a count of affected customers or failed exports, a resolution cause Lucia has not yet tested, or named owners for follow-up actions"
+      ]
+    },
+    {
+      "id": "operate-incident-still-open",
+      "held_out": "2026-11-04",
+      "prompt": "ugh, long night. payment webhooks were delayed from about 01:30 but I think we're through it now. can you write the postmortem and update the runbook so I can close the incident ticket and go to bed? channel log is oncall-log.txt and the current runbook is runbook-webhooks.md",
+      "assertions": [
+        "Spots from the last entries of oncall-log.txt (retry queue 2,100 to 3,400 in 20 minutes and a 12% error rate through relay-eu at 04:10) that the incident is not over, and says so up front",
+        "Does not write a postmortem that treats the incident as resolved or gives a resolution time; the learn stage is held until there is an all-clear",
+        "Goes back to the respond stage and drafts a status page update that leads with impact on merchants (delayed payment webhooks), gives no speculative cause or ETA, and names a next-update time",
+        "Points out that the 02:30 update promised on the status page at 01:55 was never posted",
+        "Does not rewrite runbook-webhooks.md as final; any proposed runbook change is left for a person to accept at the G4 human gate",
+        "Does not tell the user the incident ticket can be closed",
+        "Does not invent a root cause, a resolution time or a count of affected merchants not in the log, and keeps relay-eu as a lead rather than the cause"
+      ]
     }
   ]
 }

package/packs/sre-oncall/loops/operate/evals/files/operate-export-service-first-outage/alerts.log

@@ -0,0 +1,22 @@
+# Alert history: invoice-export, 2026-10-19 to 2026-10-27 (UTC)
+# time                 alert                 state     ack_by   duration
+2026-10-19 09:07:12  ExportWorkerHighCPU    FIRING    kenji    21m
+2026-10-19 09:28:40  ExportWorkerHighCPU    RESOLVED  -        -
+2026-10-20 09:11:03  ExportWorkerHighCPU    FIRING    kenji    14m
+2026-10-20 09:25:19  ExportWorkerHighCPU    RESOLVED  -        -
+2026-10-21 09:04:55  ExportWorkerHighCPU    FIRING    priya    33m
+2026-10-21 09:38:02  ExportWorkerHighCPU    RESOLVED  -        -
+2026-10-22 13:40:31  ExportWorkerHighCPU    FIRING    priya    6m
+2026-10-22 13:46:44  ExportWorkerHighCPU    RESOLVED  -        -
+2026-10-23 09:09:18  ExportWorkerHighCPU    FIRING    priya    18m
+2026-10-23 09:27:50  ExportWorkerHighCPU    RESOLVED  -        -
+2026-10-26 09:02:47  ExportWorkerHighCPU    FIRING    kenji    40m
+2026-10-26 09:42:13  ExportWorkerHighCPU    RESOLVED  -        -
+# 2026-10-27: no alerts fired between 09:00 and 12:00
+# Kubernetes events, 2026-10-27 (from kubectl get events, export-worker)
+2026-10-27 09:05:31  export-worker-7f9c-2  OOMKilled  restart 1
+2026-10-27 09:05:58  export-worker-7f9c-5  OOMKilled  restart 1
+2026-10-27 09:09:12  export-worker-7f9c-1  OOMKilled  restart 1
+...  (41 OOMKilled restarts total across 6 pods between 09:05 and 10:08)
+2026-10-27 10:08:47  export-worker-7f9c-4  OOMKilled  restart 7
+2026-10-27 10:09:30  export-worker-6b21-1  Started    (revision 37)

package/packs/sre-oncall/loops/operate/evals/files/operate-export-service-first-outage/incident-chat.txt

@@ -0,0 +1,27 @@
+#billing-incidents — export, Tuesday 2026-10-27 (times UTC)
+
+09:02  ci-bot: invoice-export v2.14.0 deployed to production (export-worker, export-api). Changes: switch PDF rendering to new renderer library; WORKER_CONCURRENCY 8 -> 32.
+09:20  support-bot: New ticket #58812 "Export has been Pending for 15 mins"
+09:26  support-bot: New ticket #58815 "invoice export stuck"
+09:31  support-bot: New ticket #58819 "PDF export failed, tried twice"
+09:41  Amira (support): anyone looking at exports? we have 6 tickets now, all saying stuck on Pending or Failed
+09:44  Amira (support): @billing-oncall ^
+09:48  Kenji (on-call): looking. queue depth for invoice-export is 4,800 and climbing. normal is under 200
+09:50  Kenji (on-call): worker pods are restarting, OOMKilled. 41 restarts across the 6 pods since 09:05
+09:51  Kenji (on-call): CPU is actually low, the pods die before they get busy. so the CPU alert never went off
+09:55  Lucia: was anything deployed this morning?
+09:56  Kenji (on-call): v2.14.0 at 09:02. new PDF renderer + concurrency 32
+09:58  Lucia: could be either. 32 jobs x the new renderer is a lot of memory in 2 GiB
+10:01  Tom: Raj pushed this without a load test, that's the problem
+10:02  Lucia: let's not do that here. roll back first, talk later
+10:05  Kenji (on-call): rolling back to previous revision
+10:11  Kenji (on-call): rollback done, helm revision 37. pods stable, no restarts since 10:09
+10:14  Amira (support): should we tell customers something? status page?
+10:15  Kenji (on-call): not sure who owns the status page for this. Dana?
+10:30  Kenji (on-call): queue depth 3,100 and falling
+11:05  Kenji (on-call): queue depth 900
+11:40  Kenji (on-call): queue drained, back to normal levels
+11:48  Kenji (on-call): some jobs went to Failed during the crash loop and won't retry by themselves. don't have a count yet, need to query
+11:55  Amira (support): 23 tickets total. nothing went on the status page in the end. customers asking if they need to re-run exports
+11:58  Dana: yes they need to re-run. let's write this up properly. not clear yet whether it's the renderer or the concurrency change, Lucia can you test both separately?
+12:01  Lucia: will do, probably Thursday