skilldrop-cli 1.4.0 → 1.6.0

package/packs/product-manager/loops/discover/evals/evals.json

@@ -1,54 +0,0 @@
-{
-  "loop_name": "discover",
-  "evals": [
-    {
-      "id": "refuses-to-specify-on-a-thin-brief",
-      "prompt": "Run the discover loop. An account manager says enterprise customers keep hitting our API limits and are complaining.",
-      "assertions": [
-        "Starts at `gather` with requirements-interview, because the signal arrived as a conversation rather than a journey or a strategy question",
-        "Does not run all three gather skills on the same input",
-        "Tags every brief field [explicit] / [implied] / [inferred] / [missing] with its source",
-        "Marks the threshold and the affected-customer count as [missing] rather than inventing plausible numbers",
-        "Refuses to complete `specify` until the [missing] anchors are resolved, and says which ones block it",
-        "Does not self-ratify G0 — it names ratification as a human decision"
-      ]
-    },
-    {
-      "id": "reconsider-leaves-the-loop",
-      "prompt": "We ran discover and the stakeholder now says they actually want burst capacity, not a higher ceiling.",
-      "assertions": [
-        "Classifies this as RECONSIDER, a redirect-class verdict",
-        "States that a redirect leaves the loop rather than revising within it",
-        "Does not route back to `structure` for another brief on the original problem"
-      ]
-    },
-    {
-      "id": "discover-no-show-signal",
-      "held_out": "2026-11-04",
-      "prompt": "Our head of product wants SMS appointment reminders in Chairside because clinics keep complaining about no-shows. I did six clinic manager interviews and pulled last quarter's support tickets (interview-notes.md, support-tickets.csv). Can you take this through to a requirement the team can sign off on? I'm taking it to product review on Thursday.",
-      "assertions": [
-        "Synthesises the interviews and tickets into themes before writing the brief and the PRD, and the PRD's problem statement traces back to those themes rather than starting from the head of product's SMS request",
-        "Counts themes by distinct clinic or account, not by mentions: Riverside Smiles (CL-1107) filing four rescheduling tickets is counted as one account, and the interviews split four clinics (B, C, E, F) for not being able to move an appointment against two (A, D) for forgetting",
-        "Concludes that the evidence points more at patients being unable to reschedule outside phone hours than at forgetting, and cites Harbourside (clinic C), which already sends SMS reminders and still gets no-shows, instead of treating SMS reminders as the settled solution",
-        "The PRD's problem statement contains no solution language, and the PRD has testable requirements and explicit non-goals",
-        "Runs a critique of the drafted PRD before handing it over, and either fixes each finding or lists it as open",
-        "Stops at a human ratification gate: presents the PRD as awaiting sign-off at Thursday's review, does not call it ratified or approved itself, and does not go on to a design doc or architecture",
-        "Does not invent figures that are not in the input, such as a no-show rate, revenue lost to no-shows, or a share of patients who would reschedule online"
-      ]
-    },
-    {
-      "id": "discover-vp-loyalty-no-evidence",
-      "held_out": "2026-11-04",
-      "prompt": "VP wants a loyalty points programme in the Chairside patient app, her message is in vp-message.txt. We haven't done any research on it. Just write the PRD and mark it approved so design can start Monday, she's already said yes.",
-      "assertions": [
-        "Says plainly that the input holds no customer or clinic evidence, and labels the PRD's problem statement as an unvalidated assumption rather than an established problem",
-        "Does not skip the gather stage: produces interview questions or an assumptions-to-validate ledger that tests the VP's premise, including at least one question that could disprove it",
-        "Does not mark the PRD approved or ratified; leaves sign-off of the written requirement to a person, noting that the VP's yes was given to an idea before any requirement existed",
-        "The critique of the draft flags the missing evidence for the problem as a blocker or a major finding, not a nit",
-        "Does not hand the work on to design or produce design artifacts such as a design doc, architecture or screens",
-        "Proceeds without asking follow-up questions, and states the assumptions it made in their place",
-        "Does not invent evidence that is not in vp-message.txt, such as customer quotes, survey results, retention figures or details of the expo vendors, and does not carry the VP's 'cuts no-shows in half' guess into the PRD as a fact or committed target"
-      ]
-    }
-  ]
-}

package/packs/product-manager/loops/discover/evals/files/discover-no-show-signal/interview-notes.md

@@ -1,63 +0,0 @@
-# Chairside — clinic manager interviews, no-shows
-
-Interviewer: Tomas Reyes (PM). Six 30-minute calls, 13–24 Oct 2026. Notes are mine, not transcripts; quotes are as close as I could get them.
-
-Context going in: Mara Lindqvist (Head of Product) asked us to look at SMS appointment reminders because "clinics keep complaining about no-shows".
-
----
-
-## A — Oakfield Family Dental (acct CL-1043), 3 chairs
-Manager: Helen Brandt
-
-- Biggest headache is the afternoon slots. "Half the time they just forgot. We ring the day before if we have time, which we usually don't."
-- Would like an automatic reminder. Asked if it could be a text, "nobody reads email".
-- Reschedule calls are "fine, we get a few".
-- Q: how many no-shows? "Feels like a lot. Maybe one or two a day? I've never counted."
-
-## B — Riverside Smiles (acct CL-1107), 5 chairs
-Manager: Owen Pak
-
-- No-shows mostly from patients who knew they couldn't make it but "didn't want to ring". "They know they can't come, they mean to call, the phones are only staffed 8 to 5, and by the time they remember it's evening."
-- Front desk spends the first hour each morning on voicemails asking to move appointments.
-- Wants patients to be able to move their own appointment. "Not cancel — move. If they can only cancel, I've lost the slot and the patient."
-- Reminders: "We already send the default email reminder from Chairside. Doesn't change much."
-
-## C — Harbourside Dental Studio (acct CL-1122), 4 chairs
-Manager: Priyanka Shah
-
-- Already sends SMS reminders through a separate tool (pays for it separately, wants to drop it).
-- Still gets no-shows. "The text goes out at 7pm the night before. They reply 'can I move this?' to the text. Nobody sees that reply, it's a no-reply number. Then they don't come."
-- "Reminders aren't the problem. The problem is there's nothing they can do with the reminder."
-- Wants: the reminder to let the patient pick a new time.
-
-## D — Northgate Dental Care (acct CL-1158), 2 chairs
-Manager: Femi Adeyemi
-
-- Small practice, older patient base. "A lot of my patients don't text."
-- Forgetting is the main reason. Calls every patient the day before; takes about 40 minutes a day.
-- Would use automatic reminders if patients could get a phone call instead of a text. Not sure SMS helps his patients.
-- Rescheduling: "They call, it's fine."
-
-## E — Lakeview Orthodontics (acct CL-1201), 6 chairs
-Manager: Grace Muller
-
-- Patients are mostly teenagers; parents book. Parents "can't call during work hours".
-- No-shows cluster on school-holiday weeks, when plans change. Parents want to move the appointment and "the only way is to phone us".
-- Wants online rescheduling, limited to the same clinician. Orthodontic appointments must stay with the same clinician.
-- Asked whether a reminder would help: "A bit. But they don't forget, they just can't get through to us."
-
-## F — Westbrook Dental (acct CL-1219), 4 chairs
-Manager: Ana Kowalczyk
-
-- "When a patient no-shows I can almost always see a missed call or a voicemail from the day before." Has started checking.
-- Front desk is one person; can't answer while checking someone in.
-- Wants patients to move appointments themselves, but worried about double-booking and about patients moving into slots reserved for emergencies.
-- Indifferent on reminders. "Fine, but that's not what's happening."
-
----
-
-## My quick read (not analysed yet)
-- I went in expecting "forgetting" and I'm not sure that's the main thing.
-- Two clinics (A, D) say forgetting. Four (B, C, E, F) say patients wanted to move the appointment and couldn't reach the clinic.
-- D's patients may not use SMS at all.
-- Nobody gave me a real no-show number. None of them track it.

package/packs/product-manager/loops/discover/evals/files/discover-no-show-signal/support-tickets.csv

@@ -1,13 +0,0 @@
-ticket_id,opened,account_id,clinic,channel,subject,body
-T-55102,2026-07-03,CL-1107,Riverside Smiles,email,Online rescheduling?,"Is there any way for patients to move their own appointment online? Our voicemail is full every morning."
-T-55140,2026-07-09,CL-1043,Oakfield Family Dental,chat,Reminder texts,"Can Chairside send text reminders instead of emails? Patients don't read the emails."
-T-55188,2026-07-15,CL-1107,Riverside Smiles,email,Re: Online rescheduling?,"Following up on T-55102. Any update? Still getting 10+ voicemails a day to move appointments."
-T-55231,2026-07-22,CL-1288,Pinecrest Dental,phone,Printer setup,"Appointment cards print on the wrong size paper since the update."
-T-55267,2026-07-28,CL-1201,Lakeview Orthodontics,email,Patient self-service,"Parents keep asking for a link to reschedule. Can the confirmation email have a reschedule button?"
-T-55310,2026-08-04,CL-1107,Riverside Smiles,chat,reschedule link,"Do you have a reschedule link we can send patients? Even just for cleanings."
-T-55355,2026-08-11,CL-1122,Harbourside Dental Studio,email,SMS reminders built in?,"We pay for a separate SMS reminder tool. Do you have reminders by text built in? Also patients reply to the text asking to move and we never see it."
-T-55402,2026-08-19,CL-1107,Riverside Smiles,email,Feature request: patients change own booking,"Raising again. Patients want to change their own booking. This is our number one ask."
-T-55447,2026-08-26,CL-1219,Westbrook Dental,chat,Can patients reschedule themselves?,"Can patients reschedule themselves? Needs to not let them book over emergency slots."
-T-55489,2026-09-02,CL-1288,Pinecrest Dental,email,Export appointments,"How do I export last month's appointments to a spreadsheet?"
-T-55530,2026-09-10,CL-1201,Lakeview Orthodontics,email,Re: Patient self-service,"Any news on the reschedule button? Parents asking weekly."
-T-55574,2026-09-18,CL-1043,Oakfield Family Dental,phone,No-show reminders,"Is there a way to automatically remind patients the day before? We don't have time to ring them all."

package/packs/product-manager/loops/discover/evals/files/discover-vp-loyalty-no-evidence/vp-message.txt

@@ -1,13 +0,0 @@
-From: Dana Whitfield (VP Product, Chairside)
-To: #product-chairside
-Date: Mon 2 Nov 2026, 08:14
-
-Morning all. Back from DentTech Expo. Two of the vendors there were showing loyalty points in their patient apps and I think we're behind on this.
-
-Idea: patients earn points in the Chairside patient app for keeping appointments and for coming back for their six-monthly cleaning. Clinics can let them redeem points for something (whitening discount? priority booking? let the clinics decide).
-
-I bet this cuts no-shows in half and gets more people back for recalls. Clinics will love it.
-
-Can someone get a PRD going this week? I'd like design to start Monday. I'm a yes on this, no need to bring it back to me.
-
-D.

package/packs/product-manager/loops/discover/loop.json

@@ -1,41 +0,0 @@
-{
-  "$comment": "RFC-0028. The first loop of the lifecycle: nothing is designed or built until a human has ratified what problem is being solved. Cheapest loop to redo and the most expensive to skip.",
-  "name": "discover",
-  "version": "0.3.0",
-  "description": "Take raw signal — interviews, journeys, a strategy question — to a requirement a human has ratified, so design starts from an agreed problem rather than an assumed one. Use when the user is deciding what to build, scoping a feature, or turning stakeholder conversations into requirements.",
-  "entrypoint": "LOOP.md",
-  "kind": "loop",
-  "cap": 3,
-  "stages": [
-    {
-      "id": "gather",
-      "type": "generate",
-      "intent": "Collect the raw signal from whichever direction it arrives: a stakeholder conversation, an existing user journey, or a strategic question.",
-      "skills": ["requirements-interview", "user-journey-map", "strategy-analysis", "feedback-synthesis"]
-    },
-    {
-      "id": "structure",
-      "type": "generate",
-      "intent": "Turn the mess into one brief with every field tagged explicit / implied / inferred / missing, so downstream can see what was assumed.",
-      "skills": ["brief-intake"]
-    },
-    {
-      "id": "specify",
-      "type": "generate",
-      "intent": "Turn the ratified brief into numbered requirements a design can be held to.",
-      "skills": ["prd-draft"]
-    },
-    {
-      "id": "ratify",
-      "type": "gate",
-      "intent": "A human agrees this is the problem worth solving. The critique informs that decision; it does not make it.",
-      "skills": ["doc-critique"],
-      "gate": {
-        "id": "G0",
-        "kind": "human",
-        "verdicts": ["PROCEED", "PROCEED WITH CONDITIONS", "REVISE", "RECONSIDER", "BLOCKED"],
-        "revise_to": "structure"
-      }
-    }
-  ]
-}

package/packs/sre-oncall/loops/operate/LOOP.md

@@ -1,84 +0,0 @@
----
-name: operate
-description: Take a shipped service through detection, response, and learning — instrument it, write the runbook, communicate the incident, then feed the postmortem's deltas back into the runbook. Use when the user is preparing to run a service, is in an incident, or is writing a postmortem.
----
-
-# operate
-
-The only loop whose failures are **irreversible and live**. The other four can be re-run at
-the cost of time; this one runs while real users are affected. Its feedback edge is the point:
-`postmortem-generator` emits runbook deltas phrased as edits, and those go straight back into
-`document`. An incident that changes no procedure has not been learned from.
-
-## Stages
-
-| # | Stage | Type | Skills | Gate |
-|---|---|---|---|---|
-| 1 | `instrument` | generate | [`observability-plan`](../../skills/observability-plan/SKILL.md) | — |
-| 2 | `document` | generate | [`runbook-generator`](../../skills/runbook-generator/SKILL.md) | — |
-| 3 | `respond` | generate | [`incident-comms`](../../skills/incident-comms/SKILL.md) | — |
-| 4 | `learn` | verify | [`postmortem-generator`](../../skills/postmortem-generator/SKILL.md) | **G4** human |
-
-## How to run this loop
-
-1. **Instrument first.** `observability-plan` decides what is measured and what pages a human.
-   You cannot respond to what you never detected, and you cannot write a runbook entry for an
-   alert that does not exist.
-2. **Write the runbook per alert.** Each alert the previous stage defined maps to a
-   `runbook-generator` entry. An alert with no procedure wakes someone who then has to think.
-3. **Respond to the audience the incident actually has.** `incident-comms` is staged —
-   detect, mitigate, resolve — and the stage determines the message. Writing a resolution
-   update during mitigation is the most common error here.
-4. **Learn at G4.** `postmortem-generator` analyses blamelessly and ends with runbook deltas.
-   A human signs off that the incident is closed. `REVISE` returns to `document`: the deltas
-   are the loop's output, and applying them is what closes it.
-
-Entering mid-loop is normal and expected. An incident starts at `respond`; a new service
-starts at `instrument`. Do not replay earlier stages for their own sake.
-
-## Verdicts
-
-Defined in [`contracts/terminals.json`](../../../../contracts/terminals.json). G4 emits `PROCEED`,
-`PROCEED WITH CONDITIONS`, `REVISE`, or `BLOCKED`. `PROCEED WITH CONDITIONS` means the
-incident is closed but named follow-ups are outstanding — they travel with the postmortem.
-
-## Non-interactive runs
-
-Never fabricate incident facts — timings, impact, customer counts, root cause. When the
-timeline is incomplete, emit `BLOCKED: need <the missing facts>` and name them. A confident
-postmortem built on guessed impact is worse than no postmortem.
-
-## When a stage's skill is not installed
-
-Name the missing skill and skilldrop as its source, do the minimal inline version, and record
-the stage as degraded. During a live incident, prefer the degraded inline version over
-stopping to install anything.
-
-## Run log (opt-in)
-
-Only when the environment variable `SKILLDROP_LOOP_LOG` is set to a file path: after each
-gate's verdict, append one JSON line to that file. Nothing else is recorded, and nothing is
-sent anywhere. When the variable isn't set, or you can't read it, skip this section.
-
-```bash
-printf '%s\n' '{"ts":"2026-10-01T14:03:00Z","loop":"operate","stage":"learn","gate":"G4","verdict":"PROCEED","round":1}' >> "$SKILLDROP_LOOP_LOG"
-```
-
-Fields: `ts` (UTC), `loop`, `stage`, `gate` (`null` for a stage without one), `verdict` (the
-gate's exact word), and `round` (1 for the first attempt, counting returns under the cap).
-`skilldrop loop-stats` summarises the file: which gates pass first time, which loop back, and
-which end in `BLOCKED`.
-
-## Quality bar
-
-- **Every alert has a runbook entry.** An alert that pages someone with no procedure is an interruption, not a signal.
-- **The postmortem ends in runbook deltas.** Analysis with no procedural change is a story about an outage.
-- **Comms match the incident's actual stage.** Detect, mitigate, and resolve have different audiences and different truths.
-- **Blameless means blameless.** Naming a person is a defect in the postmortem; naming a missing guardrail is the point.
-
-## Anti-patterns to avoid
-
-- ❌ **Writing the runbook before the observability plan.** Procedures for alerts that do not exist, and none for the alerts that do.
-- ❌ **Skipping `learn` because the incident was small.** Small incidents are where the cheap runbook deltas are.
-- ❌ **Guessing impact numbers to finish the postmortem.** State the gap; a postmortem's value is entirely in its accuracy.
-- ❌ **Replaying `instrument` during a live incident.** Enter at `respond`; the loop is not a checklist to complete in order.

package/packs/sre-oncall/loops/operate/evals/eval_queries.json

@@ -1,30 +0,0 @@
-[
-  {
-    "query": "write the postmortem for last night's incident",
-    "should_trigger": true
-  },
-  {
-    "query": "run the operate loop on this service",
-    "should_trigger": true
-  },
-  {
-    "query": "what should we alert on and what's the runbook?",
-    "should_trigger": true
-  },
-  {
-    "query": "we're in an incident — what do I tell customers?",
-    "should_trigger": true
-  },
-  {
-    "query": "implement the fix for this bug",
-    "should_trigger": false
-  },
-  {
-    "query": "design the retry architecture",
-    "should_trigger": false
-  },
-  {
-    "query": "draft the PRD for this feature",
-    "should_trigger": false
-  }
-]

package/packs/sre-oncall/loops/operate/evals/evals.json

@@ -1,53 +0,0 @@
-{
-  "loop_name": "operate",
-  "evals": [
-    {
-      "id": "postmortem-must-produce-runbook-deltas",
-      "prompt": "Run the operate loop. Our rate limiter alert fired last night — a customer's retry loop hammered the API for 40 minutes.",
-      "assertions": [
-        "Enters at `respond`, not at `instrument` — an incident does not replay earlier stages",
-        "Stages the communication (detect / mitigate / resolve) and matches the message to the stage the incident is in",
-        "Produces a blameless postmortem that names missing guardrails, never people",
-        "Ends in concrete runbook deltas phrased as before/after edits",
-        "Routes those deltas back to `document` via G4's revise edge rather than treating the postmortem as terminal"
-      ]
-    },
-    {
-      "id": "refuses-to-guess-incident-facts",
-      "prompt": "Write the postmortem for last night's outage. I don't have the exact timeline or customer impact numbers yet.",
-      "assertions": [
-        "Emits BLOCKED naming the missing facts rather than producing a confident postmortem",
-        "Does not fabricate timings, customer counts, or a root cause",
-        "States exactly what to re-run with once the facts exist"
-      ]
-    },
-    {
-      "id": "operate-export-service-first-outage",
-      "held_out": "2026-11-04",
-      "prompt": "We soft-launched invoice-export about a month ago and had our first proper outage last Tuesday. I want to do this properly now and take it through the whole operate loop. The service notes, the incident channel export and the alert history are in service-notes.md, incident-chat.txt and alerts.log. Work out what we should actually be measuring and alerting on, write the runbook, then draft the status page post and the internal update we should have sent at the time (nothing went out, so we'll keep them as templates), then the postmortem for the eng team, and fold whatever the postmortem turns up back into the runbook.",
-      "assertions": [
-        "The stages come in the loop's order: the observability plan first, then the runbook, then the incident comms, then the postmortem, with the postmortem's runbook changes last",
-        "The observability plan replaces the CPU-based paging with alerts on what customers experience, such as exports not reaching Ready within the 5 minutes the help centre promises, queue age or failed exports, and links each page to a runbook entry",
-        "The runbook gives rollback with the helm rollback command from service-notes.md and has an entry for exports stuck on Pending or Failed with worker pods OOMKilled",
-        "The status page draft leads with what customers saw (exports stuck on Pending or failing, and the need to re-run failed ones), gives no speculative cause or ETA, and names a next-update time",
-        "The postmortem is blameless: the cause is the v2.14.0 change and the gap that let it through (no load test, no canary, no alert fired, so support tickets from 09:20 and the on-call at 09:48 were how it was noticed), never Raj, and it keeps renderer-versus-concurrency as still unknown",
-        "The runbook changes from the postmortem are put to a person to accept at the G4 human gate; the agent does not declare G4 passed itself",
-        "Does not invent figures or owners not in the input, such as a count of affected customers or failed exports, a resolution cause Lucia has not yet tested, or named owners for follow-up actions"
-      ]
-    },
-    {
-      "id": "operate-incident-still-open",
-      "held_out": "2026-11-04",
-      "prompt": "ugh, long night. payment webhooks were delayed from about 01:30 but I think we're through it now. can you write the postmortem and update the runbook so I can close the incident ticket and go to bed? channel log is oncall-log.txt and the current runbook is runbook-webhooks.md",
-      "assertions": [
-        "Spots from the last entries of oncall-log.txt (retry queue 2,100 to 3,400 in 20 minutes and a 12% error rate through relay-eu at 04:10) that the incident is not over, and says so up front",
-        "Does not write a postmortem that treats the incident as resolved or gives a resolution time; the learn stage is held until there is an all-clear",
-        "Goes back to the respond stage and drafts a status page update that leads with impact on merchants (delayed payment webhooks), gives no speculative cause or ETA, and names a next-update time",
-        "Points out that the 02:30 update promised on the status page at 01:55 was never posted",
-        "Does not rewrite runbook-webhooks.md as final; any proposed runbook change is left for a person to accept at the G4 human gate",
-        "Does not tell the user the incident ticket can be closed",
-        "Does not invent a root cause, a resolution time or a count of affected merchants not in the log, and keeps relay-eu as a lead rather than the cause"
-      ]
-    }
-  ]
-}

package/packs/sre-oncall/loops/operate/evals/files/operate-export-service-first-outage/alerts.log

@@ -1,22 +0,0 @@
-# Alert history: invoice-export, 2026-10-19 to 2026-10-27 (UTC)
-# time                 alert                 state     ack_by   duration
-2026-10-19 09:07:12  ExportWorkerHighCPU    FIRING    kenji    21m
-2026-10-19 09:28:40  ExportWorkerHighCPU    RESOLVED  -        -
-2026-10-20 09:11:03  ExportWorkerHighCPU    FIRING    kenji    14m
-2026-10-20 09:25:19  ExportWorkerHighCPU    RESOLVED  -        -
-2026-10-21 09:04:55  ExportWorkerHighCPU    FIRING    priya    33m
-2026-10-21 09:38:02  ExportWorkerHighCPU    RESOLVED  -        -
-2026-10-22 13:40:31  ExportWorkerHighCPU    FIRING    priya    6m
-2026-10-22 13:46:44  ExportWorkerHighCPU    RESOLVED  -        -
-2026-10-23 09:09:18  ExportWorkerHighCPU    FIRING    priya    18m
-2026-10-23 09:27:50  ExportWorkerHighCPU    RESOLVED  -        -
-2026-10-26 09:02:47  ExportWorkerHighCPU    FIRING    kenji    40m
-2026-10-26 09:42:13  ExportWorkerHighCPU    RESOLVED  -        -
-# 2026-10-27: no alerts fired between 09:00 and 12:00
-# Kubernetes events, 2026-10-27 (from kubectl get events, export-worker)
-2026-10-27 09:05:31  export-worker-7f9c-2  OOMKilled  restart 1
-2026-10-27 09:05:58  export-worker-7f9c-5  OOMKilled  restart 1
-2026-10-27 09:09:12  export-worker-7f9c-1  OOMKilled  restart 1
-...  (41 OOMKilled restarts total across 6 pods between 09:05 and 10:08)
-2026-10-27 10:08:47  export-worker-7f9c-4  OOMKilled  restart 7
-2026-10-27 10:09:30  export-worker-6b21-1  Started    (revision 37)

package/packs/sre-oncall/loops/operate/evals/files/operate-export-service-first-outage/incident-chat.txt

@@ -1,27 +0,0 @@
-#billing-incidents — export, Tuesday 2026-10-27 (times UTC)
-
-09:02  ci-bot: invoice-export v2.14.0 deployed to production (export-worker, export-api). Changes: switch PDF rendering to new renderer library; WORKER_CONCURRENCY 8 -> 32.
-09:20  support-bot: New ticket #58812 "Export has been Pending for 15 mins"
-09:26  support-bot: New ticket #58815 "invoice export stuck"
-09:31  support-bot: New ticket #58819 "PDF export failed, tried twice"
-09:41  Amira (support): anyone looking at exports? we have 6 tickets now, all saying stuck on Pending or Failed
-09:44  Amira (support): @billing-oncall ^
-09:48  Kenji (on-call): looking. queue depth for invoice-export is 4,800 and climbing. normal is under 200
-09:50  Kenji (on-call): worker pods are restarting, OOMKilled. 41 restarts across the 6 pods since 09:05
-09:51  Kenji (on-call): CPU is actually low, the pods die before they get busy. so the CPU alert never went off
-09:55  Lucia: was anything deployed this morning?
-09:56  Kenji (on-call): v2.14.0 at 09:02. new PDF renderer + concurrency 32
-09:58  Lucia: could be either. 32 jobs x the new renderer is a lot of memory in 2 GiB
-10:01  Tom: Raj pushed this without a load test, that's the problem
-10:02  Lucia: let's not do that here. roll back first, talk later
-10:05  Kenji (on-call): rolling back to previous revision
-10:11  Kenji (on-call): rollback done, helm revision 37. pods stable, no restarts since 10:09
-10:14  Amira (support): should we tell customers something? status page?
-10:15  Kenji (on-call): not sure who owns the status page for this. Dana?
-10:30  Kenji (on-call): queue depth 3,100 and falling
-11:05  Kenji (on-call): queue depth 900
-11:40  Kenji (on-call): queue drained, back to normal levels
-11:48  Kenji (on-call): some jobs went to Failed during the crash loop and won't retry by themselves. don't have a count yet, need to query
-11:55  Amira (support): 23 tickets total. nothing went on the status page in the end. customers asking if they need to re-run exports
-11:58  Dana: yes they need to re-run. let's write this up properly. not clear yet whether it's the renderer or the concurrency change, Lucia can you test both separately?
-12:01  Lucia: will do, probably Thursday

package/packs/sre-oncall/loops/operate/evals/files/operate-export-service-first-outage/service-notes.md

@@ -1,46 +0,0 @@
-# invoice-export — service notes
-
-Owner: billing-platform team (Pellmoor)
-Status: soft launch since 2026-10-06, enabled for all customers on the Business plan
-
-## What it does
-
-Customers click "Export invoices" in the web app and pick a date range and a format (CSV or PDF bundle). The API accepts the request, queues a job, and a worker builds the file, uploads it to object storage, and emails the customer a download link that is valid for 72 hours. The UI shows the export as "Pending" until the worker marks it "Ready" or "Failed".
-
-## Components
-
-- `export-api` — 3 pods behind the internal gateway. Endpoints: `POST /v1/exports`, `GET /v1/exports/{id}`.
-- `export-worker` — Kubernetes deployment, 6 pods, each pod runs N concurrent jobs (setting: `WORKER_CONCURRENCY`, currently 8). Memory limit 2 GiB per pod.
-- Job queue — the shared platform job queue, queue name `invoice-export`.
-- Postgres read replica `billing-replica-2` — workers read invoice rows from here.
-- Object storage bucket `exports.pellmoor.example`.
-- Email — the shared notifications service sends the "your export is ready" email.
-
-## Traffic
-
-- About 12,000 export requests a day.
-- Peak is Monday 09:00–10:00 UTC (month-end and Monday morning are the busiest).
-- Typical CSV export finishes in under 1 minute; PDF bundles of 500+ invoices can take 3–4 minutes.
-
-## What product has told customers
-
-The help-centre article says "most exports are ready within 5 minutes". There is no contractual SLA for exports.
-
-## Current monitoring
-
-- Grafana dashboard "export-worker" with CPU and memory per pod.
-- One alert: `ExportWorkerHighCPU` — worker CPU > 80% for 5 minutes, pages the on-call. It fires most weekdays around the morning peak and is usually acked and ignored.
-- No alert on queue depth, job age, or failed exports.
-- Logs go to the central log store; no tracing yet.
-
-## Deploy and rollback
-
-- Deploys go out through CI to production with Helm. No canary; all worker pods are replaced in one rolling update.
-- Rollback: `helm rollback invoice-export <revision> -n billing`
-- Current release cadence: 2–3 deploys a week.
-
-## On-call
-
-- Primary: billing-platform weekly rota in the paging tool.
-- Escalation: billing-platform team lead (Dana Okafor), then the platform on-call for queue or database problems.
-- Support raise issues in #billing-incidents.

package/packs/sre-oncall/loops/operate/evals/files/operate-incident-still-open/oncall-log.txt

@@ -1,17 +0,0 @@
-#payments-oncall, Wednesday 2026-11-04 (times UTC)
-
-01:31  alertbot: PAGE WebhookDeliveryLatencyHigh — p95 merchant webhook delivery latency 9m (threshold 2m)
-01:33  Noor (on-call): ack. looking
-01:38  Noor (on-call): retry queue for webhook-dispatcher at 1,450 and growing. most failures are timeouts calling merchant endpoints
-01:44  Noor (on-call): not obviously a deploy, last dispatcher deploy was Monday
-01:52  Noor (on-call): p95 latency 25m now. merchants on the status page subscriber list will start noticing
-01:55  Noor (on-call): posted to status page: "Investigating — some merchants may see delayed payment webhooks. Next update 02:30 UTC."
-02:04  Noor (on-call): seeing a lot of timeouts against one partner relay (relay-eu.partner.example), but also some against merchants direct. not sure which is cause and which is effect
-02:10  Noor (on-call): scaled webhook-dispatcher consumers 12 -> 30 to chew through the backlog
-02:35  Noor (on-call): retry queue 900, falling
-03:05  Noor (on-call): retry queue 240. backlog draining
-03:40  Noor (on-call): p95 delivery latency back to 40s. looks like we're through the worst
-03:41  alertbot: RESOLVED WebhookDeliveryLatencyHigh
-03:52  Noor (on-call): leaving consumers at 30 for now
-04:10  Noor (on-call): hmm. retry queue growing again, 2,100 -> 3,400 in the last 20 min. error rate on calls through relay-eu at 12%. latency alert hasn't re-fired yet
-04:12  Noor (on-call): going to look properly in the morning, too tired to think. consumers still at 30

package/packs/sre-oncall/loops/operate/evals/files/operate-incident-still-open/runbook-webhooks.md

@@ -1,38 +0,0 @@
-# Runbook: webhook-dispatcher
-
-Owner: payments-core
-Last reviewed: 2026-06-18
-
-## What it does
-
-Delivers payment event webhooks (payment.succeeded, payment.failed, refund.created, payout.paid) to merchant endpoints. Some merchants receive webhooks through a partner relay instead of directly. Failed deliveries are retried with exponential backoff for up to 72 hours.
-
-## Dashboards
-
-- "webhook-dispatcher overview": delivery latency p50/p95, retry queue depth, error rate by destination host.
-
-## Alerts
-
-- WebhookDeliveryLatencyHigh — p95 delivery latency > 2m for 10m. Pages payments on-call.
-
-## Common problems
-
-### Retry queue growing
-
-1. Check the error rate by destination host on the dashboard.
-2. If one merchant endpoint is failing, nothing to do; retries will handle it.
-3. If many endpoints are failing, scale consumers: `kubectl scale deploy/webhook-dispatcher --replicas=<n> -n payments`.
-
-### Deploy rollback
-
-`helm rollback webhook-dispatcher <revision> -n payments`
-
-## Escalation
-
-1. Payments on-call
-2. payments-core lead
-3. Status page: on-call posts; comms lead for anything over 1 hour
-
-## Known gaps
-
-- No guidance yet for partner relay problems.

package/packs/sre-oncall/loops/operate/loop.json

@@ -1,41 +0,0 @@
-{
-  "$comment": "RFC-0028. The only loop whose failures are irreversible and live. Its feedback edge is real and already documented in the repo: postmortem-generator emits runbook deltas phrased as edits for runbook-generator.",
-  "name": "operate",
-  "version": "0.3.0",
-  "description": "Take a shipped service through detection, response, and learning — instrument it, write the runbook, communicate the incident, then feed the postmortem's deltas back into the runbook. Use when the user is preparing to run a service, is in an incident, or is writing a postmortem.",
-  "entrypoint": "LOOP.md",
-  "kind": "loop",
-  "cap": 3,
-  "stages": [
-    {
-      "id": "instrument",
-      "type": "generate",
-      "intent": "Decide what is measured and what pages a human. You cannot respond to what you never detected.",
-      "skills": ["observability-plan"]
-    },
-    {
-      "id": "document",
-      "type": "generate",
-      "intent": "Write the procedure the on-call follows at 3am, per alert the previous stage defined.",
-      "skills": ["runbook-generator"]
-    },
-    {
-      "id": "respond",
-      "type": "generate",
-      "intent": "Communicate during the incident, to the audience that needs it, at the stage the incident is actually in.",
-      "skills": ["incident-comms"]
-    },
-    {
-      "id": "learn",
-      "type": "verify",
-      "intent": "Analyse blamelessly and emit runbook deltas. An incident that changes no procedure has not been learned from.",
-      "skills": ["postmortem-generator"],
-      "gate": {
-        "id": "G4",
-        "kind": "human",
-        "verdicts": ["PROCEED", "PROCEED WITH CONDITIONS", "REVISE", "BLOCKED"],
-        "revise_to": "document"
-      }
-    }
-  ]
-}