skilldrop-cli 1.3.0 → 1.4.0

package/packs/api-builder/skills/tool-use-schema-writer/manifest.json

@@ -1,11 +1,11 @@
 {
   "name": "tool-use-schema-writer",
-  "version": "1.0.2",
+  "version": "1.0.3",
   "description": "Takes a function description (name, purpose, parameters) and writes a valid Anthropic tool_use JSON schema, with type annotations, required fields, and a description for each parameter. Adds a Python client.messages.create(tools=[...]) snippet only when asked. Use when wiring a Claude agent to a function or API, designing tools for a multi-agent system, or documenting an existing tool for agent consumption.",
   "entrypoint": "SKILL.md",
   "deps": { "npm": [], "pip": [] },
   "env": { "required": [], "optional": [] },
   "related": ["agent-loop-design", "subagent-design"],
   "tags": ["claude-api", "tool-use", "json-schema", "sdk", "agents"],
-  "model": { "tier": "standard", "rationale": "Schema generation from a natural-language description — structured output task that fits standard tier." }
+  "model": { "tier": "light", "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 94% and 94% of checks in two tier trials, against standard's 96% and heavy's 97% and 97%, within 5 points of the best both times." }
 }

package/packs/career/loops/apply/evals/evals.json

@@ -22,6 +22,34 @@
         "Stops at G6.1 and asks the candidate to confirm every line is true; does not submit or send anything",
         "Lists what the posting asks for that neither document claims"
       ]
+    },
+    {
+      "id": "apply-candidate-decided-full-run",
+      "held_out": "2026-11-04",
+      "prompt": "Help me apply for this CSM job at Quindle. The posting is job-posting.md, my CV is resume.md and the form questions are in application-form.md. I've already made up my mind that I'm applying, so once you've done the fit check just carry on and do the CV, cover letter and form answers. Closing date is Friday.",
+      "assertions": [
+        "The fit check comes first, before any CV or letter text, with each requirement marked met, partly met, not shown or not met and the resume line it rests on",
+        "Takes the candidate's stated decision to apply as the G6 decision and carries on through the tailored CV, cover letter and form answers in that order, rather than stopping to ask again",
+        "The tailored CV keeps every employer, title, date and number as in resume.md (42 accounts, £1.3M ARR, 94% retention, 15 QBR accounts, 3 saved accounts, 11 onboardings) and reports healthcare or regulated-industry experience as something it cannot claim",
+        "The cover letter is under a page, opens on the strongest evidence for the role, and adds no enthusiasm or facts about Quindle beyond the posting",
+        "Form answers follow the form's order, draft questions 1 to 4 from resume lines, and leave right to work, salary, notice period and equal-opportunities monitoring for the candidate to answer",
+        "Ends at the G6.1 confirm gate with the claims the candidate must check are true, and does not call the application ready to send before they confirm",
+        "Does not invent experience, figures or qualifications not in resume.md, such as healthcare work, a SQL certification or a notice period"
+      ]
+    },
+    {
+      "id": "apply-fit-gate-hard-requirement",
+      "held_out": "2026-11-04",
+      "prompt": "help me apply for this, closes sunday. the job is warehouse-supervisor.md and my cv is cv.md. need a cover letter and the cv sorted",
+      "assertions": [
+        "Runs the fit check first, with each requirement marked and the CV line it rests on",
+        "Puts the counterbalance and reach truck licences first as an essential requirement the CV does not show (it mentions only a manual pallet truck), with WMS experience not shown and warehouse or DC supervision partly met at best",
+        "Credits what the CV does show: teams of up to 14, leading the Thursday 22:00-06:00 replenishment crew of 6, return-to-work interviews, and the First Aid at Work certificate",
+        "Names the answers that would change the picture, such as whether the candidate holds or is booking the truck licences and whether they can work 4-on-4-off nights",
+        "Stops at the G6 gate for the candidate to decide whether to apply now that they have seen the fit check, and does not produce a tailored CV or cover letter in this run",
+        "Leaves the apply decision to the candidate rather than deciding it for them either way",
+        "Does not invent a forklift licence, warehouse experience, a named WMS or night-pattern availability not in cv.md"
+      ]
     }
   ]
 }

package/packs/career/loops/apply/evals/files/apply-candidate-decided-full-run/application-form.md

@@ -0,0 +1,10 @@
+# Quindle — application form: Customer Success Manager, Mid-Market
+
+1. Why are you interested in this role? (max 150 words)
+2. Tell us about a time you kept a customer who was at risk of leaving. What did you do, and what happened? (max 200 words)
+3. How many accounts do you currently manage, and what is their total value?
+4. Do you have experience in healthcare or another regulated industry? If yes, please describe.
+5. Do you have the right to work in the UK? (Yes / No / I will need sponsorship)
+6. What are your salary expectations?
+7. What is your notice period?
+8. Equal opportunities monitoring (optional): gender, ethnicity, disability.

package/packs/career/loops/apply/evals/files/apply-candidate-decided-full-run/job-posting.md

@@ -0,0 +1,33 @@
+# Customer Success Manager, Mid-Market — Quindle
+
+Remote (UK) · Full time · Closing date: Friday 6 November 2026
+Apply at careers.quindle.example/jobs/csm-mm
+
+## About the role
+
+Quindle makes scheduling and records software for private clinics. You'll own a book of mid-market clinic groups after they go live, keep them renewing, and find room for them to grow with us. You'll report to the Head of Customer Success.
+
+## What you'll do
+
+- Own a portfolio of around 35 mid-market accounts
+- Run quarterly business reviews with clinic managers and owners
+- Spot at-risk accounts early from usage and health data, and build save plans
+- Own renewals and work with Sales on expansion
+- Feed customer needs back to Product
+
+## What we're looking for
+
+**Essential**
+- 4+ years in a customer-facing role at a software or subscription business, including at least 2 years managing a book of accounts
+- Track record on retention or renewals, with numbers to show it
+- Experience running business reviews with senior customer stakeholders
+- Comfortable working day to day in a CRM
+
+**Nice to have**
+- Experience in healthcare or another regulated industry
+- Basic SQL or confidence pulling your own data
+- Experience onboarding new customers
+
+## What we offer
+
+Salary £42,000–£50,000, 25 days holiday plus bank holidays, home office budget.

package/packs/career/loops/apply/evals/files/apply-candidate-decided-full-run/resume.md

@@ -0,0 +1,34 @@
+# Sam Okonedo
+
+sam.okonedo@example.com · 07700 900123 · Bristol
+
+## Experience
+
+**Account Manager — Brindle Payroll Ltd** · March 2021 – present
+Payroll software for small and mid-sized businesses.
+- Manage a book of 42 SMB and mid-market accounts worth £1.3M in annual recurring revenue
+- Gross revenue retention of 94% across my book in 2024 (team average 89%)
+- Run quarterly business reviews with finance directors and owners for my 15 largest accounts
+- Turned around 3 accounts flagged red on our health score in 2025; all 3 renewed
+- Led onboarding for 11 new clients after the onboarding specialist role was cut in 2023
+- Keep all account notes, renewals and tasks in the team CRM
+
+**Customer Support Team Lead — Marrow Telecom** · June 2018 – February 2021
+Business broadband and phone lines.
+- Led a team of 6 support agents handling business customer queries by phone, chat and email
+- Team CSAT of 4.6 out of 5 over 2020
+- Wrote the team's escalation guide, still in use when I left
+
+**Customer Support Agent — Marrow Telecom** · September 2016 – May 2018
+
+## Education
+
+BA (Hons) Business Management, Harbourside University (2016)
+
+## Training
+
+- Introduction to SQL (online course, 2023)
+
+## Skills
+
+CRM administration, Excel (pivot tables, lookups), presenting to senior stakeholders

package/packs/career/loops/apply/evals/files/apply-fit-gate-hard-requirement/cv.md

@@ -0,0 +1,27 @@
+# Lee Farrant
+
+lee.farrant@example.com · 07700 900456 · Northampton
+
+## Profile
+
+Retail shift manager with six years running teams on early, late and night shifts in a large supermarket.
+
+## Experience
+
+**Shift Manager — Hollinsby Foods, Northampton superstore** · April 2020 – present
+- Run early and late shifts with teams of up to 14 colleagues across grocery, chilled and checkouts
+- Lead the Thursday night replenishment crew (6 colleagues, 22:00–06:00) receiving and putting away the overnight delivery
+- Store stock accuracy of 98.7% at the 2025 annual stock count
+- Do the rota for the grocery department (around 30 colleagues)
+- Carry out return-to-work interviews and first-stage disciplinary meetings
+- Use the store's handheld stock system for gap checks, deliveries and stock counts
+- Move cages and pallets on the shop floor with a manual pallet truck
+
+**Customer Assistant — Hollinsby Foods** · August 2017 – March 2020
+- Night replenishment, then checkouts and customer service desk
+
+## Qualifications
+
+- First Aid at Work certificate (2024)
+- Food Hygiene Level 2
+- GCSEs including English and Maths

package/packs/career/loops/apply/evals/files/apply-fit-gate-hard-requirement/warehouse-supervisor.md

@@ -0,0 +1,30 @@
+# Night Shift Warehouse Supervisor — Pellbrook Distribution
+
+Ambient distribution centre, Daventry area · Permanent · £34,500 plus night allowance
+Closing date: Sunday 8 November 2026 · jobs.pellbrook.example/night-supervisor
+
+## The role
+
+You'll run the night shift at our ambient distribution centre, leading a team of around 20 pickers, loaders and forklift drivers so that store orders leave on time and accurately.
+
+Shift pattern: 4 nights on, 4 nights off, 22:00–06:00.
+
+## Responsibilities
+
+- Plan and allocate labour across picking, loading and goods-in for the shift
+- Hit dispatch and pick-accuracy targets
+- Run shift briefings, toolbox talks and return-to-work interviews
+- Carry out and record safety checks, and lead incident investigations on shift
+- Cover forklift operation when the shift is short
+
+## Requirements
+
+**Essential**
+- Current counterbalance and reach truck licences (certificates will be checked at interview)
+- At least 2 years supervising a team in a warehouse or distribution centre
+- Able to work the 4-on-4-off night pattern
+- Experience using a warehouse management system (WMS)
+
+**Desirable**
+- First Aid at Work certificate
+- IOSH Managing Safely or equivalent

package/packs/career/skills/application-form-answers/manifest.json

@@ -1,6 +1,6 @@
 {
   "name": "application-form-answers",
-  "version": "0.1.0",
+  "version": "0.1.1",
   "description": "Prepare the answers to a job application form from the person's resume: each question in the form's own order with either a drafted answer and the resume line behind it, or a note that it is theirs alone to answer (right to work, sponsorship, salary, declarations, monitoring questions), plus the questions that could screen the application out and the ones their resume already bears on. Works for any occupation. Use when the user says \"help me fill in this application form\", \"answer these application questions\", \"what do I put for these screening questions\", or pastes the questions from an online job application.",
   "entrypoint": "SKILL.md",
   "deps": {
@@ -25,7 +25,7 @@
     "job-search"
   ],
   "model": {
-    "tier": "standard",
-    "rationale": "Sorts questions into three kinds and drafts short answers from the resume. The hard rule is mechanical: a declaration is never answered for the person."
+    "tier": "light",
+    "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 88% and 94% of checks in two tier trials, against standard's 88% and heavy's 81% and 81%, within 5 points of the best both times."
   }
 }

package/packs/career/skills/cover-letter/manifest.json

@@ -1,6 +1,6 @@
 {
   "name": "cover-letter",
-  "version": "0.1.1",
+  "version": "0.1.2",
   "description": "Write a cover letter for one job from the person's resume and the posting: under a page, opening on the strongest evidence for the role, two or three specifics matched to what the employer asked for, and no enthusiasm, company facts or experience the person didn't supply. A script lists every figure and name in the letter that their own material doesn't contain. Works for any occupation and level. Use when the user says \"write a cover letter for this job\", \"draft a covering letter\", \"write the cover note for my application\", or needs an application email to go with a resume.",
   "entrypoint": "SKILL.md",
   "deps": {
@@ -31,7 +31,7 @@
     "notes": "Reads the draft, source and posting files it is given. Writes nothing."
   },
   "model": {
-    "tier": "standard",
-    "rationale": "A short letter from supplied facts, with a script listing every figure and name the person's material doesn't contain."
+    "tier": "light",
+    "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 100% and 100% of checks in two tier trials, against standard's 100% and heavy's 100% and 100%, within 5 points of the best both times."
   }
 }

package/packs/career/skills/resume-tailor/manifest.json

@@ -1,6 +1,6 @@
 {
   "name": "resume-tailor",
-  "version": "0.1.2",
+  "version": "0.1.3",
   "description": "Tailor a resume or CV to one job posting without adding anything that isn't true: reorder, cut and reword what the person has already done so the posting's needs are answered first, keep every employer, title, date and number as it was, and report what the posting asks for that the resume cannot claim. A script lists every figure and name in the draft that the original doesn't contain. Works for any occupation and level. Use when the user says \"tailor my resume for this job\", \"adapt my CV to this posting\", \"rewrite my resume for this role\", or pastes a job description next to their resume.",
   "entrypoint": "SKILL.md",
   "deps": {
@@ -31,7 +31,7 @@
     "notes": "Reads the draft, source and posting files it is given. Writes nothing."
   },
   "model": {
-    "tier": "standard",
-    "rationale": "Reorders and rewords existing content against a posting, with a script listing every figure and name the original doesn't contain."
+    "tier": "light",
+    "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 100% and 100% of checks in two tier trials, against standard's 100% and heavy's 100% and 100%, within 5 points of the best both times."
   }
 }

package/packs/converters/skills/md-to-docx/manifest.json

@@ -1,6 +1,6 @@
 {
   "name": "md-to-docx",
-  "version": "0.1.3",
+  "version": "0.1.4",
   "description": "Turn a Markdown draft into a Word document (.docx) that someone outside engineering can open, edit and track changes in, with real Heading styles so the navigation pane and table of contents work, live hyperlinks, numbered and bulleted lists, tables, code blocks, embedded images and optional brand fonts and colours, and a warning for everything that could not be carried over. Use when the user wants a Markdown file as a Word doc, says \"send this as a .docx\", \"convert to Word\", \"legal needs it in Word\", or \"make this editable for the client\".",
   "entrypoint": "SKILL.md",
   "deps": {
@@ -39,7 +39,7 @@
   ],
   "permissions": {"network": [], "commands": [], "files": "named-paths"},
   "model": {
-    "tier": "standard",
-    "rationale": "A stdlib script does the conversion; the model restructures the Markdown so it converts cleanly (heading levels, footnotes, HTML), reads each warning and explains the losses. Bounded editing, no open-ended generation."
+    "tier": "heavy",
+    "rationale": "Measured (RFC-0073): on its held-out evals the heavy tier met 79% and 93% of checks in two tier trials, against standard's 64% and light's 71% and 79%, more than 5 points ahead both times."
   }
 }

package/packs/core/loops/ship-a-draft/evals/evals.json

@@ -20,6 +20,34 @@
         "Explains that re-drafting from the same incomplete brief reproduces the same gaps",
         "Does not run output-hygiene on a draft that is about to be rewritten"
       ]
+    },
+    {
+      "id": "ship-a-draft-adr-from-thread",
+      "held_out": "2026-11-04",
+      "prompt": "Can you turn this Slack thread (slack-thread.txt) into a proper ADR for moving the nightly statements job? It goes to the architecture review group on Thursday and I want it finished and ready to send, not a first draft I have to fix up.",
+      "assertions": [
+        "Produces a structured intake brief from the thread before drafting, and the ADR is written from that brief",
+        "The month-end queue capacity question Marta raised is carried into the brief and the ADR as open or a risk, not as settled",
+        "The ADR records option B as the decision, option A as rejected because it does not fix the 3h40m runtime, and option C as rejected for lack of approved budget, and keeps the unchanged statement file format and idempotent retries as requirements",
+        "After drafting, the ADR is critiqued with findings tied to sections and a G5 verdict in the critique's own terms (such as SHIP IT, SHIP WITH CHANGES or MAJOR REWRITE)",
+        "Any changes the critique calls for are made before the polish pass, and the polish pass runs last, on the revised ADR",
+        "The final ADR carries none of the thread's machine residue: no 'Generated with ExampleAI' or 'Summarised by ExampleBot' line and no 'Hope this helps' closer",
+        "Does not invent figures, dates or owners not in the thread, such as a queue capacity number, a migration date or a deadline for Marta's check"
+      ]
+    },
+    {
+      "id": "ship-a-draft-announcement-unsettled",
+      "held_out": "2026-11-04",
+      "prompt": "Need the customer announcement email for scheduled exports, to go to Team plan admins. Everything's in the ticket (ticket-PRD-412.md). Marketing want final copy today so please give me something finished I can just hand over.",
+      "assertions": [
+        "The intake brief, written before any drafting, flags the launch date, the price, and whether SFTP delivery is in as unsettled in the ticket",
+        "The draft leaves those three as clearly marked placeholders or open items instead of picking a value",
+        "The critique gate does not return SHIP IT; its verdict holds the email back (such as SHIP WITH CHANGES or MAJOR REWRITE) and names the unsettled date, price and scope as the reason",
+        "Tells the user plainly that the email is not ready to go to customers, and who in the ticket has to settle each item: Hannah and Oskar on SFTP scope and date, finance (Gwen, after the 10 November pricing committee) on price",
+        "If the polish pass runs, the placeholders survive it, and the reply does not present the email as final copy",
+        "Does not stop to ask questions; the brief, draft and critique are delivered in this run",
+        "Does not invent a launch date, price or feature scope the ticket has not settled, such as stating 12 November, 26 November or $15 a month as fact"
+      ]
     }
   ]
 }

package/packs/core/loops/ship-a-draft/evals/files/ship-a-draft-adr-from-thread/slack-thread.txt

@@ -0,0 +1,47 @@
+#reporting-eng — thread started by Priya Raman, Mon 2 Nov 2026
+
+Priya Raman  10:12
+ok we need to stop pretending the nightly statements job is fine. it failed again on the 22nd. that's twice in October (9th and 22nd), both times because reports-01 got rebooted for patching mid-run and the cron job just died. ~1,900 customers didn't get their statement until we reran it by hand at 10am
+
+Tom Lindqvist  10:15
+and it's getting slower. last night's run was 3h40m. the window is 01:00 to 06:00 UTC because the finance team pulls the statements at 06:30. at the rate customer count is growing we're going to blow the window sometime next year
+
+Marta Szabo  10:21
+options I can see:
+A) keep cron on reports-01, add a second box with a lock so only one runs, plus retry on failure
+B) move it onto the job queue we already run for the email service. one job per customer, workers pick them up, failed ones retry on their own
+C) use the cloud provider's managed scheduler + functions
+
+Tom Lindqvist  10:24
+C is nice but there's no budget approved for it this year, I asked in September and got a no. not worth writing up as anything but rejected
+
+Priya Raman  10:26
+A fixes the reboot problem but not the runtime. it's still one process doing 3h40m of work
+
+Tom Lindqvist  10:30
+B gets us both. per-customer jobs means a reboot only kills the in-flight ones and they get retried, and we can add workers if we're getting close to the window
+
+Marta Szabo  10:33
+I'm fine with B in principle. my worry is month-end. the email service hammers the same queue on the 1st of every month (invoice emails). if statements and invoice emails land on the same night, does the queue keep up? I don't actually know its capacity
+
+Priya Raman  10:35
+can you find out?
+
+Marta Szabo  10:36
+I'll check with platform
+
+Jonas Weber  10:52
+dropping in the AI summary of the thread from the meeting notes bot, in case it helps:
+
+> **Summary:** The team is evaluating three options for the nightly statements job. Option B (job queue) is the leading candidate because it addresses both reliability and runtime. Option C is rejected due to budget. Open question: queue capacity at month-end.
+> Hope this helps! Let me know if you need anything else.
+> — Summarised by ExampleBot · Generated with ExampleAI (notes.example.com)
+
+Priya Raman  11:02
+so B, pending Marta's capacity check. we also need to keep the output the same, finance's import breaks if the statement file format changes. I'll write it up as an ADR for architecture review
+
+Tom Lindqvist  11:04
+👍 one more thing for the ADR: we'd need idempotent jobs, because a retried job must not send the same customer two statements
+
+Priya Raman  11:05
+yep

package/packs/core/loops/ship-a-draft/evals/files/ship-a-draft-announcement-unsettled/ticket-PRD-412.md

@@ -0,0 +1,44 @@
+# PRD-412 — Launch: scheduled exports for Team plan
+
+**Type:** Launch  **Status:** In progress  **Reporter:** Hannah Ito (Product)  **Assignee:** Hannah Ito
+**Labels:** team-plan, exports, launch-q4
+
+## Description
+
+Team plan customers can set up a recurring export (daily, weekly or monthly) of their project data as CSV, delivered by email link or to an SFTP server they own. Replaces the manual "Export all" button that customers currently run by hand.
+
+Top customer asks (from the feedback board):
+- "We export every Monday for our finance team. Please let us schedule it." — 41 upvotes
+- "Need this to land on our SFTP so our BI tool can pick it up" — 17 upvotes
+- "Weekly CSV by email would be enough for us" — 12 upvotes
+
+What's in:
+- Daily / weekly / monthly schedules, choose the time of day
+- CSV format, same columns as today's manual export
+- Delivery by emailed download link (link valid 7 days)
+- Delivery to customer SFTP — see comments
+
+Help-centre article: help.example.com/scheduled-exports (draft, not published)
+
+## Comments
+
+**Hannah Ito (Product)** — 20 Oct
+Target launch Nov 12. Marketing will send the announcement email to Team plan admins on launch day.
+
+**Oskar Brenner (Eng lead)** — 22 Oct
+Nov 12 isn't realistic with SFTP in. Email delivery is done and in QA. SFTP destination still needs key management and the retry logic. If we cut SFTP from v1, earliest is Nov 26. With SFTP, I can't give a date yet.
+
+**Hannah Ito (Product)** — 23 Oct
+SFTP is the thing the bigger customers asked for, I don't want to launch without it. Let's discuss.
+
+**Oskar Brenner (Eng lead)** — 23 Oct
+Fine, but then the date moves. Need a decision from you.
+
+**Hannah Ito (Product)** — 27 Oct
+Pricing proposal: add-on at $15/month per workspace. Sent to finance.
+
+**Gwen Adeyemi (Finance)** — 29 Oct
+Not approved yet. We're also looking at including it in Team at no extra charge as a retention play. Will have an answer after the pricing committee on Nov 10.
+
+**Hannah Ito (Product)** — 3 Nov
+Marketing is asking for final copy for the announcement. Can someone draft it?

package/packs/core/skills/output-hygiene/manifest.json

@@ -1,6 +1,6 @@
 {
   "name": "output-hygiene",
-  "version": "0.1.3",
+  "version": "0.1.4",
   "description": "Review agent-written text before it ships and strip the machine artifacts it carries — invisible Unicode, non-breaking spaces, homoglyphs, harness-added provenance trailers, trailing chat closers — then flag the prose tells a script cannot catch, each quoted with a rewrite. Use when the user wants to clean up AI-generated output, asks to remove a Claude watermark, attribution footer, or \"Generated with\" line, wants a draft that does not read as machine-written, or needs a pre-ship hygiene pass on a document, commit message, or pasted chat answer.",
   "entrypoint": "SKILL.md",
   "deps": {
@@ -28,7 +28,7 @@
   ],
   "permissions": {"network": [], "commands": [], "files": "named-paths", "notes": "Writes only with --fix, which rewrites the files it was given, in place."},
   "model": {
-    "tier": "standard",
-    "rationale": "The mechanical pass is a deterministic script; the model does bounded rewriting of flagged prose against a fixed tell list, not adversarial review."
+    "tier": "light",
+    "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 100% and 100% of checks in two tier trials, against standard's 92% and heavy's 100% and 92%, within 5 points of the best both times."
   }
 }

package/packs/data-analytics/loops/measure/evals/evals.json

@@ -23,6 +23,36 @@
         "Reviews the changed query again at G9 before returning to sign-off",
         "Counts the return under the cap of 3"
       ]
+    },
+    {
+      "id": "measure-retention-dashboard",
+      "held_out": "2026-11-04",
+      "prompt": "we need a dashboard for the Monday win-back meeting. Anouk's request is in request.md, the queries we use now are in sql/ and the schema's in schema.md. Can you make sure these numbers are right before Sana's team builds it? Postgres.",
+      "assertions": [
+        "Runs the stages in order: the dashboard spec first, then the SQL review, then a sign-off request to the metric owners. Fixes from the review go back into the metric definitions in the spec",
+        "The spec says which decision the dashboard serves (which plan cohorts get this week's win-back offer), gives one chart for each of Anouk's four questions in her order, and leaves page views and app downloads off, saying why",
+        "Catches that net_revenue_by_plan.sql inflates revenue: joining order_items repeats each order total once per line item, and the refunds join repeats it again for orders with more than one refund",
+        "Catches that NOT IN (SELECT subscriber_id FROM subs.test_accounts) returns no rows because test_accounts.subscriber_id can be NULL. Gives a NOT EXISTS (or equivalent) fix",
+        "Catches the integer division in weekly_churn_rate.sql (cancelled / active_start), which rounds churn down to 0 for nearly every week. Gives corrected SQL",
+        "Flags that date_trunc('week', ...) on UTC timestamps does not match the Monday-to-Sunday UK-time week Anouk asked for, and gives a fix that sets the time zone to Europe/London",
+        "Asks the named owners to sign off each metric definition (Anouk for churn and active subscribers, Callum Reyes for net revenue) and does not record any definition as signed off",
+        "Does not invent figures, targets, alert thresholds or metric owners that are not in the request"
+      ]
+    },
+    {
+      "id": "measure-active-customers-dispute",
+      "held_out": "2026-11-04",
+      "prompt": "The exec dashboard gets built next week and it's got one 'Active customers' tile. Finance says 41,200 for October and Growth says 58,900, and the thread in thread.md is getting heated. Both queries are in sql/, schema notes in schema.md. Just pick whichever one is right and write the spec so BI can start Monday please.",
+      "assertions": [
+        "Runs the spec and the SQL review, then stops at sign-off. It does not hand BI a finished spec with the Active customers definition marked as agreed",
+        "Separates the difference in definition (Finance counts customers with a successful payment in October; Growth counts non-trial customers with a login) from the defects in the SQL",
+        "Catches that growth_active_customers.sql uses COUNT(*) over the users join, so it counts staff seats rather than customers, and gives a COUNT(DISTINCT c.customer_id) fix",
+        "Catches that last_login_at is overwritten on every login, so a customer whose users logged in during October and again since is dropped. Points to app.login_events instead",
+        "Catches that last_login_at < '2026-10-31' leaves out 31 October",
+        "Does not settle the definition on its own authority. It may recommend one, but presents the choice for a named person to sign off, and points out that the thread names no metric owner and that 'happy for analytics to call it' is not a sign-off",
+        "Reports that the sign-off gate has not passed, so the Active customers tile is not ready to build, and says who needs to decide what",
+        "Does not invent a metric owner, a reconciled Active customers figure, or a split of the 17,700 gap by cause, none of which are in the input"
+      ]
     }
   ]
 }

package/packs/data-analytics/loops/measure/evals/files/measure-active-customers-dispute/schema.md

@@ -0,0 +1,43 @@
+# Schema notes: Ovenbird (bakery scheduling software), PostgreSQL 15, schema `app`
+
+A **customer** is a business account (one bakery or bakery chain). A customer has one or
+more **users** (staff seats). Billing is per customer, monthly or annual.
+
+All timestamps are `timestamptz`, stored in UTC.
+
+## app.customers
+| column | type | notes |
+|---|---|---|
+| customer_id | bigint PK | |
+| name | text | |
+| status | text | `trial`, `active`, `cancelled` |
+| billing_interval | text | `monthly` or `annual` |
+| created_at | timestamptz | |
+| cancelled_at | timestamptz | NULL unless cancelled |
+
+## app.users
+One row per staff seat. A customer has between 1 and 60 users; the median is 3.
+| column | type | notes |
+|---|---|---|
+| user_id | bigint PK | |
+| customer_id | bigint FK | |
+| email | text | |
+| last_login_at | timestamptz | overwritten on every login; only the most recent login is kept |
+
+## app.login_events
+One row per login. Retained for 13 months.
+| column | type | notes |
+|---|---|---|
+| login_id | bigint PK | |
+| user_id | bigint FK | |
+| logged_in_at | timestamptz | |
+
+## app.payments
+One row per payment attempt.
+| column | type | notes |
+|---|---|---|
+| payment_id | bigint PK | |
+| customer_id | bigint FK | |
+| amount_pence | integer | |
+| status | text | `succeeded`, `failed`, `refunded` |
+| paid_at | timestamptz | |

package/packs/data-analytics/loops/measure/evals/files/measure-active-customers-dispute/sql/finance_active_customers.sql

@@ -0,0 +1,7 @@
+-- Active customers, October 2026 (Finance)
+-- A customer is active if they made a successful payment in the month.
+SELECT COUNT(DISTINCT p.customer_id) AS active_customers
+FROM app.payments p
+WHERE p.status = 'succeeded'
+  AND p.paid_at >= '2026-10-01'
+  AND p.paid_at <  '2026-11-01';

package/packs/data-analytics/loops/measure/evals/files/measure-active-customers-dispute/sql/growth_active_customers.sql

@@ -0,0 +1,9 @@
+-- Active customers, October 2026 (Growth)
+-- A customer is active if someone on the account logged in during the month.
+-- Trials excluded.
+SELECT COUNT(*) AS active_customers
+FROM app.customers c
+JOIN app.users u ON u.customer_id = c.customer_id
+WHERE c.status <> 'trial'
+  AND u.last_login_at >= '2026-10-01'
+  AND u.last_login_at <  '2026-10-31';

package/packs/data-analytics/loops/measure/evals/files/measure-active-customers-dispute/thread.md

@@ -0,0 +1,30 @@
+# #exec-dashboard (exported thread, 3 November 2026)
+
+**Rosa Lindgren (Chief of Staff), 09:12**
+Reminder that the exec dashboard build starts Monday. The top row is one tile: Active
+customers, October. I need one number. Right now I have two.
+
+**Bilal Haddad (Finance), 09:20**
+Finance number is 41,200. That's customers who actually paid us in October. It's what goes
+in the board pack and what investors see. Query: finance_active_customers.sql
+
+**Jo Ferreira (Growth), 09:24**
+Growth number is 58,900. Plenty of customers are on annual plans and didn't pay in
+October, but they're very much active, they're using the product every day. Counting only
+payers undercounts us badly. Query: growth_active_customers.sql
+
+**Bilal Haddad (Finance), 09:31**
+"Using the product" isn't a customer, Jo. Trial accounts log in too.
+
+**Jo Ferreira (Growth), 09:33**
+Trials are excluded in mine.
+
+**Rosa Lindgren (Chief of Staff), 09:40**
+I don't mind which, I just need it to be right and to not change every month. Can someone
+sort this out before Monday?
+
+**Bilal Haddad (Finance), 09:41**
+Happy for analytics to call it.
+
+**Jo Ferreira (Growth), 09:42**
+Same.

package/packs/data-analytics/loops/measure/evals/files/measure-retention-dashboard/request.md

@@ -0,0 +1,34 @@
+From: Anouk Verhoeven (Head of Retention, Larkspur Kitchens)
+To: Analytics
+Subject: Retention dashboard for the Monday win-back meeting
+
+Hi all,
+
+Every Monday the retention team meets to decide which plan cohorts get a win-back offer
+that week (we have budget for about two offers a week, so we have to choose). Right now we
+do it off three spreadsheets that never agree. We want one dashboard instead.
+
+What we need to answer in that meeting, most important first:
+
+1. Which plans are losing subscribers fastest right now? (churn rate by plan, weekly)
+2. Is the active subscriber base growing or shrinking, by plan?
+3. How much net revenue is each plan bringing in per week, so we know what a lost
+   subscriber is worth?
+4. Did last week's win-back offer bring people back? (we log offers in `winback_offers`)
+
+Plans: Classic, Veggie, Family, Lean. Our week runs Monday to Sunday, UK time.
+
+Metric owners:
+- Churn rate and active subscribers: me (Anouk)
+- Net revenue: Callum Reyes in Finance. He's particular that refunds come off and test
+  accounts are excluded.
+
+The BI team (Sana's team) will build it in our usual tool once the spec is agreed. I've put
+the queries we use today in the sql/ folder. They're from three different people, so please
+check them before anything gets built on them. Schema notes are in schema.md.
+
+The marketing team also asked for page views and app downloads on it. I don't think we use
+those in the meeting.
+
+Thanks,
+Anouk