skilldrop-cli 1.3.0 → 1.5.0

package/packs/ai-engineering/loops/adopt/evals/evals.json

@@ -1,69 +0,0 @@
-{
-  "loop_name": "adopt",
-  "evals": [
-    {
-      "id": "slice-first-then-readiness",
-      "prompt": "Run the adopt loop for our 40-person customer operations team at Larkspur Freight. Ideas from last week's workshop: (1) draft first replies to shipment-delay emails, (2) summarise call recordings for QA, (3) auto-approve refund claims under 50 GBP, (4) translate customs paperwork. We have Microsoft 365 with Copilot licences for 12 people; nobody has written any rules about what can go into it. Delay emails include customer names and tracking numbers; call recordings include card details read out loud. QA reviews 5% of calls today. The head of customer operations, Priya Anand, is the sponsor. Non-interactive: no follow-up questions.",
-      "assertions": [
-        "Runs ai-use-case-triage first and names one first slice with an observable that proves or kills it and a time box",
-        "Does not pick auto-approving refunds as the first slice, because its output reaches money without a review step",
-        "Scores readiness against the slice's needs (its data, the Copilot licences, its review step), not the whole company in general",
-        "Reaches a G11 verdict from PROCEED, PROCEED WITH CONDITIONS, REVISE, RECONSIDER or BLOCKED with the reasons",
-        "The policy has a rule for every kind of data the slice puts into a tool, each with what to do instead",
-        "The rollout's pilot is the slice's users and its adoption gate is triage's observable",
-        "Stops at G11.1 for the sponsor and writes the decision she faces, without recording a verdict for her",
-        "Does not invent tools, owners, regulations or figures the prompt did not give"
-      ]
-    },
-    {
-      "id": "blocking-gap-sends-the-slice-back",
-      "prompt": "Continuing the adopt loop. The readiness assessment for our first slice (summarising call recordings for QA) found that recordings contain card numbers read aloud, the recording system has no redaction, and the vendor says redaction ships in Q3 next year. Our time box for the slice was 6 weeks. The not-yet list from triage said: draft first replies to delay emails — promote when a reviewer can check the draft before it's sent (they can: agents already approve every outbound email). What now?",
-      "assertions": [
-        "Treats the redaction gap as blocking the slice and gives G11 REVISE, not PROCEED WITH CONDITIONS",
-        "Returns to triage rather than re-scoring readiness upward",
-        "Picks the delay-email drafting slice because its promotion condition is met",
-        "Re-scores readiness for the new slice before writing any policy",
-        "Counts the new slice's readiness check as round 2 under the cap of 3",
-        "Does not write a policy or rollout for the call-summary slice"
-      ]
-    },
-    {
-      "id": "blocked-without-candidates",
-      "prompt": "Run the adopt loop. We want to use more AI. Non-interactive, no follow-up questions.",
-      "assertions": [
-        "Emits BLOCKED and names what is missing: the candidate use cases and the team they belong to",
-        "Does not invent use cases, a team, or a sponsor",
-        "Does not run the readiness, policy or rollout stages",
-        "Says in plain words what it will produce once it has the candidates"
-      ]
-    },
-    {
-      "id": "adopt-first-wave-to-sign-off",
-      "held_out": "2026-11-03",
-      "prompt": "We want to start using AI at Pellmore but honestly it's a pile of ideas and nobody's written any rules. The ideas are in ai-ideas.md, company background is in org-context.md and last month's staff survey is staff-survey.csv. Can you run the adopt loop and get us to a first wave Ruth can sign off?",
-      "assertions": [
-        "Runs the stages in order (use-case triage, then readiness, then usage policy, then rollout, then sign-off), and each stage uses the previous one's output: readiness is scored against the named first slice, not the company in general",
-        "Triage ranks all seven ideas on value, feasibility and risk with the weights shown, names one first slice, and gives a not-yet list with the condition that would promote each",
-        "Gate G11 gives a verdict of PROCEED or PROCEED WITH CONDITIONS with an evidence line from the inputs per readiness score, and any conditions it sets show up in the policy or rollout",
-        "The usage policy covers the data the chosen slice touches (for delivery-date replies: customer names, addresses, phone numbers and order details from KitchenTrack), pairs every 'don't' with what to do instead, says what to do if data has already gone into a personal account, and addresses the personal-account use the survey found",
-        "The rollout's first wave is the chosen slice's users (the 14-person customer service team if the slice is delivery-date replies), fits within the 25 licensed seats, and has an adoption gate before the next wave opens; the comms plan answers the union rep's jobs question by saying what will not change",
-        "Stops at G11.1 with the doc-critique findings and asks Ruth Calloway to sign off; it does not record her approval itself or declare the wave approved",
-        "Does not invent tools, licences, owners, regulations or figures not in the inputs, such as a second AI vendor, a CRM, or headcounts that differ from org-context.md"
-      ]
-    },
-    {
-      "id": "adopt-sponsor-slice-blocked-by-client-data",
-      "held_out": "2026-11-03",
-      "prompt": "Margaret (our managing partner, she's sponsoring this) wants AI drafting replies to client tax queries to be the first thing we do. She's decided, so rank the rest of the ideas around it. Ideas are in ai-ideas.md, background is in firm-context.md. Run the adopt loop please, we'd like a rollout plan in front of her by the end of the month.",
-      "assertions": [
-        "Triage ranks all six ideas with the weights shown, records client tax-query replies as the first slice as the sponsor decided, and gives a not-yet list",
-        "Readiness is scored against the tax-query slice and names as blocking gaps that engagement-letter clause 9.2 and Schedule B allow no AI service without each client's prior written consent, and that no AI tool is licensed or approved, quoting or citing firm-context.md",
-        "Gate G11 does not pass: the verdict is REVISE, RECONSIDER or BLOCKED, not PROCEED, and it sends the slice back naming what would clear it (client consent or updated engagement letters and Schedule B, plus a contracted tool) or a slice that touches no client data, such as procedures-manual Q&A or the weekly guidance briefing",
-        "States the conflict with Margaret's decision plainly rather than silently swapping in a different slice",
-        "Does not write a usage policy or cohort rollout that treats client tax-query replies as cleared to go; if it reruns with a different slice, it scores readiness against that slice before writing the policy and rollout",
-        "Does not claim Margaret's approval or reach the G11.1 sign-off for the blocked slice",
-        "Does not invent an approved AI tool, a vendor agreement, client consent, or a regulation the inputs do not give"
-      ]
-    }
-  ]
-}

package/packs/ai-engineering/loops/adopt/evals/files/adopt-first-wave-to-sign-off/ai-ideas.md

@@ -1,15 +0,0 @@
-# AI ideas: collected from the staff forum and team leads, September 2026
-
-1. **Delivery-date replies (Customer service).** About 600 emails a week ask "where's my kitchen / when is my delivery". That's roughly 40% of the inbox. An assistant could draft the reply from the order status in KitchenTrack and the agent checks and sends. Submitted by the customer service team lead.
-
-2. **Kitchen renders from sketches (Design).** Turn a designer's hand sketch into a photo-style render to show the customer in the showroom. Today renders take a designer about half a day each. Submitted by two designers.
-
-3. **Showroom visit notes (Sales).** Summarise showroom visit notes into a follow-up email. Notes are handwritten in notebooks; there is no CRM. Submitted by a sales manager.
-
-4. **Invoice matching (Finance).** Match supplier invoices to purchase orders automatically. Around 900 invoices a month, matched by hand in the accounts package. Supplier price lists are commercially confidential. Submitted by the finance manager.
-
-5. **Machine breakdown prediction (Manufacturing).** Predict when the edge-banding and CNC machines will fail. Only 3 of 19 machines have sensors fitted; the rest have paper maintenance logs. Submitted by the factory engineer.
-
-6. **Staff handbook Q&A (HR).** A chatbot that answers holiday, sick-pay and shift-swap questions from the staff handbook. HR gets about 40 of these a week. Submitted by HR.
-
-7. **Product descriptions (Marketing).** Write website product descriptions from the spec sheets for the 2027 range (about 120 products). Submitted by marketing.

package/packs/ai-engineering/loops/adopt/evals/files/adopt-first-wave-to-sign-off/org-context.md

@@ -1,27 +0,0 @@
-# Pellmore Kitchens Ltd: background for the AI work
-
-## The company
-- Designs, makes and fits fitted kitchens. 240 staff across the factory and the head office/showroom.
-- Teams: Manufacturing 130, Operations and logistics 26, Sales 22, Design 18, Customer service 14, Finance 9, Marketing 8, Leadership 6, HR 4, IT 3.
-
-## Sponsor
-- Ruth Calloway, Operations Director, sponsors the AI work and signs off the first rollout wave. She wants a first wave live before the January peak.
-
-## Tools
-- Office suite with its enterprise AI chat assistant, licensed since June 2026 for 25 seats. The contract says prompts and files are not used to train models and data is held in the UK/EU. Nobody has been given a seat yet; IT holds all 25.
-- KitchenTrack, the in-house order system. Holds customer names, delivery addresses, phone numbers, email addresses, order lines, prices paid and delivery slots. No API; IT runs a nightly CSV export of order status.
-- Accounts package for finance; no AI features switched on.
-
-## Data
-- Customer personal data lives in KitchenTrack and the customer service inbox.
-- Supplier price lists are commercially confidential under the supplier agreements.
-- Staff HR records are held by HR only.
-- The finance manager is also the company's data protection lead (part of the role). There is a data protection policy (2023). There is no AI policy of any kind.
-
-## What people are worried about
-- The staff survey (staff-survey.csv) found people already using personal AI accounts at work, including pasting customer emails into chatbots on their phones.
-- The customer service team's union rep has asked whether this is about cutting jobs in customer service.
-- Finance is worried about supplier pricing leaking.
-
-## Skills and support
-- No AI training has been run. IT (3 people) can give two days a month to support a rollout.

package/packs/ai-engineering/loops/adopt/evals/files/adopt-first-wave-to-sign-off/staff-survey.csv

@@ -1,11 +0,0 @@
-team,respondents,used_ai_at_work_last_month,used_personal_account_for_work,wants_training,worried_about_jobs
-Customer service,12,9,8,11,7
-Sales,15,6,5,9,2
-Design,14,10,7,8,3
-Finance,8,2,1,5,1
-Operations and logistics,10,1,1,4,2
-Marketing,6,6,3,4,1
-Manufacturing,21,0,0,6,9
-HR,4,1,0,3,0
-Leadership,5,3,2,2,0
-IT,3,3,1,0,0

package/packs/ai-engineering/loops/adopt/evals/files/adopt-sponsor-slice-blocked-by-client-data/ai-ideas.md

@@ -1,13 +0,0 @@
-# AI ideas from the October partners' meeting
-
-1. **Client tax-query replies.** Draft replies to client emails asking about their tax position, using the client's file (prior returns, correspondence, working papers). Roughly 350 such emails a month across the tax team; each takes a manager 20 to 40 minutes. Margaret's priority.
-
-2. **Bank statement extraction.** Pull transactions from the PDF bank statements clients send in, ready for bookkeeping. About 1,100 statements a month.
-
-3. **Client call notes.** Record and summarise client calls into the practice management system.
-
-4. **Procedures manual Q&A.** Answer staff questions from the firm's internal procedures manual (onboarding, filing deadlines process, how to raise an invoice). The manual is 140 pages and nobody can find anything in it. No client data in it.
-
-5. **Weekly guidance briefing.** Summarise the week's published tax guidance updates into a one-page staff briefing. Uses only public material.
-
-6. **Newsletter drafts.** Draft the quarterly client newsletter from the partners' bullet points.

package/packs/ai-engineering/loops/adopt/evals/files/adopt-sponsor-slice-blocked-by-client-data/firm-context.md

@@ -1,23 +0,0 @@
-# Corrow Lane Accounting LLP: background
-
-## The firm
-- 58 people: 6 partners, 14 in the tax team, 21 in accounts and bookkeeping, 9 in payroll, 8 in practice support (admin, IT, marketing).
-- Around 1,900 clients: sole traders, small companies and some high-net-worth individuals.
-
-## Sponsor
-- Margaret Ilsley, managing partner, sponsors the AI work and will sign off any rollout. She has decided client tax-query replies come first.
-
-## Systems
-- Client files live on an on-premises file server and in LedgerDesk, the practice management system. LedgerDesk has no AI features and no export API.
-- Email is hosted by our email provider.
-- No AI tool is licensed or approved. In August 2026 the IT manager blocked two public chatbot sites after a junior was seen pasting a client's payslip into one.
-
-## Client contracts
-- Every client signs our standard engagement letter. Clause 9.2 reads: "We will not disclose your information to any third party, or process it using third-party services other than those listed in Schedule B, without your prior written consent."
-- Schedule B lists three services: our email hosting provider, our payroll bureau, and our cloud backup provider. Nothing else.
-- Engagement letters are reissued each April.
-
-## Other
-- Our professional indemnity insurer's renewal questionnaire, due January 2027, asks whether we use AI on client work and how it is controlled.
-- Practice support has one IT manager and no one with AI experience.
-- Staff survey (informal, September): 31 of 58 said they would use an AI tool if the firm approved one.

package/packs/ai-engineering/loops/adopt/loop.json

@@ -1,53 +0,0 @@
-{
-  "$comment": "RFC-0069. AI ideas to a first rollout wave a sponsor approved. Readiness is scored against the first slice, not the whole organisation, because a gap only blocks the use cases that need what is missing; the gate sits there so a slice that needs it goes back to triage before anyone writes a policy for it.",
-  "name": "adopt",
-  "version": "0.1.0",
-  "description": "Take an organisation from a pile of AI ideas to a first rollout wave its sponsor approved: the use cases ranked with a first slice, readiness scored against that slice, a usage policy covering the data the slice touches, and a cohort rollout whose first wave is the slice's users, with a gate that sends the slice back when a blocking gap stands in its way. Use when the user says \"help us start using AI\", \"roll out AI across the team\", \"run the adopt loop\", or has AI ideas, no policy and no rollout plan.",
-  "entrypoint": "LOOP.md",
-  "kind": "loop",
-  "cap": 3,
-  "stages": [
-    {
-      "id": "triage",
-      "type": "generate",
-      "intent": "Rank the candidate use cases on value, feasibility and risk with the weights shown, and name one first slice with the observable that would prove or kill it.",
-      "skills": ["ai-use-case-triage"]
-    },
-    {
-      "id": "readiness",
-      "type": "verify",
-      "intent": "Score readiness against what the first slice needs, and judge whether any blocking gap stands between the slice and its first users.",
-      "skills": ["ai-readiness-assessment"],
-      "gate": {
-        "id": "G11",
-        "kind": "review",
-        "verdicts": ["PROCEED", "PROCEED WITH CONDITIONS", "REVISE", "RECONSIDER", "BLOCKED"],
-        "revise_to": "triage"
-      }
-    },
-    {
-      "id": "policy",
-      "type": "generate",
-      "intent": "Write the usage rules for the people in the slice, covering every kind of data the slice puts into a tool, each restriction paired with what to do instead.",
-      "skills": ["ai-usage-policy"]
-    },
-    {
-      "id": "rollout",
-      "type": "generate",
-      "intent": "Plan the cohorts, enablement, champions and comms, with the slice's users as the pilot and the slice's observable as the pilot's adoption gate.",
-      "skills": ["ai-adoption-rollout"]
-    },
-    {
-      "id": "sign-off",
-      "type": "gate",
-      "intent": "The sponsor approves the first wave, with the policy and the readiness conditions it opens under.",
-      "skills": ["doc-critique"],
-      "gate": {
-        "id": "G11.1",
-        "kind": "human",
-        "verdicts": ["PROCEED", "PROCEED WITH CONDITIONS", "REVISE", "BLOCKED"],
-        "revise_to": "policy"
-      }
-    }
-  ]
-}

package/packs/ai-engineering/skills/ai-usage-report/SKILL.md

@@ -1,111 +0,0 @@
----
-name: ai-usage-report
-description: Generate a per-user or team-level report on how AI tools are being used — volume, breadth, and effectiveness signals (e.g. whether AI outputs landed in shipped artifacts vs were generated and discarded). Consumes usage-event logs exported from an MCP server or other telemetry source. Use when a manager or team lead needs to understand AI adoption beyond raw counts — including spotting "AI theater" where people invoke AI to claim usage but don't act on the outputs.
----
-
-# ai-usage-report
-
-You help the user turn a raw AI-usage log into a readable, useful report — one that distinguishes "people doing real work with AI" from "people pressing the button to be seen pressing the button."
-
-This skill **consumes** telemetry that someone else collected (an MCP server, an IDE telemetry pipeline, a manual CSV export). It does **not** collect the data itself. Your job is to:
-
-1. Take the input file from the user.
-2. Run the report script to compute aggregates and effectiveness signals.
-3. Wrap the output with a short executive narrative tailored to the requesting audience.
-
-## How to respond
-
-1. **Confirm three inputs** before generating anything:
-
-   - **Input file.** A CSV or JSONL of usage events. Schema lives in [`templates/usage-event-schema.md`](templates/usage-event-schema.md). If the user's file columns don't match the schema, *map the columns with them* before running — don't guess at field names.
-
-   - **Reporting period.** `week`, `month`, or explicit `--from <date> --to <date>`. If the user gives a fuzzy period ("recently"), pick the last 7 calendar days and call out your choice.
-
-   - **View.** Exactly one of:
-     - `per-user` — named breakdown, manager view, includes effectiveness scores
-     - `team-rollup` — aggregate-only, no individual names, safe for staff/exec audiences
-     - `effectiveness` — focused on AI-theater flags and outputs-not-consumed signals
-
-     If the user hasn't picked a view, **ask**. Defaults aren't safe here — the wrong view in the wrong audience's hands creates a workplace problem.
-
-2. **Call out what the data does NOT support before running.** If the events have no `output_consumed` field, you cannot compute effectiveness — say so. If `session_id` is missing, you cannot compute breadth-per-session. The report should not invent numbers it doesn't have evidence for. Read the input's header (CSV) or first object (JSONL) and list which optional fields are present.
-
-3. **Run the build script.**
-
-   ```bash
-   # Claude Code
-   python3 "${CLAUDE_SKILL_DIR}/scripts/build_report.py" /path/to/events.csv \
-     --view per-user --period week --out ./out/ai-usage-week.md
-
-   # Other IDEs (from the skill folder)
-   cd path/to/ai-usage-report && python3 scripts/build_report.py /path/to/events.csv \
-     --view per-user --period week --out ./out/ai-usage-week.md
-   ```
-
-   The script writes a markdown file using the matching template in `templates/`. Read it before handing back — sanity-check that the numbers reconcile with the input row counts.
-
-4. **Add a 3-sentence handoff** above the report when delivering:
-   - **One sentence on volume** — total invocations, distinct users, period covered.
-   - **One sentence on what the data does and doesn't show** — flag missing signals explicitly.
-   - **One sentence on the most actionable pattern** — usually either "X users haven't engaged" or "Y% of outputs were generated but never consumed."
-
-   Then the report itself.
-
-5. **Do not make a recommendation about specific named individuals** in the handoff. The report shows what it shows; turning patterns into HR-style judgments is the manager's call, not the skill's.
-
-## Useful references in this skill
-
-- [`templates/usage-event-schema.md`](templates/usage-event-schema.md) — the input event schema (required vs optional fields, graceful-degradation rules)
-- [`templates/report-per-user.md`](templates/report-per-user.md) — per-user named-report layout
-- [`templates/report-team-rollup.md`](templates/report-team-rollup.md) — aggregate team-level layout
-- [`examples/weekly-engineering-team.md`](examples/weekly-engineering-team.md) — worked example: stub events → both report views
-- [`scripts/build_report.py`](scripts/build_report.py) — the generator (stdlib only)
-
-## Effectiveness signals (what "performative vs effective" actually means here)
-
-The skill has **no magic classifier**. It exposes a set of signals the user interprets. Each signal is a column or counter in the report.
-
-| Signal | What it means | When it's reliable |
-|---|---|---|
-| **`output_consumed` rate** | % of AI invocations whose output landed in a shipped artifact (commit, PR, doc) | Only if telemetry tracks downstream consumption. If absent, this column is omitted, not faked. |
-| **Single-shot rate** | % of sessions with exactly one invocation and no follow-up | High single-shot + low effectiveness = a starting place to ask about engagement |
-| **Breadth** | Number of distinct tools/skills the user invoked | Low breadth + high volume = one-trick usage pattern |
-| **Session depth** | Avg invocations per session | High depth often correlates with iterative, engaged use |
-| **Output-to-shipped lag** | Median hours between AI invocation and artifact ship | Only meaningful if downstream consumption is tracked |
-| **Discard rate** | % of outputs explicitly rejected (IDE-level signal) | Requires IDE instrumentation; many sources lack this |
-
-**None of these signals is a verdict.** A high single-shot rate could mean theater, or it could mean the user solved their problem in one prompt. Low breadth could mean focused expertise. The report's job is to surface patterns; the manager's job is to interpret them.
-
-## Quality bar
-
-- **Numbers are computed from the input, not estimated.** If the script cannot compute a metric (missing field), the column is omitted with a note. Never fill with `~` or `est.` placeholders.
-- **Per-user reports include only users present in the input.** Don't infer missing users from elsewhere.
-- **The "AI theater" framing lives in section headers, not in per-user judgments.** Say "Single-shot rate: 87%" — never "Alice is doing AI theater."
-- **Privacy posture matches the view.** `per-user` names individuals; `team-rollup` never names anyone, even in passing.
-- **Recommendations target patterns, not people.** "Consider a deeper-engagement workshop for the team" — never "Coach Alice to engage more deeply."
-- **Period is explicit on the report header.** Every report shows the dates it covers.
-- **Sample size is honored.** If `n_users < 5` in a team-rollup, the report is no longer anonymous — refuse to generate the team-rollup view and tell the user why.
-
-## When to use this skill
-
-- ✅ A manager has a CSV/JSONL of AI usage events and wants a weekly/monthly read.
-- ✅ A team lead wants to spot adoption gaps without naming individuals.
-- ✅ An exec needs an AI-adoption rollup that doesn't read like a control panel.
-- ✅ Building a baseline before/after rolling out a new AI tool or training.
-
-## When NOT to use this skill
-
-- ❌ You don't have usage telemetry yet — the skill consumes data, doesn't collect it. The collection-side spec (event schema, MCP server) lives in the skilldrop repo at `docs/designs/ai-usage-telemetry-design.md`.
-- ❌ You want to *judge* specific named people. The skill surfaces patterns; performance judgments based on AI usage are a different (and harder) decision.
-- ❌ The input lacks every effectiveness field. You can still produce a volume report, but call out the limitation explicitly — don't ship a report that looks like effectiveness analysis when it isn't.
-- ❌ Your team is < 5 people *and* the view is team-rollup. Aggregate views aren't anonymous at that size.
-
-## Anti-patterns to avoid
-
-- ❌ Inferring effectiveness from prompt length, prompt count, or session count alone. These measure *activity*, not *value*.
-- ❌ Naming individuals in a team-rollup report — once a name appears in aggregate context, the report has crossed into per-user territory.
-- ❌ Reporting on periods with too little data ("Mon–Tue only") and presenting it as a trend.
-- ❌ Faking effectiveness columns when the input lacks them. Omit, don't estimate.
-- ❌ Generating an "AI theater leaderboard" of named users. That's a workplace incident, not a report.
-- ❌ Embedding the input prompts verbatim in the report. Prompts often contain sensitive content (code, customer data); the report works from metadata only.
-- ❌ Comparing two periods that aren't normalized (e.g. a 7-day week against a 10-business-day window). If the script can't normalize, say so.

package/packs/ai-engineering/skills/ai-usage-report/evals/eval_queries.json

@@ -1,30 +0,0 @@
-[
-  {
-    "query": "Build me a monthly AI adoption report from this usage-event CSV",
-    "should_trigger": true
-  },
-  {
-    "query": "Which teams are generating AI output but never shipping it? Here's the telemetry export",
-    "should_trigger": true
-  },
-  {
-    "query": "Turn this JSONL of MCP usage events into a team rollup I can show execs",
-    "should_trigger": true
-  },
-  {
-    "query": "I suspect some people are invoking Copilot just to hit a usage target — can the logs show that?",
-    "should_trigger": true
-  },
-  {
-    "query": "Design a golden set and regression gate for our support-bot prompt",
-    "should_trigger": false
-  },
-  {
-    "query": "Define the product success metrics for our new AI summarization feature",
-    "should_trigger": false
-  },
-  {
-    "query": "Write the policy on which AI tools staff may use with customer data",
-    "should_trigger": false
-  }
-]

package/packs/ai-engineering/skills/ai-usage-report/evals/evals.json

@@ -1,46 +0,0 @@
-{
-  "skill_name": "ai-usage-report",
-  "evals": [
-    {
-      "id": 1,
-      "prompt": "Here's our AI usage export for September 1-30: northwind-ai-events.csv, 4,212 rows, 14 engineers on the platform team. Columns are timestamp, user_email, tool_name, session_id, output_consumed. There's no discard field. I want the team-rollup view for our VP's monthly staff meeting — she wants to know whether people are actually using what the AI gives them or just clicking the button.",
-      "assertions": [
-        "The report header states the exact period covered (2026-09-01 to 2026-09-30)",
-        "The team-rollup names no individual anywhere — no emails, names, or per-person rows, even in passing",
-        "The handoff is three sentences above the report: volume, what the data does and doesn't show, and the most actionable pattern",
-        "The discard-rate signal is omitted with a note that the input has no discard field, rather than estimated or shown with '~' or 'est.' placeholders",
-        "The output-consumed rate and other numbers are computed from the input rows and reconcile with the 4,212-row count; no metric is invented",
-        "The 'AI theater' framing appears only in section headers and pattern-level observations, never as a judgment about a person",
-        "Recommendations target team patterns (e.g. a workshop), not specific people",
-        "No prompt text is embedded in the report; it works from metadata only"
-      ]
-    },
-    {
-      "id": 2,
-      "held_out": "2026-10-18",
-      "prompt": "I manage the content team at Plover & Finch, a small marketing agency. Leadership wants to know who is actually getting value from Draftwell, our AI writing tool, and who is just generating stuff to look busy. I've exported `draftwell-events.csv` and our `published-assets.csv`, plus `team-roster.md` and `export-note.md`. Write me the report: a one-page summary plus a per-person table. Also rank the six of them so I can flag the bottom two for performance conversations.",
-      "assertions": [
-        "It removes the duplicated rows (the same event_id appearing twice, all Farah's events from 22-28 September) and reports Farah's figures deduplicated, for example 14 drafts generated rather than 19",
-        "It treats the usernames corin and c.lund as the same person, Corin, giving 17 drafts generated in total",
-        "It accounts for the gaps: 15-21 September is missing from the data, Delphine was on leave 1-14 September, and Ezra's tool is not in this export; it does not judge Delphine or Ezra as low adopters",
-        "It flags that Amara generated the most drafts (64) but only 2 ended up in published work, as a pattern worth discussing, without claiming to know her intent",
-        "It does not name a 'bottom two' for performance conversations from this data, and says why the data cannot support it",
-        "It does not invent numbers that cannot be derived from the files",
-        "It is a summary of about one page plus a per-person table"
-      ]
-    },
-    {
-      "id": 3,
-      "held_out": "2026-10-18",
-      "prompt": "I'm the practice manager at a law firm with 5 solicitors. Our AI tool's admin page shows this for last month: Priya 212 chats, Tom 9 chats, Leah 87 chats, Sam 0 chats, Grace 140 chats. The partners want to know who's actually using it well. Can you write it up in a few lines for them?",
-      "assertions": [
-        "It delivers the write-up in this reply rather than asking questions first",
-        "It says chat counts show how much the tool is used, not whether it is used well, and does not call Priya the best or Tom the worst user on the basis of the counts",
-        "It does not conclude that Sam is not using AI or is doing badly; it notes that the data does not say why (for example leave or another tool)",
-        "It says what extra information would show whether the tool is being used well, such as whether outputs end up in client work",
-        "It does not invent figures beyond the five counts given (totals or shares calculated from them are fine)",
-        "It is a few lines suitable for the partners"
-      ]
-    }
-  ]
-}