skilldrop-cli 1.3.0 → 1.4.0

package/packs/sre-oncall/loops/operate/evals/files/operate-export-service-first-outage/service-notes.md

@@ -0,0 +1,46 @@
+# invoice-export — service notes
+
+Owner: billing-platform team (Pellmoor)
+Status: soft launch since 2026-10-06, enabled for all customers on the Business plan
+
+## What it does
+
+Customers click "Export invoices" in the web app and pick a date range and a format (CSV or PDF bundle). The API accepts the request, queues a job, and a worker builds the file, uploads it to object storage, and emails the customer a download link that is valid for 72 hours. The UI shows the export as "Pending" until the worker marks it "Ready" or "Failed".
+
+## Components
+
+- `export-api` — 3 pods behind the internal gateway. Endpoints: `POST /v1/exports`, `GET /v1/exports/{id}`.
+- `export-worker` — Kubernetes deployment, 6 pods, each pod runs N concurrent jobs (setting: `WORKER_CONCURRENCY`, currently 8). Memory limit 2 GiB per pod.
+- Job queue — the shared platform job queue, queue name `invoice-export`.
+- Postgres read replica `billing-replica-2` — workers read invoice rows from here.
+- Object storage bucket `exports.pellmoor.example`.
+- Email — the shared notifications service sends the "your export is ready" email.
+
+## Traffic
+
+- About 12,000 export requests a day.
+- Peak is Monday 09:00–10:00 UTC (month-end and Monday morning are the busiest).
+- Typical CSV export finishes in under 1 minute; PDF bundles of 500+ invoices can take 3–4 minutes.
+
+## What product has told customers
+
+The help-centre article says "most exports are ready within 5 minutes". There is no contractual SLA for exports.
+
+## Current monitoring
+
+- Grafana dashboard "export-worker" with CPU and memory per pod.
+- One alert: `ExportWorkerHighCPU` — worker CPU > 80% for 5 minutes, pages the on-call. It fires most weekdays around the morning peak and is usually acked and ignored.
+- No alert on queue depth, job age, or failed exports.
+- Logs go to the central log store; no tracing yet.
+
+## Deploy and rollback
+
+- Deploys go out through CI to production with Helm. No canary; all worker pods are replaced in one rolling update.
+- Rollback: `helm rollback invoice-export <revision> -n billing`
+- Current release cadence: 2–3 deploys a week.
+
+## On-call
+
+- Primary: billing-platform weekly rota in the paging tool.
+- Escalation: billing-platform team lead (Dana Okafor), then the platform on-call for queue or database problems.
+- Support raise issues in #billing-incidents.

package/packs/sre-oncall/loops/operate/evals/files/operate-incident-still-open/oncall-log.txt

@@ -0,0 +1,17 @@
+#payments-oncall, Wednesday 2026-11-04 (times UTC)
+
+01:31  alertbot: PAGE WebhookDeliveryLatencyHigh — p95 merchant webhook delivery latency 9m (threshold 2m)
+01:33  Noor (on-call): ack. looking
+01:38  Noor (on-call): retry queue for webhook-dispatcher at 1,450 and growing. most failures are timeouts calling merchant endpoints
+01:44  Noor (on-call): not obviously a deploy, last dispatcher deploy was Monday
+01:52  Noor (on-call): p95 latency 25m now. merchants on the status page subscriber list will start noticing
+01:55  Noor (on-call): posted to status page: "Investigating — some merchants may see delayed payment webhooks. Next update 02:30 UTC."
+02:04  Noor (on-call): seeing a lot of timeouts against one partner relay (relay-eu.partner.example), but also some against merchants direct. not sure which is cause and which is effect
+02:10  Noor (on-call): scaled webhook-dispatcher consumers 12 -> 30 to chew through the backlog
+02:35  Noor (on-call): retry queue 900, falling
+03:05  Noor (on-call): retry queue 240. backlog draining
+03:40  Noor (on-call): p95 delivery latency back to 40s. looks like we're through the worst
+03:41  alertbot: RESOLVED WebhookDeliveryLatencyHigh
+03:52  Noor (on-call): leaving consumers at 30 for now
+04:10  Noor (on-call): hmm. retry queue growing again, 2,100 -> 3,400 in the last 20 min. error rate on calls through relay-eu at 12%. latency alert hasn't re-fired yet
+04:12  Noor (on-call): going to look properly in the morning, too tired to think. consumers still at 30

package/packs/sre-oncall/loops/operate/evals/files/operate-incident-still-open/runbook-webhooks.md

@@ -0,0 +1,38 @@
+# Runbook: webhook-dispatcher
+
+Owner: payments-core
+Last reviewed: 2026-06-18
+
+## What it does
+
+Delivers payment event webhooks (payment.succeeded, payment.failed, refund.created, payout.paid) to merchant endpoints. Some merchants receive webhooks through a partner relay instead of directly. Failed deliveries are retried with exponential backoff for up to 72 hours.
+
+## Dashboards
+
+- "webhook-dispatcher overview": delivery latency p50/p95, retry queue depth, error rate by destination host.
+
+## Alerts
+
+- WebhookDeliveryLatencyHigh — p95 delivery latency > 2m for 10m. Pages payments on-call.
+
+## Common problems
+
+### Retry queue growing
+
+1. Check the error rate by destination host on the dashboard.
+2. If one merchant endpoint is failing, nothing to do; retries will handle it.
+3. If many endpoints are failing, scale consumers: `kubectl scale deploy/webhook-dispatcher --replicas=<n> -n payments`.
+
+### Deploy rollback
+
+`helm rollback webhook-dispatcher <revision> -n payments`
+
+## Escalation
+
+1. Payments on-call
+2. payments-core lead
+3. Status page: on-call posts; comms lead for anything over 1 hour
+
+## Known gaps
+
+- No guidance yet for partner relay problems.

package/packs/sre-oncall/skills/cloud-cost-review/SKILL.md

@@ -107,7 +107,6 @@ This looks backward at a real bill. `capacity-cost-model` looks forward: it size
 - ❌ Estimating cost before launch, or projecting at 3× or 10× growth. Use `capacity-cost-model`.
 - ❌ Designing telemetry or a logging budget from scratch. Use `observability-plan`, though this skill flags logging cost as a hot spot.
 - ❌ Writing up an incident, including a cost spike from a runaway job. Use `postmortem-generator`.
-- ❌ Reporting AI tool usage or spend by user. Use `ai-usage-report`.
 - ❌ Defining business KPIs for a feature or product. Use `success-metrics`.
 
 ## Anti-patterns to avoid

package/packs/sre-oncall/skills/cloud-cost-review/manifest.json

@@ -1,6 +1,6 @@
 {
   "name": "cloud-cost-review",
-  "version": "0.1.2",
+  "version": "0.1.3",
   "description": "Review actual cloud spend from a cost export (AWS CUR or Data Exports, GCP billing export, Azure cost details) and produce a ranked savings report — idle and unattached resources, oversized instances backed by utilisation evidence, storage tiering, data transfer hot spots, commitment coverage and untagged spend — each finding with a monthly saving computed from the bill, a confidence level and an ease rating. Use when the user shares a cost CSV and says \"where are we wasting money\", \"review our AWS bill\", \"why did the cloud bill jump this month\", \"find savings in our GCP/Azure spend\", or \"should we buy savings plans\".",
   "entrypoint": "SKILL.md",
   "deps": {
@@ -12,7 +12,6 @@
     "optional": []
   },
   "related": [
-    "ai-usage-report",
     "capacity-cost-model",
     "observability-plan",
     "postmortem-generator",

package/packs/sre-oncall/skills/delivery-metrics-report/SKILL.md

@@ -106,7 +106,6 @@ It uses the metric names DORA currently publishes on [dora.dev](https://dora.dev
 - ❌ Writing up one incident: timeline, impact, contributing factors. Use `postmortem-generator`.
 - ❌ Designing SLOs, alerts or dashboards for a service. Use `observability-plan`.
 - ❌ Product or business outcome metrics (activation, retention, revenue). Use `success-metrics`.
-- ❌ Reporting how people use AI tools. Use `ai-usage-report`.
 - ❌ Forecasting infrastructure cost or capacity. Use `capacity-cost-model`.
 
 ## Anti-patterns to avoid

package/packs/sre-oncall/skills/delivery-metrics-report/manifest.json

@@ -1,6 +1,6 @@
 {
   "name": "delivery-metrics-report",
-  "version": "0.1.4",
+  "version": "0.1.5",
   "description": "Produce a delivery metrics report for one service from deployment, incident and pull-request exports — the DORA metrics (deployment frequency, change lead time, change fail rate, failed deployment recovery time) plus flow metrics (WIP, cycle time p50/p85, throughput), computed by a script with per-week trends and explicit data gaps, then interpreted for what changed and why. Use when the user wants DORA metrics, says \"how are we doing on the four keys\", \"what's our change failure rate\", \"compute lead time and deploy frequency from this CSV\", or \"build our delivery metrics report\".",
   "entrypoint": "SKILL.md",
   "deps": {
@@ -12,7 +12,6 @@
     "optional": []
   },
   "related": [
-    "ai-usage-report",
     "capacity-cost-model",
     "observability-plan",
     "postmortem-generator",

package/packs/sre-oncall/skills/incident-comms/manifest.json

@@ -1,6 +1,6 @@
 {
   "name": "incident-comms",
-  "version": "0.1.0",
+  "version": "0.1.1",
   "description": "Draft audience-segmented updates during a live incident — customer status-page posts, internal stakeholder updates, and exec notifications — each leading with user-facing impact, never speculative cause or ETA, always naming the next-update time. Use when an incident is in progress and the user needs a status page post, customer notification, internal update, exec brief, or the resolved/all-clear message.",
   "entrypoint": "SKILL.md",
   "deps": { "npm": [], "pip": [] },
@@ -10,5 +10,5 @@
     { "to": "postmortem-generator", "when": "the incident is resolved", "purpose": "Analyse blamelessly and emit runbook deltas.", "fallback": "If postmortem-generator is absent, write the timeline, the contributing factors, and the procedural changes inline." }
   ],
   "tags": ["incident", "communication", "status-page", "outage", "stakeholder", "sre"],
-  "model": { "tier": "standard", "rationale": "Audience-segmented message generation under a strict don't-retract discipline. Same shape as exec-summary/audience-profile — translation and altitude, not adversarial reasoning." }
+  "model": { "tier": "light", "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 93% and 100% of checks in two tier trials, against standard's 85% and heavy's 92% and 93%, within 5 points of the best both times." }
 }

package/packs/sre-oncall/skills/postmortem-generator/manifest.json

@@ -1,6 +1,6 @@
 {
   "name": "postmortem-generator",
-  "version": "0.2.0",
+  "version": "0.2.1",
   "description": "Write a blameless incident postmortem from the raw material (chat scrollback, alert history, timeline, notes) for the audience and length the user names — the cause stated as the change and the gap that let it through, never a person even when the notes name one; who or what noticed first and how late; known kept apart from still-unknown; and follow-up actions that close the gap. Invents no times, figures, owners or severities. Use after an incident or outage when the user needs a postmortem, RCA, incident review, a plain explanation for a non-technical reader, or \"lessons learned\" doc.",
   "entrypoint": "SKILL.md",
   "deps": { "npm": [], "pip": [] },
@@ -10,5 +10,5 @@
     { "to": "runbook-generator", "when": "the postmortem produced runbook deltas", "purpose": "Apply the deltas as edits to the existing runbook.", "fallback": "If runbook-generator is absent, phrase each delta as a concrete before/after edit so a human can apply it." }
   ],
   "tags": ["postmortem", "incident", "rca", "sre", "blameless", "outage"],
-  "model": { "tier": "standard", "rationale": "Synthesizes a structured doc from incident notes. Contributing-factor chains need judgment but are bounded; escalate via large-input on huge scrollbacks." }
+  "model": { "tier": "light", "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 93% and 91% of checks in two tier trials, against standard's 92% and heavy's 90% and 87%, within 5 points of the best both times." }
 }

package/packs/stakeholder-comms/skills/decision-log/manifest.json

@@ -1,11 +1,11 @@
 {
   "name": "decision-log",
-  "version": "0.1.1",
+  "version": "0.1.2",
   "description": "Extract decisions, action items, owners, and due dates from meeting notes, Slack threads, design-review transcripts, or any unstructured discussion. Produces a clean structured log the user can paste into a doc, tracker, or ADR queue. Use when the user has notes/threads/transcripts and asks for a \"decision log\", \"action items\", \"what did we agree\", or \"follow-ups\".",
   "entrypoint": "SKILL.md",
   "deps": { "npm": [], "pip": [] },
   "env": { "required": [], "optional": [] },
   "related": [],
   "tags": ["meetings", "decisions", "action-items", "stakeholder-communication"],
-  "model": { "tier": "light", "rationale": "Extracts decisions/owners/dates from notes. Pattern extraction; escalates on long transcripts." }
+  "model": { "tier": "standard", "rationale": "Measured (RFC-0073): on its held-out evals the light tier fell more than 5 points behind the best in both tier trials (79% and 61% against standard's 79%); one step up, to standard." }
 }

package/packs/trackers/skills/team-status-report/manifest.json

@@ -1,6 +1,6 @@
 {
   "name": "team-status-report",
-  "version": "0.1.4",
+  "version": "0.1.5",
   "description": "Write a weekly team status report from tracker data (Jira, Linear or GitHub Projects, via CSV, JSON or an MCP tool) — shipped, in progress, blocked with the blocker and owner, slipped against plan, risks and asks — with every number computed from the data by a stdlib script, never estimated, and a RAG status that states the rule that set it. Use when the user wants a weekly status update, sprint report or stakeholder update from the tracker, or says \"write this week's status report\", \"are we on track?\".",
   "entrypoint": "SKILL.md",
   "deps": {
@@ -27,7 +27,7 @@
   ],
   "permissions": {"network": [], "commands": [], "files": "named-paths"},
   "model": {
-    "tier": "standard",
-    "rationale": "A script computes every count and the suggested RAG; the model writes plain-language summaries, attributes outside facts and frames risks and asks. Bounded writing over computed numbers."
+    "tier": "light",
+    "rationale": "Measured (RFC-0073): on its held-out evals the light tier met 93% and 93% of checks in two tier trials, against standard's 93% and heavy's 93% and 93%, within 5 points of the best both times."
   }
 }

package/packs/ai-engineering/skills/ai-usage-report/SKILL.md

@@ -1,111 +0,0 @@
----
-name: ai-usage-report
-description: Generate a per-user or team-level report on how AI tools are being used — volume, breadth, and effectiveness signals (e.g. whether AI outputs landed in shipped artifacts vs were generated and discarded). Consumes usage-event logs exported from an MCP server or other telemetry source. Use when a manager or team lead needs to understand AI adoption beyond raw counts — including spotting "AI theater" where people invoke AI to claim usage but don't act on the outputs.
----
-
-# ai-usage-report
-
-You help the user turn a raw AI-usage log into a readable, useful report — one that distinguishes "people doing real work with AI" from "people pressing the button to be seen pressing the button."
-
-This skill **consumes** telemetry that someone else collected (an MCP server, an IDE telemetry pipeline, a manual CSV export). It does **not** collect the data itself. Your job is to:
-
-1. Take the input file from the user.
-2. Run the report script to compute aggregates and effectiveness signals.
-3. Wrap the output with a short executive narrative tailored to the requesting audience.
-
-## How to respond
-
-1. **Confirm three inputs** before generating anything:
-
-   - **Input file.** A CSV or JSONL of usage events. Schema lives in [`templates/usage-event-schema.md`](templates/usage-event-schema.md). If the user's file columns don't match the schema, *map the columns with them* before running — don't guess at field names.
-
-   - **Reporting period.** `week`, `month`, or explicit `--from <date> --to <date>`. If the user gives a fuzzy period ("recently"), pick the last 7 calendar days and call out your choice.
-
-   - **View.** Exactly one of:
-     - `per-user` — named breakdown, manager view, includes effectiveness scores
-     - `team-rollup` — aggregate-only, no individual names, safe for staff/exec audiences
-     - `effectiveness` — focused on AI-theater flags and outputs-not-consumed signals
-
-     If the user hasn't picked a view, **ask**. Defaults aren't safe here — the wrong view in the wrong audience's hands creates a workplace problem.
-
-2. **Call out what the data does NOT support before running.** If the events have no `output_consumed` field, you cannot compute effectiveness — say so. If `session_id` is missing, you cannot compute breadth-per-session. The report should not invent numbers it doesn't have evidence for. Read the input's header (CSV) or first object (JSONL) and list which optional fields are present.
-
-3. **Run the build script.**
-
-   ```bash
-   # Claude Code
-   python3 "${CLAUDE_SKILL_DIR}/scripts/build_report.py" /path/to/events.csv \
-     --view per-user --period week --out ./out/ai-usage-week.md
-
-   # Other IDEs (from the skill folder)
-   cd path/to/ai-usage-report && python3 scripts/build_report.py /path/to/events.csv \
-     --view per-user --period week --out ./out/ai-usage-week.md
-   ```
-
-   The script writes a markdown file using the matching template in `templates/`. Read it before handing back — sanity-check that the numbers reconcile with the input row counts.
-
-4. **Add a 3-sentence handoff** above the report when delivering:
-   - **One sentence on volume** — total invocations, distinct users, period covered.
-   - **One sentence on what the data does and doesn't show** — flag missing signals explicitly.
-   - **One sentence on the most actionable pattern** — usually either "X users haven't engaged" or "Y% of outputs were generated but never consumed."
-
-   Then the report itself.
-
-5. **Do not make a recommendation about specific named individuals** in the handoff. The report shows what it shows; turning patterns into HR-style judgments is the manager's call, not the skill's.
-
-## Useful references in this skill
-
-- [`templates/usage-event-schema.md`](templates/usage-event-schema.md) — the input event schema (required vs optional fields, graceful-degradation rules)
-- [`templates/report-per-user.md`](templates/report-per-user.md) — per-user named-report layout
-- [`templates/report-team-rollup.md`](templates/report-team-rollup.md) — aggregate team-level layout
-- [`examples/weekly-engineering-team.md`](examples/weekly-engineering-team.md) — worked example: stub events → both report views
-- [`scripts/build_report.py`](scripts/build_report.py) — the generator (stdlib only)
-
-## Effectiveness signals (what "performative vs effective" actually means here)
-
-The skill has **no magic classifier**. It exposes a set of signals the user interprets. Each signal is a column or counter in the report.
-
-| Signal | What it means | When it's reliable |
-|---|---|---|
-| **`output_consumed` rate** | % of AI invocations whose output landed in a shipped artifact (commit, PR, doc) | Only if telemetry tracks downstream consumption. If absent, this column is omitted, not faked. |
-| **Single-shot rate** | % of sessions with exactly one invocation and no follow-up | High single-shot + low effectiveness = a starting place to ask about engagement |
-| **Breadth** | Number of distinct tools/skills the user invoked | Low breadth + high volume = one-trick usage pattern |
-| **Session depth** | Avg invocations per session | High depth often correlates with iterative, engaged use |
-| **Output-to-shipped lag** | Median hours between AI invocation and artifact ship | Only meaningful if downstream consumption is tracked |
-| **Discard rate** | % of outputs explicitly rejected (IDE-level signal) | Requires IDE instrumentation; many sources lack this |
-
-**None of these signals is a verdict.** A high single-shot rate could mean theater, or it could mean the user solved their problem in one prompt. Low breadth could mean focused expertise. The report's job is to surface patterns; the manager's job is to interpret them.
-
-## Quality bar
-
-- **Numbers are computed from the input, not estimated.** If the script cannot compute a metric (missing field), the column is omitted with a note. Never fill with `~` or `est.` placeholders.
-- **Per-user reports include only users present in the input.** Don't infer missing users from elsewhere.
-- **The "AI theater" framing lives in section headers, not in per-user judgments.** Say "Single-shot rate: 87%" — never "Alice is doing AI theater."
-- **Privacy posture matches the view.** `per-user` names individuals; `team-rollup` never names anyone, even in passing.
-- **Recommendations target patterns, not people.** "Consider a deeper-engagement workshop for the team" — never "Coach Alice to engage more deeply."
-- **Period is explicit on the report header.** Every report shows the dates it covers.
-- **Sample size is honored.** If `n_users < 5` in a team-rollup, the report is no longer anonymous — refuse to generate the team-rollup view and tell the user why.
-
-## When to use this skill
-
-- ✅ A manager has a CSV/JSONL of AI usage events and wants a weekly/monthly read.
-- ✅ A team lead wants to spot adoption gaps without naming individuals.
-- ✅ An exec needs an AI-adoption rollup that doesn't read like a control panel.
-- ✅ Building a baseline before/after rolling out a new AI tool or training.
-
-## When NOT to use this skill
-
-- ❌ You don't have usage telemetry yet — the skill consumes data, doesn't collect it. The collection-side spec (event schema, MCP server) lives in the skilldrop repo at `docs/designs/ai-usage-telemetry-design.md`.
-- ❌ You want to *judge* specific named people. The skill surfaces patterns; performance judgments based on AI usage are a different (and harder) decision.
-- ❌ The input lacks every effectiveness field. You can still produce a volume report, but call out the limitation explicitly — don't ship a report that looks like effectiveness analysis when it isn't.
-- ❌ Your team is < 5 people *and* the view is team-rollup. Aggregate views aren't anonymous at that size.
-
-## Anti-patterns to avoid
-
-- ❌ Inferring effectiveness from prompt length, prompt count, or session count alone. These measure *activity*, not *value*.
-- ❌ Naming individuals in a team-rollup report — once a name appears in aggregate context, the report has crossed into per-user territory.
-- ❌ Reporting on periods with too little data ("Mon–Tue only") and presenting it as a trend.
-- ❌ Faking effectiveness columns when the input lacks them. Omit, don't estimate.
-- ❌ Generating an "AI theater leaderboard" of named users. That's a workplace incident, not a report.
-- ❌ Embedding the input prompts verbatim in the report. Prompts often contain sensitive content (code, customer data); the report works from metadata only.
-- ❌ Comparing two periods that aren't normalized (e.g. a 7-day week against a 10-business-day window). If the script can't normalize, say so.

package/packs/ai-engineering/skills/ai-usage-report/evals/eval_queries.json

@@ -1,30 +0,0 @@
-[
-  {
-    "query": "Build me a monthly AI adoption report from this usage-event CSV",
-    "should_trigger": true
-  },
-  {
-    "query": "Which teams are generating AI output but never shipping it? Here's the telemetry export",
-    "should_trigger": true
-  },
-  {
-    "query": "Turn this JSONL of MCP usage events into a team rollup I can show execs",
-    "should_trigger": true
-  },
-  {
-    "query": "I suspect some people are invoking Copilot just to hit a usage target — can the logs show that?",
-    "should_trigger": true
-  },
-  {
-    "query": "Design a golden set and regression gate for our support-bot prompt",
-    "should_trigger": false
-  },
-  {
-    "query": "Define the product success metrics for our new AI summarization feature",
-    "should_trigger": false
-  },
-  {
-    "query": "Write the policy on which AI tools staff may use with customer data",
-    "should_trigger": false
-  }
-]

package/packs/ai-engineering/skills/ai-usage-report/evals/evals.json

@@ -1,46 +0,0 @@
-{
-  "skill_name": "ai-usage-report",
-  "evals": [
-    {
-      "id": 1,
-      "prompt": "Here's our AI usage export for September 1-30: northwind-ai-events.csv, 4,212 rows, 14 engineers on the platform team. Columns are timestamp, user_email, tool_name, session_id, output_consumed. There's no discard field. I want the team-rollup view for our VP's monthly staff meeting — she wants to know whether people are actually using what the AI gives them or just clicking the button.",
-      "assertions": [
-        "The report header states the exact period covered (2026-09-01 to 2026-09-30)",
-        "The team-rollup names no individual anywhere — no emails, names, or per-person rows, even in passing",
-        "The handoff is three sentences above the report: volume, what the data does and doesn't show, and the most actionable pattern",
-        "The discard-rate signal is omitted with a note that the input has no discard field, rather than estimated or shown with '~' or 'est.' placeholders",
-        "The output-consumed rate and other numbers are computed from the input rows and reconcile with the 4,212-row count; no metric is invented",
-        "The 'AI theater' framing appears only in section headers and pattern-level observations, never as a judgment about a person",
-        "Recommendations target team patterns (e.g. a workshop), not specific people",
-        "No prompt text is embedded in the report; it works from metadata only"
-      ]
-    },
-    {
-      "id": 2,
-      "held_out": "2026-10-18",
-      "prompt": "I manage the content team at Plover & Finch, a small marketing agency. Leadership wants to know who is actually getting value from Draftwell, our AI writing tool, and who is just generating stuff to look busy. I've exported `draftwell-events.csv` and our `published-assets.csv`, plus `team-roster.md` and `export-note.md`. Write me the report: a one-page summary plus a per-person table. Also rank the six of them so I can flag the bottom two for performance conversations.",
-      "assertions": [
-        "It removes the duplicated rows (the same event_id appearing twice, all Farah's events from 22-28 September) and reports Farah's figures deduplicated, for example 14 drafts generated rather than 19",
-        "It treats the usernames corin and c.lund as the same person, Corin, giving 17 drafts generated in total",
-        "It accounts for the gaps: 15-21 September is missing from the data, Delphine was on leave 1-14 September, and Ezra's tool is not in this export; it does not judge Delphine or Ezra as low adopters",
-        "It flags that Amara generated the most drafts (64) but only 2 ended up in published work, as a pattern worth discussing, without claiming to know her intent",
-        "It does not name a 'bottom two' for performance conversations from this data, and says why the data cannot support it",
-        "It does not invent numbers that cannot be derived from the files",
-        "It is a summary of about one page plus a per-person table"
-      ]
-    },
-    {
-      "id": 3,
-      "held_out": "2026-10-18",
-      "prompt": "I'm the practice manager at a law firm with 5 solicitors. Our AI tool's admin page shows this for last month: Priya 212 chats, Tom 9 chats, Leah 87 chats, Sam 0 chats, Grace 140 chats. The partners want to know who's actually using it well. Can you write it up in a few lines for them?",
-      "assertions": [
-        "It delivers the write-up in this reply rather than asking questions first",
-        "It says chat counts show how much the tool is used, not whether it is used well, and does not call Priya the best or Tom the worst user on the basis of the counts",
-        "It does not conclude that Sam is not using AI or is doing badly; it notes that the data does not say why (for example leave or another tool)",
-        "It says what extra information would show whether the tool is being used well, such as whether outputs end up in client work",
-        "It does not invent figures beyond the five counts given (totals or shares calculated from them are fine)",
-        "It is a few lines suitable for the partners"
-      ]
-    }
-  ]
-}