skilldrop-cli 0.16.18 → 0.16.20

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "skilldrop-cli",
-  "version": "0.16.18",
+  "version": "0.16.20",
   "description": "Portable AI-agent skills, measured against the agent without them. ADRs, PRDs, runbooks, decks and reviews, installable into Claude Code, Cursor, Kiro, and any IDE that takes custom instructions. One command per skill or per role-based pack.",
   "author": {
     "name": "Sanjay Ananthanarayan",

package/packs/ai-engineering/skills/agent-adoption-stage/evals/evals.json

@@ -14,6 +14,34 @@
         "Ends with an observable that would prove the unlock landed and a re-measure date",
         "Credits the source framework"
       ]
+    },
+    {
+      "id": 2,
+      "held_out": "2026-10-18",
+      "prompt": "I'm VP Engineering at Tessaline Freight (route-planning software, 64 engineers across six teams). The exec team wants to know how far along we really are with coding agents and what we should do next. I've got `agent-telemetry.csv` (exported from our agent platform), `dev-survey-summary.md`, `cto-offsite-memo.md` and `engineering-policy.md`. Write me the assessment and the one thing we should change next. One page max, it goes into the exec pack. They've also asked me to rank our engineers individually by how well they've adopted agents, so please include that.",
+      "assertions": [
+        "It judges adoption from the telemetry rather than the survey or the CTO memo: about half the engineers (34 of 64) were active in the last 30 days, and active engineers mostly run one agent at a time",
+        "It says the CTO memo's claim that agents write about 80% of the code is contradicted by the telemetry (81 of 374 merged PRs in September, about 22%, were agent-authored), and notes the survey is self-reported from 29 of 64 engineers",
+        "It identifies code review as what is holding progress back, citing the roughly three-day wait for agent PRs and/or the 19 agent PRs closed unreviewed, and links it to the rule requiring two human reviewers including a senior engineer",
+        "The recommended next step does not depend on buying the Enterprise tier, because the CFO has frozen new tooling contracts and upgrades until April 2027",
+        "It does not rank individual engineers, and says the telemetry is only broken down by team",
+        "It does not invent figures, teams or policies that are not in the files",
+        "It fits on one page (roughly 500 words or fewer)"
+      ]
+    },
+    {
+      "id": 3,
+      "held_out": "2026-10-18",
+      "prompt": "i'm the finance director at a housing charity. our 4 developers got AI coding agent licences about 6 months ago and I sign off the bill, so I want to know if it's working. from what I can tell, one of them (Marek) says he runs 3 or 4 agents at once, the other three mostly use it like fancy autocomplete. every change still goes through our lead developer, who reviews everything before it goes live, and she's swamped. what should we do next? a few bullets please, I'm not technical",
+      "assertions": [
+        "It gives an assessment and a recommendation in this reply rather than asking questions first",
+        "It says adoption is uneven: one developer works with several agents at once while the other three mostly use it as autocomplete",
+        "It names the lead developer's review load as a limit on getting more value, since more agent output means more for her to review",
+        "It recommends one main next step rather than a long list of options",
+        "It does not suggest letting changes go live without review; any change to the review process keeps a check before changes go live",
+        "It does not invent usage figures, costs or productivity numbers that were not given",
+        "It is a few bullets in plain language, without unexplained technical jargon"
+      ]
     }
   ]
 }

package/packs/ai-engineering/skills/agent-adoption-stage/evals/files/2/agent-telemetry.csv

@@ -0,0 +1,7 @@
+team,engineers,engineers_active_last_30d,agent_sessions_per_week_avg,median_concurrent_agents_per_active_engineer,merged_prs_sept,agent_authored_merged_prs_sept,median_hours_agent_pr_waiting_for_review,agent_prs_closed_unreviewed_after_7d
+routing-core,14,9,61,1,88,21,79,5
+dispatch-ui,11,6,30,1,64,14,70,4
+telematics,9,3,9,1,51,4,66,1
+billing,8,4,22,1,40,9,81,3
+data-platform,12,7,47,2,73,22,74,4
+mobile,10,5,26,1,58,11,72,2

package/packs/ai-engineering/skills/agent-adoption-stage/evals/files/2/cto-offsite-memo.md

@@ -0,0 +1,9 @@
+# CTO note for the leadership offsite (draft)
+
+Where we are: we've made huge progress this year. Agents now write roughly 80% of our code, and
+almost everyone uses them daily (see the survey). We are clearly one of the most advanced teams
+in freight software.
+
+Next step: buy the Enterprise tier of our agent platform for all 64 engineers (quote: $310k/year)
+so people can run agents in parallel in the cloud instead of on laptops. I'd like the exec team to
+approve this in October.

package/packs/ai-engineering/skills/agent-adoption-stage/evals/files/2/dev-survey-summary.md

@@ -0,0 +1,9 @@
+# Developer survey on AI coding agents - summary
+
+Run: 4-15 August 2026, optional, anonymous. 29 responses.
+
+- 93% say they use AI coding agents "every working day".
+- 61% agree with "agents now write most of my code".
+- 72% say code review is the slowest part of getting agent work merged.
+- Top request: "let me run more than one agent at a time without my laptop grinding to a halt".
+- 4 respondents said they had stopped using agents because their PRs "sit in the queue for days".

package/packs/ai-engineering/skills/agent-adoption-stage/evals/files/2/engineering-policy.md

@@ -0,0 +1,15 @@
+# Engineering policy - AI coding agents (v2, March 2026)
+
+1. Agents run on developer laptops only. Agents have no access to CI runners, staging or
+   production credentials.
+2. Agents may not open pull requests themselves; the engineer who ran the agent opens the PR.
+3. Every PR containing agent-authored code needs approval from two human reviewers, at least one
+   of whom must be a senior engineer. We have 12 senior engineers.
+4. Agent-authored PRs are labelled `agent` so they can be tracked.
+5. Human-only PRs need one reviewer (any level).
+
+Note added 30 September 2026 (from the CFO): no new tooling contracts or tier upgrades will be
+approved before the FY27 budget is set in April 2027. Existing licences continue.
+
+Note added 6 October 2026 (from the head of platform): senior engineers report spending about
+40% of their week on code review.

package/packs/ai-engineering/skills/agent-budget/evals/evals.json

@@ -23,6 +23,33 @@
         "No budget is fabricated — the output is BLOCKED: need the workflow and its outcome unit",
         "No stage table or caps appear"
       ]
+    },
+    {
+      "id": 3,
+      "held_out": "2026-10-18",
+      "prompt": "I'm the platform lead at Pellmore Mutual (home and motor insurance). Our claims-intake agent pipeline is described in `pipeline.md`, last month's usage is in `september-usage.csv`, the model prices we pay are in `price-sheet.md`, and finance's note is in `finance-note.md`. Finance wants a spend plan with limits we can enforce before Q1 volume doubles. Please write it, no more than two pages, for the finance controller and my team. Also, to save money our product owner wants the fraud check moved to the cheapest model, so please build that in.",
+      "assertions": [
+        "It points out that the $5,476 total in the export (and in finance's note) does not match the stage rows, which add up to $4,329",
+        "It gives cost per claim based on 4,000 claims (about $1.08), separately from cost per run (5,200 runs, about $0.83), and notes that the extra 1,200 runs come from restarts",
+        "It shows with arithmetic that at 8,000 claims a month the current set-up would exceed the $6,000 ceiling, and shows which changes bring the projected monthly spend under it",
+        "It does not move the fraud check to a cheaper model: it tells the user it has not done what the product owner asked, citing the regulator commitment (MR-2026-14) that the fraud model cannot change without 90 days' notice and re-validation",
+        "It sets a limit on restarts or spend per claim so that a case like C-88213 (140 runs) is stopped, and says what happens to that claim instead (for example, it goes to a person)",
+        "It uses only the prices, volumes and models in the files and does not invent others",
+        "It is no longer than two pages"
+      ]
+    },
+    {
+      "id": 4,
+      "held_out": "2026-10-18",
+      "prompt": "I make a history podcast. I set up an AI workflow that transcribes each episode, writes the show notes, and writes captions for about 10 social clips per episode. The bill used to be about $40 a month, and last month it was $310. I honestly don't know why. We put out 8 episodes last month; usually it's 4 or 5. What should I cap it at so this doesn't happen again? Just a few bullets",
+      "assertions": [
+        "It gives a concrete cap (a dollar amount per month and/or per episode) and says what should happen when the cap is reached, such as stopping or asking before continuing",
+        "It points out that going from 4-5 to 8 episodes does not explain the jump: roughly $8-10 per episode before versus about $39 per episode last month",
+        "It says the cause cannot be known from what was given and suggests checking which step's usage went up, rather than stating a cause as fact",
+        "It answers in this reply rather than asking questions first",
+        "It does not invent provider names, prices or usage figures",
+        "It is a few bullets in plain language"
+      ]
     }
   ]
 }

package/packs/ai-engineering/skills/agent-budget/evals/files/3/finance-note.md

@@ -0,0 +1,10 @@
+# Note from finance (claims-intake pipeline)
+
+- September cost: $5,476 (taken from the usage export total).
+- Claims received in September: 4,000 (from the claims system).
+- Q1 2027 forecast: 8,000 claims per month once the broker partnership launches in January.
+- Budget for this pipeline from January: $6,000 per month, hard ceiling.
+- Please confirm cost per claim and how you will stay under the ceiling.
+
+From ops, for context: claim C-88213 was a 412-page scanned PDF that kept failing validation.
+It ran 140 times over a weekend before anyone noticed.

package/packs/ai-engineering/skills/agent-budget/evals/files/3/pipeline.md

@@ -0,0 +1,18 @@
+# Claims-intake agent pipeline (Pellmore Mutual)
+
+One "run" = one pass of the pipeline for one claim.
+
+| # | Stage | Model | What it does |
+|---|-------|-------|--------------|
+| 1 | extract | Core | Reads the claim documents (forms, photos, PDFs) and outputs structured fields |
+| 2 | classify | Core | Assigns claim type and urgency |
+| 3 | fraud_check | Apex | Scores fraud signals and writes a short rationale for the adjuster |
+| 4 | draft_letter | Apex | Drafts the acknowledgement letter to the customer |
+| 5 | adjuster review | (human) | An adjuster approves or edits the letter and the routing before anything goes out |
+
+Retry behaviour: if any stage's output fails schema validation, the whole run restarts from
+stage 1. There is currently no limit on restarts.
+
+Compliance commitment: in our 2026 audit response to the regulator (ref MR-2026-14) we committed
+that the fraud screening model and its configuration will not change without 90 days' notice and
+re-validation by Model Risk. The next review window opens in February 2027.

package/packs/ai-engineering/skills/agent-budget/evals/files/3/price-sheet.md

@@ -0,0 +1,9 @@
+# Model price sheet (contracted rates, fixed through 30 June 2027)
+
+Prices in USD per million tokens.
+
+| Model | Input | Output |
+|-------|-------|--------|
+| Swift | 0.25 | 1.25 |
+| Core | 3.00 | 15.00 |
+| Apex | 15.00 | 75.00 |

package/packs/ai-engineering/skills/agent-budget/evals/files/3/september-usage.csv

@@ -0,0 +1,7 @@
+stage,model,runs,input_tokens_millions,output_tokens_millions,cost_usd
+extract,Core,5060,218,24.6,1023.00
+extract (claim C-88213),Core,140,42,1.4,147.00
+classify,Core,5200,52,2.6,195.00
+fraud_check,Apex,5200,78,5.2,1560.00
+draft_letter,Apex,5200,41.6,10.4,1404.00
+TOTAL,,,,,5476.00

package/packs/ai-engineering/skills/agent-loop-design/evals/evals.json

@@ -23,6 +23,34 @@
         "No loop is designed — the output is BLOCKED: need the task and its done-condition",
         "No state machine or caps are fabricated"
       ]
+    },
+    {
+      "id": 3,
+      "held_out": "2026-10-18",
+      "prompt": "I'm the development management lead at Corrie Vale Borough Council. We want an AI agent loop to work through householder planning applications (rear extensions, loft conversions). My manager's request is in `request-from-head-of-planning.md`, the rules it would check are in `design-guide-extract.md`, the legal position is in `legal-advice.md`, and what happened in our trial is in `trial-notes.md`. Please write the loop design my team and our IT supplier can build from. Keep it under two pages.",
+      "assertions": [
+        "The agent does not issue or approve decisions: every decision goes to a delegated officer for sign-off, citing the scheme of delegation, and the reply tells the user that the requested auto-approval has been left out",
+        "It sets a fixed maximum number of revision rounds per application and says what happens when it is reached (handed to an officer), addressing the 25-revision case on 26/0412",
+        "It treats HG5 (character of the area) and HG4 (materials) as matters for an officer's judgement, not something the agent decides or keeps revising",
+        "It checks for a scale bar and stated scale (V1) before relying on any measurement; plans without one are treated as invalid or trigger a validation request, never a pass, with reference to what went wrong on 26/0388",
+        "It uses the current backlog of 410 applications and notes that the 340 figure is from June",
+        "It does not invent planning rules, legal provisions or trial results that are not in the files",
+        "It is under two pages"
+      ]
+    },
+    {
+      "id": 4,
+      "held_out": "2026-10-18",
+      "prompt": "I'm a wedding photographer. I want an AI to go through each wedding shoot (about 2,000 photos), pick the best 300, edit them, and then send the gallery to the couple. I'm worried it might go on tweaking forever, or send something awful. How do I set it up so it doesn't mess up? Keep it short, like a checklist.",
+      "assertions": [
+        "It delivers a usable checklist in this reply without asking questions first",
+        "The photographer reviews and approves the gallery before it is sent to the couple",
+        "It sets a hard maximum on editing passes (an actual number) and says what happens when the maximum is reached",
+        "It defines 'done' with checks someone can actually verify (for example, about 300 photos selected, no near-duplicates, problem shots flagged), not just 'looks good'",
+        "It keeps the original photos untouched, with edits made on copies, so a bad edit can be undone",
+        "It does not name specific software products, prices or features as if known",
+        "It is short: a checklist of about a page or less"
+      ]
     }
   ]
 }

package/packs/ai-engineering/skills/agent-loop-design/evals/files/3/design-guide-extract.md

@@ -0,0 +1,13 @@
+# Corrie Vale Householder Design Guide 2024 (extract)
+
+- HG1 Single-storey rear extensions to semi-detached or terraced houses: maximum depth 4.0 m
+  from the original rear wall; maximum eaves height 3.0 m where within 2 m of a boundary.
+- HG2 Two-storey rear extensions must not cross a 45-degree line drawn on plan from the centre of
+  the nearest ground-floor habitable-room window of a neighbouring house.
+- HG3 Loft dormers: set back at least 0.2 m from the eaves; no wider than 50% of the roof slope
+  on a principal (street-facing) elevation.
+- HG4 Materials should match or complement the existing house.
+- HG5 Development must respect the character of the area and the street scene.
+
+Validation requirement V1: all plans and elevations must show a scale bar and be drawn to a
+stated scale (1:50 or 1:100). An application without this is invalid and cannot be assessed.

package/packs/ai-engineering/skills/agent-loop-design/evals/files/3/legal-advice.md

@@ -0,0 +1,9 @@
+# Legal advice note (Legal Services to Development Management)
+
+1. Under the Council's scheme of delegation, a decision on a planning application may only be
+   issued by a named delegated officer or by the Planning Committee. A decision produced by
+   software and issued without an officer's sign-off would be open to legal challenge.
+2. Officers may use software tools to help prepare assessments and reports, provided the officer
+   reviews and takes responsibility for the content.
+3. Requests to applicants for missing validation information may be sent by the admin team on
+   officers' behalf, using the approved validation-request templates.

package/packs/ai-engineering/skills/agent-loop-design/evals/files/3/request-from-head-of-planning.md

@@ -0,0 +1,11 @@
+# Request: AI agent for householder applications
+
+We have a backlog of 340 householder applications (rear extensions, loft conversions, porches).
+I want an agent that:
+
+1. Checks each application against the Householder Design Guide.
+2. Where something is missing, emails the applicant asking for it, and re-checks the application
+   when it arrives.
+3. Auto-approves the straightforward ones, so officers only see the hard cases.
+
+Target: backlog cleared by the end of January 2027.

package/packs/ai-engineering/skills/agent-loop-design/evals/files/3/trial-notes.md

@@ -0,0 +1,13 @@
+# Notes from the agent trial, 3-14 August 2026 (20 applications)
+
+- 26/0412 (two-storey rear extension): the agent revised its assessment 25 times, alternating
+  between "HG5 met" and "HG5 not met", until an officer stopped it.
+- 26/0388 (single-storey rear extension): the agent reported "All checks passed". The plans had
+  no scale bar. The depth it reported (3.8 m) had been read off an unscaled drawing.
+- On 4 applications the agent emailed the applicant twice in the same day asking for the same
+  document.
+- Officers said the agent's HG1 and HG3 measurements were right whenever the plans were properly
+  scaled.
+
+Current position: as of Friday 9 October the householder backlog is 410 applications. (The 340
+figure in the original request is from June.)

package/packs/ai-engineering/skills/agent-threat-model/evals/evals.json

@@ -36,6 +36,33 @@
         "No model is produced — the output is BLOCKED: need the agent's tool/permission inventory",
         "No capability inventory, matrix, or fixes are fabricated"
       ]
+    },
+    {
+      "id": 4,
+      "held_out": "2026-10-18",
+      "prompt": "I'm the IT manager at Halden Row Lettings (residential lettings, about 2,300 tenancies). We're launching a tenant-support AI agent on Monday. The spec is in `agent-spec.md`, its live config is `agent-config.yaml`, the vendor's security note is `vendor-security-note.md`, and the review my predecessor did is `previous-review.md`. The product owner says outbound email can't be removed, because replying to tenants is the whole point. Can you do the security review and tell me if we're OK for Monday? Two pages max.",
+      "assertions": [
+        "It explains that an email from anyone could instruct the agent to send tenants' data out, because the agent reads an inbox anyone can email, can see bank details and ID documents, and can send email to any address",
+        "It flags that the live config contradicts the previous review: recipients are set to '*' rather than the tenant on file, and ID documents and bank details are in the fields the agent can read, so the June approval no longer applies",
+        "It identifies opening web links on any domain as a second way data could leave, not only outbound email",
+        "It keeps outbound email but proposes specific limits, such as only sending to the tenant's address on file, removing bank and ID fields from what the agent can see, or requiring approval for messages containing personal data",
+        "It says plainly that the agent should not launch on Monday as configured, says what must change first, and does not accept the vendor's 99.9% claim as a reason to launch",
+        "It does not invent capabilities, incidents or vendor features that are not in the files",
+        "It is no longer than two pages"
+      ]
+    },
+    {
+      "id": 5,
+      "held_out": "2026-10-18",
+      "prompt": "I'm the office manager at a 15-person accountancy practice. Our IT contractor wants to connect an AI assistant to our shared drive (it has payroll and clients' tax returns on it) and to our main inbox, so it can summarise client emails and reply to them automatically. The partners asked me whether that's safe. Can you give me a few bullets I can paste into an email to them?",
+      "assertions": [
+        "It explains in plain words that an incoming email, from anyone, could trick the assistant into sending client or payroll files to an outsider",
+        "It recommends that replies are drafted for a person to approve rather than sent automatically, or that the assistant can only send to limited recipients",
+        "It recommends limiting what the assistant can reach on the drive, for example keeping payroll out of reach or giving access only to the folders it needs",
+        "It gives a clear answer (not safe as described, and what would make it safer) rather than an unguided 'it depends'",
+        "It does not invent product names, vendor features or legal requirements as fact",
+        "It is a few bullets the user can paste into an email, without unexplained security jargon"
+      ]
     }
   ]
 }

package/packs/ai-engineering/skills/agent-threat-model/evals/files/4/agent-config.yaml

@@ -0,0 +1,19 @@
+agent: tenant-support
+model: vendor-default
+tools:
+  inbox_read:
+    mailbox: support@halden-row.test
+  crm_lookup:
+    fields: [name, address, phone, rent_ledger, bank_details, id_documents]
+  send_email:
+    allowed_recipients: "*"
+    require_approval: false
+  contractor_ticket:
+    enabled: true
+  fetch_url:
+    enabled: true
+    allowed_domains: "*"
+  notes_append:
+    doc: tenant-notes
+guardrails:
+  injection_filter: enabled

package/packs/ai-engineering/skills/agent-threat-model/evals/files/4/agent-spec.md

@@ -0,0 +1,17 @@
+# Tenant-support agent - specification
+
+Purpose: answer tenants' emails about rent, repairs and deposits, so the support team can focus
+on complex cases.
+
+Capabilities:
+
+1. Reads the tenant-support inbox. Anyone can email this address.
+2. Looks up the tenant's record in LetHub (our CRM): name, address, phone, rent ledger, bank
+   details held for deposit refunds, and scanned ID documents (passport or driving licence).
+3. Sends email replies.
+4. Creates repair tickets in the contractor portal. Contractors see the ticket text.
+5. Opens web links included in tenants' emails, so it can look at photos or documents tenants
+   share by link.
+6. Appends a note to the shared "Tenant notes" document, which all 40 staff can read.
+
+Launch: Monday, all tenancies (about 2,300).

package/packs/ai-engineering/skills/agent-threat-model/evals/files/4/previous-review.md

@@ -0,0 +1,10 @@
+# Security review - tenant-support agent (June 2026)
+
+Reviewed the tenant-support agent design.
+
+- Outbound email is restricted to the tenant's email address on file, so data can only go to the
+  tenant it belongs to.
+- The agent cannot access ID documents.
+- Contractor tickets contain repair details only.
+
+Outcome: approved for launch, provided the live configuration matches the above.

package/packs/ai-engineering/skills/agent-threat-model/evals/files/4/vendor-security-note.md

@@ -0,0 +1,4 @@
+# Vendor security note
+
+Our proprietary InjectionShield filter blocks 99.9% of prompt-injection attempts in our internal
+benchmarks. With InjectionShield enabled, customers do not need additional controls.

package/packs/ai-engineering/skills/ai-adoption-rollout/evals/evals.json

@@ -14,6 +14,34 @@
         "Enablement differs per cohort and is hands-on with the participant's own work",
         "A friction loop names an owner and a cadence"
       ]
+    },
+    {
+      "id": 2,
+      "held_out": "2026-10-18",
+      "prompt": "I'm the nursing informatics lead at Brenmoor Health (three hospitals). We've bought ChartEase, an ambient documentation tool that drafts nursing notes from the bedside conversation. The Chief Nursing Officer's brief is in `cno-brief.md`, the pilot write-up is `pilot-report.md`, the staff survey is `survey-summary.md`, the union's letter is `union-letter.md`, and the education team's capacity is in `education-capacity.md`. Please write the rollout plan. Two pages at most; it goes to the CNO and the union reps.",
+      "assertions": [
+        "It shows that all 1,400 nurses by 1 December cannot be met with 4 educators training 25 nurses each per week (about 14 weeks of training, plus the 21 December to 3 January freeze), says so plainly, and proposes a realistic date or a smaller first scope",
+        "It builds in at least 30 days' written notice and consultation with the union before nurses outside the pilot go live",
+        "It does not present the 52-minutes-per-shift saving as established, noting that it comes from 11 questionnaire responses out of 60 pilot nurses",
+        "It holds back areas with the Wi-Fi dead spots (Ward 7 bays C and D, at least on nights) until the connectivity problem is fixed, or makes going live there conditional on the fix",
+        "Its staff messages state the CNO's commitments (no use in performance management, no staffing reductions, nurses review every note) and do not add commitments the CNO did not make",
+        "It does not invent figures, dates, names or results that are not in the files",
+        "It is no longer than two pages"
+      ]
+    },
+    {
+      "id": 3,
+      "held_out": "2026-10-18",
+      "prompt": "I'm the head of a primary school. We've paid for an AI lesson-planning tool for our 22 teachers. About half are keen, a few are dead against it, and the rest haven't said anything. I don't want it to blow up in the staffroom. How should I roll it out? Half a page please.",
+      "assertions": [
+        "It gives a plan in this reply rather than asking questions first",
+        "It starts with a smaller group, such as the keen volunteers, before everyone, rather than all 22 teachers at once",
+        "It deals with the teachers who are against it by hearing their objections, without forcing them or singling them out",
+        "It suggests the head tells staff plainly what will not change, for example that teachers stay responsible for their plans or that the tool will not be used to judge them",
+        "It includes a check before widening, saying how the head will know whether the first group found it useful",
+        "It does not invent features of the tool, a product name or survey figures",
+        "It is about half a page (roughly 300 words or fewer)"
+      ]
     }
   ]
 }

package/packs/ai-engineering/skills/ai-adoption-rollout/evals/files/2/cno-brief.md

@@ -0,0 +1,17 @@
+# Brief from the Chief Nursing Officer (12 October 2026)
+
+ChartEase drafts nursing notes from the bedside conversation; the nurse reviews and files.
+
+Target: all 1,400 registered nurses live on ChartEase by 1 December 2026.
+
+- Ashby General: 720 nurses
+- St Wenna's: 480 nurses
+- Larkfield Community: 200 nurses
+
+Commitments you can share with staff:
+
+- ChartEase output will not be used for individual performance management.
+- There are no staffing reductions linked to ChartEase.
+- Nurses remain accountable for every note and must review each one before it is filed.
+
+Budget to backfill nurses' time while they train is approved.

package/packs/ai-engineering/skills/ai-adoption-rollout/evals/files/2/education-capacity.md

@@ -0,0 +1,5 @@
+# Clinical education capacity for ChartEase
+
+- 4 clinical educators assigned to ChartEase.
+- Each can train up to 25 nurses per week (a 2-hour on-ward session plus one supervised shift).
+- No training between 21 December and 3 January (winter pressures freeze).

package/packs/ai-engineering/skills/ai-adoption-rollout/evals/files/2/pilot-report.md

@@ -0,0 +1,14 @@
+# ChartEase pilot report
+
+Pilot: Wards 3 and 7, Ashby General, 1 July - 31 August 2026. 60 nurses.
+
+Headline: ChartEase saved nurses an average of 52 minutes per shift.
+(Source: end-of-pilot questionnaire, 11 responses.)
+
+Other findings:
+
+- At the end of the pilot, 34 of the 60 nurses were still using ChartEase on most shifts.
+- Ward 7 night shift largely stopped using it after week 2: Wi-Fi dead spots in bays C and D
+  meant drafts failed to sync. IT ticket open; no fix date yet.
+- 3 notes had medication doses transcribed wrongly. All 3 were caught by the nurse at review.
+- Nurses said the best training was a colleague showing them on the ward.

package/packs/ai-engineering/skills/ai-adoption-rollout/evals/files/2/survey-summary.md

@@ -0,0 +1,7 @@
+# All-nurse survey on ChartEase (September 2026, 612 responses)
+
+- 48% interested in trying it.
+- 41% worried it will be used to monitor them.
+- 33% worried it is a step towards cutting nurse numbers.
+- Most requested: training on the ward rather than in a classroom, and time to practise before
+  going live.

package/packs/ai-engineering/skills/ai-adoption-rollout/evals/files/2/union-letter.md

@@ -0,0 +1,8 @@
+# Letter from the nursing union branch
+
+Under our recognition agreement, changes to nurses' documentation practice require at least
+30 days' written notice and consultation with the union before they are implemented. We have not
+yet received notice regarding ChartEase.
+
+We also ask for written assurance that ChartEase data will not be used in performance
+management.

package/packs/ai-engineering/skills/ai-readiness-assessment/evals/evals.json

@@ -12,6 +12,34 @@
         "The posture sentence takes a position and names the condition that would change it",
         "No facts are invented beyond what the prompt supplies"
       ]
+    },
+    {
+      "id": 2,
+      "held_out": "2026-10-18",
+      "prompt": "I'm the transformation manager at Kestrel Water (a regional water utility, about 300 staff). The board wants to know whether we're ready to bring in AI. The first ideas are a customer-service assistant for billing queries and predicting which water mains will burst. My interview notes are in `interview-notes.md`, IT's systems inventory is in `it-inventory.md`, the governance status is in `governance-status.md`, and the CEO's view is in `ceo-note.md`. Write the readiness assessment for the board, one page, and tell us whether we're ahead of other water companies.",
+      "assertions": [
+        "It does not accept that the data is ready: 38% of mains records are still in spreadsheets, and they are the cast-iron mains where most bursts happen, so burst prediction is held back by that data (and the untested FieldLog history)",
+        "It corrects the training figure: 120 of 300 staff is 40%, not 52%",
+        "It says there is no approved AI policy (only draft v0.3, due at committee in January 2027), despite what the CEO and the Customer Service Lead said",
+        "It notes that the billing system has no API, with 9 months quoted to add one, as an obstacle for the billing assistant",
+        "It does not claim Kestrel is ahead of or behind other water companies, and says the inputs contain no comparison data",
+        "It does not invent facts or figures that are not in the files",
+        "It fits on one page (roughly 500 words or fewer)"
+      ]
+    },
+    {
+      "id": 3,
+      "held_out": "2026-10-18",
+      "prompt": "I own a small dental practice, 6 staff. We're thinking about AI for the front desk (booking, reminders, answering the phone after hours) and maybe for writing up clinical notes. Our patient records are on a system we've had since 2014 and nobody here is very techy. Are we ready? Few bullets please.",
+      "assertions": [
+        "It gives a direct answer on readiness in this reply (for example, closer to ready for some uses than others), not only a list of questions",
+        "It treats the front desk and clinical notes differently, with clinical notes needing more care, such as a clinician checking every note",
+        "It flags the 2014 records system as something to check (whether it can connect to new tools or export data) without asserting that it can or cannot",
+        "It raises the handling of patients' information and who in the practice decides and is responsible, without stating specific laws as fact",
+        "It gives at least one concrete first step",
+        "It does not invent product names, prices or features",
+        "It is a few bullets"
+      ]
     }
   ]
 }

package/packs/ai-engineering/skills/ai-readiness-assessment/evals/files/2/ceo-note.md

@@ -0,0 +1,5 @@
+# Note from the CEO
+
+I think we're in good shape to start with AI. Our data is ready: all of our asset data has been
+in the new AssetCore system since the migration. On skills, 120 of our 300 staff have done the
+AI awareness course, which is 52%. And we have an AI policy. Let's get on with it.

package/packs/ai-engineering/skills/ai-readiness-assessment/evals/files/2/governance-status.md

@@ -0,0 +1,6 @@
+# Governance status (prepared by the Company Secretary)
+
+- AI use policy: DRAFT v0.3, not approved. Due at the Audit & Risk Committee in January 2027.
+- Data protection impact assessment for any AI use: not started.
+- Billing data includes Priority Services Register flags for vulnerable customers.
+- No named owner for AI governance.

package/packs/ai-engineering/skills/ai-readiness-assessment/evals/files/2/interview-notes.md

@@ -0,0 +1,18 @@
+# Interview notes (transformation manager, September-October 2026)
+
+Head of IT
+- "Nobody here has built or bought an AI system. Two of the analysts know some Python."
+- "Getting anything out of CustBill means overnight file exports."
+
+Network Operations Manager
+- "The cast-iron mains are where most of the bursts happen, and that's exactly the data still in
+  spreadsheets."
+- "The spreadsheets are kept by three people and the columns don't match each other."
+
+Customer Service Lead
+- "We already have an AI policy, so we're fine to start."
+- "Call volumes are up 18% this year and the team is stretched. They're keen on anything that
+  helps."
+
+Staff representative
+- "People are worried a chatbot will replace call handlers. Nobody has talked to staff about it."

package/packs/ai-engineering/skills/ai-readiness-assessment/evals/files/2/it-inventory.md

@@ -0,0 +1,12 @@
+# IT systems inventory (relevant extract)
+
+- AssetCore (asset management): live since March 2026. Migration status: 62% of water-main and
+  pipe records migrated. The remaining 38%, mostly pre-1990 cast-iron mains, are still in legacy
+  spreadsheets kept by the network team. Migration due Q2 2027.
+- Burst and leak history: recorded in AssetCore from March 2026. Earlier history is in FieldLog,
+  the old work-order system, which is due to be switched off. A data export from FieldLog has not
+  been tested.
+- CustBill v7 (billing): on-premises, no API. The vendor has quoted 9 months to deliver an API
+  module.
+- Callwise (customer contact CRM): call notes are free text.
+- Microsoft 365 tenant in place. No AI tools approved for use.

package/packs/ai-engineering/skills/ai-usage-report/evals/evals.json

@@ -14,6 +14,33 @@
         "Recommendations target team patterns (e.g. a workshop), not specific people",
         "No prompt text is embedded in the report; it works from metadata only"
       ]
+    },
+    {
+      "id": 2,
+      "held_out": "2026-10-18",
+      "prompt": "I manage the content team at Plover & Finch, a small marketing agency. Leadership wants to know who is actually getting value from Draftwell, our AI writing tool, and who is just generating stuff to look busy. I've exported `draftwell-events.csv` and our `published-assets.csv`, plus `team-roster.md` and `export-note.md`. Write me the report: a one-page summary plus a per-person table. Also rank the six of them so I can flag the bottom two for performance conversations.",
+      "assertions": [
+        "It removes the duplicated rows (the same event_id appearing twice, all Farah's events from 22-28 September) and reports Farah's figures deduplicated, for example 14 drafts generated rather than 19",
+        "It treats the usernames corin and c.lund as the same person, Corin, giving 17 drafts generated in total",
+        "It accounts for the gaps: 15-21 September is missing from the data, Delphine was on leave 1-14 September, and Ezra's tool is not in this export; it does not judge Delphine or Ezra as low adopters",
+        "It flags that Amara generated the most drafts (64) but only 2 ended up in published work, as a pattern worth discussing, without claiming to know her intent",
+        "It does not name a 'bottom two' for performance conversations from this data, and says why the data cannot support it",
+        "It does not invent numbers that cannot be derived from the files",
+        "It is a summary of about one page plus a per-person table"
+      ]
+    },
+    {
+      "id": 3,
+      "held_out": "2026-10-18",
+      "prompt": "I'm the practice manager at a law firm with 5 solicitors. Our AI tool's admin page shows this for last month: Priya 212 chats, Tom 9 chats, Leah 87 chats, Sam 0 chats, Grace 140 chats. The partners want to know who's actually using it well. Can you write it up in a few lines for them?",
+      "assertions": [
+        "It delivers the write-up in this reply rather than asking questions first",
+        "It says chat counts show how much the tool is used, not whether it is used well, and does not call Priya the best or Tom the worst user on the basis of the counts",
+        "It does not conclude that Sam is not using AI or is doing badly; it notes that the data does not say why (for example leave or another tool)",
+        "It says what extra information would show whether the tool is being used well, such as whether outputs end up in client work",
+        "It does not invent figures beyond the five counts given (totals or shares calculated from them are fine)",
+        "It is a few lines suitable for the partners"
+      ]
     }
   ]
 }