skilldrop-cli 0.16.18 → 0.16.20

package/packs/api-builder/skills/prompt-caching-advisor/evals/evals.json

@@ -14,6 +14,31 @@
         "Gives a cache strategy summary paragraph with what is cached, why, expected hit rate, and the dominant cost driver before and after",
         "Does not invent prompt sections, request volumes, or models beyond those in the prompt"
       ]
+    },
+    {
+      "id": 2,
+      "held_out": "2026-10-18",
+      "prompt": "Our nursing-school tutor builds every Claude request with build_request.py, and notes.md has our usage and what the team wants. Where should the cache breakpoints go? Send back a corrected build_request function and a short note explaining the changes, under 250 words.",
+      "assertions": [
+        "Identifies that the first system block holds the user's name and role plus a timestamp, which change on every request and sit at the very start, so nothing after them can be reused; the fix moves them after the cached content or out of the cached prefix",
+        "Notes that the tools list differs by role and comes before the system prompt in the cached prefix, so students and instructors get separate cache entries, or recommends one stable tools list",
+        "Does not adopt the dev lead's breakpoint-on-everything plan; the corrected function uses no more than four breakpoints and the note gives the per-request limit as the reason",
+        "Puts a breakpoint after the shared static content (handbook and few-shot examples) and one on the latest conversation turn, or the case study, so each 12-turn session reuses its earlier turns",
+        "Does not give an exact monthly saving, since the files give no daily request volume or prices; any figure is tied to stated assumptions",
+        "Provides a corrected build_request, and the note is under 250 words"
+      ]
+    },
+    {
+      "id": 3,
+      "held_out": "2026-10-18",
+      "prompt": "I use Claude through a little script my son set up to help edit my novel. Every time, it sends the whole 300-page manuscript plus my style guide, and then one question about a chapter. The bill is huge. Someone said 'prompt caching'. What do I change? Two or three sentences, I'm not technical.",
+      "assertions": [
+        "Says to keep the unchanging part (style guide and manuscript) at the start of every request and mark it to be cached, with the changing question after it",
+        "Warns that editing the manuscript between questions means the cached copy no longer matches and has to be rebuilt, or that the cache expires after a few minutes unused, so it pays to ask several questions in one sitting on an unchanged draft",
+        "Does not invent a dollar figure or percentage saving for their bill",
+        "Uses two or three sentences in plain language",
+        "Answers without first asking questions"
+      ]
     }
   ]
 }

package/packs/api-builder/skills/prompt-caching-advisor/evals/files/2/build_request.py

@@ -0,0 +1,40 @@
+"""Builds each Claude API request for Calder Tutor, the study assistant at the Calder School of
+Nursing (invented). Called once per student or instructor turn."""
+import datetime
+
+from tutor.config import MODEL
+from tutor.content import FEW_SHOT, load_case_study, load_handbook
+from tutor.tools import INSTRUCTOR_TOOLS, STUDENT_TOOLS
+
+
+def build_request(user, session, question):
+    system = [
+        {
+            "type": "text",
+            "text": (
+                f"You are Calder Tutor. Current user: {user.name} ({user.role}). "
+                f"Request time: {datetime.datetime.now().isoformat()}."
+            ),
+        },
+        # Clinical skills handbook, about 110,000 characters. Revised once per term.
+        {"type": "text", "text": load_handbook()},
+        # Eight worked question-and-answer examples, about 14,000 characters. Fixed.
+        {"type": "text", "text": FEW_SHOT},
+    ]
+
+    # Students get 3 tools; instructors get those 3 plus 2 grading tools.
+    tools = INSTRUCTOR_TOOLS if user.role == "instructor" else STUDENT_TOOLS
+
+    # The student picks one patient case study (about 6,000 characters) at the start of a session.
+    # It is sent as the first user message and stays for the whole session.
+    messages = [{"role": "user", "content": load_case_study(session.case_id)}]
+    messages += session.history  # all earlier turns, user and assistant, in order
+    messages.append({"role": "user", "content": question})
+
+    return {
+        "model": MODEL,
+        "max_tokens": 1024,
+        "system": system,
+        "tools": tools,
+        "messages": messages,
+    }

package/packs/api-builder/skills/prompt-caching-advisor/evals/files/2/notes.md

@@ -0,0 +1,10 @@
+# Calder Tutor: usage and the caching ask
+
+- About 1,800 students and 60 instructors use the tutor.
+- A typical session is 12 turns, with a turn every 1 to 3 minutes. Sessions are spread through the
+  day; there are long gaps overnight.
+- The handbook changes once per term. The few-shot examples have not changed since launch.
+- Our dev lead's plan: put a cache_control breakpoint on every block — the user line, the
+  handbook, the few-shot examples, the tools, the case study, and every user turn — "so nothing is
+  ever reprocessed".
+- Finance wants the exact monthly saving from caching before we ship it.

package/packs/api-builder/skills/token-budget-estimator/evals/evals.json

@@ -13,6 +13,33 @@
         "The rough cost estimate states its pricing-tier assumption rather than guessing silently",
         "Does not invent workflow components, session counts, or payload sizes not in the prompt"
       ]
+    },
+    {
+      "id": 2,
+      "held_out": "2026-10-18",
+      "prompt": "We're planning the grant-screening workflow in workflow.md, and pilot-usage.csv has the token usage from our pilot. Estimate the token budget and the yearly cost, and tell me whether we're inside finance's cap. One page I can send to finance.",
+      "assertions": [
+        "Catches that 1,200 applications a round for three rounds is 3,600 a year, not 4,800, and bases the yearly figure on 3,600 (or shows both and flags the error)",
+        "Notes that the pilot's scoring stage used the 2,000-character draft rubric, so its measured stage 2 input understates the final 8,000-character rubric (by roughly 1,500 tokens on every model call in the loop), and adjusts for it or flags it",
+        "Identifies the scoring stage, on the large model with the history re-sent on every tool-call turn, as the dominant cost",
+        "Uses the supplied planning prices, giving roughly $0.07 to $0.13 per application and roughly $250 to $470 a year at 3,600 applications",
+        "States whether the estimate fits the $400 annual cap, consistent with its own figures",
+        "Does not invent prices, volumes, stages or tool calls that are not in the files",
+        "Fits about one page"
+      ]
+    },
+    {
+      "id": 3,
+      "held_out": "2026-10-18",
+      "prompt": "Our school district wants to use an AI to draft report card comments. 600 students, each gets a comment of about 80 words; the teacher pastes in a few notes per student (maybe 50 words), and there's one page of instructions that goes with every request. Roughly how many tokens and how much money is that? Just a few bullets.",
+      "assertions": [
+        "Gives a rough total for the 600 comments that is consistent with the inputs: on the order of a few hundred thousand tokens in total, not millions",
+        "Points out that the page of instructions, sent with every request, makes up most of the input",
+        "Any cost figure states the price it assumes and is consistent with that price, coming to a few dollars rather than hundreds",
+        "Does not invent extra requirements such as several terms, rewrites or a number of teachers, or marks any such thing clearly as an assumption",
+        "Is a few short bullets in plain language",
+        "Answers without first asking questions"
+      ]
     }
   ]
 }

package/packs/api-builder/skills/token-budget-estimator/evals/files/2/pilot-usage.csv

@@ -0,0 +1,17 @@
+# Pilot run, September 2026, 5 applications. Stage 2 used the 2,000-character draft rubric, not the final 8,000-character rubric.
+application,stage,input_tokens,output_tokens
+A-101,extract,5180,410
+A-101,score,17420,790
+A-101,summary,4860,395
+A-102,extract,4950,388
+A-102,score,17050,812
+A-102,summary,4700,402
+A-103,extract,15820,455
+A-103,score,18300,840
+A-103,summary,15400,410
+A-104,extract,5400,402
+A-104,score,17900,775
+A-104,summary,5050,388
+A-105,extract,5010,396
+A-105,score,16880,801
+A-105,summary,4790,399

package/packs/api-builder/skills/token-budget-estimator/evals/files/2/workflow.md

@@ -0,0 +1,42 @@
+# Grant application screening: planned Claude workflow
+
+Merrow Arts Foundation (invented). Each application arrives as a PDF, is converted to plain text,
+then goes through three stages.
+
+## Stage 1: extract (small model)
+
+- Input: extraction instructions (about 3,000 characters) plus the full application text.
+  Applications average about 18,000 characters; the largest we have seen was about 60,000.
+- Output: a JSON record of the applicant, project, amount and dates, about 1,200 characters.
+- max_tokens is currently set to 4,096.
+
+## Stage 2: score (large model, agent loop)
+
+- System prompt: scoring instructions (about 4,000 characters) plus the scoring rubric
+  (about 8,000 characters).
+- First user message: the stage 1 JSON record.
+- The model calls the check_eligibility tool about 4 times per application. Each tool result is
+  about 2,000 characters of JSON. Each tool-call turn produces about 150 output tokens.
+- The full conversation history is kept and re-sent on every turn.
+- The final turn returns a score and justification of about 600 characters.
+
+## Stage 3: summary (small model)
+
+- Input: summary instructions (about 1,000 characters), the stage 2 score and justification, and
+  the full application text again.
+- Output: a 300-word summary for the grants panel.
+- max_tokens is currently set to 4,096.
+
+## Volume
+
+About 1,200 applications per funding round, three rounds a year, so about 4,800 applications a
+year.
+
+## Planning prices (from finance, per million tokens)
+
+| Model | Input | Output |
+|-------|-------|--------|
+| small | $0.80 | $4.00  |
+| large | $3.00 | $15.00 |
+
+Finance has approved a cap of $400 a year for this workflow.

package/packs/career/pack.json

@@ -1,5 +1,5 @@
 {
-  "description": "One job application, for any occupation: an honest check of your resume against the posting, a tailored resume and a cover letter that say nothing you didn't, the application form's answers drafted from your resume, and a script that lists every figure and name in a draft that your own resume doesn't contain.",
+  "description": "Job applications, for any occupation: a shortlist of which of an employer's roles to go for, an honest check of your resume against the posting, a tailored resume and a cover letter that say nothing you didn't, the application form's answers drafted from your resume, and a script that lists every figure and name in a draft that your own resume doesn't contain.",
   "display_name": "Job applications",
   "keywords": [
     "resume",

package/packs/career/skills/job-fit-analysis/SKILL.md

@@ -136,10 +136,11 @@ posting**, ask for the posting in one line: there is nothing to compare against.
 
 - ✅ Deciding whether a posting is worth applying for
 - ✅ Finding the gaps between a resume and a posting before tailoring the resume
-- ✅ Checking several postings against one resume to choose where to spend effort
+- ✅ Checking two or three postings against one resume to choose where to spend effort
 
 ## When NOT to use this skill
 
+- ❌ Sorting a long list of openings into which to apply for. Use `role-shortlist`, then come back for the first choice.
 - ❌ Rewriting the resume for the posting. Use `resume-tailor`.
 - ❌ Writing the cover letter. Use `cover-letter`.
 - ❌ Critiquing the resume as a document, with no job in mind. Use `doc-critique`.

package/packs/career/skills/job-fit-analysis/manifest.json

@@ -1,6 +1,6 @@
 {
   "name": "job-fit-analysis",
-  "version": "0.1.2",
+  "version": "0.1.3",
   "description": "Compare a job posting with a resume and say honestly whether to apply: each requirement marked met, partly met, not shown or not met, with the resume line it rests on; the requirements an employer screens on listed first; and the questions whose answers would change the picture. Works for any occupation and level. Use when the user asks \"am I a good fit for this job\", \"should I apply for this\", \"how does my resume match this posting\", or wants the gaps between a job description and their CV before applying.",
   "entrypoint": "SKILL.md",
   "deps": {
@@ -14,7 +14,8 @@
   "related": [
     "cover-letter",
     "doc-critique",
-    "resume-tailor"
+    "resume-tailor",
+    "role-shortlist"
   ],
   "tags": [
     "job-search",

package/packs/career/skills/role-shortlist/SKILL.md

@@ -0,0 +1,107 @@
+---
+name: role-shortlist
+description: Read every open role on an employer's careers page, a job-board export or a folder of saved postings, and say which ones one person should apply for — each role judged only from its own posting text, roles that couldn't be read listed as unread instead of guessed from their titles, a role whose screening requirement the person doesn't meet named as out whatever else fits, and the same role posted twice counted once. Works for any occupation. Use when the user asks "which of these jobs should I apply for", "go through this company's openings", "shortlist these postings against my CV", or has more postings than they want to check one by one.
+---
+
+# role-shortlist
+
+**The rule: a role is judged from its posting, or not at all.** A careers page lists forty
+titles. The agent's habit is to rank them from the titles, the department and what such jobs
+usually ask for, and a ranking built that way looks exactly like one built from the postings.
+"Senior Care Coordinator" may require a nursing registration; "Data Analyst" may be a night
+shift in another country. So each role is ranked only after its own posting text has been
+read. A role whose posting you couldn't open, or that is only a title in a list, goes under
+**Not read**, with no score, no guess and no reason to apply.
+
+Two things follow from the rule:
+
+- **A screening requirement the person doesn't meet takes a role out.** A licence or
+  registration, a degree the posting says is required, a minimum number of years, the right
+  to work in a country, a location or shift, a clearance, a language. One of those unmet
+  outweighs everything else that fits, so the role goes under **Skip** with that requirement
+  named, however good the rest looks.
+- **One role is one role.** The same job posted in three cities, or reposted with a new
+  reference number, is one entry with its locations listed. A posting marked closed, or past
+  its closing date, is noted and not ranked.
+
+## How to respond
+
+1. **Gather the postings.** Use what you were given: saved pages, a pasted list, an export, a
+   folder of files. When you were given a careers-page link and can open it, read the listing
+   and then each posting. When you can't open something, say so once and ask for the text or
+   saved pages; never fill it in from what you know about the employer.
+
+2. **Read the person's evidence.** Their resume, and any other file of theirs that sits beside
+   it or that they point to. Nothing is credited because people in that line of work usually
+   have it. Note any constraint they stated themselves: where they can work, hours, salary,
+   the kind of role they want.
+
+3. **Check each posting for what could screen them out first.** For every role, find the
+   requirements an employer checks before reading the rest, and mark whether the person's
+   material meets them. Any unmet one sends the role to **Skip**, named in a few words:
+   "requires HCPC registration".
+
+4. **Rank the rest from the posting's own requirements.** For each remaining role, compare
+   the required and preferred items with the person's evidence, and say in one line why it
+   fits and what is missing. Use the posting's wording for requirements and the person's for
+   evidence. Similar is not the same: a named system, qualification or setting the posting
+   requires is met only if the person's material names it.
+
+5. **Give the shortlist, answer first.** Open with the two or three roles worth applying for
+   and why, in a sentence or two. Then:
+   - **Apply**: roles where nothing screening is unmet and most of what's required is met.
+   - **Maybe**: roles worth applying for if one named question has the right answer ("if your
+     forklift licence is current").
+   - **Skip**: each with the requirement that rules it out.
+   - **Not read**: each with why (link wouldn't open, title only, page was empty).
+   - **Closed or duplicate**: one line each.
+
+   One line per role. Every role you were given appears exactly once, so the person can see
+   nothing was dropped.
+
+6. **Close with what to do next**, in two or three lines: which role to check in full first,
+   the one or two questions whose answers would move a role between groups, and which unread
+   postings to fetch.
+
+7. **Fit the size and format they asked for.** "Just the top three" gets three, with one line
+   saying how many others were reviewed. A list for a spreadsheet gets one row per role. The
+   whole answer is in the reply unless they ask for a file.
+
+8. **Deliver from what you have.** With postings and a few lines about the person, shortlist
+   against those and say a resume would settle the rest. With a resume and no postings, ask
+   for the postings in one line.
+
+## Quality bar
+
+- **No role is ranked without its posting text having been read.** Unread roles are listed as
+  unread, with no verdict.
+- **A role with an unmet screening requirement is never under Apply,** and the requirement is
+  named.
+- **Every role given appears exactly once;** duplicates are merged, closed ones are noted.
+- **Every "fits" rests on the person's own material,** and a similar thing is not counted as
+  the named one.
+- **The answer opens the reply,** and the size is the one asked for.
+
+## When to use this skill
+
+- ✅ "Here's the careers page export for this company — which should I go for?"
+- ✅ "I've saved twelve postings in this folder. Shortlist them against my CV."
+- ✅ Narrowing a long list of openings to the few worth a full application
+
+## When NOT to use this skill
+
+- ❌ A full, requirement-by-requirement check of one posting — use `job-fit-analysis` on the role this shortlist puts first
+- ❌ Rewriting the resume for a role — use `resume-tailor`
+- ❌ Searching the internet for jobs to apply for — this reads the postings you have or point to
+
+## Anti-patterns to avoid
+
+- ❌ **Ranking from titles.** "Operations Lead sounds like a good fit" with the posting unread.
+- ❌ **Guessing an unread posting** from what the employer or the job title usually means.
+- ❌ **A strong fit over an unmet licence.** Screening requirements decide first.
+- ❌ **Counting a duplicate twice,** or ranking a closed posting.
+- ❌ **Dropping roles silently.** A list that covers 9 of 12 postings without saying so.
+- ❌ **Crediting the usual.** "As a nurse you'll have used Epic" is an invention.
+- ❌ **Percentages and scores.** "82% match" hides which requirement is the problem.
+- ❌ **Judging the person.** No remarks on age, gaps, family, health or anything an employer may
+  not lawfully weigh.

package/packs/career/skills/role-shortlist/evals/eval_queries.json

@@ -0,0 +1,34 @@
+[
+  {
+    "query": "Here are all the open roles at the hospital trust, saved in postings/. Which ones should I actually apply for? My CV is cv.md",
+    "should_trigger": true
+  },
+  {
+    "query": "go through this company's careers page export and shortlist the jobs that fit my resume",
+    "should_trigger": true
+  },
+  {
+    "query": "I have 15 job ads saved, can you tell me which are worth my time",
+    "should_trigger": true
+  },
+  {
+    "query": "which of these openings match my experience? resume attached",
+    "should_trigger": true
+  },
+  {
+    "query": "Am I a good fit for this one posting? Be honest about the gaps",
+    "should_trigger": false
+  },
+  {
+    "query": "Tailor my resume for the warehouse supervisor job",
+    "should_trigger": false
+  },
+  {
+    "query": "Write a cover letter for the analyst role",
+    "should_trigger": false
+  },
+  {
+    "query": "Find me remote data jobs on the internet",
+    "should_trigger": false
+  }
+]

package/packs/career/skills/role-shortlist/evals/evals.json

@@ -0,0 +1,75 @@
+{
+  "skill_name": "role-shortlist",
+  "evals": [
+    {
+      "id": 1,
+      "prompt": "I saved the Tamsworth Valley Health Board vacancies page as careers-index.html, and every posting I could open is in the postings folder. My CV is cv.md. Which of these should I actually apply for?",
+      "assertions": [
+        "The reply opens with Patient Pathway Coordinator and Medical Secretary - Cardiology as the roles to apply for, with a reason for each, before the role-by-role list",
+        "Senior Care Coordinator is under Skip, not Apply or Maybe, and the reason named is the professional registration (nurse, occupational therapist, physiotherapist or social worker) that the CV says she does not hold",
+        "The two Patient Pathway Coordinator postings (HV-2231-B and HV-2231-E) are one entry that lists both Bramwick and Elsford, not two ranked entries",
+        "Outpatient Booking Team Leader (HV-2176) is noted as closed and is not placed under Apply, Maybe or Skip",
+        "Waiting List Validation Lead is listed as not read because it is only a title in the listing with no posting saved, and it gets no verdict, no reason to apply and no stated requirements",
+        "All 11 vacancies in careers-index.html are accounted for, each appearing exactly once (10 entries once the duplicate is merged)",
+        "Data and Performance Analyst is not under Apply, and her Excel spreadsheet work is not counted as SQL or Power BI",
+        "It gives no percentage, score or match rating, and does not credit her with any registration, qualification, system or experience that is not in cv.md"
+      ]
+    },
+    {
+      "id": 2,
+      "prompt": "This is my saved search from the job board: job-board-export.csv. Where I could open the full ad I saved it in the postings folder, named by its reference number. My CV is cv.md. I'm staying in Fenmoor and I can't do night shifts any more. Which of these are worth applying for?",
+      "assertions": [
+        "Warehouse Team Leader (FJ-10455) is under Skip with the nights-only shift named as the reason, even though the reply may note that his acting team leader experience fits",
+        "The two Reach Truck Operator rows (FJ-10422 and FJ-10587) are treated as one role, with both reference numbers noted, not ranked twice",
+        "VNA Truck Operator (FJ-10431) is not under Apply, the missing VNA licence is named, and his reach truck licence is not counted as meeting it",
+        "Forklift Driver - Days (FJ-10502), Stock Control Lead (FJ-10519) and Transport Clerk (FJ-10533) are listed as not read because only a snippet was saved, with no verdict on fit, and the reply suggests fetching their full postings",
+        "Reach Truck Operator is among the roles the reply opens with as worth applying for",
+        "For Inventory Controller (FJ-10476), the Korvex requirement is marked as not shown or as the open question, and his StockPilot experience is not presented as Korvex",
+        "All 10 rows in the export are accounted for, each appearing exactly once (9 entries once the duplicate is merged)",
+        "It does not credit him with a VNA licence, Korvex, multi-drop driving experience or any other licence or experience not in cv.md, and gives no percentage or match score"
+      ]
+    },
+    {
+      "id": 3,
+      "prompt": "Which should I go for, just the top two? I've worked in hotels for six years, front desk first and night manager for the last two at a 70-room hotel. I run a team of four on nights and do the rota and the end-of-night reports. I want to get back to working days.\n\n1. Guest Services Manager, The Marlowe, Penhallow - lead the front desk and concierge team at a 120-room hotel.\n2. Night Manager, Harbourside Lodge - overnight sole charge of a 90-room hotel, four nights on, four off.\n3. Reservations Supervisor, Ferrand Hotels - run a team of five handling group and online bookings, Monday to Friday days.\n4. Bar and Events Manager, The Old Mill Inn - personal licence holder required; run the bar and the weekend weddings.\n5. Revenue Analyst, Ferrand Hotels - set room rates using our revenue system; degree in a numerate subject preferred.\n6. Housekeeping Supervisor, The Marlowe, Penhallow - supervise 12 room attendants, early starts from 6am.",
+      "assertions": [
+        "It names exactly two roles as its picks, not a ranked list of all six",
+        "Night Manager is not one of the two picks, because the person said they want to work days",
+        "Bar and Events Manager is not a pick unless the reply names the personal licence as a question to settle first, since the person did not say they hold one",
+        "It says the picks rest on one-line descriptions rather than full postings, and that the full postings should be read before applying",
+        "It does not state any requirement, duty, pay or hours for a role beyond what that role's one line says",
+        "It says in a line or so that the other four were looked at, without a full write-up of each",
+        "It delivers the two picks from what was given, and does not refuse or stop to ask for a CV or the postings first",
+        "It gives no percentage or match score, and does not credit the person with a qualification, licence or system they did not mention"
+      ]
+    },
+    {
+      "id": 4,
+      "held_out": "2026-10-18",
+      "prompt": "I'm a heavy diesel tech looking to move on from my current workshop. I saved Brackwater Freight's careers page as brackwater-careers.html and exported my saved TradesBoard listings as tradesboard-export.csv. My CV is mere-castell-cv.md. Which of these should I apply for? I'm staying in Tarnhaven: Marrowby and Port Halloran are both under an hour's drive, Kestrel Flats is about three and a half hours away. Give me my top three at most, ranked, and just a line on each of the others so I know why not. I'll be reading it on my phone at smoko so keep it tight.",
+      "assertions": [
+        "It recommends no more than three roles, and covers each of the other roles in about a line, not a full write-up",
+        "Every role it recommends is one of the Brackwater Heavy Diesel Technician at Tarnhaven (HDT-0412), the Tolley Earthmoving Hydraulics Technician, or the Marrowby Truck & Trailer Brake & Suspension Technician, and it does not set the Tolley or Marrowby role aside over items those postings list only as preferred or desirable (excavator experience, a forklift certificate, CoF inspection experience)",
+        "It rules out the Brackwater Senior Fleet Technician because night shifts are a required part of its roster and the CV says day shift only, even though the pay and experience fit; and it rules out the Halloran Haulage Mobile Service Technician (Class 4 licence required at start, she holds Class 2), the Kestrel Flats Workshop Supervisor (must live within 30 minutes of the site, no relocation, and she is staying in Tarnhaven), and the Fleet Reliability Engineer (a mechanical engineering degree is required and the CV shows a trade certificate)",
+        "It treats the Brackwater Heavy Diesel Technician on the careers page and the TradesBoard 'Heavy Diesel Mechanic (Tarnhaven depot)' listing as one role (both are ref HDT-0412), not two, while keeping the Port Halloran Heavy Diesel Technician (HDT-0431) as a separate role",
+        "For the Port Halloran Heavy Diesel Technician (HDT-0431) it says the saved page has only the title and a link, not the posting, and neither recommends it nor rules it out on the title alone; it suggests reading the full posting first",
+        "It accounts for all eight distinct roles across the two files: the three Brackwater featured roles, the Port Halloran listing, and the four other TradesBoard listings",
+        "It does not credit Mere with anything her CV does not show, such as a Class 4 licence, an engineering degree, a forklift certificate, CoF inspection experience, supervisory experience beyond mentoring one apprentice, or availability for nights, and it does not state what the Port Halloran posting says"
+      ]
+    },
+    {
+      "id": 5,
+      "held_out": "2026-10-18",
+      "prompt": "ok so my cousin sent me these from the local jobs group, which 2 should i actually go for?? 2 max, i dont have time to do more applications than that\n\nme: line cook 3 yrs at a pub kitchen in Dunmore Quay, food safety level 2 cert, no car so its the bus or nothing, and im on notice til 15 nov so cant start before that\n\n1. Commis chef - The Lantern Room, Dunmore Quay. 4 nights a wk, €15.10/hr, 1 yr kitchen exp, start asap\n2. Breakfast cook, Gullhaven Hotel (Gullhaven). 6am starts, own transport essential (no bus out there before 7). 2 yrs exp, €15.80\n3. chef de partie - Saltmarsh Bistro, Dunmore Quay. 2+ yrs line experience, food safety level 2. starting december. €16.40/hr\n4. Lantern Room Dunmore Quay - commis chef, 4 nights/week, 15.10 ph, kitchen exp 1yr+, start ASAP!!\n5. Head chef, Copperline Catering. food safety level 3 required + 5 yrs supervisory experience, €48k\n6. kitchen porter/cook - Brindle Café, he only sent the link: https://jobs.example.com/brindle-kp",
+      "assertions": [
+        "It picks no more than two roles, as asked, and keeps the reply short to match the quick question",
+        "It recognises that postings 1 and 4 are the same Lantern Room commis chef job and does not count them as two roles or use them as both picks",
+        "Its picks include the Saltmarsh Bistro chef de partie, and neither pick is the Gullhaven Hotel, Copperline Catering or Brindle Café job",
+        "It says the Gullhaven Hotel job is out because own transport is essential with 6am starts and there is no bus before 7, and the Copperline head chef job is out because it requires a level 3 food safety certificate and five years' supervisory experience, which the person doesn't have",
+        "For the Brindle Café job it says only a link was given, so the posting hasn't been read and the job can't be judged from its title; it neither recommends it nor rules it out",
+        "It points out that the Lantern Room wants someone to start as soon as possible while the person can't start until after 15 November, as something to raise or weigh, rather than treating the start date as fine",
+        "It does not credit the person with anything they didn't say, such as a car or driving licence, a level 3 food safety certificate, supervisory or chef de partie experience, or more than three years in kitchens, and it does not claim to know what the Brindle Café posting says"
+      ]
+    }
+  ]
+}

package/packs/career/skills/role-shortlist/evals/files/1/careers-index.html

@@ -0,0 +1,22 @@
+<!DOCTYPE html>
+<html lang="en">
+<head><meta charset="utf-8"><title>Vacancies | Tamsworth Valley Health Board</title></head>
+<body>
+<header><h1>Tamsworth Valley Health Board - Current vacancies</h1></header>
+<p>Showing 11 vacancies. Saved 15 October 2026.</p>
+<ul class="vacancy-list">
+  <li><a href="https://careers.tamsworth-valley.test/jobs/HV-2231-B">Patient Pathway Coordinator - Bramwick General Hospital</a></li>
+  <li><a href="https://careers.tamsworth-valley.test/jobs/HV-2231-E">Patient Pathway Coordinator - Elsford Community Hospital</a></li>
+  <li><a href="https://careers.tamsworth-valley.test/jobs/HV-2240">Senior Care Coordinator - Community Services</a></li>
+  <li><a href="https://careers.tamsworth-valley.test/jobs/HV-2198">Medical Secretary - Cardiology, Elsford</a></li>
+  <li><a href="https://careers.tamsworth-valley.test/jobs/HV-2176">Outpatient Booking Team Leader - Bramwick</a></li>
+  <li><a href="https://careers.tamsworth-valley.test/jobs/HV-2244">Clinical Coding Officer - Bramwick</a></li>
+  <li><a href="https://careers.tamsworth-valley.test/jobs/HV-2251">Data and Performance Analyst - Headquarters</a></li>
+  <li><a href="https://careers.tamsworth-valley.test/jobs/HV-2253">Healthcare Support Worker (Nights) - Corrin Bay</a></li>
+  <li><a href="https://careers.tamsworth-valley.test/jobs/HV-2257">Reception and Admin Assistant - Corrin Bay Health Centre</a></li>
+  <li><a href="https://careers.tamsworth-valley.test/jobs/HV-2259">Estates Maintenance Electrician - Bramwick</a></li>
+  <li><a href="https://careers.tamsworth-valley.test/jobs/HV-2262">Waiting List Validation Lead - Elsford</a></li>
+</ul>
+<footer><p>Tamsworth Valley Health Board is an equal opportunities employer.</p></footer>
+</body>
+</html>

package/packs/career/skills/role-shortlist/evals/files/1/cv.md

@@ -0,0 +1,44 @@
+# Rhiannon Talbot
+
+Elsford, Tamsworth Valley
+
+## Profile
+
+Medical secretary and outpatient administrator with six years in hospital outpatients. I manage
+referral-to-treatment pathways, type clinic letters and keep consultants' clinics running on time.
+
+## Experience
+
+**Medical Secretary to the Orthopaedic Team**, Northmoor Health Board, Garrow Hospital
+April 2022 - present
+- Secretary to three orthopaedic consultants; type clinic letters and discharge summaries from
+  digital dictation (DictaScribe), turnaround under 5 working days.
+- Track referral-to-treatment (RTT) pathways for about 1,400 patients in MedFlow PAS and flag
+  breach risks to the service manager each week.
+- Validate the orthopaedic waiting list each month: contact patients, remove duplicates and
+  record outcomes in MedFlow.
+- Book and rebook clinics and theatre lists; handle calls from patients and GP practices.
+- Keep the clinic tracking spreadsheet in Excel (filters, conditional formatting).
+
+**Outpatient Administrator**, Northmoor Health Board, Garrow Hospital
+September 2019 - April 2022
+- Booked outpatient appointments in MedFlow PAS for dermatology and rheumatology.
+- Prepared clinic lists and case notes; managed cancellations and the partial booking list.
+- Covered main outpatients reception one day a week.
+
+**Receptionist**, Carrick Lane Dental Practice, Garrow
+2017 - 2019
+
+## Qualifications
+
+- Level 3 Diploma in Medical Administration, 2019
+- Audio typing, 70 wpm, tested 2023
+- GCSEs including English and Maths
+
+## Training
+
+- Information governance and data protection, renewed 2026
+- Safeguarding adults and children, level 1, 2026
+- Customer care in healthcare (in-house, one day), 2021
+
+No professional registration. Full driving licence and own car.

package/packs/career/skills/role-shortlist/evals/files/1/postings/clinical-coding-officer-bramwick.md

@@ -0,0 +1,23 @@
+# Clinical Coding Officer (Trainee)
+
+**Tamsworth Valley Health Board** - Bramwick General Hospital
+Reference: HV-2244 | Band 3, rising to Band 4 on completing training | Permanent, full time
+Salary: £24,100 - £25,600 | Closing date: 20 November 2026
+
+## About the role
+
+You will read inpatient case notes and translate diagnoses and procedures into ICD-10 and OPCS-4
+codes. Full training is provided, including the national clinical coding standards course in your
+first year.
+
+## Essential
+
+- Good knowledge of medical terminology and anatomy
+- High level of accuracy and attention to detail
+- Experience of working with confidential patient records
+- Able to work to monthly coding deadlines
+
+## Desirable
+
+- Previous experience of clinical coding using ICD-10 and OPCS-4
+- Experience using MedFlow PAS

package/packs/career/skills/role-shortlist/evals/files/1/postings/data-and-performance-analyst.html

@@ -0,0 +1,28 @@
+<!DOCTYPE html>
+<html lang="en">
+<head><meta charset="utf-8"><title>Data and Performance Analyst | Tamsworth Valley Health Board</title></head>
+<body>
+<h1>Data and Performance Analyst</h1>
+<p><strong>Tamsworth Valley Health Board</strong> - Headquarters, Bramwick (hybrid, 2 days in office)</p>
+<p>Reference: HV-2251 | Band 6 | Permanent, full time | Salary: £37,300 - £44,900<br>
+Closing date: 9 November 2026</p>
+
+<h2>About the role</h2>
+<p>You will produce the board's weekly waiting-time and RTT performance reports, build
+dashboards for service managers and answer ad hoc data requests.</p>
+
+<h2>Essential</h2>
+<ul>
+  <li>Degree in a numerate subject, or equivalent experience in a data analysis role</li>
+  <li>Writing SQL queries against a reporting database</li>
+  <li>Building dashboards in Power BI</li>
+  <li>Understanding of RTT and waiting-time reporting</li>
+</ul>
+
+<h2>Desirable</h2>
+<ul>
+  <li>Experience with Python or R</li>
+  <li>Experience of health board performance reporting</li>
+</ul>
+</body>
+</html>

package/packs/career/skills/role-shortlist/evals/files/1/postings/estates-maintenance-electrician-bramwick.md

@@ -0,0 +1,20 @@
+# Estates Maintenance Electrician
+
+**Tamsworth Valley Health Board** - Estates Department, Bramwick General Hospital
+Reference: HV-2259 | Band 5 | Permanent, full time, with on-call rota
+Salary: £30,100 - £36,500 | Closing date: 12 November 2026
+
+## About the role
+
+Planned and reactive electrical maintenance across the Bramwick hospital site, including
+emergency lighting, fire alarm and generator testing.
+
+## Essential
+
+- Level 3 electrical installation qualification and current wiring regulations certificate
+- Time-served apprenticeship as an electrician
+- Experience of maintenance in a commercial or healthcare building
+
+## Desirable
+
+- Experience of healthcare electrical systems

package/packs/career/skills/role-shortlist/evals/files/1/postings/healthcare-support-worker-nights-corrin-bay.md

@@ -0,0 +1,20 @@
+# Healthcare Support Worker (Nights)
+
+**Tamsworth Valley Health Board** - Corrin Bay Community Hospital, Ward 2
+Reference: HV-2253 | Band 3 | Permanent, 30 hours a week, nights only
+Salary: £24,100 - £25,600 plus unsocial hours enhancement | Closing date: 4 November 2026
+
+## About the role
+
+Supporting registered nurses to deliver personal care to older inpatients overnight: washing,
+dressing, repositioning, taking observations and recording them accurately.
+
+## Essential
+
+- Care Certificate, or willingness to complete it within 12 weeks of starting
+- Experience of providing personal care in a hospital, care home or home care setting
+- Able to work nights only, including weekends and bank holidays
+
+## Desirable
+
+- Level 2 or 3 qualification in health and social care