skilldrop-cli 0.16.19 → 0.16.20

package/packs/infra-as-code/skills/terraform-plan-review/evals/files/3/plan.txt

@@ -0,0 +1,87 @@
+aws_db_instance.tracking: Refreshing state... [id=tidewater-tracking]
+aws_ebs_volume.gps_archive: Refreshing state... [id=vol-0abc1234def567890]
+aws_security_group.tracking_db: Refreshing state... [id=sg-0123abcd4567ef890]
+aws_instance.ingest: Refreshing state... [id=i-0fedcba9876543210]
+
+Note: Objects have changed outside of Terraform
+
+Terraform detected the following changes made outside of Terraform since the
+last "terraform apply" which may have affected this plan:
+
+  # aws_security_group.tracking_db has changed
+  ~ resource "aws_security_group" "tracking_db" {
+        id      = "sg-0123abcd4567ef890"
+      ~ ingress = [
+          + {
+              + cidr_blocks = ["0.0.0.0/0"]
+              + description = "temp vendor access"
+              + from_port   = 5432
+              + protocol    = "tcp"
+              + to_port     = 5432
+            },
+            # (1 unchanged element hidden)
+        ]
+        name    = "tidewater-tracking-db"
+        # (6 unchanged attributes hidden)
+    }
+
+Unless you have made equivalent changes to your configuration, or ignored the
+relevant attributes using ignore_changes, the following plan may include
+actions to undo or respond to these changes.
+
+─────────────────────────────────────────────────────────────────────────────
+
+Terraform used the selected providers to generate the following execution
+plan. Resource actions are indicated with the following symbols:
+  + create
+  ~ update in-place
+-/+ destroy and then create replacement
+
+Terraform will perform the following actions:
+
+  # aws_db_instance.tracking will be updated in-place
+  ~ resource "aws_db_instance" "tracking" {
+      + allow_major_version_upgrade = true
+      ~ apply_immediately           = false -> true
+      ~ engine_version              = "14.9" -> "15.4"
+        id                          = "tidewater-tracking"
+      ~ instance_class              = "db.r6g.large" -> "db.r6g.xlarge"
+        # (38 unchanged attributes hidden)
+    }
+
+  # aws_ebs_volume.gps_archive must be replaced
+-/+ resource "aws_ebs_volume" "gps_archive" {
+      ~ arn               = "arn:aws:ec2:eu-west-1:111111111111:volume/vol-0abc1234def567890" -> (known after apply)
+      ~ availability_zone = "eu-west-1a" -> "eu-west-1b" # forces replacement
+      ~ id                = "vol-0abc1234def567890" -> (known after apply)
+        size              = 2000
+        type              = "gp3"
+        # (4 unchanged attributes hidden)
+    }
+
+  # aws_security_group.tracking_db will be updated in-place
+  ~ resource "aws_security_group" "tracking_db" {
+        id      = "sg-0123abcd4567ef890"
+      ~ ingress = [
+          - {
+              - cidr_blocks = ["0.0.0.0/0"]
+              - description = "temp vendor access"
+              - from_port   = 5432
+              - protocol    = "tcp"
+              - to_port     = 5432
+            },
+            # (1 unchanged element hidden)
+        ]
+        name    = "tidewater-tracking-db"
+        # (6 unchanged attributes hidden)
+    }
+
+  # aws_volume_attachment.gps_archive will be created
+  + resource "aws_volume_attachment" "gps_archive" {
+      + device_name = "/dev/sdf"
+      + id          = (known after apply)
+      + instance_id = "i-0fedcba9876543210"
+      + volume_id   = (known after apply)
+    }
+
+Plan: 1 to add, 2 to change, 0 to destroy.

package/packs/product-manager/skills/business-case/evals/evals.json

@@ -14,6 +14,33 @@
         "Names the flip-assumption that would reverse the recommendation, with a de-risking step if its confidence is low",
         "Recommends exactly one option and states the runner-up's strongest argument fairly"
       ]
+    },
+    {
+      "id": 2,
+      "held_out": "2026-10-18",
+      "prompt": "The board of our food bank network needs to decide whether to get a refrigerated van. Everything I have is in hollin_vale_inputs.md. Write the business case for the November 20 board meeting. Keep it to what fits on two printed pages — the board reads it on paper.",
+      "assertions": [
+        "Uses the warehouse log's total (about 10.4 tonnes a year) for spoilage, or explicitly reconciles it against the chair's 18-tonne figure, rather than silently using 18 tonnes",
+        "Catches that the lease quote's annual figure is wrong: $1,850 x 12 is $22,200, not $19,200",
+        "Recognises that the grant can fund only the van purchase, so the lease and all running costs (including the paid driver) must come from the $20,000-a-year operating headroom, and notes that even the buy option's recurring costs (about $33,500 a year) exceed that headroom",
+        "Treats keeping the volunteer drivers as a real option to compare against, and accounts for 3 of the 11 regular drivers leaving by spring",
+        "Does not present a number of extra families fed as a sourced fact; any conversion from produce to families is flagged as an assumption not in the input, or the gap is named",
+        "Does not invent costs, funding sources, or figures not in the input file without labelling them as assumptions",
+        "Fits roughly two printed pages (about 1,000 words or less) and makes a clear recommendation the board can vote on"
+      ]
+    },
+    {
+      "id": 3,
+      "held_out": "2026-10-18",
+      "prompt": "my dental practice is thinking about getting one of those 3D intraoral scanners for crowns instead of goopy impressions, its like $30k?? my partner thinks its a waste of money. we do maybe 8 crowns a month. can you give me a few bullets on whether its worth it, dont need an essay",
+      "assertions": [
+        "Gives a usable answer on whether the scanner is worth it instead of only asking questions back",
+        "Stays to a few bullets — no more than about 10 bullets or 250 words",
+        "Compares buying against keeping traditional impressions, not just the scanner's benefits in isolation",
+        "Any figures the user did not give (lab fees, chair time saved, impression material costs, financing) are labelled as estimates or assumptions, not stated as facts",
+        "Names what the answer most depends on (for example, savings per crown or crown volume) so the owner knows what to check before deciding",
+        "Written in plain language for a dentist, without unexplained finance jargon such as NPV or IRR"
+      ]
     }
   ]
 }

package/packs/product-manager/skills/business-case/evals/files/2/hollin_vale_inputs.md

@@ -0,0 +1,46 @@
+# Hollin Vale Food Network — inputs for the cold-chain decision
+
+Compiled by the operations lead for the November 20 board meeting.
+
+## Background
+
+We collect surplus produce from 14 farms and 6 supermarkets and distribute it through 22 partner pantries. Today volunteers collect in their own cars. Produce can sit unrefrigerated for up to 5 hours between pickup and our warehouse.
+
+## Spoilage
+
+- Board chair's memo (September): "We lose about 18 tonnes of produce a year to spoilage in transit."
+- Warehouse intake log, last 12 months, kg discarded on arrival as spoiled:
+
+| Month | kg discarded |
+|---|---|
+| Oct | 820 |
+| Nov | 610 |
+| Dec | 400 |
+| Jan | 350 |
+| Feb | 380 |
+| Mar | 560 |
+| Apr | 790 |
+| May | 1,050 |
+| Jun | 1,320 |
+| Jul | 1,480 |
+| Aug | 1,410 |
+| Sep | 1,230 |
+
+- For our annual report we value donated produce at $1.90 per kg.
+
+## Options
+
+**A. Buy a refrigerated 3.5-tonne van.** Dealer quote: $68,000. Insurance, maintenance and fuel estimated at $9,500 a year. It needs a paid driver three days a week: $24,000 a year. Expected life 8 years.
+
+**B. Lease the same van.** Dealer quote: "$1,850 per month, i.e. $19,200 per year", 3-year term, maintenance included. Insurance and fuel estimated at $6,000 a year. Same paid driver as option A.
+
+**C. Keep volunteer drivers as today.** No new cost. The volunteer coordinator reports that 3 of our 11 regular drivers are leaving by spring.
+
+## Funding
+
+- Mereford Community Trust grant: up to $70,000, for capital purchases only (vehicles or equipment the network will own). It cannot pay for leases, salaries, or running costs. Application deadline: December 1.
+- The treasurer says the operating budget has $20,000 a year of headroom for new recurring costs.
+
+## Board chair's request
+
+"Make the case, and tell the board how many more families we'll feed."

package/packs/product-manager/skills/okr-cascade/evals/evals.json

@@ -23,6 +23,33 @@
         "The BLOCKED output states what to rerun with (the top 3-5 objectives and their KRs, plus teams in scope)",
         "No cascade tables, gap registry, or metric tree appear"
       ]
+    },
+    {
+      "id": 3,
+      "held_out": "2026-10-18",
+      "prompt": "I'm the planning lead for our library system. quillfield_2027.md has the board-approved system OKRs for 2027 and each team's draft OKRs. Cascade the system OKRs into team OKRs, fix what's wrong with the drafts, and tell me where the plan doesn't hold together. The director wants it to fit on two pages, with no more than 3 objectives per team.",
+      "assertions": [
+        "Flags that the 16–24 sign-up key result has no team that credibly owns it (the Teen Services post was eliminated), rather than quietly assigning it to a team that lacks the capacity",
+        "Catches that the branch visit targets add up to 1.28M, short of the system target of 1.38M",
+        "Catches that Eastgate is closed March through August 2027 yet its draft target is unchanged at 200,000 visits",
+        "Flags the social media impressions key result as not moving e-book and audiobook lending (or as a vanity metric) and replaces or drops it",
+        "Catches that 120 sessions in 4-session courses of 10 seats allows at most about 300 completions, far short of the 1,500 target, or otherwise shows the Adult Learning draft cannot deliver it",
+        "Does not invent teams, staff, budgets, or baseline figures not in the input",
+        "Keeps each team to at most 3 objectives and the whole within roughly two pages"
+      ]
+    },
+    {
+      "id": 4,
+      "held_out": "2026-10-18",
+      "prompt": "my sister and i run a bakery, just us plus 2 staff. our accountant says we should do 'OKRs' for next year?? we want to start opening sundays and get more cafes buying our bread wholesale. can you just give us the table, nothing fancy",
+      "assertions": [
+        "Delivers the OKRs without stopping to ask clarifying questions first",
+        "The reply is essentially just a table, with at most a line or two of text around it",
+        "Covers both goals the user named: opening on Sundays and winning more wholesale café customers",
+        "Key results are measurable results (for example Sunday takings or number of wholesale accounts) rather than tasks such as making a flyer",
+        "Does not invent current figures (revenue, customer counts) as facts; missing baselines are left to fill in or labelled as assumptions",
+        "Sized for a four-person business — a small number of objectives, not a multi-department corporate cascade"
+      ]
     }
   ]
 }

package/packs/product-manager/skills/okr-cascade/evals/files/3/quillfield_2027.md

@@ -0,0 +1,45 @@
+# Quillfield Public Libraries — 2027 OKRs and team drafts
+
+## System OKRs for 2027 (approved by the library board)
+
+**O1. Make the library part of more residents' weekly routine.**
+- KR1.1 In-person visits across all branches rise from 1.20M (2026) to 1.38M (+15%).
+- KR1.2 Active cardholders (card used in the last 12 months) rise from 41,000 to 47,000.
+- KR1.3 New card sign-ups among residents aged 16–24 rise 25% (2026: 2,400).
+
+**O2. Grow digital lending without losing print readers.**
+- KR2.1 E-book and audiobook loans rise from 310,000 to 400,000.
+- KR2.2 Print circulation does not fall below 900,000 (2026: 940,000).
+
+**O3. Close the digital skills gap for older residents.**
+- KR3.1 1,500 residents aged 65+ complete a digital skills course (2026: 600).
+
+## Teams and their draft OKRs
+
+**Branch Services** (runs the four branches: Central, Northside, Millbrook, Eastgate)
+Draft KR: hit these 2027 visit targets —
+
+| Branch | 2026 visits | 2027 target |
+|---|---|---|
+| Central | 480,000 | 520,000 |
+| Northside | 280,000 | 300,000 |
+| Millbrook | 240,000 | 260,000 |
+| Eastgate | 200,000 | 200,000 |
+
+**Digital Services**
+- Reach 1 million impressions on library social media channels.
+- Launch the new library app by Q3.
+- E-book and audiobook loans reach 400,000.
+
+**Adult Learning** (runs computer classes)
+- Run 120 digital skills sessions for seniors.
+- Note from the team lead: each course is a series of 4 sessions with 10 seats.
+
+**Collections & Acquisitions**
+- Grow the e-book catalogue by 20%.
+- Keep print circulation above 900,000.
+
+## Other notes
+
+- Facilities: Eastgate branch is closed for renovation from March through August 2027.
+- HR: the Teen Services librarian position was eliminated in the 2026 budget. No team currently runs programming for ages 16–24.

package/packs/product-manager/skills/prd-draft/evals/evals.json

@@ -12,6 +12,33 @@
         "Each goal carries a success line stating how it will be measured",
         "Claims that need evidence are marked rather than asserted as fact"
       ]
+    },
+    {
+      "id": 2,
+      "held_out": "2026-10-18",
+      "prompt": "Our VP wants a leak-alert feature for water customers. Her brief is in brief.md; I also pulled call_center_notes.md and meter_facts.md. Write the PRD for v1 so engineering can start. Keep it under three pages.",
+      "assertions": [
+        "Does not promise real-time or within-minutes alerts; the requirements reflect that meter readings reach the utility only once a day",
+        "Uses or reconciles the call-centre figure (2,140 of 18,600 calls, about 11.5%) rather than repeating the brief's 30% as fact",
+        "Does not adopt the 20% network water-loss reduction as a target or fact without flagging that nothing in the inputs supports it",
+        "Respects the opt-in rule for SMS: does not switch SMS alerts on for all 140,000 customers, and accounts for app reach being only 9%",
+        "Deals explicitly with the 18% of homes on manually read meters (for example excluding them from v1), rather than assuming they get alerts",
+        "Does not invent figures, systems, or stakeholders not in the three input files",
+        "Stays under about three pages (roughly 1,500 words)"
+      ]
+    },
+    {
+      "id": 3,
+      "held_out": "2026-10-18",
+      "prompt": "im on the PTA at my kids school. we want parents to be able to sign up for volunteer slots at events (bake sale, book fair etc) online instead of the paper sheet at the front office. i got asked to 'write up the requirements' for one of the dads who builds websites. no idea what that means honestly. half a page please",
+      "assertions": [
+        "Produces the requirements write-up rather than only asking questions",
+        "About half a page — no more than roughly 300 words",
+        "Plain language that both a PTA parent and a hobbyist web developer can follow, without unexplained product-management jargon",
+        "Requirements are concrete enough to check (for example parents can see open slots, sign up, and the organiser can see who is coming), not vague aims",
+        "Keeps the scope small enough for one volunteer to build — does not make payments, accounts, or a mobile app must-haves",
+        "Does not invent specifics presented as facts (school name, number of events, existing systems, budget); any assumptions are marked"
+      ]
     }
   ]
 }

package/packs/product-manager/skills/prd-draft/evals/files/2/brief.md

@@ -0,0 +1,7 @@
+# Leak alerts — brief from the VP of Customer Experience, Fenmoor Water
+
+Customers keep getting shocked by huge bills caused by leaks they didn't know about. Leak-related high-bill complaints are 30% of our call volume. I want a feature that pushes a notification from our app the moment a customer's usage spikes, so they can find the leak within minutes. This will cut water loss across our network by 20%.
+
+We should switch it on for all 140,000 customers at launch so everyone benefits.
+
+Constraints from engineering: v1 must ship by March with 2 engineers. No hardware changes.

package/packs/product-manager/skills/prd-draft/evals/files/2/call_center_notes.md

@@ -0,0 +1,7 @@
+# Call centre summary, last 12 months (Fenmoor Water)
+
+- Total inbound calls: 18,600
+- Calls tagged "high bill — leak suspected or confirmed": 2,140
+- Average time from leak start to customer noticing (from 40 follow-up calls): 26 days, usually when the bill arrives
+- Most common customer quote: "Why didn't you tell me sooner?"
+- 9% of customers have our mobile app installed. 71% have a mobile number on file.

package/packs/product-manager/skills/prd-draft/evals/files/2/meter_facts.md

@@ -0,0 +1,6 @@
+# Meter and messaging facts (from Operations and Legal)
+
+- 82% of homes have automated meters. They log hourly readings, but the readings reach our systems once a day, in a batch at 6 a.m.
+- The other 18% of homes have older meters that are read manually every quarter. There is no daily data for them.
+- State consumer rule: utilities may send SMS alerts only to customers who have opted in to SMS.
+- Email alerts are permitted for any customer with an email on file (64% of customers).

package/packs/product-manager/skills/prfaq/evals/evals.json

@@ -27,6 +27,32 @@
         "The BLOCKED output names the missing anchors (customer, problem, differentiator) and what to rerun with",
         "No invented product, customer segment, or problem statement appears"
       ]
+    },
+    {
+      "id": 3,
+      "held_out": "2026-10-18",
+      "prompt": "Our farming co-op board wants a working-backwards PR/FAQ for a shared drone spraying service before we commit money. My working notes are in drone_notes.md. Write the PR/FAQ. Keep the press release under 400 words.",
+      "assertions": [
+        "Does not claim that 72% of members want the service; reflects that only 58 of 410 members responded",
+        "Catches that the per-acre costs add up to $12.00, not $10.20, so the margin at $14 is about 14%, not 27%",
+        "Does not promise members they can book a drone for any field; addresses the 50 m waterway rule affecting about 35% of member acreage",
+        "Flags that an April 2027 launch conflicts with a 9–12 month certification process that has not started, rather than writing as if spring is achievable without comment",
+        "The press release is under 400 words",
+        "Does not invent data (member counts, prices, dates, yields) not in the notes; any new figure is labelled as an assumption"
+      ]
+    },
+    {
+      "id": 4,
+      "held_out": "2026-10-18",
+      "prompt": "im the volunteer coordinator for our youth soccer league. we want to start a carpool matching thing for away games so parents arent all driving separately. someone on the board said we should 'write the press release first'?? can u do a short one, like half a page, plus a few faqs",
+      "assertions": [
+        "Delivers a press release and FAQs without stopping to ask questions first",
+        "The press release is about half a page (no more than roughly 300 words) and there are only a few FAQs (no more than about 8)",
+        "Written as an announcement of the carpool matching as if it exists, with the benefit to families stated plainly",
+        "The FAQs take on at least one hard practical question a parent or the league would ask, such as driver vetting and child safety, insurance or liability, or what happens when a driver cancels",
+        "Does not invent statistics, sponsors, or partner organisations presented as fact (for example 'cuts driving by 40%')",
+        "Plain language suited to parents in a volunteer-run league, without corporate or tech jargon"
+      ]
     }
   ]
 }

package/packs/product-manager/skills/prfaq/evals/files/3/drone_notes.md

@@ -0,0 +1,25 @@
+# Tamsin Valley Growers Cooperative — shared drone spraying service: working notes
+
+## Member survey (September)
+- Sent to all 410 member farms. 58 responded.
+- 42 of the 58 respondents said they were "interested" or "very interested" (72%).
+- Top reasons given: labour shortage at spraying time, wet fields that ground rigs can't enter, wanting less soil compaction.
+
+## Proposed service
+- The co-op buys 3 spraying drones and employs 2 certified pilots. Members book a slot through the co-op office.
+- Proposed price to members: $14 per acre.
+
+## Cost per acre (finance committee estimate)
+- Drone purchase spread over 5 years: $4.10
+- Pilot time: $5.20
+- Chemical handling and cleaning: $2.70
+- Total: $10.20 per acre, giving a 27% margin at $14.
+
+## Regulation
+- Drone spraying is not permitted within 50 m of a waterway.
+- Our agronomist estimates about 35% of member acreage lies within 50 m of the Tamsin River or its creeks.
+- Drone spraying needs an operator certificate. The regulator's published processing time is 9 to 12 months from application. We have not applied yet.
+
+## Board's wish
+- Launch for the spring 2027 spraying season (April).
+- The board would like the press release to say members can "book a drone for any field, any day".

package/packs/product-manager/skills/requirements-interview/evals/evals.json

@@ -11,6 +11,33 @@
         "Every script contains a kill-question, and the kit states what a kill-answer would mean",
         "Stakeholders are chosen and named by role, with a reason each one is on the list"
       ]
+    },
+    {
+      "id": 2,
+      "held_out": "2026-10-18",
+      "prompt": "I'm leading a project to speed up patient discharge at our hospital. The brief is in project_brief.md. Prepare the interview kits for the four people listed there.",
+      "assertions": [
+        "Does not plan to interview patients directly in the next two weeks, given ethics approval takes about six weeks; the patient view is sought another way (such as the Patient Liaison Officer's complaint themes) or deferred",
+        "Questions probe the pharmacy wait (median 3.1 hours) rather than accepting the sponsor's view that late discharge summaries are the main cause, and at least one question tests the sponsor's theory",
+        "Does not spend questions on adding staff, given there is no budget for new staff, or notes that this is already decided",
+        "Catches that 14% of 1,200 monthly discharges is 168, not 280",
+        "Each stakeholder's set of questions fits a 20-minute interview (no more than about 7 questions each)",
+        "Covers all four named stakeholders",
+        "Does not invent data, stakeholders, or hospital systems not in the brief"
+      ]
+    },
+    {
+      "id": 3,
+      "held_out": "2026-10-18",
+      "prompt": "i own a small dog grooming salon, 3 groomers + 1 person on the front desk. thinking of paying a freelancer to build online booking. what should i ask my groomers and my front desk person before we build it? just a few bullets each",
+      "assertions": [
+        "Gives questions for both the groomers and the front-desk person without asking the user clarifying questions first",
+        "A few bullets per person — no more than about 7 questions each — and short overall",
+        "Questions get at how the salon works today and what could go wrong (for example how long different dogs take, no-shows, deposits, walk-ins, special handling), not just 'what features do you want'",
+        "At least one question tests the owner's assumption that online booking will help, for example by asking what actually goes wrong with bookings today",
+        "Questions are worded so a groomer can answer them easily, without software jargon",
+        "Does not invent facts about the salon (prices, customer counts, current software) presented as known"
+      ]
     }
   ]
 }

package/packs/product-manager/skills/requirements-interview/evals/files/2/project_brief.md

@@ -0,0 +1,29 @@
+# Faster discharge — project brief, St Aldric's General Hospital
+
+Sponsor: Chief Financial Officer
+Prepared by: Patient Flow programme lead
+
+## Problem as the sponsor sees it
+"Discharges are slow because doctors write discharge summaries late. If we fix the summaries, beds free up before noon."
+
+## What the data says (discharge audit, Ward 7, last 3 months)
+- Median wait from "medically fit for discharge" to leaving the ward: 5.4 hours.
+- Median wait for the discharge summary to be signed: 1.2 hours.
+- Median wait for take-home medicines from pharmacy: 3.1 hours.
+- Median wait for patient transport (for patients who need it): 1.6 hours.
+- Discharges before noon: 14% of 1,200 monthly discharges, i.e. 280 patients.
+
+## Constraints
+- The CFO has confirmed there is no budget for new staff this year. Any change must use existing staff.
+- We need the requirements by the end of next month, so interviews must happen in the next two weeks.
+- Each interview slot is 20 minutes.
+- Interviewing patients directly needs ethics approval, which takes about six weeks. The Patient Liaison Officer can share themes from discharge-related complaints.
+
+## Who we will interview
+1. The CFO (sponsor)
+2. Ward 7 nurse manager
+3. Pharmacy lead
+4. Patient transport supervisor
+
+## Ask
+Prepare the interview kits for these four people.

package/packs/product-manager/skills/strategy-analysis/evals/evals.json

@@ -31,6 +31,33 @@
         "Assumptions replacing the capped clarifying questions (e.g. competitive set) are tagged [assumption] and stated at the top",
         "Market facts drawn from general knowledge are tagged [market: ...], distinguishable from user-supplied facts and assumptions"
       ]
+    },
+    {
+      "id": 4,
+      "held_out": "2026-10-18",
+      "prompt": "Our coach company's board meets next week on whether to launch an airport shuttle. My notes are in market_notes.md. Give me the analysis and a recommendation — one page, the board wants a recommendation, not a textbook.",
+      "assertions": [
+        "Uses or reconciles the airport authority's passenger figures (roughly 3–4% annual growth) rather than relying on the consultant's 12%",
+        "Catches the market-size arithmetic error: 1.2M trips x $28 is $33.6M, not $46M",
+        "Recognises that the incumbent's exclusive curbside rights until June 2029 block or limit a curbside service now, and the recommendation reflects it",
+        "Does not give a year-1 market-share number as a finding; says the inputs cannot support one, or labels any figure as a hedged assumption",
+        "Gives a clear recommendation (for example enter now, enter a different way, or wait) tied to the evidence",
+        "Does not invent competitors, figures, or regulations not in the notes",
+        "About one page (roughly 500 words or less)"
+      ]
+    },
+    {
+      "id": 5,
+      "held_out": "2026-10-18",
+      "prompt": "i run a swim school for kids, we rent 2 pools by the hour. a big gym chain just opened 10 min away with its own pool and kids lessons. what do i do?? a few bullets pls, im not a business person",
+      "assertions": [
+        "Gives concrete advice without stopping to ask questions first",
+        "Short — no more than about 10 bullets or 250 words",
+        "Uses plain language a non-business owner can act on; any framework terms are explained or avoided",
+        "Identifies where a specialist swim school can realistically compete with or differ from a gym chain's lessons, rather than only generic advice",
+        "Points out that depending on rented pool hours is a vulnerability or a factor to manage",
+        "Does not invent facts about the gym chain (its prices, class sizes, quality) or the local market as known; such points are framed as things to find out"
+      ]
     }
   ]
 }

package/packs/product-manager/skills/strategy-analysis/evals/files/4/market_notes.md

@@ -0,0 +1,21 @@
+# Corrin Coachlines — airport shuttle market: notes for the board
+
+Question from the CEO: "Should we launch an airport shuttle between the city and Wrenmoor Field Airport? And what market share will we win in year 1?"
+
+## Demand
+- Consultant's deck (Halvard Advisory): "Passenger traffic at Wrenmoor Field is growing 12% a year."
+- Airport authority annual reports, total passengers: 2023: 4.1M, 2024: 4.3M, 2025: 4.4M.
+- Consultant's market size: "1.2M shuttle-eligible trips a year x $28 average fare = $46M market."
+
+## Competition
+- Bluewing Shuttle (incumbent): runs curbside shuttles every 20 minutes, fare $32.
+- Ride-hailing: typical city-to-airport fare about $45.
+- City rail link: none; a rail extension has been discussed for years with no funding.
+
+## Access
+- The airport's ground-transport concession gives Bluewing Shuttle exclusive curbside pickup and drop-off rights until June 2029.
+- Off-airport options: a park-and-ride lot 1.5 km from the terminal, linked by the airport's free terminal bus.
+
+## Our position
+- 38 coaches, mostly used for school runs and weekend charters; about 9 coaches sit idle between 9:00 and 15:00 on weekdays.
+- Strong safety record; brand known for school transport, not for travellers.

package/packs/product-manager/skills/success-metrics/evals/evals.json

@@ -22,6 +22,34 @@
         "The BLOCKED output routes to prd-draft (or names the goal as what to rerun with)",
         "No primary metric, instrumentation table, or decision rule appears"
       ]
+    },
+    {
+      "id": 3,
+      "held_out": "2026-10-18",
+      "prompt": "We're launching SMS reminders to get first-time blood donors to come back. The brief is in brief.md. Answer the ask in it — how will we know this worked? One page.",
+      "assertions": [
+        "Does not set a target of donating again within 30 days; recognises the 12-week minimum gap between donations and uses a timeframe consistent with it",
+        "Uses or reconciles the export's baseline (4,410 of 15,200, about 29%) rather than taking the brief's 35% as fact",
+        "Deals with the TV campaign starting the same day as a confound — for example a holdout group or comparison that separates the SMS effect — rather than relying on a simple before-and-after",
+        "Includes at least one measure that would show the reminders causing harm, such as opt-outs or complaints",
+        "Acknowledges the stated data gaps (no click data from the SMS vendor; phone bookings not tagged by channel) and does not rely on data that is not collected",
+        "Does not invent baselines or figures not in the brief; any new number is labelled as an assumption or a target",
+        "Fits on about one page"
+      ]
+    },
+    {
+      "id": 4,
+      "held_out": "2026-10-18",
+      "prompt": "we're switching our school lunch line to a pre-order system — kids pick their lunch on a tablet in homeroom in the morning. the principal asked how we'd tell if it worked. few bullets, plain english, im a cafeteria manager not a data person",
+      "assertions": [
+        "Gives a short set of measures without asking questions first",
+        "A few bullets — no more than about 8 bullets or 200 words",
+        "Picks one main measure tied to a plausible purpose of pre-ordering (for example less food waste or shorter lines) and says what change would count as success, rather than listing many equal metrics",
+        "Includes something to watch for that would mean the change is hurting, such as fewer students eating lunch or students who miss the morning order going without",
+        "Measures are things a cafeteria manager can actually count without new data systems (for example meals prepared versus taken, or timing the line)",
+        "Does not invent current numbers (meals served, waste amounts) as facts",
+        "Plain English with no data jargon"
+      ]
     }
   ]
 }

package/packs/product-manager/skills/success-metrics/evals/files/3/brief.md

@@ -0,0 +1,21 @@
+# Donor return reminders — feature brief, Kestrel Blood Service
+
+## The feature
+After a first-time donor gives blood, we send SMS reminders with a link to book their next donation. Phone booking stays available.
+
+## Goal (from the donor programmes manager)
+"Our repeat donation rate is 35%. Target: 50% of first-time donors donate again within 30 days of their first donation."
+
+## Data export (donor database, last full year)
+- First-time donors: 15,200
+- Of those, donated again within 12 months: 4,410
+
+## Facts and constraints
+- Whole-blood donors must wait at least 12 weeks between donations.
+- The SMS vendor reports delivery only. It does not report link clicks.
+- The booking system records the channel ("web" vs "phone") only for web bookings; phone bookings have no channel field.
+- Launch date: March 1. The communications team's national TV campaign "Give Again" also starts on March 1.
+- Donors can reply STOP to opt out of SMS; the vendor logs opt-outs.
+
+## Ask
+One page: how will we know this worked?

package/packs/product-manager/skills/user-journey-map/evals/evals.json

@@ -25,6 +25,32 @@
         "The BLOCKED output names both missing anchors and what to rerun with",
         "No invented persona, stages, or Mermaid diagram appear"
       ]
+    },
+    {
+      "id": 3,
+      "held_out": "2026-10-18",
+      "prompt": "We run the city's emergency rent relief programme. Our research notes are in research_notes.md. Do what the ask at the bottom says.",
+      "assertions": [
+        "Identifies landlord confirmation (3,100 to 1,500) as the largest drop in the funnel, rather than adopting the programme manager's view that document upload is the main problem",
+        "Does not propose removing the landlord-confirmation step, which the funding rules require; improvements work within it",
+        "Reports the overall success rate correctly as about 1 in 4 (1,180 of 4,800), catching the manager's '1 in 3'",
+        "Flags that all six interviews were in English while about 40% of applications were started in Spanish, so the experience of Spanish-language applicants is not evidenced",
+        "Pains and emotions trace to the interview notes or funnel data; does not invent quotes, statistics, or research not in the notes",
+        "Stays within about two pages"
+      ]
+    },
+    {
+      "id": 4,
+      "held_out": "2026-10-18",
+      "prompt": "im a solo wedding photographer. couples keep telling me booking me was 'stressful' even though they love the photos. can you map out what it's like for them from first inquiry to getting their photos back? one page max, i just want to know where to fix things",
+      "assertions": [
+        "Delivers the map without asking questions first",
+        "Covers the whole span from first inquiry to the couple receiving their photos",
+        "Fits on one page",
+        "Ends with a clear pointer to the few places to fix first, specific to a solo wedding photographer (for example slow replies to inquiries, contract and deposit, or the long wait for photos)",
+        "Does not present invented client quotes, survey results, or statistics as real findings; pains the user did not mention are framed as likely or as things to check",
+        "Plain language for a solo business owner, without UX jargon"
+      ]
     }
   ]
 }

package/packs/product-manager/skills/user-journey-map/evals/files/3/research_notes.md

@@ -0,0 +1,30 @@
+# Calder City Rent Relief — research notes for the tenant journey map
+
+## Programme manager's view
+"Our biggest problem is document upload. We lose a huge share of applicants there. Overall only 1,180 of 4,800 applicants were paid — a 1 in 3 success rate."
+
+## Application funnel (last 6 months)
+| Step | Applicants reaching it |
+|---|---|
+| Started application | 4,800 |
+| Finished eligibility check | 4,350 |
+| Uploaded all documents | 3,100 |
+| Landlord confirmation received | 1,500 |
+| Approved | 1,320 |
+| Paid to landlord | 1,180 |
+
+About 40% of applications were started in Spanish.
+
+## Tenant interviews (6 tenants, all interviewed in English)
+- T1: "The eligibility questions were fine. Took ten minutes."
+- T2: "I uploaded my pay stubs on my phone. Two got rejected as blurry, so I did them again at the library."
+- T3: "My landlord never answered the email from the city. I called him three times. I had no idea what was happening for weeks."
+- T4: "I didn't know if my application was stuck or just slow. Nobody told me anything."
+- T5: "I was scared we'd be evicted before the money came. The notice said 14 days."
+- T6: "When it got approved I cried. But then it was another two weeks before the landlord got paid and he was still angry."
+
+## Funding rules
+- The landlord confirmation step is required by the state funding rules and cannot be removed.
+
+## Ask
+Map the tenant's journey from finding out about the programme to the rent being paid, and tell us where to focus. Keep it to two pages.