skilldrop-cli 0.16.6 → 0.16.7

package/packs/career/skills/resume-tailor/evals/files/6/posting.md

@@ -0,0 +1,25 @@
+# Finance Analyst
+
+**Orrell Leisure Group** · Birmingham · £36,000 – £40,000
+
+Orrell runs 22 gyms across the Midlands. The Finance Analyst reports to the Financial Controller and produces the monthly management pack for site managers.
+
+## What you'll do
+
+- Build the monthly management accounts pack: variance to budget by site, with commentary
+- Own the month-end close for payroll and utilities accruals
+- Maintain the site-level budget model and reforecast each quarter
+- Build dashboards in Power BI for the operations team
+
+## What we need
+
+- Part-qualified ACCA or CIMA, or AAT Level 4 complete
+- At least 2 years in a management accounts or analyst role
+- Advanced Excel (pivot tables, lookups, modelling)
+- Experience preparing variance analysis and budget commentary
+- Experience with Power BI or a similar reporting tool
+
+## Nice to have
+
+- Experience in a multi-site business
+- Experience with Sage or NetSuite

package/packs/career/skills/resume-tailor/evals/files/6/resume.md

@@ -0,0 +1,29 @@
+# Grace Mbeki
+
+Birmingham · grace.mbeki@example.com · 07700 900455
+
+## Work
+
+**Accounts Payable Clerk** — Halden Building Supplies, Birmingham
+February 2021 – present
+
+- Process around 400 supplier invoices a month in Sage 200, matching each to its purchase order and delivery note.
+- Reconcile 60 supplier statements each month and chase the differences.
+- Prepare the fortnightly payment run for the Finance Manager to approve.
+- Built an Excel workbook (pivot tables and lookups) that lists invoices on hold by reason; the purchasing team now uses it every week.
+- Support month-end by posting accruals for goods received and not yet invoiced.
+
+**Finance Administrator** — Corrin Dental Group, Solihull
+September 2018 – January 2021
+
+- Raised sales invoices, banked receipts and kept the petty cash for four practices.
+
+## Qualifications
+
+- AAT Level 3 Diploma in Accounting, 2020
+- Studying AAT Level 4; two of five units passed
+- A levels: Mathematics (B), Economics (C), 2018
+
+## Tools
+
+Sage 200, Excel, Microsoft Teams

package/packs/career/skills/resume-tailor/evals/files/7/posting.md

@@ -0,0 +1,18 @@
+Head Chef
+Calder House Hotel, Peebles - 62 bedrooms, restaurant and events
+
+You will lead the kitchen for our 120-cover restaurant and for weddings and events of up to 180 guests.
+
+Requirements
+- Level 3 Food Safety (Supervising Food Safety in Catering)
+- Experience as a head chef, or as a sous chef with full responsibility for a kitchen
+- Experience leading a team of 12 or more
+- Menu design and costing to a gross profit target of 70%
+- Experience of banqueting and events
+- Stock control and supplier management
+
+Desirable
+- Experience in a hotel kitchen
+- Experience with Procure Wizard or a similar ordering system
+
+Salary 48,000 pounds plus tronc. Live-in available.

package/packs/career/skills/resume-tailor/evals/files/7/resume.md

@@ -0,0 +1,23 @@
+LEO TANAKA
+Edinburgh | leo.tanaka@example.com | 07700 900271
+
+Sous Chef, The Larder at Queensgate (80-cover restaurant), Edinburgh
+May 2022 - present
+- Run the kitchen on the head chef's two days off each week.
+- Lead a brigade of 6 on service.
+- Write and cost the weekly specials; do the fish and vegetable orders.
+- Trained 3 commis chefs, one now a chef de partie.
+- Keep the allergen matrix up to date for every menu change.
+- Environmental health inspection in 2025: rated 5.
+
+Chef de Partie, Harbour Rooms Hotel, Leith
+March 2019 - April 2022
+- Ran the sauce section; covered banqueting for weddings in summer.
+
+Commis Chef, Harbour Rooms Hotel, Leith
+August 2017 - March 2019
+
+TRAINING
+SVQ Level 3 Professional Cookery, Edinburgh College, 2019
+Level 2 Food Safety and Hygiene for Catering, renewed 2024
+Allergen awareness training, 2024

package/packs/career/skills/resume-tailor/manifest.json

@@ -0,0 +1,37 @@
+{
+  "name": "resume-tailor",
+  "version": "0.1.0",
+  "description": "Tailor a resume or CV to one job posting without adding anything that isn't true: reorder, cut and reword what the person has already done so the posting's needs are answered first, keep every employer, title, date and number as it was, and report what the posting asks for that the resume cannot claim. A script lists every figure and name in the draft that the original doesn't contain. Works for any occupation and level. Use when the user says \"tailor my resume for this job\", \"adapt my CV to this posting\", \"rewrite my resume for this role\", or pastes a job description next to their resume.",
+  "entrypoint": "SKILL.md",
+  "deps": {
+    "npm": [],
+    "pip": []
+  },
+  "env": {
+    "required": [],
+    "optional": []
+  },
+  "related": [
+    "cover-letter",
+    "doc-critique",
+    "job-fit-analysis"
+  ],
+  "tags": [
+    "resume",
+    "cv",
+    "job-application",
+    "tailoring",
+    "career",
+    "job-search"
+  ],
+  "permissions": {
+    "network": [],
+    "commands": [],
+    "files": "named-paths",
+    "notes": "Reads the draft, source and posting files it is given. Writes nothing."
+  },
+  "model": {
+    "tier": "standard",
+    "rationale": "Reorders and rewords existing content against a posting, with a script listing every figure and name the original doesn't contain."
+  }
+}

package/packs/career/skills/resume-tailor/scripts/claim_check.py

@@ -0,0 +1,247 @@
+#!/usr/bin/env python3
+"""List what a tailored resume or cover letter says that the candidate's own material doesn't.
+
+  python3 claim_check.py DRAFT --source RESUME [--source NOTES ...] [--posting POSTING] [--json]
+
+DRAFT is the text you are about to send. Each --source is something the candidate wrote or
+told you: the original resume, notes from the conversation. --posting is the job posting.
+
+It reports four kinds of finding, each with the line it is on:
+
+  number       a figure in the draft that no source contains ("35%", "$2M", "team of 8")
+  name         a named thing in the draft that no source contains: a tool, an employer, a
+               certification, a degree ("Kubernetes", "Epic", "Green Belt")
+  combination  capitalised words that each appear in a source but never together
+               ("Senior Manager" when the source has "Senior Analyst" and "Office Manager")
+  borrowed     wording the draft shares with the posting and no source uses. Fine when the
+               source shows the same thing in other words; a false claim when it doesn't.
+
+A name or number that the posting contains is marked "in posting": right when the draft is
+talking about the employer, wrong when it is claimed as the candidate's own.
+
+It reads text, so it cannot tell a true claim from a false one. It tells you where to look.
+Exit status: 0 nothing found, 1 findings, 2 a file could not be read. Standard library only.
+"""
+import argparse
+import json
+import re
+import sys
+
+MULT = {"k": 1e3, "thousand": 1e3, "m": 1e6, "mm": 1e6, "million": 1e6,
+        "bn": 1e9, "b": 1e9, "billion": 1e9}
+NUMBER = re.compile(r"(?<![\w.])(\d[\d,]*(?:\.\d+)?)(?:(k|mm|m|bn|b)\b|\s?(thousand|million|billion)\b)?"
+                    r"(\s?%|\s?percent\b|\s?per cent\b)?", re.I)
+WORD = re.compile(r"\.?[A-Za-z][A-Za-z0-9+#']*(?:\.[A-Za-z0-9]+)*")
+LIST_MARK = re.compile(r"^\s*(?:[-*•–>]+|\d+[.)]|#+|\|)\s*")
+LINK = re.compile(r"\[([^\]]*)\]\([^)]*\)")
+SUFFIXES = ("ations", "ation", "ments", "ment", "ings", "ing", "ers", "ies", "ed", "er", "es", "s", "ly")
+
+# Words that start a line or a sentence with a capital letter and name nothing.
+PLAIN = set("""
+a an and are as at be been both but by for from had has have he her his i if in into is it its
+me my no not of on or our she so that the their them then there these they this to up us was
+we were what when where which who why with you your dear sincerely regards kind best yours
+faithfully hiring manager team committee recruiter sir madam re subject date enclosed
+summary profile objective experience employment work history education skills tools
+certifications certification qualifications training projects awards languages interests
+references available request professional career key core technical additional other
+volunteering volunteer contact details achievements highlights about present current
+january february march april may june july august september october november december
+jan feb mar apr jun jul aug sep sept oct nov dec monday tuesday wednesday thursday friday
+saturday sunday full part time application applying while although though because since
+during after before however also please thank thanks between over across within having most each every one two three
+""".split())
+DATE_LINE = re.compile(r"^\s*(?:\d{1,2}(?:st|nd|rd|th)?\s+)?[A-Z][a-z]{2,8}\.?\s+(?:\d{1,2}(?:st|nd|rd|th)?,?\s+)?\d{4}\s*$")
+# The joining words allowed inside a capitalised phrase: "Head of Geography".
+JOINERS = {"of", "and", "for", "the", "in", "at", "&", "de"}
+STOP = PLAIN | set("""
+able across after also been before being between can could did does doing each every experience
+experienced including into just like more most much need needs over role roles some such than
+them through under using very well will within without work worked working would years year
+have with that this from your their about must should strong good great team teams including
+ability skills skill knowledge new help helps support supports ensure make makes
+""".split())
+
+
+def read(path):
+    try:
+        with open(path, encoding="utf-8", errors="replace") as f:
+            return f.read()
+    except OSError as e:
+        print("claim_check: cannot read %s: %s" % (path, e.strerror), file=sys.stderr)
+        sys.exit(2)
+
+
+def clean(line):
+    """A line with its markdown removed, so `**Go**` and `Go` read the same."""
+    line = LINK.sub(r"\1", line)
+    line = line.replace("’", "'").replace("‘", "'").replace("–", " - ").replace("—", " - ")
+    return re.sub(r"[*_`~]", "", line)
+
+
+def numbers(text):
+    """{(value, is_percent)} for every figure in the text. "$2M" and "2 million" are one value."""
+    found = set()
+    for m in NUMBER.finditer(text):
+        try:
+            value = float(m.group(1).replace(",", ""))
+        except ValueError:
+            continue
+        unit = (m.group(2) or m.group(3) or "").lower()
+        found.add((value * MULT.get(unit, 1), bool(m.group(4))))
+    return found
+
+
+def stem(word):
+    w = word.lower().strip("'.")
+    if w.endswith("'s"):
+        w = w[:-2]
+    for s in SUFFIXES:
+        if w.endswith(s) and len(w) - len(s) >= 4:
+            return w[: -len(s)]
+    return w
+
+
+def key(word):
+    """How a name is compared: case, dots, a plural, a possessive or a contraction don't matter."""
+    w = word.lower().strip("'")
+    w = re.sub(r"'(?:s|m|d|ve|ll|re|t)$", "", w)
+    w = w.replace(".", "") or w
+    return w[:-1] if len(w) > 4 and w.endswith("s") and not w.endswith("ss") else w
+
+
+def vocabulary(text):
+    names, stems = set(), set()
+    for line in text.splitlines():
+        for w in WORD.findall(clean(line)):
+            names.add(key(w))
+            stems.add(stem(w))
+    flat = " ".join(key(w) for line in text.splitlines() for w in WORD.findall(clean(line)))
+    return names, stems, " " + flat + " "
+
+
+def looks_named(word):
+    """True for a word that is a name whatever its position: AWS, JavaScript, C++, k8s, .NET."""
+    body = word.lstrip(".")
+    return bool(re.search(r"[A-Z]", body[1:]) or re.search(r"[0-9+#]", body) or word.startswith("."))
+
+
+def phrases(line):
+    """Runs of capitalised or name-shaped words in one line, as lists of words.
+
+    The first word of a line or sentence is capitalised by grammar, not because it is a name,
+    so it only counts when it is name-shaped itself or opens a longer capitalised run."""
+    text = LIST_MARK.sub("", clean(line))
+    out, run, start_of_sentence = [], [], True
+    tokens = re.findall(r"\.?[A-Za-z][A-Za-z0-9+#']*(?:\.[A-Za-z0-9]+)*|&|[.!?;:]\s|[,|·/()]|\s-\s", text + " ")
+    for i, tok in enumerate(tokens):
+        if not (tok[0].isalpha() or tok[0] in ".&") or tok.strip() in (".", "!", "?", ";", ":"):
+            if run:
+                out.append(run)
+            run = []
+            start_of_sentence = tok.strip() in (".", "!", "?")
+            continue
+        if "'" in tok and key(tok) in PLAIN:  # I'm, I've, we'd
+            tok = key(tok)
+        capital = tok[0].isupper() or looks_named(tok)
+        if capital and not (start_of_sentence and not looks_named(tok) and tok.lower() in PLAIN):
+            run.append((tok, start_of_sentence))
+        elif run and tok.lower() in JOINERS and i + 1 < len(tokens) and tokens[i + 1][:1].isupper():
+            run.append((tok, False))
+        else:
+            if run:
+                out.append(run)
+            run = []
+        start_of_sentence = False
+    if run:
+        out.append(run)
+    cleaned = []
+    for r in out:
+        # A lone sentence-opening word ("Led", "Built") is grammar, not a name.
+        if len(r) == 1 and r[0][1] and not looks_named(r[0][0]):
+            continue
+        # A sentence-opening verb in front of a name ("Led AWS migration") is dropped from the run.
+        if len(r) > 1 and r[0][1] and not looks_named(r[0][0]) and r[0][0].lower() not in PLAIN:
+            r = r[1:] if r[0][0].lower().endswith(("ed", "ing")) else r
+        words = [w for w, _ in r]
+        while words and words[-1].lower() in JOINERS:
+            words.pop()
+        if words:
+            cleaned.append(words)
+    return cleaned
+
+
+def check(draft, sources, posting):
+    src_names, src_stems, src_flat = vocabulary("\n".join(sources))
+    src_numbers = numbers("\n".join(clean(s) for s in sources))
+    post_names, post_stems, _ = vocabulary(posting or "")
+    post_numbers = numbers(clean(posting or ""))
+    findings, seen = [], set()
+
+    def add(kind, what, n, line, in_posting=False):
+        if (kind, what.lower()) in seen:
+            return
+        seen.add((kind, what.lower()))
+        findings.append({"kind": kind, "text": what, "line": n, "context": line.strip()[:160],
+                         "in_posting": in_posting})
+
+    for n, raw in enumerate(draft.splitlines(), 1):
+        line = clean(raw)
+        if not line.strip() or DATE_LINE.match(LIST_MARK.sub("", line)):
+            continue  # a letter's own date line claims nothing
+        body = LIST_MARK.sub("", line)
+        for m in NUMBER.finditer(body):
+            got = next(iter(numbers(m.group(0))), None)
+            if got and got not in src_numbers:
+                add("number", m.group(0).strip(), n, raw, got in post_numbers)
+        for words in phrases(raw):
+            content = [w for w in words if w.lower() not in JOINERS]
+            missing = [w for w in content if key(w) not in src_names and w.lower() not in PLAIN]
+            if missing:
+                add("name", " ".join(words), n, raw, all(key(w) in post_names for w in missing))
+            elif len(content) > 1 and all(w.lower() not in PLAIN for w in content):
+                flat = " ".join(key(w) for w in words)
+                if " " + flat + " " not in src_flat:
+                    add("combination", " ".join(words), n, raw)
+        if posting:
+            for w in WORD.findall(body):
+                s = stem(w)
+                if (len(s) >= 4 and w.lower() not in STOP and s in post_stems and s not in src_stems
+                        and key(w) not in src_names and not w[0].isupper() and not looks_named(w)):
+                    add("borrowed", w.lower(), n, raw, True)
+    return findings
+
+
+def main():
+    ap = argparse.ArgumentParser(description="List what a draft says that the candidate's own material doesn't.")
+    ap.add_argument("draft")
+    ap.add_argument("--source", action="append", required=True, metavar="FILE",
+                    help="the candidate's own resume or notes; repeat for more than one")
+    ap.add_argument("--posting", metavar="FILE", help="the job posting")
+    ap.add_argument("--json", action="store_true")
+    a = ap.parse_args()
+
+    findings = check(read(a.draft), [read(p) for p in a.source], read(a.posting) if a.posting else None)
+    if a.json:
+        print(json.dumps({"draft": a.draft, "findings": findings}, indent=1))
+        return 1 if findings else 0
+    if not findings:
+        print("PASS  %s: every figure and name is in the source material." % a.draft)
+        return 0
+    labels = {"number": "Figures not in the source", "name": "Names not in the source",
+              "combination": "Words the source has, but not together",
+              "borrowed": "Wording taken from the posting"}
+    print("REVIEW  %s: %d item(s) to check against the source material." % (a.draft, len(findings)))
+    for kind in ("number", "name", "combination", "borrowed"):
+        rows = [f for f in findings if f["kind"] == kind]
+        if not rows:
+            continue
+        print("\n%s (%d)" % (labels[kind], len(rows)))
+        for f in rows:
+            note = "  [in posting]" if f["in_posting"] and kind != "borrowed" else ""
+            print("  line %d: %s%s\n      %s" % (f["line"], f["text"], note, f["context"]))
+    return 1
+
+
+if __name__ == "__main__":
+    sys.exit(main())

package/packs/data-analytics/skills/metric-definition/evals/evals.json

@@ -14,6 +14,36 @@
         "Includes at least three runnable sanity checks, including a known-answer fixture or a reconciliation with billing",
         "Does not invent tables, columns or an owner not given in the prompt; anything assumed is tagged [assumption] and listed under open decisions"
       ]
+    },
+    {
+      "id": 2,
+      "held_out": "2026-10-05",
+      "prompt": "Finance and CX keep turning up to the Monday meeting with different refund rates and it's getting silly. I've pasted the Slack thread into thread.md and the table descriptions into tables.md. Can you write down one proper definition of refund rate that we can all use, with the SQL? We're on Postgres.",
+      "assertions": [
+        "It separates the share of orders refunded from the share of revenue refunded as two differently named measures, or picks one as 'refund rate' and names the other as a separate measure; it does not blend them into one formula",
+        "It points out that 312 refunded orders out of 7,480 is about 4.2%, not the 4.6% quoted in the thread",
+        "The order-based SQL counts each refunded order once (COUNT DISTINCT on order_id, EXISTS, or refunds aggregated per order before joining), so orders with several refund rows are not double-counted",
+        "It states one rule for which month a refund belongs to (the order's month or the refund's month) and says what happens to a February order refunded in March",
+        "It says a single money-based rate across both stores cannot be computed from these tables as they stand because there is no exchange-rate data, and it does not invent an exchange rate or an exchange-rate table",
+        "Test orders are excluded with a condition on orders.is_test, and staff orders are flagged as not identifiable from the listed columns; it does not use a staff flag column that does not exist",
+        "Month boundaries are set in UK time (Europe/London) against timestamps stored in UTC",
+        "It lists whether cancelled orders count in the denominator and who owns the metric as open decisions, without naming an owner the thread did not name"
+      ]
+    },
+    {
+      "id": 3,
+      "held_out": "2026-10-05",
+      "prompt": "need a proper definition of 'time to first response' for our support team. we use a helpdesk tool and I can export tickets to a spreadsheet. my boss wants it on the weekly report. there's 8 agents. not sure if weekends should count, we don't work them",
+      "assertions": [
+        "It delivers a written definition in the reply and does not stop at a list of questions",
+        "It defines the start and the end of the clock (ticket created to the first reply from a human agent) and says automatic acknowledgement emails do not count as the first response, or flags that as a decision",
+        "Whether weekends and out-of-hours time count is given as an open decision with a recommended default or with both versions described; it is not decided silently",
+        "It says which statistic goes on the report (median, mean or a percentile), not just 'average' left undefined",
+        "It says how tickets that have had no reply yet when the report is run are treated",
+        "It says which week a ticket is counted in (the week it was created or the week it was answered)",
+        "Any export column names it uses are marked as assumed, since no export layout was given",
+        "It does not state a response-time target or SLA figure as the team's own, since none was given"
+      ]
     }
   ]
 }

package/packs/data-analytics/skills/metric-definition/evals/files/2/tables.md

@@ -0,0 +1,42 @@
+# Tallow & Thyme: tables (Postgres 15)
+
+All timestamps are `timestamptz`, stored in UTC. The business reports on UK time.
+
+## orders
+
+| Column | Type | Notes |
+|---|---|---|
+| order_id | bigint | primary key |
+| customer_id | bigint | references customers |
+| store | text | 'uk' or 'eu' |
+| currency | text | 'GBP' for the uk store, 'EUR' for the eu store |
+| gross_amount_minor | integer | order total in pence (GBP) or cents (EUR) |
+| placed_at | timestamptz | when the order was placed |
+| status | text | 'placed', 'delivered', 'cancelled' |
+| is_test | boolean | true for orders made by the test harness |
+
+## refunds
+
+| Column | Type | Notes |
+|---|---|---|
+| refund_id | bigint | primary key |
+| order_id | bigint | references orders |
+| amount_minor | integer | in the same currency as the order |
+| reason | text | free text typed by the CX agent |
+| created_at | timestamptz | when the refund was issued |
+
+An order can have more than one refund row: a missing ingredient is refunded, then a second
+problem with the same box is refunded later. About 1 in 9 refunded orders has two or more
+rows.
+
+## customers
+
+| Column | Type | Notes |
+|---|---|---|
+| customer_id | bigint | primary key |
+| email | text | |
+| created_at | timestamptz | |
+
+## Not in the database
+
+There is no exchange-rate table. Finance converts EUR to GBP by hand in a spreadsheet.

package/packs/data-analytics/skills/metric-definition/evals/files/2/thread.md

@@ -0,0 +1,40 @@
+# #ops-metrics, exported 3 March 2026
+
+**Hattie Blum** (CX lead) 09:12
+Feb refund rate is 4.6%. 312 refunded orders out of 7,480 orders.
+
+**Rupert Osei** (Finance) 09:20
+I get 3.5%. £9,870 refunded on £284,300 gross.
+
+**Hattie Blum** 09:24
+Well mine is orders, yours is money. I count an order as refunded if it shows up in the
+refunds table. I go by the date the refund was issued, so that's refunds issued in Feb.
+
+**Rupert Osei** 09:31
+I go by the month the order was placed. If a Feb order gets refunded in March it goes back
+against Feb, that's how the P&L works. Also mine is the UK store only. The EU store is in a
+separate sheet and I convert the euros by hand at whatever the rate is that day, I haven't
+done Feb yet.
+
+**Hattie Blum** 09:33
+Mine is both stores.
+
+**Mina Castellanos** (founder) 09:47
+This is why the Monday meeting goes round in circles. I want ONE refund rate for the whole
+business, both stores, monthly, on UK time. And take out test orders, and staff orders too.
+Staff get free boxes and we refund them through the system so they make it look worse than
+it is.
+
+**Hattie Blum** 09:52
+There's an is_test flag on orders. No idea how you'd spot staff orders though, we just know
+the names.
+
+**Rupert Osei** 09:55
+Do cancelled orders count as orders? I leave them in, they were placed.
+
+**Hattie Blum** 09:58
+I take them out.
+
+**Mina Castellanos** 10:05
+Someone write it down properly please. Who owns it after that, tbd, let's get the definition
+sorted first.

package/packs/dev-team/skills/contribution-wizard/evals/evals.json

@@ -14,6 +14,35 @@
         "Ends with next steps that place files under packs/sre-oncall/skills/sla-credit-calculator/ and require python3 validate.py to pass before opening a PR",
         "Does not invent inputs, deps, or related skills the author did not give"
       ]
+    },
+    {
+      "id": 2,
+      "held_out": "2026-10-05",
+      "prompt": "I want to add a skill to our team's skills repo. The repo is this folder (CONTRIBUTING.md, AGENTS.md, existing skills under skills/). The new one is carrier-invoice-audit: you give it a carrier's freight invoice and the rate confirmation for that load, and it lists every line where the carrier billed more than we agreed, with the difference. Ignore differences of $5.00 or less per line. It goes under billing. Related skills are rate-con-extractor and bol-parser. And if it finds a detention charge on the invoice it should kick off detention-claim-letter automatically at the end. Write everything I need for the pull request.",
+      "assertions": [
+        "The skill is delivered as skills/carrier-invoice-audit/ containing skill.yaml, INSTRUCTIONS.md and tests/cases.yaml, the layout CONTRIBUTING.md requires, and not as some other layout such as manifest.json plus SKILL.md",
+        "The related list contains rate-con-extractor and does not contain bol-parser, and the reply says bol-parser is not a skill in the repo",
+        "The skill's instructions do not make it run or hand off to detention-claim-letter, and the reply says the repo's rule that a skill never calls another skill is the reason",
+        "The owner field is a team that appears in OWNERS.md (billing-ops, dispatch or claims), and the reply tells the author the owner was not given and needs confirming",
+        "tests/cases.yaml has at least five cases, one of which covers a missing input such as no rate confirmation supplied, following AGENTS.md's rule over the three cases CONTRIBUTING.md asks for",
+        "The $5.00 per-line tolerance appears as an exact figure in INSTRUCTIONS.md, and no other threshold or percentage is introduced",
+        "A CATALOG.md row for carrier-invoice-audit is provided for the billing table",
+        "The reply tells the author to run make check before opening the pull request"
+      ]
+    },
+    {
+      "id": 3,
+      "held_out": "2026-10-05",
+      "prompt": "need a new skill for our skills repo. takes the standup notes people paste from slack during the week and writes the friday update that goes to our director. not sure on the name yet, weekly-digest or friday-rollup. Priya wanted it to pull ticket status from Jira too but we don't have API access sorted so leave that out for now i guess. don't know which folder it should live in, whoever set the repo up is on leave. can you write the skill up plus a few test cases so i can open a PR when she's back",
+      "assertions": [
+        "It delivers a drafted instruction file for the skill and at least three test cases, and does not stop at a list of questions",
+        "It says the repo's conventions (required files, metadata format or folder) were not available and that the file layout used is an assumption to check",
+        "It uses one name for the skill throughout and tells the author the choice between weekly-digest and friday-rollup is still theirs to make",
+        "The skill as written does not fetch from Jira and declares no Jira credential, API key or dependency; Jira is mentioned only as left out for now",
+        "The skill's instructions tell it to report only what is in the pasted standup notes and not to add progress, blockers or ticket status the notes do not contain",
+        "It does not invent the director's name, the team's name, team members other than Priya, or a specific repo folder stated as fact",
+        "The test cases use realistic pasted standup notes as input, and at least one covers thin or missing notes"
+      ]
     }
   ]
 }

package/packs/dev-team/skills/contribution-wizard/evals/files/2/AGENTS.md

@@ -0,0 +1,9 @@
+# Instructions for agents working in this repo
+
+Read `CONTRIBUTING.md` first. The rules below were added after it and win where the two differ.
+
+1. **A skill never calls, starts or hands off to another skill.** Each folder has to work when it is the only one installed. If a second skill is the natural follow-up, say so to the user in the final line of the output and let them run it.
+2. **Five test cases minimum, not three.** At least one case must cover missing input: the user leaves out something listed under `inputs`, and the expected behaviour is that the skill says what is missing and does not guess.
+3. **No customer or carrier names from real accounts** in instructions or tests. Use made-up ones.
+4. **Money rules are written as numbers.** If a skill applies a threshold or tolerance, the exact figure goes in `INSTRUCTIONS.md`. "A reasonable tolerance" is not acceptable.
+5. Do not edit `OWNERS.md`. If no listed team fits, ask in the pull request.

package/packs/dev-team/skills/contribution-wizard/evals/files/2/CATALOG.md

@@ -0,0 +1,18 @@
+# Skill catalogue
+
+## billing
+
+| skill | what it does | owner |
+|---|---|---|
+| [`rate-con-extractor`](skills/rate-con-extractor/INSTRUCTIONS.md) | Pulls the agreed rate, fuel surcharge and accessorials out of a rate confirmation into a table. | billing-ops |
+
+## dispatch
+
+| skill | what it does | owner |
+|---|---|---|
+
+## claims
+
+| skill | what it does | owner |
+|---|---|---|
+| [`detention-claim-letter`](skills/detention-claim-letter/INSTRUCTIONS.md) | Drafts a detention claim letter to a shipper or receiver from the gate timestamps and the free-time terms. | claims |

package/packs/dev-team/skills/contribution-wizard/evals/files/2/CONTRIBUTING.md

@@ -0,0 +1,45 @@
+# Contributing to larkfield-agent-skills
+
+Internal skills for the Larkfield Freight operations and billing teams. One folder per skill under `skills/`.
+
+## What a skill folder contains
+
+```
+skills/<skill-name>/
+  skill.yaml          # metadata, see below
+  INSTRUCTIONS.md     # what the agent does
+  tests/cases.yaml    # test cases
+```
+
+`<skill-name>` is lowercase kebab-case and matches `name` in `skill.yaml`.
+
+## skill.yaml
+
+All of these are required:
+
+| field | notes |
+|---|---|
+| `name` | same as the folder |
+| `summary` | one sentence, 160 characters or fewer |
+| `owner` | a team listed in `OWNERS.md` — the team that answers when the skill is wrong |
+| `category` | `billing`, `dispatch` or `claims` |
+| `inputs` | list of what the user must supply |
+| `related` | list of other skills in this repo (may be empty). Every entry must be a folder under `skills/` |
+| `network` | `false` unless the security team has signed off |
+
+## INSTRUCTIONS.md
+
+Three sections, in this order: `## Purpose`, `## Steps`, `## Never`.
+`## Never` lists at least two things the skill must not do.
+
+## Tests
+
+`tests/cases.yaml` needs at least 3 cases. Each case has `name`, `input` and `expect` (a list of things that must be true of the output).
+
+## Catalogue
+
+Add one row for the skill to `CATALOG.md`, in the table for its category, keeping the rows in alphabetical order.
+
+## Before you open a pull request
+
+Run `make check`. It validates every `skill.yaml`, checks that `related` entries exist and counts the test cases. A pull request with a failing `make check` is not reviewed.

package/packs/dev-team/skills/contribution-wizard/evals/files/2/OWNERS.md

@@ -0,0 +1,7 @@
+# Owning teams
+
+| team | covers | contact |
+|---|---|---|
+| `billing-ops` | carrier payables, customer invoicing | Ines Okonkwo-Reyes |
+| `dispatch` | load planning, carrier assignment | Tobias Vrana |
+| `claims` | detention, damage and shortage claims | Marguerite Dlamini |

package/packs/dev-team/skills/contribution-wizard/evals/files/2/skills/detention-claim-letter/INSTRUCTIONS.md

@@ -0,0 +1,20 @@
+# detention-claim-letter
+
+## Purpose
+
+When a driver is held at a facility past the agreed free time, Larkfield claims detention from the party that caused the delay. This skill drafts that letter with the arithmetic shown so the recipient can check it.
+
+## Steps
+
+1. Work out time on site: departure minus arrival, in hours and minutes.
+2. Subtract the free time. If the result is zero or less, say that no detention is owed and stop.
+3. Bill the remainder in the increments the terms state. If the terms do not state an increment, bill in 15-minute increments rounded down.
+4. Apply the cap if the terms state one.
+5. Write the letter: load number, facility, the timestamps, the calculation line by line, the amount claimed, and a reply-by date 14 days from the letter date.
+6. End the output with one line listing what evidence to attach (the signed bill of lading with in and out times).
+
+## Never
+
+- Never claim an amount without showing the timestamps it came from.
+- Never round time on site up.
+- Never state that payment is overdue; this is a first claim.