skilldrop-cli 0.16.11 → 0.16.13
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bin/skilldrop.js +36 -11
- package/catalogue.json +8 -6
- package/contracts/catalogue.schema.json +15 -1
- package/model-routing.json +1 -17
- package/package.json +1 -1
- package/packs/ai-engineering/skills/ai-readiness-assessment/SKILL.md +1 -1
- package/packs/ai-engineering/skills/ai-readiness-assessment/manifest.json +2 -3
- package/packs/ai-engineering/skills/llm-eval-harness/SKILL.md +1 -1
- package/packs/ai-engineering/skills/llm-eval-harness/manifest.json +2 -2
- package/packs/api-builder/pack.json +1 -1
- package/packs/api-builder/skills/eval-harness-generator/SKILL.md +1 -1
- package/packs/api-builder/skills/eval-harness-generator/manifest.json +2 -2
- package/packs/api-builder/skills/tool-use-schema-writer/SKILL.md +3 -3
- package/packs/api-builder/skills/tool-use-schema-writer/evals/evals.json +1 -1
- package/packs/api-builder/skills/tool-use-schema-writer/manifest.json +2 -2
- package/packs/core/skills/brief-intake/SKILL.md +2 -2
- package/packs/core/skills/brief-intake/manifest.json +3 -3
- package/packs/core/skills/brief-intake/templates/brief-tech-comparison.md +1 -1
- package/packs/core/skills/council-review/SKILL.md +2 -2
- package/packs/core/skills/council-review/manifest.json +2 -2
- package/packs/core/skills/council-review/reference.md +0 -1
- package/packs/core/skills/doc-critique/SKILL.md +1 -1
- package/packs/core/skills/doc-critique/manifest.json +2 -2
- package/packs/core/skills/doc-critique/rubrics/tech-comparison.md +1 -1
- package/packs/data-analytics/pack.json +8 -14
- package/packs/data-analytics/skills/dashboard-spec/SKILL.md +2 -2
- package/packs/data-analytics/skills/dashboard-spec/examples/support-staffing.md +1 -1
- package/packs/data-analytics/skills/dashboard-spec/manifest.json +1 -10
- package/packs/data-analytics/skills/dashboard-spec/templates/dashboard-spec.md +1 -1
- package/packs/data-analytics/skills/sql-review/SKILL.md +2 -2
- package/packs/data-analytics/skills/sql-review/manifest.json +1 -11
- package/packs/dev-team/skills/migration-plan/SKILL.md +1 -1
- package/packs/dev-team/skills/migration-plan/manifest.json +2 -2
- package/packs/product-manager/skills/business-case/SKILL.md +2 -2
- package/packs/product-manager/skills/business-case/manifest.json +3 -3
- package/packs/product-manager/skills/strategy-analysis/SKILL.md +3 -3
- package/packs/product-manager/skills/strategy-analysis/evals/evals.json +1 -1
- package/packs/product-manager/skills/strategy-analysis/manifest.json +2 -3
- package/packs/research/skills/hypothesis-comparison/SKILL.md +3 -3
- package/packs/research/skills/hypothesis-comparison/manifest.json +2 -3
- package/packs/research/skills/research-plan/SKILL.md +0 -1
- package/packs/research/skills/research-plan/manifest.json +2 -3
- package/packs/skill-engineering/skills/skill-author/SKILL.md +1 -1
- package/packs/skill-engineering/skills/skill-author/manifest.json +1 -2
- package/packs/skill-engineering/skills/skill-review/SKILL.md +1 -1
- package/packs/skill-engineering/skills/skill-review/manifest.json +1 -2
- package/packs/solution-architect/skills/api-contract-draft/SKILL.md +1 -1
- package/packs/solution-architect/skills/api-contract-draft/manifest.json +2 -2
- package/packs/data-analytics/skills/metric-definition/SKILL.md +0 -92
- package/packs/data-analytics/skills/metric-definition/evals/eval_queries.json +0 -34
- package/packs/data-analytics/skills/metric-definition/evals/evals.json +0 -79
- package/packs/data-analytics/skills/metric-definition/evals/files/2/tables.md +0 -42
- package/packs/data-analytics/skills/metric-definition/evals/files/2/thread.md +0 -40
- package/packs/data-analytics/skills/metric-definition/evals/files/4/tables.md +0 -32
- package/packs/data-analytics/skills/metric-definition/evals/files/4/thread.md +0 -31
- package/packs/data-analytics/skills/metric-definition/examples/refund-rate.md +0 -104
- package/packs/data-analytics/skills/metric-definition/manifest.json +0 -41
- package/packs/data-analytics/skills/metric-definition/reference.md +0 -67
- package/packs/data-analytics/skills/metric-definition/templates/metric-spec.md +0 -70
- package/packs/dev-team/skills/contribution-wizard/SKILL.md +0 -142
- package/packs/dev-team/skills/contribution-wizard/evals/eval_queries.json +0 -30
- package/packs/dev-team/skills/contribution-wizard/evals/evals.json +0 -78
- package/packs/dev-team/skills/contribution-wizard/evals/files/2/AGENTS.md +0 -9
- package/packs/dev-team/skills/contribution-wizard/evals/files/2/CATALOG.md +0 -18
- package/packs/dev-team/skills/contribution-wizard/evals/files/2/CONTRIBUTING.md +0 -45
- package/packs/dev-team/skills/contribution-wizard/evals/files/2/OWNERS.md +0 -7
- package/packs/dev-team/skills/contribution-wizard/evals/files/2/skills/detention-claim-letter/INSTRUCTIONS.md +0 -20
- package/packs/dev-team/skills/contribution-wizard/evals/files/2/skills/detention-claim-letter/skill.yaml +0 -11
- package/packs/dev-team/skills/contribution-wizard/evals/files/2/skills/detention-claim-letter/tests/cases.yaml +0 -25
- package/packs/dev-team/skills/contribution-wizard/evals/files/2/skills/rate-con-extractor/INSTRUCTIONS.md +0 -20
- package/packs/dev-team/skills/contribution-wizard/evals/files/2/skills/rate-con-extractor/skill.yaml +0 -8
- package/packs/dev-team/skills/contribution-wizard/evals/files/2/skills/rate-con-extractor/tests/cases.yaml +0 -32
- package/packs/dev-team/skills/contribution-wizard/evals/files/4/CONTRIBUTING.md +0 -63
- package/packs/dev-team/skills/contribution-wizard/evals/files/4/registry.json +0 -15
- package/packs/dev-team/skills/contribution-wizard/evals/files/4/skills/flag-overdue-holds/PROMPT.md +0 -10
- package/packs/dev-team/skills/contribution-wizard/evals/files/4/skills/flag-overdue-holds/examples/01-expected.md +0 -2
- package/packs/dev-team/skills/contribution-wizard/evals/files/4/skills/flag-overdue-holds/examples/01-input.md +0 -3
- package/packs/dev-team/skills/contribution-wizard/evals/files/4/skills/flag-overdue-holds/examples/02-expected.md +0 -1
- package/packs/dev-team/skills/contribution-wizard/evals/files/4/skills/flag-overdue-holds/examples/02-input.md +0 -2
- package/packs/dev-team/skills/contribution-wizard/evals/files/4/skills/flag-overdue-holds/examples/03-expected.md +0 -1
- package/packs/dev-team/skills/contribution-wizard/evals/files/4/skills/flag-overdue-holds/examples/03-input.md +0 -2
- package/packs/dev-team/skills/contribution-wizard/evals/files/4/skills/flag-overdue-holds/meta.json +0 -7
- package/packs/dev-team/skills/contribution-wizard/evals/files/4/skills/summarise-reading-list/PROMPT.md +0 -11
- package/packs/dev-team/skills/contribution-wizard/evals/files/4/skills/summarise-reading-list/examples/01-expected.md +0 -4
- package/packs/dev-team/skills/contribution-wizard/evals/files/4/skills/summarise-reading-list/examples/01-input.md +0 -4
- package/packs/dev-team/skills/contribution-wizard/evals/files/4/skills/summarise-reading-list/examples/02-expected.md +0 -6
- package/packs/dev-team/skills/contribution-wizard/evals/files/4/skills/summarise-reading-list/examples/02-input.md +0 -2
- package/packs/dev-team/skills/contribution-wizard/evals/files/4/skills/summarise-reading-list/examples/03-expected.md +0 -1
- package/packs/dev-team/skills/contribution-wizard/evals/files/4/skills/summarise-reading-list/examples/03-input.md +0 -1
- package/packs/dev-team/skills/contribution-wizard/evals/files/4/skills/summarise-reading-list/skill.toml +0 -6
- package/packs/dev-team/skills/contribution-wizard/manifest.json +0 -11
- package/packs/solution-architect/skills/db-schema-design/SKILL.md +0 -76
- package/packs/solution-architect/skills/db-schema-design/evals/eval_queries.json +0 -34
- package/packs/solution-architect/skills/db-schema-design/evals/evals.json +0 -75
- package/packs/solution-architect/skills/db-schema-design/evals/files/2/current_tables.sql +0 -29
- package/packs/solution-architect/skills/db-schema-design/evals/files/2/notes.md +0 -53
- package/packs/solution-architect/skills/db-schema-design/evals/files/4/finance-email.md +0 -18
- package/packs/solution-architect/skills/db-schema-design/evals/files/4/members_sample.csv +0 -5
- package/packs/solution-architect/skills/db-schema-design/evals/files/4/points_txn_sample.csv +0 -17
- package/packs/solution-architect/skills/db-schema-design/evals/files/4/requirements.md +0 -28
- package/packs/solution-architect/skills/db-schema-design/manifest.json +0 -11
- package/packs/solution-architect/skills/db-schema-design/reference.md +0 -56
- package/packs/solution-architect/skills/db-schema-design/templates/schema-design.md +0 -75
- package/packs/solution-architect/skills/tech-comparison-matrix/SKILL.md +0 -64
- package/packs/solution-architect/skills/tech-comparison-matrix/evals/eval_queries.json +0 -30
- package/packs/solution-architect/skills/tech-comparison-matrix/evals/evals.json +0 -76
- package/packs/solution-architect/skills/tech-comparison-matrix/evals/files/2/requirements.md +0 -20
- package/packs/solution-architect/skills/tech-comparison-matrix/evals/files/2/vendor-notes.md +0 -29
- package/packs/solution-architect/skills/tech-comparison-matrix/evals/files/4/mobile-options.md +0 -45
- package/packs/solution-architect/skills/tech-comparison-matrix/examples/queue-selection.md +0 -41
- package/packs/solution-architect/skills/tech-comparison-matrix/manifest.json +0 -11
- package/packs/solution-architect/skills/tech-comparison-matrix/templates/matrix.md +0 -45
|
@@ -1,79 +0,0 @@
|
|
|
1
|
-
{
|
|
2
|
-
"skill_name": "metric-definition",
|
|
3
|
-
"evals": [
|
|
4
|
-
{
|
|
5
|
-
"id": 1,
|
|
6
|
-
"prompt": "Product and sales at Northwind both report 'active customers' and the numbers never match. Product counts any account with a login in the last 30 days; sales counts accounts with a paid subscription. We use BigQuery: `analytics.logins` (account_id, login_ts TIMESTAMP), `billing.subscriptions` (account_id, status, plan, started_at, ended_at), and `crm.accounts` (account_id, segment, is_internal BOOL). Leadership wants one monthly number for the board, reported in London time. Write the definition.",
|
|
7
|
-
"assertions": [
|
|
8
|
-
"Splits the two meanings into separately named metrics (for example login-based and paid-subscription-based) instead of one name with a footnote, and states which one the board number uses or lists it as an open decision",
|
|
9
|
-
"Writes the formula as an aggregation over the named columns, using COUNT(DISTINCT account_id)",
|
|
10
|
-
"States the time column, a half-open window and the timezone as Europe/London",
|
|
11
|
-
"States that the metric is not additive and that monthly or segment values must be recomputed, not summed",
|
|
12
|
-
"Excludes internal accounts with an explicit predicate on crm.accounts.is_internal and gives a reason",
|
|
13
|
-
"Provides reference SQL in BigQuery syntax that converts the window boundaries to Europe/London",
|
|
14
|
-
"Includes at least three runnable sanity checks, including a known-answer fixture or a reconciliation with billing",
|
|
15
|
-
"Does not invent tables, columns or an owner not given in the prompt; anything assumed is tagged [assumption] and listed under open decisions"
|
|
16
|
-
]
|
|
17
|
-
},
|
|
18
|
-
{
|
|
19
|
-
"id": 2,
|
|
20
|
-
"held_out": "2026-10-05",
|
|
21
|
-
"prompt": "Finance and CX keep turning up to the Monday meeting with different refund rates and it's getting silly. I've pasted the Slack thread into thread.md and the table descriptions into tables.md. Can you write down one proper definition of refund rate that we can all use, with the SQL? We're on Postgres.",
|
|
22
|
-
"assertions": [
|
|
23
|
-
"It separates the share of orders refunded from the share of revenue refunded as two differently named measures, or picks one as 'refund rate' and names the other as a separate measure; it does not blend them into one formula",
|
|
24
|
-
"It points out that 312 refunded orders out of 7,480 is about 4.2%, not the 4.6% quoted in the thread",
|
|
25
|
-
"The order-based SQL counts each refunded order once (COUNT DISTINCT on order_id, EXISTS, or refunds aggregated per order before joining), so orders with several refund rows are not double-counted",
|
|
26
|
-
"It states one rule for which month a refund belongs to (the order's month or the refund's month) and says what happens to a February order refunded in March",
|
|
27
|
-
"It says a single money-based rate across both stores cannot be computed from these tables as they stand because there is no exchange-rate data, and it does not invent an exchange rate or an exchange-rate table",
|
|
28
|
-
"Test orders are excluded with a condition on orders.is_test, and staff orders are flagged as not identifiable from the listed columns; it does not use a staff flag column that does not exist",
|
|
29
|
-
"Month boundaries are set in UK time (Europe/London) against timestamps stored in UTC",
|
|
30
|
-
"It lists whether cancelled orders count in the denominator and who owns the metric as open decisions, without naming an owner the thread did not name"
|
|
31
|
-
]
|
|
32
|
-
},
|
|
33
|
-
{
|
|
34
|
-
"id": 3,
|
|
35
|
-
"held_out": "2026-10-05",
|
|
36
|
-
"prompt": "need a proper definition of 'time to first response' for our support team. we use a helpdesk tool and I can export tickets to a spreadsheet. my boss wants it on the weekly report. there's 8 agents. not sure if weekends should count, we don't work them",
|
|
37
|
-
"assertions": [
|
|
38
|
-
"It delivers a written definition in the reply and does not stop at a list of questions",
|
|
39
|
-
"It defines the start and the end of the clock (ticket created to the first reply from a human agent) and says automatic acknowledgement emails do not count as the first response, or flags that as a decision",
|
|
40
|
-
"Whether weekends and out-of-hours time count is given as an open decision with a recommended default or with both versions described; it is not decided silently",
|
|
41
|
-
"It says which statistic goes on the report (median, mean or a percentile), not just 'average' left undefined",
|
|
42
|
-
"It says how tickets that have had no reply yet when the report is run are treated",
|
|
43
|
-
"It says which week a ticket is counted in (the week it was created or the week it was answered)",
|
|
44
|
-
"Any export column names it uses are marked as assumed, since no export layout was given",
|
|
45
|
-
"It does not state a response-time target or SLA figure as the team's own, since none was given"
|
|
46
|
-
]
|
|
47
|
-
},
|
|
48
|
-
{
|
|
49
|
-
"id": 4,
|
|
50
|
-
"held_out": "2026-10-06",
|
|
51
|
-
"prompt": "Ops and customer care at Tarn Valley Heating both report first-time fix rate and they're miles apart. The Slack thread is in thread.md and the tables are in tables.md (we're on Snowflake). The FD wants one number for the board pack, ready on the 1st of each month, shown next to last September. Can you write the definition — one page, with the SQL underneath?",
|
|
52
|
-
"assertions": [
|
|
53
|
-
"It points out that 3,912 out of 4,830 is 81%, not the 84% quoted in the thread",
|
|
54
|
-
"The definition counts a repair as not fixed first time when there is a return visit within a stated number of days, and does not rely only on the engineer's follow_up_required flag",
|
|
55
|
-
"It explains the gap between the two teams' figures: 630 of the 3,912 repairs with no follow-up flag had a return visit within 30 days, leaving 3,282 (68%)",
|
|
56
|
-
"A return visit is counted whether it was logged as a 'recall' or as a new 'repair', and recall visits are not themselves added to the denominator as fresh repairs",
|
|
57
|
-
"It says the month's figure cannot be final on the 1st of the next month because the return window has not yet run for late-month repairs, and proposes a provisional figure, a lag or a shorter window",
|
|
58
|
-
"It says a like-for-like comparison with September 2025 is not possible by appliance because appliance_id is empty before 3 March 2026, and says what to do instead (match by address for both periods, or mark the comparison as not comparable)",
|
|
59
|
-
"The SQL uses only the tables and columns listed in tables.md, restricts the denominator to completed repairs, and does not invent a column linking a return visit to its original job",
|
|
60
|
-
"It leaves the metric owner as an open decision, noting the previous owner left in August, and says whether subcontractor jobs are included"
|
|
61
|
-
]
|
|
62
|
-
},
|
|
63
|
-
{
|
|
64
|
-
"id": 5,
|
|
65
|
-
"held_out": "2026-10-06",
|
|
66
|
-
"prompt": "I run a coffee subscription box, just me and one part-timer. An investor asked what our monthly churn is and I realised I've never actually defined it. Billing is on Stripe and I keep a Google Sheet of subscribers. September: started the month with 312 subscribers, 21 cancelled, 9 paused (people can pause for a month or two instead of cancelling), 34 new signed up. Can you give me a quick definition I can stick to every month, and the September number for the investor update?",
|
|
67
|
-
"assertions": [
|
|
68
|
-
"It delivers a written definition and a September figure in the reply, and does not stop at a list of questions",
|
|
69
|
-
"The denominator is stated as subscribers active at the start of the month (312), and the 34 new sign-ups are not netted off against cancellations",
|
|
70
|
-
"It gives September churn on cancellations alone as 21 out of 312, about 6.7%",
|
|
71
|
-
"It does not decide silently whether pauses count: it states the choice made and gives the figure with pauses included, 30 out of 312, about 9.6%",
|
|
72
|
-
"It says revenue churn cannot be worked out from what was given because no prices or plan amounts were provided, and it does not invent a price",
|
|
73
|
-
"It says whether a cancellation is counted in the month the subscriber cancels or the month their paid period ends",
|
|
74
|
-
"It does not quote an industry-average or benchmark churn figure as fact",
|
|
75
|
-
"Any Stripe field names or sheet column names it uses are marked as assumed, since none were given"
|
|
76
|
-
]
|
|
77
|
-
}
|
|
78
|
-
]
|
|
79
|
-
}
|
|
@@ -1,42 +0,0 @@
|
|
|
1
|
-
# Tallow & Thyme: tables (Postgres 15)
|
|
2
|
-
|
|
3
|
-
All timestamps are `timestamptz`, stored in UTC. The business reports on UK time.
|
|
4
|
-
|
|
5
|
-
## orders
|
|
6
|
-
|
|
7
|
-
| Column | Type | Notes |
|
|
8
|
-
|---|---|---|
|
|
9
|
-
| order_id | bigint | primary key |
|
|
10
|
-
| customer_id | bigint | references customers |
|
|
11
|
-
| store | text | 'uk' or 'eu' |
|
|
12
|
-
| currency | text | 'GBP' for the uk store, 'EUR' for the eu store |
|
|
13
|
-
| gross_amount_minor | integer | order total in pence (GBP) or cents (EUR) |
|
|
14
|
-
| placed_at | timestamptz | when the order was placed |
|
|
15
|
-
| status | text | 'placed', 'delivered', 'cancelled' |
|
|
16
|
-
| is_test | boolean | true for orders made by the test harness |
|
|
17
|
-
|
|
18
|
-
## refunds
|
|
19
|
-
|
|
20
|
-
| Column | Type | Notes |
|
|
21
|
-
|---|---|---|
|
|
22
|
-
| refund_id | bigint | primary key |
|
|
23
|
-
| order_id | bigint | references orders |
|
|
24
|
-
| amount_minor | integer | in the same currency as the order |
|
|
25
|
-
| reason | text | free text typed by the CX agent |
|
|
26
|
-
| created_at | timestamptz | when the refund was issued |
|
|
27
|
-
|
|
28
|
-
An order can have more than one refund row: a missing ingredient is refunded, then a second
|
|
29
|
-
problem with the same box is refunded later. About 1 in 9 refunded orders has two or more
|
|
30
|
-
rows.
|
|
31
|
-
|
|
32
|
-
## customers
|
|
33
|
-
|
|
34
|
-
| Column | Type | Notes |
|
|
35
|
-
|---|---|---|
|
|
36
|
-
| customer_id | bigint | primary key |
|
|
37
|
-
| email | text | |
|
|
38
|
-
| created_at | timestamptz | |
|
|
39
|
-
|
|
40
|
-
## Not in the database
|
|
41
|
-
|
|
42
|
-
There is no exchange-rate table. Finance converts EUR to GBP by hand in a spreadsheet.
|
|
@@ -1,40 +0,0 @@
|
|
|
1
|
-
# #ops-metrics, exported 3 March 2026
|
|
2
|
-
|
|
3
|
-
**Hattie Blum** (CX lead) 09:12
|
|
4
|
-
Feb refund rate is 4.6%. 312 refunded orders out of 7,480 orders.
|
|
5
|
-
|
|
6
|
-
**Rupert Osei** (Finance) 09:20
|
|
7
|
-
I get 3.5%. £9,870 refunded on £284,300 gross.
|
|
8
|
-
|
|
9
|
-
**Hattie Blum** 09:24
|
|
10
|
-
Well mine is orders, yours is money. I count an order as refunded if it shows up in the
|
|
11
|
-
refunds table. I go by the date the refund was issued, so that's refunds issued in Feb.
|
|
12
|
-
|
|
13
|
-
**Rupert Osei** 09:31
|
|
14
|
-
I go by the month the order was placed. If a Feb order gets refunded in March it goes back
|
|
15
|
-
against Feb, that's how the P&L works. Also mine is the UK store only. The EU store is in a
|
|
16
|
-
separate sheet and I convert the euros by hand at whatever the rate is that day, I haven't
|
|
17
|
-
done Feb yet.
|
|
18
|
-
|
|
19
|
-
**Hattie Blum** 09:33
|
|
20
|
-
Mine is both stores.
|
|
21
|
-
|
|
22
|
-
**Mina Castellanos** (founder) 09:47
|
|
23
|
-
This is why the Monday meeting goes round in circles. I want ONE refund rate for the whole
|
|
24
|
-
business, both stores, monthly, on UK time. And take out test orders, and staff orders too.
|
|
25
|
-
Staff get free boxes and we refund them through the system so they make it look worse than
|
|
26
|
-
it is.
|
|
27
|
-
|
|
28
|
-
**Hattie Blum** 09:52
|
|
29
|
-
There's an is_test flag on orders. No idea how you'd spot staff orders though, we just know
|
|
30
|
-
the names.
|
|
31
|
-
|
|
32
|
-
**Rupert Osei** 09:55
|
|
33
|
-
Do cancelled orders count as orders? I leave them in, they were placed.
|
|
34
|
-
|
|
35
|
-
**Hattie Blum** 09:58
|
|
36
|
-
I take them out.
|
|
37
|
-
|
|
38
|
-
**Mina Castellanos** 10:05
|
|
39
|
-
Someone write it down properly please. Who owns it after that, tbd, let's get the definition
|
|
40
|
-
sorted first.
|
|
@@ -1,32 +0,0 @@
|
|
|
1
|
-
# Tables (Snowflake, database TARNVALLEY)
|
|
2
|
-
|
|
3
|
-
## FIELD.JOBS — one row per visit
|
|
4
|
-
| column | type | notes |
|
|
5
|
-
|---|---|---|
|
|
6
|
-
| job_id | NUMBER | primary key |
|
|
7
|
-
| address_id | NUMBER | always filled |
|
|
8
|
-
| appliance_id | NUMBER | NULL for every job before 2026-03-03 |
|
|
9
|
-
| job_type | VARCHAR | 'repair', 'recall', 'annual_service', 'install' |
|
|
10
|
-
| status | VARCHAR | 'completed', 'cancelled', 'no_access' |
|
|
11
|
-
| follow_up_required | BOOLEAN | ticked by the engineer on closing the job |
|
|
12
|
-
| engineer_id | NUMBER | |
|
|
13
|
-
| created_at | TIMESTAMP_NTZ | UK local time, when the job was booked |
|
|
14
|
-
| completed_at | TIMESTAMP_NTZ | UK local time, NULL unless status = 'completed' |
|
|
15
|
-
|
|
16
|
-
There is no column that links a recall, or any later visit, to the job it follows.
|
|
17
|
-
|
|
18
|
-
## FIELD.APPLIANCES
|
|
19
|
-
| column | type | notes |
|
|
20
|
-
|---|---|---|
|
|
21
|
-
| appliance_id | NUMBER | primary key |
|
|
22
|
-
| address_id | NUMBER | |
|
|
23
|
-
| make | VARCHAR | |
|
|
24
|
-
| model | VARCHAR | |
|
|
25
|
-
| installed_on | DATE | often NULL |
|
|
26
|
-
|
|
27
|
-
## FIELD.ENGINEERS
|
|
28
|
-
| column | type | notes |
|
|
29
|
-
|---|---|---|
|
|
30
|
-
| engineer_id | NUMBER | primary key |
|
|
31
|
-
| region | VARCHAR | 'north', 'central', 'coast' |
|
|
32
|
-
| employment_type | VARCHAR | 'employed', 'subcontractor' |
|
|
@@ -1,31 +0,0 @@
|
|
|
1
|
-
# #ops-reporting — export, 1–2 October 2026
|
|
2
|
-
|
|
3
|
-
**Gethin Aldous (Regional Ops Manager), 1 Oct 09:12**
|
|
4
|
-
September first-time fix is in. 3,912 of 4,830 completed repairs closed with no follow-up flag. That's 84%. Best month this year.
|
|
5
|
-
|
|
6
|
-
**Roisin Fairweather (Customer Care Lead), 1 Oct 09:40**
|
|
7
|
-
We get 68%. 3,282 of 4,830. We only count a repair as fixed if nobody had to go back within 30 days.
|
|
8
|
-
|
|
9
|
-
**Gethin Aldous, 1 Oct 09:44**
|
|
10
|
-
The engineer ticks follow_up_required when the job needs another visit. That is the record. If they didn't tick it, it was fixed.
|
|
11
|
-
|
|
12
|
-
**Roisin Fairweather, 1 Oct 09:58**
|
|
13
|
-
They don't tick it when they think it's fixed and it isn't. 630 of your 3,912 had someone back out within 30 days. Those customers rang us, not you.
|
|
14
|
-
And you can't just count recalls either. The contact centre logs about half of the go-backs as a brand new repair instead of a recall, because that's the default in the booking screen.
|
|
15
|
-
|
|
16
|
-
**Gethin Aldous, 1 Oct 10:03**
|
|
17
|
-
For what it's worth 4,830 already leaves out cancelled and no-access visits. We agree on that at least.
|
|
18
|
-
|
|
19
|
-
**Nkechi Brandt (BI Analyst), 1 Oct 10:20**
|
|
20
|
-
I pulled both sets of numbers this morning. Two things from the data side:
|
|
21
|
-
- appliance_id has only been filled in since 3 March 2026 when the new engineer app went live. Before that it's NULL and all you have is address_id.
|
|
22
|
-
- Some addresses have more than one appliance (HMOs, small commercial units). So "same address" and "same appliance" are not the same test.
|
|
23
|
-
|
|
24
|
-
**Owen Prys (Finance Director), 2 Oct 08:15**
|
|
25
|
-
The board pack needs one number, ready on the 1st of each month, with last September next to it for comparison. Subcontractors in or out, I don't mind, just tell me which. And who owns this going forward?
|
|
26
|
-
|
|
27
|
-
**Gethin Aldous, 2 Oct 08:31**
|
|
28
|
-
It was Marguerite's before she left in August. Nobody has picked it up.
|
|
29
|
-
|
|
30
|
-
**Roisin Fairweather, 2 Oct 08:40**
|
|
31
|
-
Installs and annual services shouldn't be anywhere near this, by the way. It's a repair measure.
|
|
@@ -1,104 +0,0 @@
|
|
|
1
|
-
# Worked example: order refund rate
|
|
2
|
-
|
|
3
|
-
## Input
|
|
4
|
-
|
|
5
|
-
> Finance says our September refund rate was one number and support says another. Can you write a proper definition? We're on Postgres. Orders are in `orders` (`order_id`, `customer_id`, `created_at` timestamptz, `amount_usd`), refunds in `refunds` (`refund_id`, `order_id`, `status`, `created_at`), and `test_accounts` lists QA customers. Finance reports monthly in New York time. Finance counts an order as refunded if any refund on it went through; support counts refund tickets. The finance analytics lead at Acme owns finance metrics.
|
|
6
|
-
|
|
7
|
-
No questions asked: the decision (monthly finance reporting), the sources and the owner are all in the prompt. The dispute itself shows the two teams are measuring different things, so the spec splits them.
|
|
8
|
-
|
|
9
|
-
## Output
|
|
10
|
-
|
|
11
|
-
# Metric: `order_refund_rate` (v1, draft)
|
|
12
|
-
|
|
13
|
-
**Business question:** What share of the orders placed in a month had at least one refund go through, by region? Read monthly by finance to track refund exposure.
|
|
14
|
-
**Owner:** Finance analytics lead, Acme finance · **Status:** draft · **Last changed:** 2026-10-01
|
|
15
|
-
|
|
16
|
-
**Why support's number differs:** support counts refund tickets, which is a different metric (`refund_tickets`, a count of tickets, not a share of orders). An order with two partial refunds is one refunded order here and two tickets there. Both are legitimate; they need separate names.
|
|
17
|
-
|
|
18
|
-
## Formula
|
|
19
|
-
|
|
20
|
-
| Part | In words | As an aggregation |
|
|
21
|
-
|---|---|---|
|
|
22
|
-
| Numerator | Orders in the window with at least one refund whose status is not `reversed` | `COUNT(DISTINCT o.order_id)` where a qualifying refund exists |
|
|
23
|
-
| Denominator | Orders created in the window, excluding test accounts | `COUNT(DISTINCT o.order_id)` |
|
|
24
|
-
|
|
25
|
-
- **Type:** ratio
|
|
26
|
-
- **Unit and format:** %, 1 decimal
|
|
27
|
-
|
|
28
|
-
## Grain and time
|
|
29
|
-
|
|
30
|
-
- **Entity:** order
|
|
31
|
-
- **Time column:** `orders.created_at`, so the refund is attributed to the month the order was placed. A September order refunded in October counts toward September.
|
|
32
|
-
- **Window:** calendar month, half-open
|
|
33
|
-
- **Timezone:** `America/New_York`
|
|
34
|
-
- **Partial periods:** the current month is shown and labelled "month to date"
|
|
35
|
-
- **Additivity:** not additive. A quarterly or all-region rate is recomputed from the base rows, never averaged from monthly or regional rates.
|
|
36
|
-
|
|
37
|
-
## Filters and exclusions
|
|
38
|
-
|
|
39
|
-
| Exclusion | Predicate | Reason |
|
|
40
|
-
|---|---|---|
|
|
41
|
-
| Test accounts | `NOT EXISTS (SELECT 1 FROM test_accounts t WHERE t.customer_id = o.customer_id)` | QA places real orders in production |
|
|
42
|
-
| Reversed refunds | `r.status IS DISTINCT FROM 'reversed'` | A reversed refund returned no money |
|
|
43
|
-
|
|
44
|
-
## Dimensions
|
|
45
|
-
|
|
46
|
-
| Dimension | Source column | Join path | Cardinality | Allowed? |
|
|
47
|
-
|---|---|---|---|---|
|
|
48
|
-
| Region | `customers.region` | `orders.customer_id = customers.customer_id` | many-to-one `[assumption: customer_id unique]` | yes |
|
|
49
|
-
| Product category | `order_items` → `products.category` | through `order_items` | one-to-many: fans out | no. An order with two categories would count in both. |
|
|
50
|
-
|
|
51
|
-
## Edge cases
|
|
52
|
-
|
|
53
|
-
| Case | Rule |
|
|
54
|
-
|---|---|
|
|
55
|
-
| Refund with NULL status | Counts as a refund `[assumption]` |
|
|
56
|
-
| Zero orders in a region-month | Rate is NULL, shown blank |
|
|
57
|
-
| Late data | Refunds arrive weeks after the order, so a month's rate keeps rising. The month locks for reporting 60 days after month end `[assumption]`; before that, label it "provisional". |
|
|
58
|
-
| Currency | Not applicable: the metric counts orders, not money |
|
|
59
|
-
| DST | Not applicable at month grain; boundaries are converted to New York time |
|
|
60
|
-
| Duplicate refund rows | Handled: the numerator counts distinct orders |
|
|
61
|
-
|
|
62
|
-
## Reference SQL (Postgres)
|
|
63
|
-
|
|
64
|
-
```sql
|
|
65
|
-
WITH base AS ( -- orders placed in the month, no test accounts
|
|
66
|
-
SELECT o.order_id, o.customer_id
|
|
67
|
-
FROM orders o
|
|
68
|
-
WHERE o.created_at >= TIMESTAMP '2026-09-01' AT TIME ZONE 'America/New_York'
|
|
69
|
-
AND o.created_at < TIMESTAMP '2026-10-01' AT TIME ZONE 'America/New_York'
|
|
70
|
-
AND NOT EXISTS (SELECT 1 FROM test_accounts t WHERE t.customer_id = o.customer_id)
|
|
71
|
-
),
|
|
72
|
-
refunded AS ( -- one row per order with a qualifying refund
|
|
73
|
-
SELECT DISTINCT r.order_id
|
|
74
|
-
FROM refunds r
|
|
75
|
-
WHERE r.status IS DISTINCT FROM 'reversed'
|
|
76
|
-
)
|
|
77
|
-
SELECT
|
|
78
|
-
c.region,
|
|
79
|
-
COUNT(rf.order_id)::numeric / NULLIF(COUNT(*), 0) AS order_refund_rate
|
|
80
|
-
FROM base b
|
|
81
|
-
JOIN customers c ON c.customer_id = b.customer_id
|
|
82
|
-
LEFT JOIN refunded rf ON rf.order_id = b.order_id
|
|
83
|
-
GROUP BY c.region;
|
|
84
|
-
```
|
|
85
|
-
|
|
86
|
-
## Sanity checks
|
|
87
|
-
|
|
88
|
-
1. **Join integrity:** `SELECT COUNT(*) FROM base` equals the sum of per-region order counts from the query above with `COUNT(*)` added. If it's lower, orders without a matching customer are being dropped.
|
|
89
|
-
2. **Bounds:** every region's rate is between 0 and 1, or NULL.
|
|
90
|
-
3. **Customer key uniqueness:** `SELECT customer_id FROM customers GROUP BY customer_id HAVING COUNT(*) > 1` returns no rows.
|
|
91
|
-
4. **Known-answer fixture:** five orders in September 2026 New York time: one with no refund, one with a single refund, one with two partial refunds, one with only a reversed refund, and one placed by a test account and refunded. Expected rate: 2 of 4 = 50.0%.
|
|
92
|
-
|
|
93
|
-
## Change log
|
|
94
|
-
|
|
95
|
-
| Version | Date | Change | Approved by |
|
|
96
|
-
|---|---|---|---|
|
|
97
|
-
| v1 | 2026-10-01 | Initial definition | pending: finance analytics lead |
|
|
98
|
-
|
|
99
|
-
## Open decisions
|
|
100
|
-
|
|
101
|
-
- [ ] Does a refund with NULL status count? Defaulted to yes. Confirm with the finance analytics lead.
|
|
102
|
-
- [ ] Lock period of 60 days after month end. Confirm with the finance analytics lead against the refund policy window.
|
|
103
|
-
- [ ] Is `customers.customer_id` unique? Check 3 answers it.
|
|
104
|
-
- [ ] Support's `refund_tickets` metric needs its own spec, owned by support.
|
|
@@ -1,41 +0,0 @@
|
|
|
1
|
-
{
|
|
2
|
-
"name": "metric-definition",
|
|
3
|
-
"version": "0.1.1",
|
|
4
|
-
"description": "Write a metric spec precise enough for a semantic layer or dbt metric to implement and for two analysts to get the same number — name, business question, exact numerator and denominator, grain, time window and timezone, filters and exclusions (test accounts, refunds), allowed dimensions, edge cases (NULLs, late data, currency), owner, reference SQL, and the sanity checks that catch it going wrong. Use when the user says \"define this metric\", \"how exactly do we calculate X\", \"finance and product get different numbers for X\", \"write the metric spec for our semantic layer\", or \"what counts as an active user\".",
|
|
5
|
-
"entrypoint": "SKILL.md",
|
|
6
|
-
"deps": {
|
|
7
|
-
"npm": [],
|
|
8
|
-
"pip": []
|
|
9
|
-
},
|
|
10
|
-
"env": {
|
|
11
|
-
"required": [],
|
|
12
|
-
"optional": []
|
|
13
|
-
},
|
|
14
|
-
"related": [
|
|
15
|
-
"dashboard-spec",
|
|
16
|
-
"data-contract",
|
|
17
|
-
"observability-plan",
|
|
18
|
-
"sql-review",
|
|
19
|
-
"success-metrics"
|
|
20
|
-
],
|
|
21
|
-
"handoff": [
|
|
22
|
-
{
|
|
23
|
-
"to": "sql-review",
|
|
24
|
-
"when": "the reference SQL is written and the metric is about to be certified",
|
|
25
|
-
"purpose": "Check the reference SQL for fan-out, NULL and timezone bugs before it becomes the tie-breaker.",
|
|
26
|
-
"fallback": "If sql-review is absent, run the spec's join-integrity and known-answer checks and note that the SQL has not been independently reviewed."
|
|
27
|
-
}
|
|
28
|
-
],
|
|
29
|
-
"tags": [
|
|
30
|
-
"metrics",
|
|
31
|
-
"semantic-layer",
|
|
32
|
-
"dbt",
|
|
33
|
-
"analytics",
|
|
34
|
-
"data-quality",
|
|
35
|
-
"definitions"
|
|
36
|
-
],
|
|
37
|
-
"model": {
|
|
38
|
-
"tier": "standard",
|
|
39
|
-
"rationale": "Structured specification against a fixed checklist (formula, grain, window, exclusions, edge cases, checks). Needs care with SQL and additivity but no adversarial judgment."
|
|
40
|
-
}
|
|
41
|
-
}
|
|
@@ -1,67 +0,0 @@
|
|
|
1
|
-
# metric-definition reference
|
|
2
|
-
|
|
3
|
-
## Metric types and their traps
|
|
4
|
-
|
|
5
|
-
| Type | Example | Additive across time and dimensions? | Trap |
|
|
6
|
-
|---|---|---|---|
|
|
7
|
-
| Count | orders | Yes | Counting rows of a joined table instead of the entity |
|
|
8
|
-
| Sum | revenue_usd | Yes, within one currency | Mixed currencies; refunds netted in some places and not others |
|
|
9
|
-
| Distinct count | active_customers | **No** | Summing daily or regional counts double-counts entities present in several |
|
|
10
|
-
| Ratio | refund_rate | **No** | Averaging per-day or per-segment ratios instead of dividing the sums |
|
|
11
|
-
| Average | avg_order_value | **No** | It's a ratio (sum / count): roll up the parts, not the averages |
|
|
12
|
-
| Percentile | p90_delivery_hours | **No** | Percentiles can't be combined; recompute from raw rows |
|
|
13
|
-
| Cumulative | customers_to_date | No (it is a running total) | Restating history when late rows land in an old period |
|
|
14
|
-
|
|
15
|
-
For every non-additive metric, the spec says: *"To roll up, recompute from the base rows; never sum or average the reported values."* A semantic layer that stores the numerator and denominator separately can roll up ratios correctly; one that stores only the ratio can't.
|
|
16
|
-
|
|
17
|
-
## Exclusion checklist
|
|
18
|
-
|
|
19
|
-
Ask about each. Every one that applies gets a predicate and a reason in the spec.
|
|
20
|
-
|
|
21
|
-
- Test, QA, demo and internal/staff accounts
|
|
22
|
-
- Refunds, chargebacks, cancellations and voids (and partial refunds)
|
|
23
|
-
- Fraudulent or blocked transactions
|
|
24
|
-
- Deleted, merged or anonymised records (including after a privacy erasure request)
|
|
25
|
-
- Zero-value, free-tier or 100%-discount transactions
|
|
26
|
-
- Duplicates from retries or event replays
|
|
27
|
-
- Records outside the product's scope (wholesale orders, a sister brand)
|
|
28
|
-
|
|
29
|
-
## Edge-case checklist
|
|
30
|
-
|
|
31
|
-
| Case | The spec must say |
|
|
32
|
-
|---|---|
|
|
33
|
-
| NULL in a formula column | Counted, excluded, or defaulted, and to what |
|
|
34
|
-
| Zero denominator | Show NULL (blank), not 0. 0% claims something happened. |
|
|
35
|
-
| Late-arriving data | How many days a closed period may change; when it locks; whether restatements are announced |
|
|
36
|
-
| Currency | Source of rates, which date's rate (transaction date or period end), and the reporting currency |
|
|
37
|
-
| Timezone and DST | IANA zone name; what a "day" means on 23- and 25-hour days |
|
|
38
|
-
| Week definition | ISO week (Monday start) or another start day, and how week 1 is numbered |
|
|
39
|
-
| Backfills | Whether historical values change when a backfill runs, and who is told |
|
|
40
|
-
| Definition change | New version number, change-log entry, cutover date; old version kept queryable for comparison |
|
|
41
|
-
|
|
42
|
-
## Sanity-check catalogue
|
|
43
|
-
|
|
44
|
-
Pick at least three. Each is written as a runnable query or test, not a description.
|
|
45
|
-
|
|
46
|
-
1. **Reconciliation with a system of record.** The metric's total for a closed period matches the ledger, billing system or source report within a stated tolerance.
|
|
47
|
-
2. **Bounds.** Rates within 0 and 1; counts not negative; revenue not negative unless refunds are netted.
|
|
48
|
-
3. **Additivity.** For additive metrics only, the sum over a dimension equals the total. For distinct counts, the sum over a dimension is **at least** the total.
|
|
49
|
-
4. **Join integrity.** Row count of the base population before and after each dimension join is equal.
|
|
50
|
-
5. **Known-answer fixture.** 5–10 hand-built rows covering each exclusion and edge case, with the expected result worked out by hand. Run it in CI (a dbt unit test or a plain SQL assertion).
|
|
51
|
-
6. **Period-over-period alert.** A change beyond a stated threshold (for example ±30% week over week) opens a ticket to check before anyone reports it.
|
|
52
|
-
7. **Freshness.** The latest timestamp in the source is recent enough for the window being reported.
|
|
53
|
-
|
|
54
|
-
## Mapping to a semantic layer
|
|
55
|
-
|
|
56
|
-
Semantic layers differ in syntax but share the same parts. Map the spec's fields like this, then write the definition in your tool's own syntax and check it against that tool's documentation.
|
|
57
|
-
|
|
58
|
-
| Spec field | Semantic layer concept |
|
|
59
|
-
|---|---|
|
|
60
|
-
| Entity | Primary entity or key of the model |
|
|
61
|
-
| Time column and window | Aggregation time dimension and granularity |
|
|
62
|
-
| Numerator, denominator | Measures (an aggregation plus an expression), combined in a ratio metric |
|
|
63
|
-
| Filters and exclusions | A filter on the metric or measure, or exclusion done in the underlying model |
|
|
64
|
-
| Allowed dimensions | Dimensions reachable through declared joins or entity relationships |
|
|
65
|
-
| Owner, status, version | Metadata or `meta` fields, plus the repo's code owners |
|
|
66
|
-
|
|
67
|
-
dbt's MetricFlow, for example, defines measures inside semantic models and offers simple, ratio, cumulative, derived and conversion metric types; a ratio metric there names a numerator metric and a denominator metric. Cube, LookML and other tools use their own terms for the same parts. Do the exclusions in the modelled table where you can, so every metric built on it inherits them, rather than repeating the filter in each metric.
|
|
@@ -1,70 +0,0 @@
|
|
|
1
|
-
# Metric: `<metric_name>` (v<N>, <draft | certified>)
|
|
2
|
-
|
|
3
|
-
**Business question:** <one sentence: what decision this number informs, and for whom>
|
|
4
|
-
**Owner:** <role, team> · **Status:** <draft | certified> · **Last changed:** <YYYY-MM-DD>
|
|
5
|
-
|
|
6
|
-
## Formula
|
|
7
|
-
|
|
8
|
-
| Part | In words | As an aggregation |
|
|
9
|
-
|---|---|---|
|
|
10
|
-
| Numerator | <what is counted or summed> | `<COUNT(DISTINCT x) / SUM(y) ...>` |
|
|
11
|
-
| Denominator | <population it is divided by, or "none"> | `<...>` |
|
|
12
|
-
|
|
13
|
-
- **Type:** <count | sum | distinct count | ratio | average | percentile | cumulative>
|
|
14
|
-
- **Unit and format:** <%, 1 decimal | USD, 0 decimals | count>
|
|
15
|
-
|
|
16
|
-
## Grain and time
|
|
17
|
-
|
|
18
|
-
- **Entity:** <order | customer | account | session>
|
|
19
|
-
- **Time column:** `<table.column>`, because <why this timestamp and not another>
|
|
20
|
-
- **Window:** <calendar month | ISO week | rolling 28 days | trailing 7 complete days>, half-open (`>= start AND < end`)
|
|
21
|
-
- **Timezone:** `<IANA name>`
|
|
22
|
-
- **Partial periods:** <shown and marked "to date" | hidden until complete>
|
|
23
|
-
- **Additivity:** <additive across time and dimensions | not additive: recompute from base rows to roll up>
|
|
24
|
-
|
|
25
|
-
## Filters and exclusions
|
|
26
|
-
|
|
27
|
-
| Exclusion | Predicate | Reason |
|
|
28
|
-
|---|---|---|
|
|
29
|
-
| <test accounts> | `<SQL>` | <why> |
|
|
30
|
-
|
|
31
|
-
## Dimensions
|
|
32
|
-
|
|
33
|
-
| Dimension | Source column | Join path | Cardinality | Allowed? |
|
|
34
|
-
|---|---|---|---|---|
|
|
35
|
-
| <region> | `<customers.region>` | `<orders.customer_id = customers.customer_id>` | <many-to-one> | <yes> |
|
|
36
|
-
| <product category> | `<...>` | `<...>` | <one-to-many: fans out> | <no, or allocation rule> |
|
|
37
|
-
|
|
38
|
-
## Edge cases
|
|
39
|
-
|
|
40
|
-
| Case | Rule |
|
|
41
|
-
|---|---|
|
|
42
|
-
| NULLs | <...> |
|
|
43
|
-
| Zero denominator | <NULL, not 0> |
|
|
44
|
-
| Late data | <closed period can change for N days; locks on day N+1> |
|
|
45
|
-
| Currency | <rate source and date> |
|
|
46
|
-
| Timezone and DST | <...> |
|
|
47
|
-
| Duplicates | <...> |
|
|
48
|
-
|
|
49
|
-
## Reference SQL (<dialect>)
|
|
50
|
-
|
|
51
|
-
```sql
|
|
52
|
-
-- base population, exclusions, numerator, denominator: one CTE each
|
|
53
|
-
```
|
|
54
|
-
|
|
55
|
-
## Sanity checks
|
|
56
|
-
|
|
57
|
-
1. <reconciliation with system of record: query and tolerance>
|
|
58
|
-
2. <bounds>
|
|
59
|
-
3. <join integrity or additivity>
|
|
60
|
-
4. <known-answer fixture>
|
|
61
|
-
|
|
62
|
-
## Change log
|
|
63
|
-
|
|
64
|
-
| Version | Date | Change | Approved by |
|
|
65
|
-
|---|---|---|---|
|
|
66
|
-
| v1 | <YYYY-MM-DD> | Initial definition | <owner> |
|
|
67
|
-
|
|
68
|
-
## Open decisions
|
|
69
|
-
|
|
70
|
-
- [ ] <choice made by default> — confirm with <role>
|