skilldrop-cli 1.2.0 → 1.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +5 -5
- package/catalogue.json +4 -4
- package/model-routing.json +62 -70
- package/package.json +1 -1
- package/packs/ai-engineering/loops/adopt/LOOP.md +120 -0
- package/packs/ai-engineering/loops/adopt/evals/eval_queries.json +10 -0
- package/packs/ai-engineering/loops/adopt/evals/evals.json +69 -0
- package/packs/ai-engineering/loops/adopt/evals/files/adopt-first-wave-to-sign-off/ai-ideas.md +15 -0
- package/packs/ai-engineering/loops/adopt/evals/files/adopt-first-wave-to-sign-off/org-context.md +27 -0
- package/packs/ai-engineering/loops/adopt/evals/files/adopt-first-wave-to-sign-off/staff-survey.csv +11 -0
- package/packs/ai-engineering/loops/adopt/evals/files/adopt-sponsor-slice-blocked-by-client-data/ai-ideas.md +13 -0
- package/packs/ai-engineering/loops/adopt/evals/files/adopt-sponsor-slice-blocked-by-client-data/firm-context.md +23 -0
- package/packs/ai-engineering/loops/adopt/loop.json +53 -0
- package/packs/ai-engineering/loops/ship-agent/LOOP.md +128 -0
- package/packs/ai-engineering/loops/ship-agent/evals/eval_queries.json +10 -0
- package/packs/ai-engineering/loops/ship-agent/evals/evals.json +70 -0
- package/packs/ai-engineering/loops/ship-agent/evals/files/ship-agent-auto-reply-exfil-open/agent-brief.md +33 -0
- package/packs/ai-engineering/loops/ship-agent/evals/files/ship-agent-changelog-config-reopens-path/agent-brief.md +33 -0
- package/packs/ai-engineering/loops/ship-agent/evals/files/ship-agent-changelog-config-reopens-path/claude-settings.json +25 -0
- package/packs/ai-engineering/loops/ship-agent/evals/files/ship-agent-changelog-config-reopens-path/sample-merged-prs.md +30 -0
- package/packs/ai-engineering/loops/ship-agent/loop.json +53 -0
- package/packs/ai-engineering/skills/agent-adoption-stage/SKILL.md +0 -1
- package/packs/ai-engineering/skills/agent-adoption-stage/manifest.json +3 -4
- package/packs/ai-engineering/skills/ai-adoption-rollout/SKILL.md +0 -1
- package/packs/ai-engineering/skills/ai-adoption-rollout/manifest.json +3 -4
- package/packs/ai-engineering/skills/ai-readiness-assessment/SKILL.md +0 -1
- package/packs/ai-engineering/skills/ai-readiness-assessment/manifest.json +1 -2
- package/packs/ai-engineering/skills/ai-use-case-triage/manifest.json +3 -3
- package/packs/ai-engineering/skills/judge-audit/SKILL.md +1 -2
- package/packs/ai-engineering/skills/judge-audit/manifest.json +1 -2
- package/packs/ai-engineering/skills/llm-eval-harness/SKILL.md +1 -2
- package/packs/ai-engineering/skills/llm-eval-harness/manifest.json +3 -3
- package/packs/api-builder/skills/tool-use-schema-writer/manifest.json +2 -2
- package/packs/career/loops/apply/evals/evals.json +28 -0
- package/packs/career/loops/apply/evals/files/apply-candidate-decided-full-run/application-form.md +10 -0
- package/packs/career/loops/apply/evals/files/apply-candidate-decided-full-run/job-posting.md +33 -0
- package/packs/career/loops/apply/evals/files/apply-candidate-decided-full-run/resume.md +34 -0
- package/packs/career/loops/apply/evals/files/apply-fit-gate-hard-requirement/cv.md +27 -0
- package/packs/career/loops/apply/evals/files/apply-fit-gate-hard-requirement/warehouse-supervisor.md +30 -0
- package/packs/career/skills/application-form-answers/manifest.json +3 -3
- package/packs/career/skills/cover-letter/manifest.json +3 -3
- package/packs/career/skills/resume-tailor/manifest.json +3 -3
- package/packs/converters/skills/md-to-docx/manifest.json +3 -3
- package/packs/core/loops/ship-a-draft/evals/evals.json +28 -0
- package/packs/core/loops/ship-a-draft/evals/files/ship-a-draft-adr-from-thread/slack-thread.txt +47 -0
- package/packs/core/loops/ship-a-draft/evals/files/ship-a-draft-announcement-unsettled/ticket-PRD-412.md +44 -0
- package/packs/core/skills/output-hygiene/manifest.json +3 -3
- package/packs/data-analytics/loops/measure/evals/evals.json +30 -0
- package/packs/data-analytics/loops/measure/evals/files/measure-active-customers-dispute/schema.md +43 -0
- package/packs/data-analytics/loops/measure/evals/files/measure-active-customers-dispute/sql/finance_active_customers.sql +7 -0
- package/packs/data-analytics/loops/measure/evals/files/measure-active-customers-dispute/sql/growth_active_customers.sql +9 -0
- package/packs/data-analytics/loops/measure/evals/files/measure-active-customers-dispute/thread.md +30 -0
- package/packs/data-analytics/loops/measure/evals/files/measure-retention-dashboard/request.md +34 -0
- package/packs/data-analytics/loops/measure/evals/files/measure-retention-dashboard/schema.md +58 -0
- package/packs/data-analytics/loops/measure/evals/files/measure-retention-dashboard/sql/active_subscribers.sql +17 -0
- package/packs/data-analytics/loops/measure/evals/files/measure-retention-dashboard/sql/net_revenue_by_plan.sql +15 -0
- package/packs/data-analytics/loops/measure/evals/files/measure-retention-dashboard/sql/weekly_churn_rate.sql +34 -0
- package/packs/data-analytics/skills/dashboard-spec/SKILL.md +0 -1
- package/packs/data-analytics/skills/dashboard-spec/manifest.json +1 -2
- package/packs/design/skills/slide-outliner/manifest.json +2 -2
- package/packs/dev-team/loops/build/evals/evals.json +28 -0
- package/packs/dev-team/loops/build/evals/files/build-promo-codes/.github/workflows/ci.yml.fixture +19 -0
- package/packs/dev-team/loops/build/evals/files/build-promo-codes/cartcalc/__init__.py +4 -0
- package/packs/dev-team/loops/build/evals/files/build-promo-codes/cartcalc/pricing.py +47 -0
- package/packs/dev-team/loops/build/evals/files/build-promo-codes/pyproject.toml.fixture +9 -0
- package/packs/dev-team/loops/build/evals/files/build-promo-codes/story.md +61 -0
- package/packs/dev-team/loops/build/evals/files/build-promo-codes/tests/__init__.py +0 -0
- package/packs/dev-team/loops/build/evals/files/build-promo-codes/tests/test_pricing.py +49 -0
- package/packs/dev-team/loops/build/evals/files/build-rate-limiter-not-ready/.github/workflows/ci.yml.fixture +19 -0
- package/packs/dev-team/loops/build/evals/files/build-rate-limiter-not-ready/bug-report.md +25 -0
- package/packs/dev-team/loops/build/evals/files/build-rate-limiter-not-ready/pr-description.md +42 -0
- package/packs/dev-team/loops/build/evals/files/build-rate-limiter-not-ready/pyproject.toml.fixture +9 -0
- package/packs/dev-team/loops/build/evals/files/build-rate-limiter-not-ready/ratelimit/__init__.py +4 -0
- package/packs/dev-team/loops/build/evals/files/build-rate-limiter-not-ready/ratelimit/bucket.py +49 -0
- package/packs/dev-team/loops/build/evals/files/build-rate-limiter-not-ready/tests/__init__.py +0 -0
- package/packs/dev-team/loops/build/evals/files/build-rate-limiter-not-ready/tests/test_bucket.py +63 -0
- package/packs/dev-team/loops/modernize/evals/evals.json +30 -0
- package/packs/dev-team/loops/modernize/evals/files/modernize-late-fee-batch-port/.github/workflows/ci.yml.fixture +11 -0
- package/packs/dev-team/loops/modernize/evals/files/modernize-late-fee-batch-port/README.md +17 -0
- package/packs/dev-team/loops/modernize/evals/files/modernize-late-fee-batch-port/docs/adr/0004-late-fee-calculation.md +25 -0
- package/packs/dev-team/loops/modernize/evals/files/modernize-late-fee-batch-port/legacy/feecalc.py +74 -0
- package/packs/dev-team/loops/modernize/evals/files/modernize-late-fee-batch-port/samples/accounts_2026-10.txt +10 -0
- package/packs/dev-team/loops/modernize/evals/files/modernize-late-fee-batch-port/samples/accounts_edge.txt +11 -0
- package/packs/dev-team/loops/modernize/evals/files/modernize-late-fee-batch-port/tests/test_feecalc_smoke.py +21 -0
- package/packs/dev-team/loops/modernize/evals/files/modernize-late-fee-rewrite-not-equivalent/.github/workflows/ci.yml.fixture +11 -0
- package/packs/dev-team/loops/modernize/evals/files/modernize-late-fee-rewrite-not-equivalent/PR-412.md +24 -0
- package/packs/dev-team/loops/modernize/evals/files/modernize-late-fee-rewrite-not-equivalent/README.md +17 -0
- package/packs/dev-team/loops/modernize/evals/files/modernize-late-fee-rewrite-not-equivalent/docs/adr/0004-late-fee-calculation.md +25 -0
- package/packs/dev-team/loops/modernize/evals/files/modernize-late-fee-rewrite-not-equivalent/hartwell_fees/__init__.py +1 -0
- package/packs/dev-team/loops/modernize/evals/files/modernize-late-fee-rewrite-not-equivalent/hartwell_fees/cli.py +33 -0
- package/packs/dev-team/loops/modernize/evals/files/modernize-late-fee-rewrite-not-equivalent/hartwell_fees/extract.py +33 -0
- package/packs/dev-team/loops/modernize/evals/files/modernize-late-fee-rewrite-not-equivalent/hartwell_fees/rules.py +39 -0
- package/packs/dev-team/loops/modernize/evals/files/modernize-late-fee-rewrite-not-equivalent/legacy/feecalc.py +74 -0
- package/packs/dev-team/loops/modernize/evals/files/modernize-late-fee-rewrite-not-equivalent/samples/accounts_2026-10.txt +10 -0
- package/packs/dev-team/loops/modernize/evals/files/modernize-late-fee-rewrite-not-equivalent/samples/accounts_edge.txt +11 -0
- package/packs/dev-team/loops/modernize/evals/files/modernize-late-fee-rewrite-not-equivalent/tests/test_feecalc_smoke.py +21 -0
- package/packs/dev-team/loops/release/evals/eval_queries.json +10 -0
- package/packs/dev-team/loops/release/evals/evals.json +34 -0
- package/packs/dev-team/loops/release/evals/files/release-irreversible-tomorrow/0061_switch_to_haldrin_tokens.sql +22 -0
- package/packs/dev-team/loops/release/evals/files/release-irreversible-tomorrow/eng-notes.md +40 -0
- package/packs/dev-team/loops/release/evals/files/release-irreversible-tomorrow/merged-prs.md +13 -0
- package/packs/dev-team/loops/release/evals/files/release-saved-searches/merged-prs.md +13 -0
- package/packs/dev-team/loops/release/evals/files/release-saved-searches/readiness-evidence.md +27 -0
- package/packs/dev-team/loops/release/evals/files/release-saved-searches/traffic-baseline.md +23 -0
- package/packs/dev-team/skills/feature-implement-loop/manifest.json +2 -2
- package/packs/dev-team/skills/pr-description-writer/manifest.json +3 -3
- package/packs/dev-team/skills/pre-merge-review/manifest.json +2 -2
- package/packs/dev-team/skills/release-notes/manifest.json +2 -2
- package/packs/dev-team/skills/test-plan-generator/manifest.json +2 -2
- package/packs/experience-design/loops/clarify/LOOP.md +114 -0
- package/packs/experience-design/loops/clarify/evals/eval_queries.json +10 -0
- package/packs/experience-design/loops/clarify/evals/evals.json +68 -0
- package/packs/experience-design/loops/clarify/evals/files/clarify-garden-waste-no-complaint-data/form-fields.csv +12 -0
- package/packs/experience-design/loops/clarify/evals/files/clarify-garden-waste-no-complaint-data/garden-waste-page.md +29 -0
- package/packs/experience-design/loops/clarify/evals/files/clarify-parking-permit-with-call-data/backoffice-notes.md +14 -0
- package/packs/experience-design/loops/clarify/evals/files/clarify-parking-permit-with-call-data/calls-q3.csv +8 -0
- package/packs/experience-design/loops/clarify/evals/files/clarify-parking-permit-with-call-data/form-fields.csv +12 -0
- package/packs/experience-design/loops/clarify/evals/files/clarify-parking-permit-with-call-data/permit-page.md +27 -0
- package/packs/experience-design/loops/clarify/loop.json +35 -0
- package/packs/experience-design/skills/service-blueprint/manifest.json +3 -3
- package/packs/experience-design/skills/ux-writing/manifest.json +3 -3
- package/packs/grc/loops/comply/evals/evals.json +29 -0
- package/packs/grc/loops/comply/evals/files/comply-chat-vendor-launch/vendor-brief.md +72 -0
- package/packs/grc/loops/comply/evals/files/comply-loyalty-app-unowned-risks/launch-note.md +31 -0
- package/packs/grc/loops/comply/evals/files/comply-loyalty-app-unowned-risks/risk-register.csv +6 -0
- package/packs/infra-as-code/loops/provision/LOOP.md +128 -0
- package/packs/infra-as-code/loops/provision/evals/eval_queries.json +9 -0
- package/packs/infra-as-code/loops/provision/evals/evals.json +67 -0
- package/packs/infra-as-code/loops/provision/evals/files/provision-rds-no-plan-yet/.terraform.lock.hcl.fixture +24 -0
- package/packs/infra-as-code/loops/provision/evals/files/provision-rds-no-plan-yet/network.md +36 -0
- package/packs/infra-as-code/loops/provision/evals/files/provision-replica-plan-replaces-primary/main.tf +74 -0
- package/packs/infra-as-code/loops/provision/evals/files/provision-replica-plan-replaces-primary/plan.json +901 -0
- package/packs/infra-as-code/loops/provision/evals/files/provision-replica-plan-replaces-primary/variables.tf +47 -0
- package/packs/infra-as-code/loops/provision/loop.json +41 -0
- package/packs/infra-as-code/skills/terraform-module/manifest.json +3 -3
- package/packs/product-manager/loops/discover/evals/evals.json +28 -0
- package/packs/product-manager/loops/discover/evals/files/discover-no-show-signal/interview-notes.md +63 -0
- package/packs/product-manager/loops/discover/evals/files/discover-no-show-signal/support-tickets.csv +13 -0
- package/packs/product-manager/loops/discover/evals/files/discover-vp-loyalty-no-evidence/vp-message.txt +13 -0
- package/packs/product-manager/skills/user-journey-map/manifest.json +2 -2
- package/packs/research/loops/research/evals/evals.json +28 -0
- package/packs/research/loops/research/evals/files/research-ticket-spike-confounded/sources/t1-ticket-volume.md +15 -0
- package/packs/research/loops/research/evals/files/research-ticket-spike-confounded/sources/t2-release-notes-4-2.md +11 -0
- package/packs/research/loops/research/evals/files/research-ticket-spike-confounded/sources/t3-billing-migration.md +12 -0
- package/packs/research/loops/research/evals/files/research-ticket-spike-confounded/sources/t4-agent-poll.md +11 -0
- package/packs/research/loops/research/evals/files/research-ticket-spike-confounded/sources/t5-app-store-reviews.md +12 -0
- package/packs/research/loops/research/evals/files/research-trial-conversion-drop/sources/s1-analytics-export.md +29 -0
- package/packs/research/loops/research/evals/files/research-trial-conversion-drop/sources/s2-email-delivery-report.md +25 -0
- package/packs/research/loops/research/evals/files/research-trial-conversion-drop/sources/s3-pricing-changelog.md +9 -0
- package/packs/research/loops/research/evals/files/research-trial-conversion-drop/sources/s4-trial-interview-notes.md +21 -0
- package/packs/research/loops/research/evals/files/research-trial-conversion-drop/sources/s5-competitor-note.md +9 -0
- package/packs/research/loops/research/evals/files/research-trial-conversion-drop/sources/s6-weekly-growth-recap.md +7 -0
- package/packs/research/skills/research-plan/manifest.json +3 -3
- package/packs/research/skills/source-synthesis/manifest.json +3 -3
- package/packs/skill-engineering/skills/skill-author/manifest.json +3 -3
- package/packs/solution-architect/loops/design/evals/evals.json +28 -0
- package/packs/solution-architect/loops/design/evals/files/design-bulk-export/adr/0001-postgres-system-of-record.md +23 -0
- package/packs/solution-architect/loops/design/evals/files/design-bulk-export/adr/0002-background-work-via-job-queue.md +23 -0
- package/packs/solution-architect/loops/design/evals/files/design-bulk-export/adr/0003-tenant-isolation-row-level.md +25 -0
- package/packs/solution-architect/loops/design/evals/files/design-bulk-export/prd-bulk-export.md +50 -0
- package/packs/solution-architect/loops/design/evals/files/design-draft-prd-webhooks/adr/0002-background-work-via-job-queue.md +23 -0
- package/packs/solution-architect/loops/design/evals/files/design-draft-prd-webhooks/adr/0004-new-stateful-infra-needs-platform-signoff.md +22 -0
- package/packs/solution-architect/loops/design/evals/files/design-draft-prd-webhooks/prd-partner-webhooks.md +36 -0
- package/packs/solution-architect/skills/api-contract-draft/manifest.json +3 -3
- package/packs/solution-architect/skills/architecture-diagrams/manifest.json +3 -3
- package/packs/solution-architect/skills/design-doc/manifest.json +2 -2
- package/packs/solution-architect/skills/figma-diagrams/manifest.json +3 -3
- package/packs/sre-oncall/loops/operate/evals/evals.json +28 -0
- package/packs/sre-oncall/loops/operate/evals/files/operate-export-service-first-outage/alerts.log +22 -0
- package/packs/sre-oncall/loops/operate/evals/files/operate-export-service-first-outage/incident-chat.txt +27 -0
- package/packs/sre-oncall/loops/operate/evals/files/operate-export-service-first-outage/service-notes.md +46 -0
- package/packs/sre-oncall/loops/operate/evals/files/operate-incident-still-open/oncall-log.txt +17 -0
- package/packs/sre-oncall/loops/operate/evals/files/operate-incident-still-open/runbook-webhooks.md +38 -0
- package/packs/sre-oncall/skills/cloud-cost-review/SKILL.md +0 -1
- package/packs/sre-oncall/skills/cloud-cost-review/manifest.json +1 -2
- package/packs/sre-oncall/skills/delivery-metrics-report/SKILL.md +0 -1
- package/packs/sre-oncall/skills/delivery-metrics-report/manifest.json +1 -2
- package/packs/sre-oncall/skills/incident-comms/manifest.json +2 -2
- package/packs/sre-oncall/skills/postmortem-generator/manifest.json +2 -2
- package/packs/stakeholder-comms/skills/decision-log/manifest.json +2 -2
- package/packs/trackers/skills/team-status-report/manifest.json +3 -3
- package/packs/ai-engineering/skills/ai-usage-report/SKILL.md +0 -111
- package/packs/ai-engineering/skills/ai-usage-report/evals/eval_queries.json +0 -30
- package/packs/ai-engineering/skills/ai-usage-report/evals/evals.json +0 -46
- package/packs/ai-engineering/skills/ai-usage-report/evals/files/1/northwind-ai-events.csv +0 -4213
- package/packs/ai-engineering/skills/ai-usage-report/evals/files/2/draftwell-events.csv +0 -249
- package/packs/ai-engineering/skills/ai-usage-report/evals/files/2/export-note.md +0 -9
- package/packs/ai-engineering/skills/ai-usage-report/evals/files/2/published-assets.csv +0 -44
- package/packs/ai-engineering/skills/ai-usage-report/evals/files/2/team-roster.md +0 -10
- package/packs/ai-engineering/skills/ai-usage-report/examples/weekly-engineering-team.md +0 -133
- package/packs/ai-engineering/skills/ai-usage-report/manifest.json +0 -12
- package/packs/ai-engineering/skills/ai-usage-report/scripts/build_report.py +0 -453
- package/packs/ai-engineering/skills/ai-usage-report/templates/report-per-user.md +0 -42
- package/packs/ai-engineering/skills/ai-usage-report/templates/report-team-rollup.md +0 -49
- package/packs/ai-engineering/skills/ai-usage-report/templates/usage-event-schema.md +0 -76
- package/packs/product-manager/skills/lean-canvas/SKILL.md +0 -162
- package/packs/product-manager/skills/lean-canvas/evals/eval_queries.json +0 -42
- package/packs/product-manager/skills/lean-canvas/evals/evals.json +0 -82
- package/packs/product-manager/skills/lean-canvas/evals/files/1/pawpass-notes.md +0 -54
- package/packs/product-manager/skills/lean-canvas/evals/files/3/lunchloop-pitch.md +0 -30
- package/packs/product-manager/skills/lean-canvas/evals/files/4/clubkit-notes.md +0 -44
- package/packs/product-manager/skills/lean-canvas/evals/files/5/cost-model.csv +0 -5
- package/packs/product-manager/skills/lean-canvas/evals/files/5/proposal.md +0 -40
- package/packs/product-manager/skills/lean-canvas/evals/files/5/stakeholder-emails.txt +0 -18
- package/packs/product-manager/skills/lean-canvas/examples/bike-repair.md +0 -72
- package/packs/product-manager/skills/lean-canvas/manifest.json +0 -34
- package/packs/product-manager/skills/lean-canvas/reference.md +0 -102
- package/packs/product-manager/skills/lean-canvas/scripts/unit_economics.py +0 -211
package/README.md
CHANGED
|
@@ -15,13 +15,13 @@ npx skilldrop-cli install --pack dev-team # or product-manager, solution-arc
|
|
|
15
15
|
|
|
16
16
|
From a held-out eval, written blind from the skill's description:
|
|
17
17
|
|
|
18
|
-
> *"
|
|
18
|
+
> *"Our design agency sent the Figma file for Ostwick Museum's new signage as a JSON export… Can you (1) list the pages and the frames on each, (2) check the Final page against what the Brief page asks for, and (3) draft the comments I should leave for the agency on any frames with problems — one line each."*
|
|
19
19
|
|
|
20
|
-
| Plain Claude Code | With [`
|
|
20
|
+
| Plain Claude Code | With [`figma-diagrams`](https://sananthanarayan.github.io/skilldrop/skills/figma-diagrams/) |
|
|
21
21
|
|---|---|
|
|
22
|
-
| Finds the
|
|
22
|
+
| Finds the same three problems, then drafts comments of two sentences or more, and adds a fourth one about the Archive page that the file doesn't support. | *"The cover email is wrong on both counts. The Sign Types page has 11 frames, not 12, and Type 07 is missing."* Then one line per frame with a problem. |
|
|
23
23
|
|
|
24
|
-
|
|
24
|
+
Both blind judges picked the skill's answer: *"A keeps each comment to a single sentence as asked, while B's main comments run to two sentences and its fourth comment is speculative."*
|
|
25
25
|
|
|
26
26
|
## The receipts
|
|
27
27
|
|
|
@@ -80,7 +80,7 @@ Install one pack. Every role pack also brings `core` (intake, critique, review c
|
|
|
80
80
|
## How it works
|
|
81
81
|
|
|
82
82
|
- **A skill produces one file you own.** Each ships a quality bar, named anti-patterns and acceptance evals, so the output is an artifact, not a conversation. See [every skill, by category](guides/reference/skill-catalogue.md).
|
|
83
|
-
- **Loops order skills, and a gate decides when work moves on.** Five loops cover the lifecycle (`discover`, `design`, `build`, `release`, `operate`), plus the `ship-a-draft` wrapper for any document and `
|
|
83
|
+
- **Loops order skills, and a gate decides when work moves on.** Five loops cover the lifecycle (`discover`, `design`, `build`, `release`, `operate`), plus the `ship-a-draft` wrapper for any document and loops for jobs beside it, such as `research`, `comply`, `adopt` for a team starting with AI, `ship-agent` for an AI agent and `provision` for a Terraform change. Each gate is a script, a review panel or a person, chosen by how expensive the mistake is to undo. See [the operating model](guides/explanation/operating-model.md) for every stage and gate on one page, [the loops, stage by stage](guides/reference/loops.md), and [why loops](guides/explanation/loops.md).
|
|
84
84
|
- **Measured, not asserted.** `run_bench.py` runs each acceptance eval with the skill and without it, in a sandbox, and reports the lift, a blind preference from two judges, and the cost per run. The results are published whether or not they flatter the skills. See [how skills are checked](https://sananthanarayan.github.io/skilldrop/evals/) and [RFC-0040](docs/rfcs/0040-skill-benchmark.md).
|
|
85
85
|
- **Copy, never transform.** A skill is a plain `SKILL.md` folder in the [Agent Skills](https://agentskills.io) format, at `packs/<pack>/skills/<name>/`. What runs in your agent is byte-identical to what is reviewed here. There is no runtime, and the tooling has zero dependencies.
|
|
86
86
|
- **Updates keep your edits.** `npx skilldrop-cli update` replaces files you haven't touched. For any file you changed, it leaves the new version beside it as `<file>.upstream`. See [upgrade installed skills](guides/how-to/upgrade-skills.md).
|
package/catalogue.json
CHANGED
|
@@ -18,8 +18,7 @@
|
|
|
18
18
|
"success-metrics",
|
|
19
19
|
"user-story-splitter",
|
|
20
20
|
"user-journey-map",
|
|
21
|
-
"feedback-synthesis"
|
|
22
|
-
"lean-canvas"
|
|
21
|
+
"feedback-synthesis"
|
|
23
22
|
]
|
|
24
23
|
},
|
|
25
24
|
"design-the-system": {
|
|
@@ -124,7 +123,6 @@
|
|
|
124
123
|
"ai-adoption-rollout",
|
|
125
124
|
"ai-usage-policy",
|
|
126
125
|
"agent-adoption-stage",
|
|
127
|
-
"ai-usage-report",
|
|
128
126
|
"agent-config-review",
|
|
129
127
|
"agent-incident-response"
|
|
130
128
|
]
|
|
@@ -217,6 +215,8 @@
|
|
|
217
215
|
"md-to-html": { "in": "1.2.0", "why": "On its held-out evals, in each of two trials and with both blind judges, the agent without it made the better page: the skill's script showed raw HTML as text and its security policy blocked embeds, and it adds no rule the agent breaks." },
|
|
218
216
|
"mermaid-render": { "in": "1.2.0", "why": "On its held-out evals, in each of two trials and with both blind judges, the agent without it fixed the same diagrams, ran the Mermaid CLI itself and said plainly what it had not rendered; the skill adds no rule it breaks." },
|
|
219
217
|
"marketing-flyer": { "in": "1.2.0", "why": "On its held-out evals, in each of two trials and with both blind judges, the agent without it built the whole flyer with placeholders where the skill stopped or used a generic template; it adds no rule the agent breaks." },
|
|
220
|
-
"deck-builder": { "in": "1.2.0", "why": "On its held-out evals the agent without it built the better deck in each of two trials; rebuilt rule-first, with its script able to run, it still lost on fresh evals (judges 2 and 1 of 4; 69% of checks against 81%).", "instead": "md-to-pptx" }
|
|
218
|
+
"deck-builder": { "in": "1.2.0", "why": "On its held-out evals the agent without it built the better deck in each of two trials; rebuilt rule-first, with its script able to run, it still lost on fresh evals (judges 2 and 1 of 4; 69% of checks against 81%).", "instead": "md-to-pptx" },
|
|
219
|
+
"ai-usage-report": { "in": "1.4.0", "why": "On its held-out evals, over three trials and with both blind judges, the agent without it made the better report: it kept every rule the skill teaches (counts show activity, not quality; missing data is not zero; no leaderboard), while the skill refused to rank when asked, hid names the manager had supplied, and its script read neither eval's export." },
|
|
220
|
+
"lean-canvas": { "in": "1.4.0", "why": "On its held-out evals, in each of three trials and with both blind judges, the agent without it made the better canvas: it caught the same errors and went further, while the skill's lead on checks came from its own format and its script's break-even left out the cost of replacing lost customers.", "instead": "business-case" }
|
|
221
221
|
}
|
|
222
222
|
}
|
package/model-routing.json
CHANGED
|
@@ -107,8 +107,8 @@
|
|
|
107
107
|
"rationale": "Structures a decision into MADR/Nygard. Some judgment on title/consequences; not adversarial."
|
|
108
108
|
},
|
|
109
109
|
"agent-adoption-stage": {
|
|
110
|
-
"tier": "
|
|
111
|
-
"rationale": "
|
|
110
|
+
"tier": "light",
|
|
111
|
+
"rationale": "Measured (RFC-0073): on its held-out evals the light tier met 79% and 86% of checks in two tier trials, against standard's 82% and heavy's 79% and 79%, within 5 points of the best both times."
|
|
112
112
|
},
|
|
113
113
|
"agent-budget": {
|
|
114
114
|
"tier": "standard",
|
|
@@ -135,8 +135,8 @@
|
|
|
135
135
|
"rationale": "Repo inspection plus synthesis against a fixed section order and line budget — the same shape as reverse-architecture. The judgment is the counterfactual test on each convention line, which is bounded; escalate via large-input on monorepos."
|
|
136
136
|
},
|
|
137
137
|
"ai-adoption-rollout": {
|
|
138
|
-
"tier": "
|
|
139
|
-
"rationale": "
|
|
138
|
+
"tier": "light",
|
|
139
|
+
"rationale": "Measured (RFC-0073): on its held-out evals the light tier met 79% and 71% of checks in two tier trials, against standard's 75% and heavy's 64% and 64%, within 5 points of the best both times."
|
|
140
140
|
},
|
|
141
141
|
"ai-readiness-assessment": {
|
|
142
142
|
"tier": "standard",
|
|
@@ -146,25 +146,21 @@
|
|
|
146
146
|
"tier": "standard",
|
|
147
147
|
"rationale": "Drafts a policy document against a fixed structure (tiers, behaviours, review matrix, exception route). Bounded generation; escalate via ambiguous-input where a regulatory regime is named or contested."
|
|
148
148
|
},
|
|
149
|
-
"ai-usage-report": {
|
|
150
|
-
"tier": "standard",
|
|
151
|
-
"rationale": "Tabulates telemetry into a report. Mostly mechanical, but the 'AI theater' effectiveness call needs judgment."
|
|
152
|
-
},
|
|
153
149
|
"ai-use-case-triage": {
|
|
154
|
-
"tier": "
|
|
155
|
-
"rationale": "
|
|
150
|
+
"tier": "light",
|
|
151
|
+
"rationale": "Measured (RFC-0073): on its held-out evals the light tier met 83% and 92% of checks in two tier trials, against standard's 81% and heavy's 85% and 92%, within 5 points of the best both times."
|
|
156
152
|
},
|
|
157
153
|
"api-contract-draft": {
|
|
158
|
-
"tier": "
|
|
159
|
-
"rationale": "
|
|
154
|
+
"tier": "light",
|
|
155
|
+
"rationale": "Measured (RFC-0073): on its held-out evals the light tier met 76% and 92% of checks in two tier trials, against standard's 69% and heavy's 69% and 76%, within 5 points of the best both times."
|
|
160
156
|
},
|
|
161
157
|
"application-form-answers": {
|
|
162
|
-
"tier": "
|
|
163
|
-
"rationale": "
|
|
158
|
+
"tier": "light",
|
|
159
|
+
"rationale": "Measured (RFC-0073): on its held-out evals the light tier met 88% and 94% of checks in two tier trials, against standard's 88% and heavy's 81% and 81%, within 5 points of the best both times."
|
|
164
160
|
},
|
|
165
161
|
"architecture-diagrams": {
|
|
166
|
-
"tier": "
|
|
167
|
-
"rationale": "
|
|
162
|
+
"tier": "light",
|
|
163
|
+
"rationale": "Measured (RFC-0073): on its held-out evals the light tier met 93% and 100% of checks in two tier trials, against standard's 93% and heavy's 93% and 93%, within 5 points of the best both times."
|
|
168
164
|
},
|
|
169
165
|
"audience-profile": {
|
|
170
166
|
"tier": "light",
|
|
@@ -219,8 +215,8 @@
|
|
|
219
215
|
"rationale": "Multi-perspective deliberation + weighted reconciliation across distinct expert seats. Cheaper models collapse the seats into one voice and either manufacture or miss the real cruxes. Never downgrade."
|
|
220
216
|
},
|
|
221
217
|
"cover-letter": {
|
|
222
|
-
"tier": "
|
|
223
|
-
"rationale": "
|
|
218
|
+
"tier": "light",
|
|
219
|
+
"rationale": "Measured (RFC-0073): on its held-out evals the light tier met 100% and 100% of checks in two tier trials, against standard's 100% and heavy's 100% and 100%, within 5 points of the best both times."
|
|
224
220
|
},
|
|
225
221
|
"dashboard-spec": {
|
|
226
222
|
"tier": "standard",
|
|
@@ -231,8 +227,8 @@
|
|
|
231
227
|
"rationale": "Schema + SLA synthesis against fixed quality dimensions and an evolution matrix. Semantics capture needs care; bounded by the catalog. Escalate via large-input on wide schemas."
|
|
232
228
|
},
|
|
233
229
|
"decision-log": {
|
|
234
|
-
"tier": "
|
|
235
|
-
"rationale": "
|
|
230
|
+
"tier": "standard",
|
|
231
|
+
"rationale": "Measured (RFC-0073): on its held-out evals the light tier fell more than 5 points behind the best in both tier trials (79% and 61% against standard's 79%); one step up, to standard."
|
|
236
232
|
},
|
|
237
233
|
"delivery-metrics-report": {
|
|
238
234
|
"tier": "standard",
|
|
@@ -243,8 +239,8 @@
|
|
|
243
239
|
"rationale": "Reads a version diff, changelogs and CI evidence against a fixed checklist; the merge verdict follows from what is verified, not open-ended judgment."
|
|
244
240
|
},
|
|
245
241
|
"design-doc": {
|
|
246
|
-
"tier": "
|
|
247
|
-
"rationale": "
|
|
242
|
+
"tier": "light",
|
|
243
|
+
"rationale": "Measured (RFC-0073): on its held-out evals the light tier met 88% and 88% of checks in two tier trials, against standard's 92% and heavy's 91% and 91%, within 5 points of the best both times."
|
|
248
244
|
},
|
|
249
245
|
"design-system-spec": {
|
|
250
246
|
"tier": "standard",
|
|
@@ -276,16 +272,16 @@
|
|
|
276
272
|
"rationale": "Compression to one page. Needs to keep the right signal; escalate on very long source docs."
|
|
277
273
|
},
|
|
278
274
|
"feature-implement-loop": {
|
|
279
|
-
"tier": "
|
|
280
|
-
"rationale": "
|
|
275
|
+
"tier": "light",
|
|
276
|
+
"rationale": "Measured (RFC-0073): on its held-out evals the light tier met 85% and 85% of checks in two tier trials, against standard's 85% and heavy's 76% and 85%, within 5 points of the best both times."
|
|
281
277
|
},
|
|
282
278
|
"feedback-synthesis": {
|
|
283
279
|
"tier": "standard",
|
|
284
280
|
"rationale": "Coding feedback into themes is bounded judgment; the counts by distinct person and segment come from its script."
|
|
285
281
|
},
|
|
286
282
|
"figma-diagrams": {
|
|
287
|
-
"tier": "
|
|
288
|
-
"rationale": "
|
|
283
|
+
"tier": "light",
|
|
284
|
+
"rationale": "Measured (RFC-0073): on its held-out evals the light tier met 92% and 92% of checks in two tier trials, against standard's 92% and heavy's 75% and 92%, within 5 points of the best both times."
|
|
289
285
|
},
|
|
290
286
|
"file-to-markdown": {
|
|
291
287
|
"tier": "light",
|
|
@@ -300,8 +296,8 @@
|
|
|
300
296
|
"rationale": "Judgment-heavy: generating real rivals, rating every evidence-hypothesis cell honestly and resisting the favourite theory is where a fast pass fails; the script only does the arithmetic."
|
|
301
297
|
},
|
|
302
298
|
"incident-comms": {
|
|
303
|
-
"tier": "
|
|
304
|
-
"rationale": "
|
|
299
|
+
"tier": "light",
|
|
300
|
+
"rationale": "Measured (RFC-0073): on its held-out evals the light tier met 93% and 100% of checks in two tier trials, against standard's 85% and heavy's 92% and 93%, within 5 points of the best both times."
|
|
305
301
|
},
|
|
306
302
|
"information-architecture": {
|
|
307
303
|
"tier": "standard",
|
|
@@ -323,13 +319,9 @@
|
|
|
323
319
|
"tier": "standard",
|
|
324
320
|
"rationale": "Checks evidence against a fixed checklist. Bounded synthesis, not adversarial review."
|
|
325
321
|
},
|
|
326
|
-
"lean-canvas": {
|
|
327
|
-
"tier": "standard",
|
|
328
|
-
"rationale": "Filling the canvas is structured; the value is in ranking assumptions by risk and naming the cheapest test that could kill the riskiest one."
|
|
329
|
-
},
|
|
330
322
|
"llm-eval-harness": {
|
|
331
|
-
"tier": "
|
|
332
|
-
"rationale": "
|
|
323
|
+
"tier": "light",
|
|
324
|
+
"rationale": "Measured (RFC-0073): on its held-out evals the light tier met 71% and 71% of checks in two tier trials, against standard's 71% and heavy's 71% and 71%, within 5 points of the best both times."
|
|
333
325
|
},
|
|
334
326
|
"mcp-pairing": {
|
|
335
327
|
"tier": "standard",
|
|
@@ -340,8 +332,8 @@
|
|
|
340
332
|
"rationale": "Mechanical conversion to Confluence storage format done by its script, with an optional publish; the model checks what was dropped and what will be overwritten."
|
|
341
333
|
},
|
|
342
334
|
"md-to-docx": {
|
|
343
|
-
"tier": "
|
|
344
|
-
"rationale": "
|
|
335
|
+
"tier": "heavy",
|
|
336
|
+
"rationale": "Measured (RFC-0073): on its held-out evals the heavy tier met 79% and 93% of checks in two tier trials, against standard's 64% and light's 71% and 79%, more than 5 points ahead both times."
|
|
345
337
|
},
|
|
346
338
|
"md-to-pptx": {
|
|
347
339
|
"tier": "light",
|
|
@@ -368,24 +360,24 @@
|
|
|
368
360
|
"rationale": "Structured derivation against a fixed roll-up discipline; the causal-sentence and gap judgments need care but the template bounds them. Escalate via ambiguous-input when the company objectives themselves are contested."
|
|
369
361
|
},
|
|
370
362
|
"output-hygiene": {
|
|
371
|
-
"tier": "
|
|
372
|
-
"rationale": "
|
|
363
|
+
"tier": "light",
|
|
364
|
+
"rationale": "Measured (RFC-0073): on its held-out evals the light tier met 100% and 100% of checks in two tier trials, against standard's 92% and heavy's 100% and 92%, within 5 points of the best both times."
|
|
373
365
|
},
|
|
374
366
|
"postmortem-generator": {
|
|
375
|
-
"tier": "
|
|
376
|
-
"rationale": "
|
|
367
|
+
"tier": "light",
|
|
368
|
+
"rationale": "Measured (RFC-0073): on its held-out evals the light tier met 93% and 91% of checks in two tier trials, against standard's 92% and heavy's 90% and 87%, within 5 points of the best both times."
|
|
377
369
|
},
|
|
378
370
|
"pr-description-writer": {
|
|
379
|
-
"tier": "
|
|
380
|
-
"rationale": "
|
|
371
|
+
"tier": "light",
|
|
372
|
+
"rationale": "Measured (RFC-0073): on its held-out evals the light tier met 85% and 85% of checks in two tier trials, against standard's 85% and heavy's 77% and 85%, within 5 points of the best both times."
|
|
381
373
|
},
|
|
382
374
|
"prd-draft": {
|
|
383
375
|
"tier": "standard",
|
|
384
376
|
"rationale": "Synthesizes a structured requirements doc from a brief. The solution-language discipline needs care; bounded by the template. Escalate via ambiguous-input on contested scope."
|
|
385
377
|
},
|
|
386
378
|
"pre-merge-review": {
|
|
387
|
-
"tier": "
|
|
388
|
-
"rationale": "
|
|
379
|
+
"tier": "light",
|
|
380
|
+
"rationale": "Measured (RFC-0073): on its held-out evals the light tier met 93% and 93% of checks in two tier trials, against standard's 86% and heavy's 93% and 93%, within 5 points of the best both times."
|
|
389
381
|
},
|
|
390
382
|
"prfaq": {
|
|
391
383
|
"tier": "standard",
|
|
@@ -400,8 +392,8 @@
|
|
|
400
392
|
"rationale": "Sequencing behaviour-preserving steps follows known patterns (characterise, parallel change, strangle); escalate via ambiguous-input for contested rewrites."
|
|
401
393
|
},
|
|
402
394
|
"release-notes": {
|
|
403
|
-
"tier": "
|
|
404
|
-
"rationale": "
|
|
395
|
+
"tier": "light",
|
|
396
|
+
"rationale": "Measured (RFC-0073): on its held-out evals the light tier met 96% and 96% of checks in two tier trials, against standard's 91% and heavy's 100% and 98%, within 5 points of the best both times."
|
|
405
397
|
},
|
|
406
398
|
"rendering-strategy": {
|
|
407
399
|
"tier": "standard",
|
|
@@ -412,12 +404,12 @@
|
|
|
412
404
|
"rationale": "Question banks are rubric-shaped, but ranking by design impact and deriving falsification conditions for the specific feature is synthesis — above light."
|
|
413
405
|
},
|
|
414
406
|
"research-plan": {
|
|
415
|
-
"tier": "
|
|
416
|
-
"rationale": "
|
|
407
|
+
"tier": "light",
|
|
408
|
+
"rationale": "Measured (RFC-0073): on its held-out evals the light tier met 79% and 79% of checks in two tier trials, against standard's 79% and heavy's 79% and 79%, within 5 points of the best both times."
|
|
417
409
|
},
|
|
418
410
|
"resume-tailor": {
|
|
419
|
-
"tier": "
|
|
420
|
-
"rationale": "
|
|
411
|
+
"tier": "light",
|
|
412
|
+
"rationale": "Measured (RFC-0073): on its held-out evals the light tier met 100% and 100% of checks in two tier trials, against standard's 100% and heavy's 100% and 100%, within 5 points of the best both times."
|
|
421
413
|
},
|
|
422
414
|
"reverse-architecture": {
|
|
423
415
|
"tier": "standard",
|
|
@@ -444,20 +436,20 @@
|
|
|
444
436
|
"rationale": "Matches questions to evidence and writes short answers. The hard rule is mechanical: no evidence, no answer."
|
|
445
437
|
},
|
|
446
438
|
"service-blueprint": {
|
|
447
|
-
"tier": "
|
|
448
|
-
"rationale": "
|
|
439
|
+
"tier": "light",
|
|
440
|
+
"rationale": "Measured (RFC-0073): on its held-out evals the light tier met 100% and 92% of checks in two tier trials, against standard's 92% and heavy's 85% and 92%, within 5 points of the best both times."
|
|
449
441
|
},
|
|
450
442
|
"skill-author": {
|
|
451
|
-
"tier": "
|
|
452
|
-
"rationale": "
|
|
443
|
+
"tier": "light",
|
|
444
|
+
"rationale": "Measured (RFC-0073): on its held-out evals the light tier met 100% and 93% of checks in two tier trials, against standard's 93% and heavy's 100% and 92%, within 5 points of the best both times."
|
|
453
445
|
},
|
|
454
446
|
"skill-review": {
|
|
455
447
|
"tier": "heavy",
|
|
456
448
|
"rationale": "Adversarial audit with a verdict: routing collisions and paraphrased safety issues are what a fast pass misses, and the lint script covers only the mechanical half."
|
|
457
449
|
},
|
|
458
450
|
"slide-outliner": {
|
|
459
|
-
"tier": "
|
|
460
|
-
"rationale": "
|
|
451
|
+
"tier": "light",
|
|
452
|
+
"rationale": "Measured (RFC-0073): on its held-out evals the light tier met 86% and 90% of checks in two tier trials, against standard's 83% and heavy's 83% and 93%, within 5 points of the best both times."
|
|
461
453
|
},
|
|
462
454
|
"slo-spec": {
|
|
463
455
|
"tier": "standard",
|
|
@@ -476,8 +468,8 @@
|
|
|
476
468
|
"rationale": "Formats Sonar API findings into a report. The hard analysis is Sonar's; this is synthesis. Escalate on very large diffs."
|
|
477
469
|
},
|
|
478
470
|
"source-synthesis": {
|
|
479
|
-
"tier": "
|
|
480
|
-
"rationale": "
|
|
471
|
+
"tier": "light",
|
|
472
|
+
"rationale": "Measured (RFC-0073): on its held-out evals the light tier met 83% and 78% of checks in two tier trials, against standard's 76% and heavy's 85% and 82%, within 5 points of the best both times."
|
|
481
473
|
},
|
|
482
474
|
"sql-review": {
|
|
483
475
|
"tier": "heavy",
|
|
@@ -496,24 +488,24 @@
|
|
|
496
488
|
"rationale": "Structured measurement design; the primary-metric and counter-metric choices need judgment but the template bounds it. Escalate via ambiguous-input when the goal itself is contested."
|
|
497
489
|
},
|
|
498
490
|
"team-status-report": {
|
|
499
|
-
"tier": "
|
|
500
|
-
"rationale": "
|
|
491
|
+
"tier": "light",
|
|
492
|
+
"rationale": "Measured (RFC-0073): on its held-out evals the light tier met 93% and 93% of checks in two tier trials, against standard's 93% and heavy's 93% and 93%, within 5 points of the best both times."
|
|
501
493
|
},
|
|
502
494
|
"tech-debt-register": {
|
|
503
495
|
"tier": "standard",
|
|
504
496
|
"rationale": "Structures and ranks items against stated evidence. Judgment is bounded: a cost with no evidence is recorded as unknown, not estimated."
|
|
505
497
|
},
|
|
506
498
|
"terraform-module": {
|
|
507
|
-
"tier": "
|
|
508
|
-
"rationale": "
|
|
499
|
+
"tier": "light",
|
|
500
|
+
"rationale": "Measured (RFC-0073): on its held-out evals the light tier met 100% and 100% of checks in two tier trials, against standard's 92% and heavy's 83% and 100%, within 5 points of the best both times."
|
|
509
501
|
},
|
|
510
502
|
"terraform-plan-review": {
|
|
511
503
|
"tier": "heavy",
|
|
512
504
|
"rationale": "Adversarial review of a destructive change: judging which replacements lose data, which access changes widen exposure, and what is out of scope needs careful reasoning; a missed destroy costs far more than the model."
|
|
513
505
|
},
|
|
514
506
|
"test-plan-generator": {
|
|
515
|
-
"tier": "
|
|
516
|
-
"rationale": "
|
|
507
|
+
"tier": "light",
|
|
508
|
+
"rationale": "Measured (RFC-0073): on its held-out evals the light tier met 76% and 92% of checks in two tier trials, against standard's 76% and heavy's 61% and 54%, within 5 points of the best both times."
|
|
517
509
|
},
|
|
518
510
|
"threat-model": {
|
|
519
511
|
"tier": "heavy",
|
|
@@ -524,24 +516,24 @@
|
|
|
524
516
|
"rationale": "Arithmetic breakdown plus heuristic judgment on dominant cost drivers — standard fits."
|
|
525
517
|
},
|
|
526
518
|
"tool-use-schema-writer": {
|
|
527
|
-
"tier": "
|
|
528
|
-
"rationale": "
|
|
519
|
+
"tier": "light",
|
|
520
|
+
"rationale": "Measured (RFC-0073): on its held-out evals the light tier met 94% and 94% of checks in two tier trials, against standard's 96% and heavy's 97% and 97%, within 5 points of the best both times."
|
|
529
521
|
},
|
|
530
522
|
"tracker-brief-sync": {
|
|
531
523
|
"tier": "standard",
|
|
532
524
|
"rationale": "Maps doc sections to items and writes acceptance criteria that restate requirements; a script checks the spec, writes the import file and computes drift. Bounded transformation with a deterministic check."
|
|
533
525
|
},
|
|
534
526
|
"user-journey-map": {
|
|
535
|
-
"tier": "
|
|
536
|
-
"rationale": "
|
|
527
|
+
"tier": "heavy",
|
|
528
|
+
"rationale": "Measured (RFC-0073): on its held-out evals the heavy tier met 83% and 83% of checks in two tier trials, against standard's 75% and light's 67% and 75%, more than 5 points ahead both times."
|
|
537
529
|
},
|
|
538
530
|
"user-story-splitter": {
|
|
539
531
|
"tier": "standard",
|
|
540
532
|
"rationale": "Decomposes an epic into INVEST slices with testable ACs. Structured synthesis; escalation rules cover large or contradictory PRDs."
|
|
541
533
|
},
|
|
542
534
|
"ux-writing": {
|
|
543
|
-
"tier": "
|
|
544
|
-
"rationale": "
|
|
535
|
+
"tier": "light",
|
|
536
|
+
"rationale": "Measured (RFC-0073): on its held-out evals the light tier met 92% and 92% of checks in two tier trials, against standard's 83% and heavy's 75% and 83%, within 5 points of the best both times."
|
|
545
537
|
},
|
|
546
538
|
"web-performance-review": {
|
|
547
539
|
"tier": "standard",
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "skilldrop-cli",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.4.0",
|
|
4
4
|
"description": "Portable AI-agent skills, measured against the agent without them. ADRs, PRDs, runbooks, decks and reviews, installable into Claude Code, Cursor, Kiro, and any IDE that takes custom instructions. One command per skill or per role-based pack.",
|
|
5
5
|
"author": {
|
|
6
6
|
"name": "Sanjay Ananthanarayan",
|
|
@@ -0,0 +1,120 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: adopt
|
|
3
|
+
description: Take an organisation from a pile of AI ideas to a first rollout wave its sponsor approved: the use cases ranked with a first slice, readiness scored against that slice, a usage policy covering the data the slice touches, and a cohort rollout whose first wave is the slice's users, with a gate that sends the slice back when a blocking gap stands in its way. Use when the user says "help us start using AI", "roll out AI across the team", "run the adopt loop", or has AI ideas, no policy and no rollout plan.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# adopt
|
|
7
|
+
|
|
8
|
+
One first slice, from a list of ideas to a wave of people using it. The four skills each work
|
|
9
|
+
alone; in order, each one narrows the next. Triage picks one use case, so readiness is scored
|
|
10
|
+
against what that use case needs instead of against the whole organisation. Readiness names the
|
|
11
|
+
gaps, so the policy covers the data the slice really touches. The rollout's pilot is the slice's
|
|
12
|
+
own users, and its adoption gate is the observable triage said would prove or kill the slice.
|
|
13
|
+
|
|
14
|
+
## Stages
|
|
15
|
+
|
|
16
|
+
| # | Stage | Type | Skills | Gate |
|
|
17
|
+
|---|---|---|---|---|
|
|
18
|
+
| 1 | `triage` | generate | [`ai-use-case-triage`](../../skills/ai-use-case-triage/SKILL.md) | — |
|
|
19
|
+
| 2 | `readiness` | verify | [`ai-readiness-assessment`](../../skills/ai-readiness-assessment/SKILL.md) | **G11** review |
|
|
20
|
+
| 3 | `policy` | generate | [`ai-usage-policy`](../../skills/ai-usage-policy/SKILL.md) | — |
|
|
21
|
+
| 4 | `rollout` | generate | [`ai-adoption-rollout`](../../skills/ai-adoption-rollout/SKILL.md) | — |
|
|
22
|
+
| 5 | `sign-off` | gate | [`doc-critique`](../../../core/skills/doc-critique/SKILL.md) | **G11.1** human |
|
|
23
|
+
|
|
24
|
+
## How to run this loop
|
|
25
|
+
|
|
26
|
+
1. **Triage the ideas.** Run `ai-use-case-triage` on whatever the person brought: a brainstorm,
|
|
27
|
+
a backlog, interview notes. Save it as `triage.md`. Carry three things from it into every
|
|
28
|
+
later stage: the first slice (one workflow and the people who do it), the observable that
|
|
29
|
+
proves or kills it, and the time box.
|
|
30
|
+
2. **Score readiness for the slice.** Run `ai-readiness-assessment` scoped to the slice's team
|
|
31
|
+
and the decision "can this slice start". Each dimension is read against what the slice
|
|
32
|
+
needs: Data is the data the slice reads, Tooling is the tool the slice uses, Process is the
|
|
33
|
+
review step the slice's output passes through. Save it as `readiness.md`.
|
|
34
|
+
3. **Judge the slice at G11.** Ask three questions, then choose the verdict below:
|
|
35
|
+
- Does any gap the assessment ranks as blocking sit on something the slice needs?
|
|
36
|
+
- Can each remaining gap be closed inside the slice's time box, with an owner role named?
|
|
37
|
+
- Is there a review step where a person checks the slice's output before anyone acts on it?
|
|
38
|
+
4. **On `REVISE`, change the slice, not the scores.** Return to `triage` with the blocking gap
|
|
39
|
+
as a constraint, and pick the highest-ranked candidate that does not need what is missing.
|
|
40
|
+
The not-yet list's promotion conditions say which one. Each return counts against the cap.
|
|
41
|
+
5. **Write the policy for the slice's data.** Run `ai-usage-policy` for the slice's people,
|
|
42
|
+
naming every kind of data the slice puts into a tool (from triage and the Data line of
|
|
43
|
+
readiness). Every kind of data the slice touches gets a rule; a rule for data the slice
|
|
44
|
+
never touches stays out of this version. Save it as `policy.md`.
|
|
45
|
+
6. **Plan the rollout from the slice.** Run `ai-adoption-rollout` with the slice's users as the
|
|
46
|
+
pilot cohort and triage's observable as the pilot's adoption gate. G11's conditions become
|
|
47
|
+
preconditions the pilot cannot open without. Save it as `rollout.md`.
|
|
48
|
+
7. **Ask the sponsor at G11.1.** Run `doc-critique` over the policy and the rollout together,
|
|
49
|
+
then put the first wave, the conditions and the critique's open findings to the person who
|
|
50
|
+
sponsors the slice. Only they choose the verdict. In a non-interactive run, stop here and
|
|
51
|
+
write the decision they need to make; never record their verdict for them.
|
|
52
|
+
8. **Hand over the plan, then the verdict.** Lead with the slice, who starts using it and when,
|
|
53
|
+
and what would end it. The scorecard, the policy and the cohort plan go underneath. Close
|
|
54
|
+
with each gate on its own line, `G11: <VERDICT>` and `G11.1: <VERDICT>` (or
|
|
55
|
+
`G11.1: awaiting <role>`), and one sentence of reasons.
|
|
56
|
+
|
|
57
|
+
## Verdicts
|
|
58
|
+
|
|
59
|
+
Defined in [`contracts/terminals.json`](../../../../contracts/terminals.json). G11 emits:
|
|
60
|
+
|
|
61
|
+
- `PROCEED`: no blocking gap sits on what the slice needs, and its output has a review step.
|
|
62
|
+
- `PROCEED WITH CONDITIONS`: a gap sits on the slice but can close inside the time box, with
|
|
63
|
+
an owner role. Each condition travels into the rollout as a precondition for the pilot.
|
|
64
|
+
- `REVISE`: a blocking gap sits on the slice and won't close inside the time box. Return to
|
|
65
|
+
`triage` with that gap as a constraint.
|
|
66
|
+
- `RECONSIDER`: the gap blocks every candidate, such as no rule for what may go into any tool
|
|
67
|
+
or no tool access at all. Fixing it is the work, so the loop ends here with that gap's first
|
|
68
|
+
action and owner role.
|
|
69
|
+
- `BLOCKED`: no candidates to triage, or no team the slice belongs to.
|
|
70
|
+
|
|
71
|
+
G11.1 emits `PROCEED`, `PROCEED WITH CONDITIONS`, `REVISE` (back to `policy`, when the sponsor
|
|
72
|
+
rejects a rule or a cohort) or `BLOCKED` (no sponsor named).
|
|
73
|
+
|
|
74
|
+
On the cap-th `REVISE` at G11, emit `RECONSIDER`: three slices in a row hit blocking gaps, so
|
|
75
|
+
the gaps are the work.
|
|
76
|
+
|
|
77
|
+
## Non-interactive runs
|
|
78
|
+
|
|
79
|
+
With no candidates, `ai-use-case-triage` emits `BLOCKED`; carry it through as the loop's
|
|
80
|
+
verdict and run no further stages. Missing weights and readiness evidence degrade to the
|
|
81
|
+
skills' own `[assumption]` and `[insufficient evidence]` tags. G11.1 always stops at the
|
|
82
|
+
sponsor: write `G11.1: awaiting <role>` with the decision they face, never a verdict.
|
|
83
|
+
|
|
84
|
+
## When a stage's skill is not installed
|
|
85
|
+
|
|
86
|
+
Name the missing skill and skilldrop as its source, do the minimal inline version of that
|
|
87
|
+
stage, and record that it ran degraded. A degraded `readiness` still scores Data, Tooling,
|
|
88
|
+
Governance and Process against the slice with an evidence line each, because G11 can't be
|
|
89
|
+
judged without them. Never skip G11 or G11.1.
|
|
90
|
+
|
|
91
|
+
## Run log (opt-in)
|
|
92
|
+
|
|
93
|
+
Only when the environment variable `SKILLDROP_LOOP_LOG` is set to a file path: after each
|
|
94
|
+
gate's verdict, append one JSON line to that file. Nothing else is recorded, and nothing is
|
|
95
|
+
sent anywhere. When the variable isn't set, or you can't read it, skip this section.
|
|
96
|
+
|
|
97
|
+
```bash
|
|
98
|
+
printf '%s\n' '{"ts":"2026-11-03T10:15:00Z","loop":"adopt","stage":"readiness","gate":"G11","verdict":"REVISE","round":1}' >> "$SKILLDROP_LOOP_LOG"
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
Fields: `ts` (UTC), `loop`, `stage`, `gate` (`null` for a stage without one), `verdict` (the
|
|
102
|
+
gate's exact word), and `round` (1 for the first attempt, counting returns under the cap).
|
|
103
|
+
The log holds verdicts only: never the use cases, the scores or the policy.
|
|
104
|
+
`skilldrop loop-stats` summarises the file.
|
|
105
|
+
|
|
106
|
+
## Quality bar
|
|
107
|
+
|
|
108
|
+
- **Readiness is scored against the slice.** An organisation-wide score hides the one gap that stops the slice and flags gaps that stop nothing yet.
|
|
109
|
+
- **A `REVISE` changes the slice.** It never re-scores a dimension upward to let the same slice through.
|
|
110
|
+
- **The policy covers every kind of data the slice touches,** each restriction with what to do instead.
|
|
111
|
+
- **The pilot is the slice's users, and its adoption gate is triage's observable.** A pilot of volunteers from elsewhere proves nothing about the slice.
|
|
112
|
+
- **G11.1 is the sponsor's call.** The loop writes the decision they face; it never records their verdict.
|
|
113
|
+
|
|
114
|
+
## Anti-patterns to avoid
|
|
115
|
+
|
|
116
|
+
- ❌ **Scoring the whole company's readiness first.** The assessment then ranks gaps by how bad they look, not by what they block.
|
|
117
|
+
- ❌ **Writing a policy for every tool and every kind of data before anything ships.** It is long, it is late, and nobody reads the part that matters to the pilot.
|
|
118
|
+
- ❌ **Passing G11 with a blocking gap and a promise.** A gap that won't close inside the time box is a `REVISE`, however good the slice looks.
|
|
119
|
+
- ❌ **A pilot gate that is "the pilot went well".** The gate is triage's observable, with its threshold and time box.
|
|
120
|
+
- ❌ **Running all four skills before the first gate.** A policy and a rollout written for a slice that then fails G11 are thrown away.
|
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
[
|
|
2
|
+
{"query": "help us start using AI in the finance team, we have ideas but no rules and no plan", "should_trigger": true},
|
|
3
|
+
{"query": "run the adopt loop on these AI use cases from the workshop", "should_trigger": true},
|
|
4
|
+
{"query": "we want to roll out AI across the support team: pick where to start, check we're ready, write the rules and plan the waves", "should_trigger": true},
|
|
5
|
+
{"query": "take this list of AI ideas to a first pilot our VP can sign off", "should_trigger": true},
|
|
6
|
+
{"query": "rank these twelve AI use cases by value and risk", "should_trigger": false},
|
|
7
|
+
{"query": "write an AI acceptable-use policy for our staff", "should_trigger": false},
|
|
8
|
+
{"query": "how far along is our engineering team with coding agents?", "should_trigger": false},
|
|
9
|
+
{"query": "design an eval harness for our support chatbot", "should_trigger": false}
|
|
10
|
+
]
|