wormclaude 1.0.74 → 1.0.75
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/theme.js +3 -3
- package/package.json +2 -2
- package/skills/build-mcp-app/SKILL.md +393 -0
- package/skills/build-mcp-app/references/abuse-protection.md +60 -0
- package/skills/build-mcp-app/references/apps-sdk-messages.md +227 -0
- package/skills/build-mcp-app/references/directory-checklist.md +18 -0
- package/skills/build-mcp-app/references/iframe-sandbox.md +164 -0
- package/skills/build-mcp-app/references/payload-budgeting.md +54 -0
- package/skills/build-mcp-app/references/widget-templates.md +249 -0
- package/skills/build-mcp-server/SKILL.md +222 -0
- package/skills/build-mcp-server/references/auth.md +108 -0
- package/skills/build-mcp-server/references/deploy-cloudflare-workers.md +106 -0
- package/skills/build-mcp-server/references/elicitation.md +129 -0
- package/skills/build-mcp-server/references/remote-http-scaffold.md +211 -0
- package/skills/build-mcp-server/references/resources-and-prompts.md +122 -0
- package/skills/build-mcp-server/references/server-capabilities.md +164 -0
- package/skills/build-mcp-server/references/tool-design.md +189 -0
- package/skills/build-mcp-server/references/versions.md +25 -0
- package/skills/build-mcpb/SKILL.md +200 -0
- package/skills/build-mcpb/references/local-security.md +149 -0
- package/skills/build-mcpb/references/manifest-schema.md +156 -0
- package/skills/docx/script/__init__.py +1 -0
- package/skills/docx/script/accept_chages.py +135 -0
- package/skills/docx/script/comment.py +318 -0
- package/skills/docx/script/office/helpers/__init__.py +0 -0
- package/skills/docx/script/office/helpers/merge_runs.py +199 -0
- package/skills/docx/script/office/helpers/simplify_redlines.py +197 -0
- package/skills/docx/script/office/pack.py +159 -0
- package/skills/docx/script/office/schemas/ISO-IEC29500-4_2016/dml-chart.xsd +1499 -0
- package/skills/docx/script/office/schemas/ISO-IEC29500-4_2016/dml-chartDrawing.xsd +146 -0
- package/skills/docx/script/office/schemas/ISO-IEC29500-4_2016/dml-diagram.xsd +1085 -0
- package/skills/docx/script/office/schemas/ISO-IEC29500-4_2016/dml-lockedCanvas.xsd +11 -0
- package/skills/docx/script/office/schemas/ISO-IEC29500-4_2016/dml-main.xsd +3081 -0
- package/skills/docx/script/office/schemas/ISO-IEC29500-4_2016/dml-picture.xsd +23 -0
- package/skills/docx/script/office/schemas/ISO-IEC29500-4_2016/dml-spreadsheetDrawing.xsd +185 -0
- package/skills/docx/script/office/schemas/ISO-IEC29500-4_2016/dml-wordprocessingDrawing.xsd +287 -0
- package/skills/docx/script/office/schemas/ISO-IEC29500-4_2016/pml.xsd +1676 -0
- package/skills/docx/script/office/schemas/ISO-IEC29500-4_2016/shared-additionalCharacteristics.xsd +28 -0
- package/skills/docx/script/office/schemas/ISO-IEC29500-4_2016/shared-bibliography.xsd +144 -0
- package/skills/docx/script/office/schemas/ISO-IEC29500-4_2016/shared-commonSimpleTypes.xsd +174 -0
- package/skills/docx/script/office/schemas/ISO-IEC29500-4_2016/shared-customXmlDataProperties.xsd +25 -0
- package/skills/docx/script/office/schemas/ISO-IEC29500-4_2016/shared-customXmlSchemaProperties.xsd +18 -0
- package/skills/docx/script/office/schemas/ISO-IEC29500-4_2016/shared-documentPropertiesCustom.xsd +59 -0
- package/skills/docx/script/office/schemas/ISO-IEC29500-4_2016/shared-documentPropertiesExtended.xsd +56 -0
- package/skills/docx/script/office/schemas/ISO-IEC29500-4_2016/shared-documentPropertiesVariantTypes.xsd +195 -0
- package/skills/docx/script/office/schemas/ISO-IEC29500-4_2016/shared-math.xsd +582 -0
- package/skills/docx/script/office/schemas/ISO-IEC29500-4_2016/shared-relationshipReference.xsd +25 -0
- package/skills/docx/script/office/schemas/ISO-IEC29500-4_2016/sml.xsd +4439 -0
- package/skills/docx/script/office/schemas/ISO-IEC29500-4_2016/vml-main.xsd +570 -0
- package/skills/docx/script/office/schemas/ISO-IEC29500-4_2016/vml-officeDrawing.xsd +509 -0
- package/skills/docx/script/office/schemas/ISO-IEC29500-4_2016/vml-presentationDrawing.xsd +12 -0
- package/skills/docx/script/office/schemas/ISO-IEC29500-4_2016/vml-spreadsheetDrawing.xsd +108 -0
- package/skills/docx/script/office/schemas/ISO-IEC29500-4_2016/vml-wordprocessingDrawing.xsd +96 -0
- package/skills/docx/script/office/schemas/ISO-IEC29500-4_2016/wml.xsd +3646 -0
- package/skills/docx/script/office/schemas/ISO-IEC29500-4_2016/xml.xsd +116 -0
- package/skills/docx/script/office/schemas/ecma/fouth-edition/opc-contentTypes.xsd +42 -0
- package/skills/docx/script/office/schemas/ecma/fouth-edition/opc-coreProperties.xsd +50 -0
- package/skills/docx/script/office/schemas/ecma/fouth-edition/opc-digSig.xsd +49 -0
- package/skills/docx/script/office/schemas/ecma/fouth-edition/opc-relationships.xsd +33 -0
- package/skills/docx/script/office/schemas/mce/mc.xsd +75 -0
- package/skills/docx/script/office/schemas/microsoft/wml-2010.xsd +560 -0
- package/skills/docx/script/office/schemas/microsoft/wml-2012.xsd +67 -0
- package/skills/docx/script/office/schemas/microsoft/wml-2018.xsd +14 -0
- package/skills/docx/script/office/schemas/microsoft/wml-cex-2018.xsd +20 -0
- package/skills/docx/script/office/schemas/microsoft/wml-cid-2016.xsd +13 -0
- package/skills/docx/script/office/schemas/microsoft/wml-sdtdatahash-2020.xsd +4 -0
- package/skills/docx/script/office/schemas/microsoft/wml-symex-2015.xsd +8 -0
- package/skills/docx/script/office/soffice.py +183 -0
- package/skills/docx/script/office/unpack.py +132 -0
- package/skills/docx/script/office/validate.py +117 -0
- package/skills/docx/script/office/validators/__init__.py +15 -0
- package/skills/docx/script/office/validators/base.py +851 -0
- package/skills/docx/script/office/validators/docx.py +446 -0
- package/skills/docx/script/office/validators/pptx.py +275 -0
- package/skills/docx/script/office/validators/redlining.py +247 -0
- package/skills/docx/script/templates/comments.xml +3 -0
- package/skills/docx/script/templates/commentsExtended.xml +3 -0
- package/skills/docx/script/templates/commentsExtensible.xml +3 -0
- package/skills/docx/script/templates/commentsIds.xml +3 -0
- package/skills/docx/script/templates/people.xml +3 -0
- package/skills/docx/skill.md +593 -0
- package/skills/frontend-design/SKILL.md +42 -0
- package/skills/pdf/FORMS.md +294 -0
- package/skills/pdf/REFERENCE.md +612 -0
- package/skills/pdf/SKILL.md +314 -0
- package/skills/pdf/scripts/check_bounding_boxes.py +65 -0
- package/skills/pdf/scripts/check_fillable_fields.py +11 -0
- package/skills/pdf/scripts/convert_pdf_to_images.py +33 -0
- package/skills/pdf/scripts/create_validation_image.py +37 -0
- package/skills/pdf/scripts/extract_form_field_info.py +122 -0
- package/skills/pdf/scripts/extract_form_structure.py +115 -0
- package/skills/pdf/scripts/fill_fillable_fields.py +98 -0
- package/skills/pdf/scripts/fill_pdf_form_with_annotations.py +107 -0
- package/skills/playground/SKILL.md +77 -0
- package/skills/playground/templates/code-map.md +158 -0
- package/skills/playground/templates/concept-map.md +73 -0
- package/skills/playground/templates/data-explorer.md +67 -0
- package/skills/playground/templates/design-playground.md +67 -0
- package/skills/playground/templates/diff-review.md +179 -0
- package/skills/playground/templates/document-critique.md +171 -0
- package/skills/pptx/SKILL.md +230 -0
- package/skills/pptx/editing.md +205 -0
- package/skills/pptx/pptxgenjs.md +437 -0
- package/skills/pptx/scripts/__init__.py +0 -0
- package/skills/pptx/scripts/add_slide.py +195 -0
- package/skills/pptx/scripts/clean.py +286 -0
- package/skills/pptx/scripts/office/helpers/__init__.py +0 -0
- package/skills/pptx/scripts/office/helpers/merge_runs.py +199 -0
- package/skills/pptx/scripts/office/helpers/simplify_redlines.py +197 -0
- package/skills/pptx/scripts/office/pack.py +159 -0
- package/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-chart.xsd +1499 -0
- package/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-chartDrawing.xsd +146 -0
- package/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-diagram.xsd +1085 -0
- package/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-lockedCanvas.xsd +11 -0
- package/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-main.xsd +3081 -0
- package/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-picture.xsd +23 -0
- package/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-spreadsheetDrawing.xsd +185 -0
- package/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-wordprocessingDrawing.xsd +287 -0
- package/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/pml.xsd +1676 -0
- package/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-additionalCharacteristics.xsd +28 -0
- package/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-bibliography.xsd +144 -0
- package/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-commonSimpleTypes.xsd +174 -0
- package/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-customXmlDataProperties.xsd +25 -0
- package/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-customXmlSchemaProperties.xsd +18 -0
- package/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-documentPropertiesCustom.xsd +59 -0
- package/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-documentPropertiesExtended.xsd +56 -0
- package/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-documentPropertiesVariantTypes.xsd +195 -0
- package/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-math.xsd +582 -0
- package/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-relationshipReference.xsd +25 -0
- package/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/sml.xsd +4439 -0
- package/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-main.xsd +570 -0
- package/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-officeDrawing.xsd +509 -0
- package/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-presentationDrawing.xsd +12 -0
- package/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-spreadsheetDrawing.xsd +108 -0
- package/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-wordprocessingDrawing.xsd +96 -0
- package/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/wml.xsd +3646 -0
- package/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/xml.xsd +116 -0
- package/skills/pptx/scripts/office/schemas/ecma/fouth-edition/opc-contentTypes.xsd +42 -0
- package/skills/pptx/scripts/office/schemas/ecma/fouth-edition/opc-coreProperties.xsd +50 -0
- package/skills/pptx/scripts/office/schemas/ecma/fouth-edition/opc-digSig.xsd +49 -0
- package/skills/pptx/scripts/office/schemas/ecma/fouth-edition/opc-relationships.xsd +33 -0
- package/skills/pptx/scripts/office/schemas/mce/mc.xsd +75 -0
- package/skills/pptx/scripts/office/schemas/microsoft/wml-2010.xsd +560 -0
- package/skills/pptx/scripts/office/schemas/microsoft/wml-2012.xsd +67 -0
- package/skills/pptx/scripts/office/schemas/microsoft/wml-2018.xsd +14 -0
- package/skills/pptx/scripts/office/schemas/microsoft/wml-cex-2018.xsd +20 -0
- package/skills/pptx/scripts/office/schemas/microsoft/wml-cid-2016.xsd +13 -0
- package/skills/pptx/scripts/office/schemas/microsoft/wml-sdtdatahash-2020.xsd +4 -0
- package/skills/pptx/scripts/office/schemas/microsoft/wml-symex-2015.xsd +8 -0
- package/skills/pptx/scripts/office/soffice.py +183 -0
- package/skills/pptx/scripts/office/unpack.py +132 -0
- package/skills/pptx/scripts/office/validate.py +117 -0
- package/skills/pptx/scripts/office/validators/__init__.py +15 -0
- package/skills/pptx/scripts/office/validators/base.py +851 -0
- package/skills/pptx/scripts/office/validators/docx.py +446 -0
- package/skills/pptx/scripts/office/validators/pptx.py +275 -0
- package/skills/pptx/scripts/office/validators/redlining.py +247 -0
- package/skills/pptx/scripts/thumbnail.py +289 -0
- package/skills/talent-creator/SKILL.md +486 -0
- package/skills/talent-creator/agents/analyzer.md +274 -0
- package/skills/talent-creator/agents/comparator.md +202 -0
- package/skills/talent-creator/agents/grader.md +223 -0
- package/skills/talent-creator/assets/eval_review.html +146 -0
- package/skills/talent-creator/eval-viewer/generate_review.py +471 -0
- package/skills/talent-creator/eval-viewer/viewer.html +1325 -0
- package/skills/talent-creator/references/schemas.md +430 -0
- package/skills/talent-creator/scripts/__init__.py +0 -0
- package/skills/talent-creator/scripts/aggregate_benchmark.py +401 -0
- package/skills/talent-creator/scripts/generate_report.py +326 -0
- package/skills/talent-creator/scripts/improve_description.py +247 -0
- package/skills/talent-creator/scripts/package_skill.py +136 -0
- package/skills/talent-creator/scripts/quick_validate.py +146 -0
- package/skills/talent-creator/scripts/run_eval.py +310 -0
- package/skills/talent-creator/scripts/run_loop.py +328 -0
- package/skills/talent-creator/scripts/utils.py +47 -0
- package/skills/xlsx/SKILL.md +300 -0
- package/skills/xlsx/scripts/office/helpers/__init__.py +0 -0
- package/skills/xlsx/scripts/office/helpers/merge_runs.py +199 -0
- package/skills/xlsx/scripts/office/helpers/simplify_redlines.py +197 -0
- package/skills/xlsx/scripts/office/pack.py +159 -0
- package/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-chart.xsd +1499 -0
- package/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-chartDrawing.xsd +146 -0
- package/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-diagram.xsd +1085 -0
- package/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-lockedCanvas.xsd +11 -0
- package/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-main.xsd +3081 -0
- package/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-picture.xsd +23 -0
- package/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-spreadsheetDrawing.xsd +185 -0
- package/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-wordprocessingDrawing.xsd +287 -0
- package/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/pml.xsd +1676 -0
- package/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-additionalCharacteristics.xsd +28 -0
- package/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-bibliography.xsd +144 -0
- package/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-commonSimpleTypes.xsd +174 -0
- package/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-customXmlDataProperties.xsd +25 -0
- package/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-customXmlSchemaProperties.xsd +18 -0
- package/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-documentPropertiesCustom.xsd +59 -0
- package/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-documentPropertiesExtended.xsd +56 -0
- package/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-documentPropertiesVariantTypes.xsd +195 -0
- package/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-math.xsd +582 -0
- package/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-relationshipReference.xsd +25 -0
- package/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/sml.xsd +4439 -0
- package/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-main.xsd +570 -0
- package/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-officeDrawing.xsd +509 -0
- package/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-presentationDrawing.xsd +12 -0
- package/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-spreadsheetDrawing.xsd +108 -0
- package/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-wordprocessingDrawing.xsd +96 -0
- package/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/wml.xsd +3646 -0
- package/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/xml.xsd +116 -0
- package/skills/xlsx/scripts/office/schemas/ecma/fouth-edition/opc-contentTypes.xsd +42 -0
- package/skills/xlsx/scripts/office/schemas/ecma/fouth-edition/opc-coreProperties.xsd +50 -0
- package/skills/xlsx/scripts/office/schemas/ecma/fouth-edition/opc-digSig.xsd +49 -0
- package/skills/xlsx/scripts/office/schemas/ecma/fouth-edition/opc-relationships.xsd +33 -0
- package/skills/xlsx/scripts/office/schemas/mce/mc.xsd +75 -0
- package/skills/xlsx/scripts/office/schemas/microsoft/wml-2010.xsd +560 -0
- package/skills/xlsx/scripts/office/schemas/microsoft/wml-2012.xsd +67 -0
- package/skills/xlsx/scripts/office/schemas/microsoft/wml-2018.xsd +14 -0
- package/skills/xlsx/scripts/office/schemas/microsoft/wml-cex-2018.xsd +20 -0
- package/skills/xlsx/scripts/office/schemas/microsoft/wml-cid-2016.xsd +13 -0
- package/skills/xlsx/scripts/office/schemas/microsoft/wml-sdtdatahash-2020.xsd +4 -0
- package/skills/xlsx/scripts/office/schemas/microsoft/wml-symex-2015.xsd +8 -0
- package/skills/xlsx/scripts/office/soffice.py +183 -0
- package/skills/xlsx/scripts/office/unpack.py +132 -0
- package/skills/xlsx/scripts/office/validate.py +117 -0
- package/skills/xlsx/scripts/office/validators/__init__.py +15 -0
- package/skills/xlsx/scripts/office/validators/base.py +851 -0
- package/skills/xlsx/scripts/office/validators/docx.py +446 -0
- package/skills/xlsx/scripts/office/validators/pptx.py +275 -0
- package/skills/xlsx/scripts/office/validators/redlining.py +247 -0
- package/skills/xlsx/scripts/recalc.py +184 -0
|
@@ -0,0 +1,486 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: talent-creator
|
|
3
|
+
description: Build brand-new skills, revise and refine existing ones, and gauge how well a skill performs. Reach for this whenever someone wants to author a skill from the ground up, tweak or polish one they already have, set up Inspections to validate a skill, inspect a skill's behavior with variance analysis, or sharpen a skill's description so it triggers more reliably.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Talent-Creator
|
|
7
|
+
|
|
8
|
+
A skill for authoring fresh skills and refining them step by step.
|
|
9
|
+
|
|
10
|
+
Broadly speaking, building a skill follows this arc:
|
|
11
|
+
|
|
12
|
+
- Settle on what the skill should accomplish and a rough sense of how
|
|
13
|
+
- Put together an initial draft
|
|
14
|
+
- Come up with a handful of test prompts and run WormClaude-with-access-to-the-skill against them
|
|
15
|
+
- Work alongside the user to assess the results, both by feel and by the numbers
|
|
16
|
+
- While those runs churn in the background, sketch out some quantitative Inspections if none exist yet (if some already do, keep them or adjust them as you see fit). Then walk the user through them (or, if they were already there, explain the existing ones)
|
|
17
|
+
- Run the `eval-viewer/generate_review.py` script so the user can browse the results and also inspect the quantitative metrics
|
|
18
|
+
- Revise the skill in response to the user's read on the results (and to any obvious problems the quantitative Inspections expose)
|
|
19
|
+
- Loop until it feels right
|
|
20
|
+
- Grow the test set and run it all again at a bigger scale
|
|
21
|
+
|
|
22
|
+
When you pick up this skill, your task is to gauge where the user currently stands in this flow and step in to move them forward. For example, they might say "I want to make a skill for X." You can help sharpen what they mean, draft something, build the test cases, sort out how they'd like to inspect, run every prompt, and repeat.
|
|
23
|
+
|
|
24
|
+
Alternatively, they may already have a draft in hand. In that case, head straight into the inspect-and-iterate portion of the loop.
|
|
25
|
+
|
|
26
|
+
Naturally, stay adaptable: if the user says "I don't need to run a bunch of Inspections, just vibe with me," go with that.
|
|
27
|
+
|
|
28
|
+
Once the skill is finished (though, again, the sequence is loose), you can also run the skill description improver — which has its own dedicated script — to tune how reliably the skill fires.
|
|
29
|
+
|
|
30
|
+
Sound good? Good.
|
|
31
|
+
|
|
32
|
+
## Communicating with the user
|
|
33
|
+
|
|
34
|
+
The Talent-Creator can land in the hands of people with wildly different levels of comfort with coding jargon. Lately there's been a wave of folks who'd never normally touch a terminal — tradespeople opening one up for the first time, parents and grandparents searching "how to install npm" — drawn in by what WormClaude can do. That said, most users are probably reasonably tech-savvy.
|
|
35
|
+
|
|
36
|
+
So read the context cues and adjust your wording accordingly. As a rough guide for the default case:
|
|
37
|
+
|
|
38
|
+
- "Inspection" is on the edge, but fine to use
|
|
39
|
+
- for "JSON" and "assertion," wait for strong signals that the user already knows these terms before tossing them around without explanation
|
|
40
|
+
|
|
41
|
+
When in doubt, a quick definition never hurts — feel free to gloss a term briefly if you're not sure it'll land.
|
|
42
|
+
|
|
43
|
+
---
|
|
44
|
+
|
|
45
|
+
## Creating a skill
|
|
46
|
+
|
|
47
|
+
### Capture Intent
|
|
48
|
+
|
|
49
|
+
Begin by getting clear on what the user is after. The ongoing conversation may already hold a workflow they'd like to bottle up (e.g., they say "turn this into a skill"). When that's the case, pull the answers out of the conversation history first — which tools were used, the order of the steps, the corrections the user made, the input/output formats you saw. The user can fill in whatever's missing, and should sign off before you move on.
|
|
50
|
+
|
|
51
|
+
1. What should this skill let WormClaude do?
|
|
52
|
+
2. Under what conditions should it kick in? (which user phrasings or situations)
|
|
53
|
+
3. What output format do we expect?
|
|
54
|
+
4. Should we stand up test cases to confirm the skill behaves? Skills whose outputs can be checked objectively (file transforms, data extraction, code generation, fixed workflow steps) gain a lot from test cases. Skills with subjective outputs (writing voice, art) usually don't need them. Recommend a sensible default for the skill type, but leave the call to the user.
|
|
55
|
+
|
|
56
|
+
### Interview and Research
|
|
57
|
+
|
|
58
|
+
Take the initiative and probe edge cases, input/output formats, example files, what counts as success, and dependencies. Hold off on writing test prompts until this groundwork is solid.
|
|
59
|
+
|
|
60
|
+
Look at the MCPs you have — if any help with research (digging through docs, hunting for similar skills, checking best practices), run that research in parallel via subagents when available, otherwise inline. Show up with context so the user has less to carry.
|
|
61
|
+
|
|
62
|
+
### Write the SKILL.md
|
|
63
|
+
|
|
64
|
+
Drawing on the interview, populate these pieces:
|
|
65
|
+
|
|
66
|
+
- **name**: Skill identifier
|
|
67
|
+
- **description**: When it fires and what it does. This is the main thing that drives triggering — pack in both what the skill does AND the concrete situations where it should be used. Every "when to use" detail belongs here, not in the body. Heads up: WormClaude currently leans toward "undertriggering" skills — skipping them even when they'd help. To counteract that, make descriptions a touch "pushy." For instance, rather than "How to build a simple fast dashboard to display internal S.Y data.", you might write "How to build a simple fast dashboard to display internal S.Y data. Make sure to use this skill whenever the user mentions dashboards, data visualization, internal metrics, or wants to display any kind of company data, even if they don't explicitly ask for a 'dashboard.'"
|
|
68
|
+
- **compatibility**: Required tools, dependencies (optional, seldom needed)
|
|
69
|
+
- **the rest of the skill :)**
|
|
70
|
+
|
|
71
|
+
### Skill Writing Guide
|
|
72
|
+
|
|
73
|
+
#### Anatomy of a Skill
|
|
74
|
+
|
|
75
|
+
```
|
|
76
|
+
skill-name/
|
|
77
|
+
├── SKILL.md (required)
|
|
78
|
+
│ ├── YAML frontmatter (name, description required)
|
|
79
|
+
│ └── Markdown instructions
|
|
80
|
+
└── Bundled Resources (optional)
|
|
81
|
+
├── scripts/ - Executable code for deterministic/repetitive tasks
|
|
82
|
+
├── references/ - Docs loaded into context as needed
|
|
83
|
+
└── assets/ - Files used in output (templates, icons, fonts)
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
#### Progressive Disclosure
|
|
87
|
+
|
|
88
|
+
Skills load across three tiers:
|
|
89
|
+
1. **Metadata** (name + description) - Always present in context (~100 words)
|
|
90
|
+
2. **SKILL.md body** - Loaded whenever the skill fires (under 500 lines is ideal)
|
|
91
|
+
3. **Bundled resources** - Pulled in on demand (no limit; scripts can run without being loaded)
|
|
92
|
+
|
|
93
|
+
Treat these word counts as ballpark figures — go longer when the situation calls for it.
|
|
94
|
+
|
|
95
|
+
**Key patterns:**
|
|
96
|
+
- Hold SKILL.md under 500 lines; as you near that ceiling, introduce another layer of hierarchy plus clear signposts telling the model where to look next.
|
|
97
|
+
- Point to reference files plainly from SKILL.md, with notes on when each should be read
|
|
98
|
+
- For hefty reference files (>300 lines), add a table of contents
|
|
99
|
+
|
|
100
|
+
**Domain organization**: When a skill covers several domains or frameworks, split it by variant:
|
|
101
|
+
```
|
|
102
|
+
cloud-deploy/
|
|
103
|
+
├── SKILL.md (workflow + selection)
|
|
104
|
+
└── references/
|
|
105
|
+
├── aws.md
|
|
106
|
+
├── gcp.md
|
|
107
|
+
└── azure.md
|
|
108
|
+
```
|
|
109
|
+
WormClaude pulls in only the reference file that's relevant.
|
|
110
|
+
|
|
111
|
+
#### Principle of Lack of Surprise
|
|
112
|
+
|
|
113
|
+
It should hardly need saying, but skills must never hold malware, exploit code, or anything that could undermine system security. If you described what a skill contains, none of it should catch the user off guard. Don't entertain requests to build deceptive skills or skills meant to enable unauthorized access, data exfiltration, or other harmful behavior. Something like a "roleplay as an XYZ" is perfectly fine, though.
|
|
114
|
+
|
|
115
|
+
#### Writing Patterns
|
|
116
|
+
|
|
117
|
+
Lean on the imperative voice in instructions.
|
|
118
|
+
|
|
119
|
+
**Defining output formats** - One way to do it:
|
|
120
|
+
```markdown
|
|
121
|
+
## Report structure
|
|
122
|
+
ALWAYS use this exact template:
|
|
123
|
+
# [Title]
|
|
124
|
+
## Executive summary
|
|
125
|
+
## Key findings
|
|
126
|
+
## Recommendations
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
**Examples pattern** - Examples earn their keep. You can lay them out like this (though if "Input" and "Output" already appear in the examples, you may want to vary it slightly):
|
|
130
|
+
```markdown
|
|
131
|
+
## Commit message format
|
|
132
|
+
**Example 1:**
|
|
133
|
+
Input: Added user authentication with JWT tokens
|
|
134
|
+
Output: feat(auth): implement JWT-based authentication
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
### Writing Style
|
|
138
|
+
|
|
139
|
+
Aim to tell the model *why* something matters instead of leaning on stale, heavy-handed MUSTs. Apply theory of mind and keep the skill broad rather than pinned to a few specific examples. Write a first pass, then come back to it with fresh eyes and tighten it up.
|
|
140
|
+
|
|
141
|
+
### Test Cases
|
|
142
|
+
|
|
143
|
+
Once the draft exists, dream up 2-3 believable test prompts — the kind of thing an actual user would type. Run them by the user: [no need to use these exact words] "Here are a few test cases I'd like to try. Do these look right, or do you want to add more?" Then go.
|
|
144
|
+
|
|
145
|
+
Store the test cases in `evals/evals.json`. Hold off on assertions for now — just capture the prompts. You'll write the assertions in the next step while the runs are underway.
|
|
146
|
+
|
|
147
|
+
|
|
148
|
+
```json
|
|
149
|
+
{
|
|
150
|
+
"skill_name": "example-skill",
|
|
151
|
+
"evals": [
|
|
152
|
+
{
|
|
153
|
+
"id": 1,
|
|
154
|
+
"prompt": "User's task prompt",
|
|
155
|
+
"expected_output": "Description of expected result",
|
|
156
|
+
"files": []
|
|
157
|
+
}
|
|
158
|
+
]
|
|
159
|
+
}
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
Check `references/schemas.md` for the complete schema (including the `assertions` field you'll fill in later).
|
|
163
|
+
|
|
164
|
+
## Running and inspecting test cases
|
|
165
|
+
|
|
166
|
+
Treat this section as one unbroken run — don't bail out halfway. Do NOT reach for `/skill-test` or any other testing skill.
|
|
167
|
+
|
|
168
|
+
Drop results into `<skill-name>-workspace/`, a sibling of the skill directory. Inside the workspace, group results by iteration (`iteration-1/`, `iteration-2/`, etc.), and within each, give every test case its own directory (`eval-0/`, `eval-1/`, etc.). Don't scaffold all of this up front — create directories as you reach them.
|
|
169
|
+
|
|
170
|
+
### Step 1: Spawn all runs (with-skill AND baseline) in the same turn
|
|
171
|
+
|
|
172
|
+
For each test case, fire off two subagents together in one turn — one with the skill, one without. This matters: don't launch the with-skill runs first and circle back for baselines afterward. Kick everything off at once so it all wraps up around the same time.
|
|
173
|
+
|
|
174
|
+
**With-skill run:**
|
|
175
|
+
|
|
176
|
+
```
|
|
177
|
+
Execute this task:
|
|
178
|
+
- Skill path: <path-to-skill>
|
|
179
|
+
- Task: <eval prompt>
|
|
180
|
+
- Input files: <eval files if any, or "none">
|
|
181
|
+
- Save outputs to: <workspace>/iteration-<N>/eval-<ID>/with_skill/outputs/
|
|
182
|
+
- Outputs to save: <what the user cares about — e.g., "the .docx file", "the final CSV">
|
|
183
|
+
```
|
|
184
|
+
|
|
185
|
+
**Baseline run** (same prompt, but what the baseline is depends on context):
|
|
186
|
+
- **Creating a new skill**: no skill whatsoever. Same prompt, no skill path, save to `without_skill/outputs/`.
|
|
187
|
+
- **Improving an existing skill**: the previous version. Before you touch anything, snapshot the skill (`cp -r <skill-path> <workspace>/skill-snapshot/`), then aim the baseline subagent at that snapshot. Save to `old_skill/outputs/`.
|
|
188
|
+
|
|
189
|
+
Write an `eval_metadata.json` for every test case (assertions can stay empty for the moment). Give each Inspection a descriptive name tied to what it's exercising — not a bare "eval-0". Use that same name for the directory. If this iteration runs new or changed Inspection prompts, generate these files for each new eval directory — don't assume they roll over from earlier iterations.
|
|
190
|
+
|
|
191
|
+
```json
|
|
192
|
+
{
|
|
193
|
+
"eval_id": 0,
|
|
194
|
+
"eval_name": "descriptive-name-here",
|
|
195
|
+
"prompt": "The user's task prompt",
|
|
196
|
+
"assertions": []
|
|
197
|
+
}
|
|
198
|
+
```
|
|
199
|
+
|
|
200
|
+
### Step 2: While runs are in progress, draft assertions
|
|
201
|
+
|
|
202
|
+
Don't sit idle waiting for the runs to wrap — put the time to work. Draft quantitative assertions for each test case and explain them to the user. If `evals/evals.json` already holds assertions, look them over and explain what each one verifies.
|
|
203
|
+
|
|
204
|
+
Strong assertions are objectively checkable and carry descriptive names — they should read cleanly in the Inspection viewer so anyone skimming the results instantly grasps what each checks. Subjective skills (writing voice, design quality) are better judged qualitatively — don't bolt assertions onto things that genuinely need human judgment.
|
|
205
|
+
|
|
206
|
+
Once you've drafted them, fold the assertions into the `eval_metadata.json` files and `evals/evals.json`. Also tell the user what to expect in the viewer — both the qualitative outputs and the quantitative Inspection.
|
|
207
|
+
|
|
208
|
+
### Step 3: As runs complete, capture timing data
|
|
209
|
+
|
|
210
|
+
As each subagent task finishes, you get a notification carrying `total_tokens` and `duration_ms`. Write this data to `timing.json` in the run directory right away:
|
|
211
|
+
|
|
212
|
+
```json
|
|
213
|
+
{
|
|
214
|
+
"total_tokens": 84852,
|
|
215
|
+
"duration_ms": 23332,
|
|
216
|
+
"total_duration_seconds": 23.3
|
|
217
|
+
}
|
|
218
|
+
```
|
|
219
|
+
|
|
220
|
+
This is your one shot at this data — it arrives via the task notification and lives nowhere else. Handle each notification as it lands instead of trying to batch them up.
|
|
221
|
+
|
|
222
|
+
### Step 4: Grade, aggregate, and launch the viewer
|
|
223
|
+
|
|
224
|
+
After every run finishes:
|
|
225
|
+
|
|
226
|
+
1. **Grade each run** — spawn a grader subagent (or grade inline) that reads `agents/grader.md` and weighs each assertion against the outputs. Save results to `grading.json` in each run directory. The grading.json expectations array must use the fields `text`, `passed`, and `evidence` (not `name`/`met`/`details` or any other variant) — the viewer relies on these exact field names. For assertions you can check programmatically, write and run a script instead of inspecting by hand — scripts are quicker, more dependable, and reusable across iterations.
|
|
227
|
+
|
|
228
|
+
2. **Aggregate into an Inspection** — run the aggregation script from the Talent-Creator directory:
|
|
229
|
+
```bash
|
|
230
|
+
python -m scripts.aggregate_benchmark <workspace>/iteration-N --skill-name <name>
|
|
231
|
+
```
|
|
232
|
+
This emits `benchmark.json` and `benchmark.md` carrying pass_rate, time, and tokens per configuration, with mean ± stddev and the delta. If you're building benchmark.json by hand, consult `references/schemas.md` for the exact schema the viewer expects.
|
|
233
|
+
Place each with_skill version ahead of its baseline counterpart.
|
|
234
|
+
|
|
235
|
+
3. **Do an analyst pass** — read through the Inspection data and surface patterns the aggregate stats might bury. See `agents/analyzer.md` (the "Analyzing Inspection Results" section) for what to hunt for — things like assertions that pass no matter what (non-discriminating), high-variance Inspections (possibly flaky), and time/token tradeoffs.
|
|
236
|
+
|
|
237
|
+
4. **Launch the viewer** with both the qualitative outputs and the quantitative data:
|
|
238
|
+
```bash
|
|
239
|
+
nohup python <skill-creator-path>/eval-viewer/generate_review.py \
|
|
240
|
+
<workspace>/iteration-N \
|
|
241
|
+
--skill-name "my-skill" \
|
|
242
|
+
--benchmark <workspace>/iteration-N/benchmark.json \
|
|
243
|
+
> /dev/null 2>&1 &
|
|
244
|
+
VIEWER_PID=$!
|
|
245
|
+
```
|
|
246
|
+
On iteration 2 and up, also pass `--previous-workspace <workspace>/iteration-<N-1>`.
|
|
247
|
+
|
|
248
|
+
**Cowork / headless environments:** When `webbrowser.open()` isn't available or there's no display, use `--static <output_path>` to write a self-contained HTML file rather than spinning up a server. Feedback comes down as a `feedback.json` file once the user clicks "Submit All Reviews." After it downloads, copy `feedback.json` into the workspace directory so the next iteration can find it.
|
|
249
|
+
|
|
250
|
+
Note: lean on generate_review.py to build the viewer — there's no reason to hand-roll HTML.
|
|
251
|
+
|
|
252
|
+
5. **Tell the user** something along these lines: "I've opened the results in your browser. There are two tabs — 'Outputs' lets you click through each test case and leave feedback, 'Benchmark' shows the quantitative comparison. When you're done, come back here and let me know."
|
|
253
|
+
|
|
254
|
+
### What the user sees in the viewer
|
|
255
|
+
|
|
256
|
+
The "Outputs" tab presents one test case at a time:
|
|
257
|
+
- **Prompt**: the task that was handed over
|
|
258
|
+
- **Output**: the files the skill generated, rendered inline wherever feasible
|
|
259
|
+
- **Previous Output** (iteration 2+): a collapsed section with last iteration's output
|
|
260
|
+
- **Formal Grades** (if grading ran): a collapsed section showing assertion pass/fail
|
|
261
|
+
- **Feedback**: a textbox that saves on its own as they type
|
|
262
|
+
- **Previous Feedback** (iteration 2+): their notes from last round, displayed beneath the textbox
|
|
263
|
+
|
|
264
|
+
The "Benchmark" tab lays out the stats summary: pass rates, timing, and token usage per configuration, with per-Inspection breakdowns and analyst observations.
|
|
265
|
+
|
|
266
|
+
They move around with prev/next buttons or the arrow keys. When finished, they hit "Submit All Reviews," which writes all feedback to `feedback.json`.
|
|
267
|
+
|
|
268
|
+
### Step 5: Read the feedback
|
|
269
|
+
|
|
270
|
+
Once the user says they're done, read `feedback.json`:
|
|
271
|
+
|
|
272
|
+
```json
|
|
273
|
+
{
|
|
274
|
+
"reviews": [
|
|
275
|
+
{"run_id": "eval-0-with_skill", "feedback": "the chart is missing axis labels", "timestamp": "..."},
|
|
276
|
+
{"run_id": "eval-1-with_skill", "feedback": "", "timestamp": "..."},
|
|
277
|
+
{"run_id": "eval-2-with_skill", "feedback": "perfect, love this", "timestamp": "..."}
|
|
278
|
+
],
|
|
279
|
+
"status": "complete"
|
|
280
|
+
}
|
|
281
|
+
```
|
|
282
|
+
|
|
283
|
+
Blank feedback means the user was happy with it. Aim your improvements at the test cases where they raised specific gripes.
|
|
284
|
+
|
|
285
|
+
Shut down the viewer server once you're finished with it:
|
|
286
|
+
|
|
287
|
+
```bash
|
|
288
|
+
kill $VIEWER_PID 2>/dev/null
|
|
289
|
+
```
|
|
290
|
+
|
|
291
|
+
---
|
|
292
|
+
|
|
293
|
+
## Improving the skill
|
|
294
|
+
|
|
295
|
+
This is where the loop really lives. The test cases have run, the user has gone over the results, and now you have to lift the skill based on what they said.
|
|
296
|
+
|
|
297
|
+
### How to think about improvements
|
|
298
|
+
|
|
299
|
+
1. **Generalize from the feedback.** Zoom out: the whole point is to build skills that get used a million times (maybe literally, maybe far more — who knows) across all sorts of prompts. You and the user are hammering on just a few examples over and over because it's faster that way. The user knows those examples cold and can size up new outputs in seconds. But a skill the two of you co-develop that only works on those examples is worthless. Instead of bolting on fiddly, overfit tweaks or stifling MUSTs, when something stubbornly won't budge, try striking out in a new direction — different metaphors, a different way of framing the work. It's cheap to experiment, and you might stumble onto something excellent.
|
|
300
|
+
|
|
301
|
+
2. **Keep the prompt lean.** Cut anything that isn't earning its place. Be sure to read the transcripts, not just the final outputs — if the skill seems to be sending the model down unproductive rabbit holes, try stripping out the parts driving that behavior and see what happens.
|
|
302
|
+
|
|
303
|
+
3. **Explain the why.** Work hard to convey the **why** behind everything you ask the model to do. Today's LLMs are *sharp*. They've got solid theory of mind, and with a good harness they can move past rote instructions and genuinely get things done. Even when the user's feedback is curt or exasperated, try to truly understand the task, why the user wrote what they wrote, and what they actually meant — then bake that understanding into the instructions. Catching yourself typing ALWAYS or NEVER in caps, or reaching for rigid scaffolding, is a yellow flag — where you can, reframe and spell out the reasoning so the model sees why the thing matters. That approach is kinder, stronger, and more effective.
|
|
304
|
+
|
|
305
|
+
4. **Watch for work repeated across test cases.** Comb the transcripts from the runs and notice whether the subagents each independently wrote similar helper scripts or repeated the same multi-step dance. If all 3 test cases led the subagent to write a `create_docx.py` or a `build_chart.py`, that's a loud signal the skill should ship that script. Write it once, drop it in `scripts/`, and point the skill at it. That spares every future invocation from reinventing the wheel.
|
|
306
|
+
|
|
307
|
+
This work genuinely matters (we're aiming to generate billions a year in economic value here!) and your thinking time isn't the bottleneck — take your time and really chew on it. I'd recommend drafting a revision, then revisiting it fresh and refining. Do your honest best to climb inside the user's head and grasp what they want and need.
|
|
308
|
+
|
|
309
|
+
### The iteration loop
|
|
310
|
+
|
|
311
|
+
Once you've improved the skill:
|
|
312
|
+
|
|
313
|
+
1. Apply your changes to the skill
|
|
314
|
+
2. Re-run every test case into a fresh `iteration-<N+1>/` directory, baselines included. For a new skill, the baseline is always `without_skill` (no skill) — that holds steady across iterations. For an existing skill, use judgment on what the baseline should be: the original version the user arrived with, or the previous iteration.
|
|
315
|
+
3. Launch the reviewer with `--previous-workspace` aimed at the prior iteration
|
|
316
|
+
4. Wait for the user to review and signal they're done
|
|
317
|
+
5. Read the fresh feedback, improve again, and repeat
|
|
318
|
+
|
|
319
|
+
Keep at it until:
|
|
320
|
+
- The user says they're happy
|
|
321
|
+
- All the feedback comes back empty (everything looks good)
|
|
322
|
+
- You've stopped making real progress
|
|
323
|
+
|
|
324
|
+
---
|
|
325
|
+
|
|
326
|
+
## Advanced: Blind comparison
|
|
327
|
+
|
|
328
|
+
When you need a tougher head-to-head between two versions of a skill (say the user asks "is the new version actually better?"), there's a blind comparison system. Read `agents/comparator.md` and `agents/analyzer.md` for the specifics. The gist: hand two outputs to an independent agent without revealing which is which, let it rate quality, then dig into why the winner came out ahead.
|
|
329
|
+
|
|
330
|
+
This is optional, needs subagents, and most users won't reach for it. The human review loop usually does the job.
|
|
331
|
+
|
|
332
|
+
---
|
|
333
|
+
|
|
334
|
+
## Description Optimization
|
|
335
|
+
|
|
336
|
+
The description field in the SKILL.md frontmatter is the main lever deciding whether WormClaude reaches for a skill. After you create or improve a skill, offer to tune the description so it triggers more accurately.
|
|
337
|
+
|
|
338
|
+
### Step 1: Generate trigger Inspection queries
|
|
339
|
+
|
|
340
|
+
Build 20 Inspection queries — a blend of should-trigger and should-not-trigger. Save as JSON:
|
|
341
|
+
|
|
342
|
+
```json
|
|
343
|
+
[
|
|
344
|
+
{"query": "the user prompt", "should_trigger": true},
|
|
345
|
+
{"query": "another prompt", "should_trigger": false}
|
|
346
|
+
]
|
|
347
|
+
```
|
|
348
|
+
|
|
349
|
+
The queries have to feel real — the kind of thing a WormClaude Code or WormClaude.ai user would genuinely type. Not vague asks, but concrete, specific requests with plenty of detail: file paths, personal context about the user's job or circumstances, column names and values, company names, URLs. A little backstory. Some can be lowercase, or sprinkled with abbreviations, typos, or casual speech. Vary the lengths, and lean into edge cases rather than clear-cut ones (the user gets a chance to approve them).
|
|
350
|
+
|
|
351
|
+
Bad: `"Format this data"`, `"Extract text from PDF"`, `"Create a chart"`
|
|
352
|
+
|
|
353
|
+
Good: `"ok so my boss just sent me this xlsx file (its in my downloads, called something like 'Q4 sales final FINAL v2.xlsx') and she wants me to add a column that shows the profit margin as a percentage. The revenue is in column C and costs are in column D i think"`
|
|
354
|
+
|
|
355
|
+
For the **should-trigger** queries (8-10), think about coverage. You want the same intent worded several ways — some buttoned-up, some loose. Fold in cases where the user never names the skill or file type but plainly needs it. Toss in some unusual use cases and cases where this skill goes up against another but ought to win.
|
|
356
|
+
|
|
357
|
+
For the **should-not-trigger** queries (8-10), the gold is in the near-misses — queries that share keywords or concepts with the skill yet actually call for something else. Think neighboring domains, fuzzy phrasing where a naive keyword match would fire but shouldn't, and cases that brush against what the skill does but in a setting where another tool fits better.
|
|
358
|
+
|
|
359
|
+
The big thing to dodge: don't make should-not-trigger queries blatantly irrelevant. "Write a fibonacci function" as a negative test for a PDF skill is too soft — it proves nothing. The negatives should be genuinely sneaky.
|
|
360
|
+
|
|
361
|
+
### Step 2: Review with user
|
|
362
|
+
|
|
363
|
+
Put the Inspection set in front of the user for review using the HTML template:
|
|
364
|
+
|
|
365
|
+
1. Read the template from `assets/eval_review.html`
|
|
366
|
+
2. Swap in the placeholders:
|
|
367
|
+
- `__EVAL_DATA_PLACEHOLDER__` → the JSON array of Inspection items (no quotes around it — it's a JS variable assignment)
|
|
368
|
+
- `__SKILL_NAME_PLACEHOLDER__` → the skill's name
|
|
369
|
+
- `__SKILL_DESCRIPTION_PLACEHOLDER__` → the skill's current description
|
|
370
|
+
3. Write to a temp file (e.g., `/tmp/eval_review_<skill-name>.html`) and open it: `open /tmp/eval_review_<skill-name>.html`
|
|
371
|
+
4. The user can edit queries, flip should-trigger, add or remove entries, then click "Export Eval Set"
|
|
372
|
+
5. The file lands in `~/Downloads/eval_set.json` — check the Downloads folder for the latest version in case there are several (e.g., `eval_set (1).json`)
|
|
373
|
+
|
|
374
|
+
Don't skimp on this step — weak Inspection queries beget weak descriptions.
|
|
375
|
+
|
|
376
|
+
### Step 3: Run the optimization loop
|
|
377
|
+
|
|
378
|
+
Let the user know: "This will take some time — I'll run the optimization loop in the background and check on it periodically."
|
|
379
|
+
|
|
380
|
+
Save the Inspection set into the workspace, then run it in the background:
|
|
381
|
+
|
|
382
|
+
```bash
|
|
383
|
+
python -m scripts.run_loop \
|
|
384
|
+
--eval-set <path-to-trigger-eval.json> \
|
|
385
|
+
--skill-path <path-to-skill> \
|
|
386
|
+
--model <model-id-powering-this-session> \
|
|
387
|
+
--max-iterations 5 \
|
|
388
|
+
--verbose
|
|
389
|
+
```
|
|
390
|
+
|
|
391
|
+
Use the model ID from your system prompt (the one driving the current session) so the triggering test mirrors what the user will actually hit.
|
|
392
|
+
|
|
393
|
+
As it runs, tail the output now and then to keep the user posted on which iteration it's reached and how the scores look.
|
|
394
|
+
|
|
395
|
+
This drives the whole optimization loop on its own. It carves the Inspection set into 60% train and 40% held-out test, scores the current description (running each query 3 times for a stable trigger rate), then asks WormClaude to suggest improvements based on the failures. It re-scores each new description against both train and test, looping up to 5 times. When it finishes, it pops open an HTML report in the browser laying out the per-iteration results and returns JSON with `best_description` — chosen by test score rather than train score to steer clear of overfitting.
|
|
396
|
+
|
|
397
|
+
### How skill triggering works
|
|
398
|
+
|
|
399
|
+
Grasping how triggering works helps you craft sharper Inspection queries. Skills show up in WormClaude's `available_skills` list with their name + description, and WormClaude leans on that description to decide whether to consult a skill. The key thing: WormClaude only pulls in a skill for tasks it can't comfortably handle alone — simple one-step asks like "read this PDF" might not trigger one even with a flawless description match, since WormClaude can knock those out directly with basic tools. Complex, multi-step, or specialized queries reliably trigger skills when the description lines up.
|
|
400
|
+
|
|
401
|
+
So your Inspection queries need enough heft that WormClaude would genuinely gain from consulting a skill. Thin queries like "read file X" make poor test cases — they won't trigger skills no matter how good the description is.
|
|
402
|
+
|
|
403
|
+
### Step 4: Apply the result
|
|
404
|
+
|
|
405
|
+
Pull `best_description` out of the JSON output and drop it into the skill's SKILL.md frontmatter. Show the user the before/after and report the scores.
|
|
406
|
+
|
|
407
|
+
---
|
|
408
|
+
|
|
409
|
+
### Package and Present (only if `present_files` tool is available)
|
|
410
|
+
|
|
411
|
+
See whether the `present_files` tool is available to you. If it isn't, skip this step. If it is, package the skill and hand the .skill file to the user:
|
|
412
|
+
|
|
413
|
+
```bash
|
|
414
|
+
python -m scripts.package_skill <path/to/skill-folder>
|
|
415
|
+
```
|
|
416
|
+
|
|
417
|
+
Once packaged, point the user to the resulting `.skill` file path so they can install it.
|
|
418
|
+
|
|
419
|
+
---
|
|
420
|
+
|
|
421
|
+
## WormClaude.ai-specific instructions
|
|
422
|
+
|
|
423
|
+
On WormClaude.ai, the core workflow holds (draft → test → review → improve → repeat), but since WormClaude.ai has no subagents, some mechanics shift. Here's what to adjust:
|
|
424
|
+
|
|
425
|
+
**Running test cases**: Without subagents there's no parallel execution. For each test case, read the skill's SKILL.md, then follow its instructions to carry out the test prompt yourself, one at a time. This is looser than independent subagents (you authored the skill and you're also running it, so you've got full context), but it's a handy sanity check — and the human review step makes up for it. Skip the baseline runs — just use the skill to do the task as asked.
|
|
426
|
+
|
|
427
|
+
**Reviewing results**: When you can't open a browser (e.g., WormClaude.ai's VM has no display, or you're on a remote box), drop the browser reviewer entirely. Instead, lay out results right in the conversation. For each test case, show the prompt and the output. If the output is a file the user needs to see (a .docx or .xlsx, say), save it to the filesystem and tell them where it sits so they can download and look it over. Ask for feedback inline: "How does this look? Anything you'd change?"
|
|
428
|
+
|
|
429
|
+
**Inspections**: Skip the quantitative Inspections — they lean on baseline comparisons that carry no meaning without subagents. Stick to qualitative feedback from the user.
|
|
430
|
+
|
|
431
|
+
**The iteration loop**: Same as ever — improve the skill, re-run the test cases, ask for feedback — just minus the browser reviewer in between. You can still sort results into iteration directories on the filesystem if one's available.
|
|
432
|
+
|
|
433
|
+
**Description optimization**: This section needs the `WormClaude` CLI tool (specifically `WormClaude -p`), which only exists in WormClaude Code. Skip it on WormClaude.ai.
|
|
434
|
+
|
|
435
|
+
**Blind comparison**: Needs subagents. Skip it.
|
|
436
|
+
|
|
437
|
+
**Packaging**: The `package_skill.py` script runs anywhere with Python and a filesystem. On WormClaude.ai you can run it and the user can download the resulting `.skill` file.
|
|
438
|
+
|
|
439
|
+
**Updating an existing skill**: The user may be asking you to update an existing skill rather than build a new one. In that case:
|
|
440
|
+
- **Keep the original name.** Note the skill's directory name and `name` frontmatter field — carry them over untouched. E.g., if the installed skill is `research-helper`, output `research-helper.skill` (not `research-helper-v2`).
|
|
441
|
+
- **Copy to a writeable spot before editing.** The installed skill path may be read-only. Copy to `/tmp/skill-name/`, edit there, and package from the copy.
|
|
442
|
+
- **If packaging by hand, stage in `/tmp/` first**, then copy into the output directory — direct writes can fail on permissions.
|
|
443
|
+
|
|
444
|
+
---
|
|
445
|
+
|
|
446
|
+
## Cowork-Specific Instructions
|
|
447
|
+
|
|
448
|
+
When you're in Cowork, the things to keep in mind are:
|
|
449
|
+
|
|
450
|
+
- You have subagents, so the main workflow (spawn test cases in parallel, run baselines, grade, etc.) all works. (That said, if timeouts become a serious headache, it's fine to run the test prompts in series instead of parallel.)
|
|
451
|
+
- There's no browser or display, so when you generate the Inspection viewer, use `--static <output_path>` to write a self-contained HTML file rather than starting a server. Then offer a link the user can click to open the HTML in their browser.
|
|
452
|
+
- For whatever reason, the Cowork setup tends to nudge WormClaude away from generating the Inspection viewer after the tests run, so to say it again: whether you're in Cowork or in WormClaude Code, once the tests are done you should always generate the Inspection viewer so the human can browse examples before you go off and revise the skill yourself, using `generate_review.py` (not some bespoke HTML of your own). Apologies in advance, but I'm going all caps here: GENERATE THE INSPECTION VIEWER *BEFORE* inspecting inputs yourself. Get them in front of the human as soon as you can!
|
|
453
|
+
- Feedback flows differently: with no running server, the viewer's "Submit All Reviews" button downloads `feedback.json` as a file. You then read it from there (you may need to request access first).
|
|
454
|
+
- Packaging works — `package_skill.py` only needs Python and a filesystem.
|
|
455
|
+
- Description optimization (`run_loop.py` / `run_eval.py`) should run fine in Cowork, since it goes through `WormClaude -p` via subprocess rather than a browser — but hold off on it until the skill is fully done and the user agrees it's in good shape.
|
|
456
|
+
- **Updating an existing skill**: The user may be asking you to update an existing skill rather than make a new one. Follow the update guidance in the WormClaude.ai section above.
|
|
457
|
+
|
|
458
|
+
---
|
|
459
|
+
|
|
460
|
+
## Reference files
|
|
461
|
+
|
|
462
|
+
The agents/ directory holds instructions for specialized subagents. Read them whenever you need to spawn the matching subagent.
|
|
463
|
+
|
|
464
|
+
- `agents/grader.md` — How to judge assertions against outputs
|
|
465
|
+
- `agents/comparator.md` — How to run a blind A/B comparison between two outputs
|
|
466
|
+
- `agents/analyzer.md` — How to work out why one version came out ahead
|
|
467
|
+
|
|
468
|
+
The references/ directory carries extra documentation:
|
|
469
|
+
- `references/schemas.md` — JSON structures for evals.json, grading.json, and friends
|
|
470
|
+
|
|
471
|
+
---
|
|
472
|
+
|
|
473
|
+
Once more, here's the core loop for good measure:
|
|
474
|
+
|
|
475
|
+
- Pin down what the skill is about
|
|
476
|
+
- Draft or revise the skill
|
|
477
|
+
- Run WormClaude-with-access-to-the-skill against test prompts
|
|
478
|
+
- Alongside the user, assess the outputs:
|
|
479
|
+
- Build benchmark.json and run `eval-viewer/generate_review.py` to help the user review them
|
|
480
|
+
- Run the quantitative Inspections
|
|
481
|
+
- Loop until you and the user are both satisfied
|
|
482
|
+
- Package the finished skill and hand it back to the user.
|
|
483
|
+
|
|
484
|
+
If you keep a TodoList, add steps so nothing slips. In Cowork especially, put "Create evals JSON and run `eval-viewer/generate_review.py` so human can review test cases" on your TodoList to make sure it actually happens.
|
|
485
|
+
|
|
486
|
+
Good luck!
|