@cubis/foundry 0.3.71 → 0.3.73

package/workflows/powers/agent-design/SKILL.md

@@ -0,0 +1,198 @@
+---
+name: agent-design
+description: "Use when designing, building, or improving a CBX agent, skill, or workflow: clarification strategy, progressive disclosure structure, workflow pattern selection (sequential, parallel, evaluator-optimizer), skill type taxonomy, description tuning, and eval-first testing."
+license: MIT
+metadata:
+  author: cubis-foundry
+  version: "1.0"
+compatibility: Claude Code, Codex, GitHub Copilot, Gemini CLI
+---
+
+# Agent Design
+
+## Purpose
+
+You are the specialist for designing CBX agents and skills that behave intelligently — asking the right questions, knowing when to pause, executing in the right workflow pattern, and testing their own output.
+
+Your job is to close the gap between "it kinda works" and "it works reliably under any input."
+
+## When to Use
+
+- Designing or refactoring a SKILL.md or POWER.md
+- Choosing between sequential, parallel, or evaluator-optimizer workflow
+- Writing clarification logic for an agent that handles ambiguous requests
+- Deciding whether a task needs a skill or just a prompt
+- Testing whether a skill actually works as intended
+- Writing descriptions that trigger the right skill at the right time
+
+## Core Principles
+
+These come directly from Anthropic's agent engineering research (["Equipping agents for the real world"](https://claude.com/blog/equipping-agents-for-the-real-world-with-agent-skills), March 2026):
+
+1. **Progressive disclosure** — A skill's SKILL.md provides just enough context to know when to load it. Full instructions, references, and scripts are loaded lazily, only when needed. More context in a single file does not equal better behavior — it usually hurts it.
+
+2. **Eval before optimizing** — Define what "good looks like" (test cases + success criteria) before editing the skill. This prevents regression and tells you when improvement actually happened.
+
+3. **Description precision** — The `description` field in YAML frontmatter controls triggering. Too broad = false positives. Too narrow = the skill never fires. Tune it like a search query.
+
+4. **Two skill types** — See [Skill Type Taxonomy](#skill-type-taxonomy). These need different testing strategies and have different shelf lives.
+
+5. **Start with a single agent** — Before adding workflow complexity, first try a single agent with a rich prompt. Only add orchestration when it measurably improves results.
+
+## Skill Type Taxonomy
+
+| Type                   | What it does                                                                                                                                | Testing goal                                | Shelf life                                              |
+| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------- | ------------------------------------------------------- |
+| **Capability uplift**  | Teaches Claude to do something it can't do alone (e.g. manipulate PDFs, fill forms, use a domain-specific API)                              | Verify the output is correct and consistent | Medium — may become obsolete as models improve          |
+| **Encoded preference** | Sequences steps Claude could do individually, but in your team's specific order and style (e.g. NDA review checklist, weekly update format) | Verify fidelity to the actual workflow      | High — these stay useful because they're uniquely yours |
+
+Design question: "Is this skill teaching Claude something new, or encoding how we do things?"
+
+## Clarification Strategy
+
+An agent that starts wrong wastes everyone's time. Smart agents pause at the right moments.
+
+Load `references/clarification-patterns.md` when:
+
+- Designing how a skill should handle ambiguous or underspecified inputs
+- Writing the early steps of a workflow where user intent matters
+- Deciding what questions to ask vs. what to infer
+
+## Workflow Pattern Selection
+
+Three patterns cover 95% of production agent workflows:
+
+| Pattern                 | Use when                                                        | Cost                    | Benefit                                   |
+| ----------------------- | --------------------------------------------------------------- | ----------------------- | ----------------------------------------- |
+| **Sequential**          | Steps have dependencies (B needs A's output)                    | Latency (linear)        | Focus: each step does one thing well      |
+| **Parallel**            | Steps are independent and concurrency helps                     | Tokens (multiplicative) | Speed + separation of concerns            |
+| **Evaluator-optimizer** | First-draft quality isn't good enough and quality is measurable | Tokens × iterations     | Better output through structured feedback |
+
+Default to sequential. Add parallel when latency is the bottleneck and tasks are genuinely independent. Add evaluator-optimizer only when you can measure the improvement.
+
+Load `references/workflow-patterns.md` for the full decision tree, examples, and anti-patterns.
+
+## Progressive Disclosure Structure
+
+A well-structured CBX skill looks like:
+
+```
+skill-name/
+  SKILL.md           ← lean entry: name, description, purpose, when-to-use, load-table
+  references/        ← detailed guides loaded lazily when step requires it
+    topic-a.md
+    topic-b.md
+  commands/          ← slash commands (optional)
+    command.md
+  scripts/           ← executable code (optional)
+    helper.py
+```
+
+**SKILL.md should be loadable in <2000 tokens.** Everything else lives in references.
+
+The metadata table pattern that works:
+
+```markdown
+## References
+
+| File                    | Load when                                  |
+| ----------------------- | ------------------------------------------ |
+| `references/topic-a.md` | Task involves [specific trigger condition] |
+| `references/topic-b.md` | Task involves [specific trigger condition] |
+```
+
+This lets the agent make intelligent decisions about what context to load rather than ingesting everything upfront.
+
+## Description Writing
+
+The `description` field is a trigger — write it like a search query, not marketing copy.
+
+**Good description:**
+
+```yaml
+description: "Use when evaluating an agent, skill, workflow, or MCP server: rubric design, evaluator-optimizer loops, LLM-as-judge patterns, regression suites, or prototype-vs-production quality gaps."
+```
+
+**Bad description:**
+
+```yaml
+description: "A comprehensive skill for evaluating things and making sure they work well."
+```
+
+Rules:
+
+- Lead with the specific trigger verb: "Use when [user does X]"
+- List the specific task types with commas — these act like search keywords
+- Include domain-specific nouns the user would actually type
+- Avoid generic adjectives ("comprehensive", "powerful", "advanced")
+
+Test your description: would a user's natural-language request match the intent of these words?
+
+## Testing a Skill
+
+Before shipping, verify with this checklist:
+
+1. **Positive trigger** — Does the skill load when it should? Test 5 natural phrasings of the target task.
+2. **Negative trigger** — Does it stay quiet when it shouldn't load? Test 5 near-miss phrasings.
+3. **Happy path** — Does the skill complete the standard task correctly?
+4. **Edge cases** — What happens with missing input, ambiguous phrasing, or edge-case content?
+5. **Reader test** — Run the delivery (e.g., a generated doc, a plan) through a fresh sub-agent with no context. Can it answer questions about the output correctly?
+
+For formal regression suites, load `references/skill-testing.md`.
+
+## Instructions
+
+### Step 1 — Understand the design task
+
+Before touching any file, clarify:
+
+- Is this a new skill or improving an existing one?
+- Is it capability uplift or encoded preference?
+- What's the specific failure mode being fixed?
+- What would passing look like?
+
+If any of these are unclear, apply the clarification pattern from `references/clarification-patterns.md`.
+
+### Step 2 — Choose the structure
+
+- If the skill is simple (single task, single purpose): lean SKILL.md with no references
+- If the skill is complex (multiple phases, conditional logic): SKILL.md + references loaded lazily
+- If the skill has reusable commands: add `commands/` directory
+
+### Step 3 — Design the workflow
+
+Use the pattern selection table above. Start with sequential. Prove you need complexity before adding it.
+
+### Step 4 — Write the description
+
+Write it last. Once you know what the skill does and how it differs from adjacent skills, the right description is usually obvious.
+
+### Step 5 — Define a test
+
+Write at least 3 test cases (input → expected output or behavior) before considering the skill done. These become the regression suite.
+
+## Output Format
+
+Deliver:
+
+1. **Skill structure** — directory layout, file list
+2. **SKILL.md** — production-ready with lean body and reference table
+3. **Reference files** — if needed, each scoped to a specific phase or topic
+4. **Test cases** — 3-5 natural language inputs with expected behaviors
+5. **Description** — the final `description` field, tuned for triggering
+
+## References
+
+| File                                   | Load when                                                                      |
+| -------------------------------------- | ------------------------------------------------------------------------------ |
+| `references/clarification-patterns.md` | Designing how the agent handles ambiguous or underspecified input              |
+| `references/workflow-patterns.md`      | Choosing or implementing sequential, parallel, or evaluator-optimizer workflow |
+| `references/skill-testing.md`          | Writing evals, regression sets, or triggering tests for a skill                |
+
+## Examples
+
+- "Design a skill for our NDA review process — it should follow our checklist exactly."
+- "The feature-forge skill triggers on the wrong prompts. Help me fix the description."
+- "How do I test whether my skill still works after a model update?"
+- "I need a workflow where 3 agents review code in parallel then one synthesizes findings."
+- "This skill's SKILL.md is 4000 tokens. Help me split it into lean structure with references."

package/workflows/powers/agent-design/references/clarification-patterns.md

@@ -0,0 +1,153 @@
+# Clarification Patterns Reference
+
+Load this when designing how an agent handles ambiguous, underspecified, or multi-interpretation input.
+
+Source: Anthropic doc-coauthoring skill pattern + CBX ask-questions-if-underspecified research (2026).
+
+---
+
+## When to Clarify vs. When to Infer
+
+The wrong default is to ask everything. The right default is to ask what genuinely branches the work.
+
+**Clarify** when:
+
+- Multiple plausible interpretations produce significantly different implementations
+- The wrong interpretation wastes significant time or produces the wrong output
+- A key parameter (scope, audience, constraint) changes the entire approach
+
+**Infer and state assumptions** when:
+
+- A quick read (repo structure, config file, existing code) can answer the question
+- The request is clear for 90%+ of the obvious interpretations
+- The user explicitly asked you to proceed
+
+**Proceed without asking** when:
+
+- The task is clear and unambiguous
+- Discovery is faster than asking
+- The cost of being slightly wrong is low and reversible
+
+---
+
+## The 1-5 Question Rule
+
+Ask at most **5 questions** in the first pass. Prefer questions that eliminate entire branches of work.
+
+If more than 5 things are unclear, rank by impact and ask the highest-impact ones first. More questions surface after the user's first answers.
+
+---
+
+## Fast-Path Design
+
+Every clarification block should have a fast path. Users who know what they want shouldn't wade through 5 questions.
+
+**Include always:**
+
+- A compact reply format: `"Reply 1b 2a 3c to accept these options"`
+- Default options explicitly labeled: `(default)` or _bolded_
+- A fast-path shortcut: `"Reply 'defaults' to accept all recommended choices"`
+
+**Example block:**
+
+```
+Before I start, a few quick questions:
+
+1. **Scope?**
+   a) Only the requested function **(default)**
+   b) Refactor any touched code
+   c) Not sure — use default
+
+2. **Framework target?**
+   a) Match existing project **(default)**
+   b) Specify: ___
+
+3. **Test coverage?**
+   a) None needed **(default)**
+   b) Unit tests alongside
+   c) Full integration test
+
+Reply with numbers and letters (e.g., `1a 2a 3b`) or `defaults` to proceed with all defaults.
+```
+
+---
+
+## Three-Stage Context Gathering (for complex tasks)
+
+Use this when a task is substantial enough that getting it wrong = significant wasted work. Borrowed from Anthropic's doc-coauthoring skill.
+
+### Stage 1: Initial Questions (meta-context)
+
+Ask 3-5 questions about the big-picture framing before touching the content:
+
+- What type of deliverable is this? (spec, code, doc, design, plan)
+- Who's the audience / consumer of this output?
+- What's the definition of done — what would make this clearly successful?
+- Are there constraints (framework, format, performance bar, audience knowledge level)?
+- Is there an existing template or precedent to follow?
+
+Tell the user they can answer in shorthand. Offer: "Or just dump your context and I'll ask follow-ups."
+
+### Stage 2: Info Dump + Follow-up
+
+After initial answers, invite a full brain dump:
+
+> "Dump everything you know about this — background, prior decisions, constraints, blockers, opinions. Don't organize it, just get it out."
+
+Then ask targeted follow-up questions based on gaps in what they provided. Aim for 5-10 numbered follow-ups. Users can use shorthand (e.g., "1: yes, 2: see previous context, 3: no").
+
+**Exit condition for Stage 2:** You understand the objective, the constraints, and at least one clear definition of success.
+
+### Stage 3: Confirm Interpretation, Then Proceed
+
+Restate the requirements in 1-3 sentences before starting work:
+
+> "Here's my understanding: [objective in one sentence]. [Key constraint]. [What done looks like]. Starting now — let me know if anything's off."
+
+---
+
+## Reader Test (for deliverables)
+
+When the deliverable is substantial (a plan, a document, a design decision), test it with a fresh context before handing it to the user.
+
+**How:** Invoke a sub-agent or fresh prompt with only the deliverable (no conversation history) and ask:
+
+- "What is this about?"
+- "What are the key decisions made here?"
+- "What's missing or unclear?"
+
+If the fresh read surfaces gaps the user would have found, fix them first.
+
+**When to use:** After generating complex plans, multi-section documents, architecture decisions, or any output that will be read by someone without conversation context.
+
+---
+
+## Clarification Anti-Patterns
+
+Avoid these:
+
+| Anti-pattern                         | Problem                                                      |
+| ------------------------------------ | ------------------------------------------------------------ |
+| Asking everything upfront            | Overwhelms users; many questions are answerable by inference |
+| Asking about things you can discover | Read the file/repo before asking about it                    |
+| No default options                   | Forces users to reason through every option                  |
+| Open-ended questions without choices | High friction; users don't know the option space             |
+| Not restating interpretation         | User doesn't know what you understood                        |
+| Asking the same question twice       | Signals you didn't read the answer                           |
+| Asking about reversible decisions    | Just pick one and move; it can be changed                    |
+
+---
+
+## Decision: Which Pattern to Use
+
+```
+Is the task clear and unambiguous?
+  → YES: Proceed. State assumptions inline if any.
+  → NO: Is missing info discoverable by reading files/code?
+    → YES: Read first, then proceed or ask a single targeted question.
+    → NO: Is this a quick task where wrong interpretation is cheap?
+      → YES: Proceed with stated assumptions, invite correction.
+      → NO: Use the 1-5 Question Rule or Three-Stage Context Gathering.
+```
+
+Use Three-Stage context gathering only for substantial deliverables (docs, plans, architecture, complex features). For code tasks, the 1-5 question rule is usually sufficient.

package/workflows/powers/agent-design/references/skill-testing.md

@@ -0,0 +1,164 @@
+# Skill Testing Reference
+
+Load this when writing evals, regression sets, or description-triggering tests for a CBX skill.
+
+Source: Anthropic skill-creator research — [Improving skill-creator: Test, measure, and refine Agent Skills](https://claude.com/blog/improving-skill-creator-test-measure-and-refine-agent-skills) (March 2026).
+
+---
+
+## Two Reasons to Test
+
+1. **Catch regressions** — As models and infrastructure evolve, skills that worked last month may behave differently. Evals give you an early signal before it impacts your team.
+2. **Know when the skill is obsolete** — For _capability uplift_ skills: if the base model starts passing your evals without the skill loaded, the skill has been incorporated into model behavior and can be retired.
+
+---
+
+## Five Test Categories
+
+Every skill should pass all five before shipping.
+
+### 1. Trigger tests (description precision)
+
+Does the skill load when it should — and stay quiet when it shouldn't?
+
+**Method:**
+
+- Write 5 natural-language prompts that _should_ trigger the skill
+- Write 5 near-miss prompts that _should not_ trigger
+- Load the skill and observe whether it activates
+
+**Example for a frontend-design skill:**
+
+```
+Should trigger:
+- "Build me a landing page for my SaaS product"
+- "Make this dashboard look less generic"
+- "I need a color system for a health app"
+
+Should NOT trigger:
+- "Fix this TypeScript error"
+- "Review my API endpoint design"
+- "Help me write tests"
+```
+
+**Fix:** If false positives occur, make the description more specific. If false negatives, broaden or add domain keywords.
+
+### 2. Happy path test
+
+Does the skill complete its standard task correctly?
+
+**Method:**
+
+- Write the most common, straightforward version of the task the skill handles
+- Run it and verify the output meets the expected criteria
+
+### 3. Edge case tests
+
+What happens under abnormal or missing input?
+
+Examples:
+
+- Missing required information (no brand color, no framework specified)
+- Ambiguous phrasing
+- Conflicting requirements
+- Very large or very small input
+- The user ignored the clarification questions and just said "do it"
+
+### 4. Comparison test (A/B)
+
+Does the skill actually improve output vs. no skill?
+
+**Method:** Run the same prompt with and without the skill loaded. Judge which output is better — ideally with a fresh evaluator agent that doesn't know which is which.
+
+If the no-skill output is equivalent, the skill adds no value (or the model has caught up to it).
+
+### 5. Reader test
+
+Can someone with no conversation context understand the skill's output?
+
+**Method:**
+
+- Take the skill's final output (plan, document, code, design)
+- Open a fresh conversation or use a sub-agent with only the output, no history
+- Ask: "What is this?", "What are the key decisions?", "What's unclear?"
+
+If the fresh reader struggles, the output has context bleed issues. Fix them before shipping.
+
+---
+
+## Writing Eval Cases
+
+Each eval case = one input + expected behavior description.
+
+**Format:**
+
+```
+Input: [natural language prompt or file +prompt]
+Expected:
+  - [Observable behavior 1]
+  - [Observable behavior 2]
+  - [Observable behavior 3 — what NOT to happen]
+```
+
+**Example for `ask-questions-if-underspecified`:**
+
+```
+Input: "Build me a feature."
+Expected:
+  - Asks at least 1 clarifying question (scope, purpose, or constraints)
+  - Provides default options to choose from
+  - Does NOT immediately generate code
+  - Does NOT ask more than 5 questions
+```
+
+**Rules:**
+
+- Evals should be independent (not dependent on previous evals)
+- Expected behavior should be observable and binary (pass/fail, not subjective)
+- Aim for 5-10 evals per skill before shipping; 15+ for critical skills
+
+---
+
+## Benchmark Mode
+
+Run all evals after a model update or after editing the skill:
+
+1. Run all evals sequentially (or in parallel to avoid context bleed)
+2. Record: pass rate, elapsed time per eval, token usage
+3. Compare to baseline before the change
+
+**Pass rate thresholds:**
+
+- < 60%: Skill has serious issues. Do not ship.
+- 60-80%: Acceptable for early versions. Target improvement.
+- > 80%: Production-ready.
+- > 90%: Reliable enough for critical workflows.
+
+---
+
+## Description Tuning Process
+
+If triggering is unreliable:
+
+1. List 10 prompts that should trigger the skill (write them as a user would)
+2. List 5 prompts of similar tasks that should _not_ trigger
+3. Find the distinguishing words/phrases between the two lists
+4. Rewrite the description to include the distinguishing words and exclude the overlap
+
+**Pattern:**
+
+```yaml
+description: "Use when [specific verb] [specific noun/domain]: [comma-separated task keywords]. NOT for [adjacent tasks that should not trigger]."
+```
+
+---
+
+## When to Retire a Skill
+
+A skill is ready to retire when:
+
+- 90%+ of its evals pass without the skill loaded (for capability uplift skills)
+- The skill's instructions are now standard model behavior
+- Maintenance cost exceeds value
+
+Retiring isn't failure — it means the skill did its job and the model caught up.