@cubis/foundry 0.3.70 → 0.3.72

package/workflows/powers/webapp-testing/SKILL.md

@@ -1,187 +1,66 @@
 ---
 name: webapp-testing
-description: Web application testing principles. E2E, Playwright, deep audit strategies.
-allowed-tools: Read, Write, Edit, Glob, Grep, Bash
+description: "Use when choosing test depth for a web product, balancing unit, integration, browser, accessibility, and contract checks, or reviewing whether a change has the right verification shape before release."
+license: MIT
+metadata:
+  author: cubis-foundry
+  version: "1.0"
+compatibility: Claude Code, Codex, GitHub Copilot
 ---
 
-# Web App Testing
+# Webapp Testing
 
-> Discover and test everything. Leave no route untested.
+## Purpose
 
-## 🔧 Runtime Scripts
+Use when choosing test depth for a web product, balancing unit, integration, browser, accessibility, and contract checks, or reviewing whether a change has the right verification shape before release.
 
-**Execute these for automated browser testing:**
+## When to Use
 
-| Script | Purpose | Usage |
-|--------|---------|-------|
-| `scripts/playwright_runner.py` | Basic browser test | `python scripts/playwright_runner.py https://example.com` |
-| | With screenshot | `python scripts/playwright_runner.py <url> --screenshot` |
-| | Accessibility check | `python scripts/playwright_runner.py <url> --a11y` |
+- Planning verification depth for a web feature, refactor, bug fix, or release candidate.
+- Choosing what belongs in unit, component, integration, contract, or browser coverage.
+- Reviewing gaps in coverage, flaky suites, or low-signal tests.
+- Auditing whether a frontend or API change has enough evidence to merge safely.
 
-**Requires:** `pip install playwright && playwright install chromium`
+## Instructions
 
----
-
-## 1. Deep Audit Approach
-
-### Discovery First
-
-| Target | How to Find |
-|--------|-------------|
-| Routes | Scan app/, pages/, router files |
-| API endpoints | Grep for HTTP methods |
-| Components | Find component directories |
-| Features | Read documentation |
-
-### Systematic Testing
-
-1. **Map** - List all routes/APIs
-2. **Scan** - Verify they respond
-3. **Test** - Cover critical paths
-
----
-
-## 2. Testing Pyramid for Web
-
-```
-        /\          E2E (Few)
-       /  \         Critical user flows
-      /----\
-     /      \       Integration (Some)
-    /--------\      API, data flow
-   /          \
-  /------------\    Component (Many)
-                    Individual UI pieces
-```
-
----
-
-## 3. E2E Test Principles
-
-### What to Test
-
-| Priority | Tests |
-|----------|-------|
-| 1 | Happy path user flows |
-| 2 | Authentication flows |
-| 3 | Critical business actions |
-| 4 | Error handling |
-
-### E2E Best Practices
-
-| Practice | Why |
-|----------|-----|
-| Use data-testid | Stable selectors |
-| Wait for elements | Avoid flaky tests |
-| Clean state | Independent tests |
-| Avoid implementation details | Test user behavior |
-
----
-
-## 4. Playwright Principles
-
-### Core Concepts
-
-| Concept | Use |
-|---------|-----|
-| Page Object Model | Encapsulate page logic |
-| Fixtures | Reusable test setup |
-| Assertions | Built-in auto-wait |
-| Trace Viewer | Debug failures |
+1. Map the change surface to business risk, user-visible impact, and regression blast radius.
+2. Put the cheapest reliable check at the lowest layer that can prove the behavior.
+3. Add browser coverage only for flows that need cross-layer confidence.
+4. Pair coverage with deterministic fixtures, contract clarity, and failure evidence.
+5. Call out what remains manual, unverified, or risky instead of hiding gaps behind suite size.
 
-### Configuration
+### Baseline standards
 
-| Setting | Recommendation |
-|---------|----------------|
-| Retries | 2 on CI |
-| Trace | on-first-retry |
-| Screenshots | on-failure |
-| Video | retain-on-failure |
+- Test behavior users or dependent systems care about, not implementation trivia.
+- Keep component and integration checks faster and more numerous than browser tests.
+- Include accessibility and error-state coverage in critical paths, not only happy paths.
+- Prefer contract or integration checks for API correctness before adding UI duplication.
+- Treat flaky tests as defects in the product, test, or environment, not a permanent condition.
 
----
-
-## 5. Visual Testing
-
-### When to Use
-
-| Scenario | Value |
-|----------|-------|
-| Design system | High |
-| Marketing pages | High |
-| Component library | Medium |
-| Dynamic content | Lower |
-
-### Strategy
-
-- Baseline screenshots
-- Compare on changes
-- Review visual diffs
-- Update intentional changes
-
----
-
-## 6. API Testing Principles
-
-### Coverage Areas
-
-| Area | Tests |
-|------|-------|
-| Status codes | 200, 400, 404, 500 |
-| Response shape | Matches schema |
-| Error messages | User-friendly |
-| Edge cases | Empty, large, special chars |
+### Constraints
 
----
-
-## 7. Test Organization
-
-### File Structure
+- Avoid one-size-fits-all coverage targets with no risk model.
+- Avoid using browser tests to compensate for missing contract or unit design.
+- Avoid counting snapshots or shallow assertions as evidence of behavior correctness.
+- Avoid treating manual QA as a substitute for repeatable regression protection on critical paths.
 
-```
-tests/
-├── e2e/           # Full user flows
-├── integration/   # API, data
-├── component/     # UI units
-└── fixtures/      # Shared data
-```
+## Output Format
 
-### Naming Convention
-
-| Pattern | Example |
-|---------|---------|
-| Feature-based | `login.spec.ts` |
-| Descriptive | `user-can-checkout.spec.ts` |
-
----
+Provide implementation guidance, code examples, and configuration as appropriate to the task.
 
-## 8. CI Integration
+## References
 
-### Pipeline Steps
+Load on demand. Do not preload all reference files.
 
-1. Install dependencies
-2. Install browsers
-3. Run tests
-4. Upload artifacts (traces, screenshots)
+| File | Load when |
+| --- | --- |
+| `references/browser-api-state-checklist.md` | You need a deeper playbook for test-layer selection, accessibility coverage, API-vs-UI duplication, flaky-suite triage, or release gating for web apps. |
 
-### Parallelization
+## Scripts
 
-| Strategy | Use |
-|----------|-----|
-| Per file | Playwright default |
-| Sharding | Large suites |
-| Workers | Multiple browsers |
+No helper scripts are required for this skill right now. Keep execution in `SKILL.md` and `references/` unless repeated automation becomes necessary.
 
----
-
-## 9. Anti-Patterns
-
-| ❌ Don't | ✅ Do |
-|----------|-------|
-| Test implementation | Test behavior |
-| Hardcode waits | Use auto-wait |
-| Skip cleanup | Isolate tests |
-| Ignore flaky tests | Fix root cause |
-
----
+## Examples
 
-> **Remember:** E2E tests are expensive. Use them for critical paths only.
+- "Help me with webapp testing best practices in this project"
+- "Review my webapp testing implementation for issues"

package/workflows/skills/agent-design/SKILL.md

@@ -0,0 +1,198 @@
+---
+name: agent-design
+description: "Use when designing, building, or improving a CBX agent, skill, or workflow: clarification strategy, progressive disclosure structure, workflow pattern selection (sequential, parallel, evaluator-optimizer), skill type taxonomy, description tuning, and eval-first testing."
+license: MIT
+metadata:
+  author: cubis-foundry
+  version: "1.0"
+compatibility: Claude Code, Codex, GitHub Copilot, Gemini CLI
+---
+
+# Agent Design
+
+## Purpose
+
+You are the specialist for designing CBX agents and skills that behave intelligently — asking the right questions, knowing when to pause, executing in the right workflow pattern, and testing their own output.
+
+Your job is to close the gap between "it kinda works" and "it works reliably under any input."
+
+## When to Use
+
+- Designing or refactoring a SKILL.md or POWER.md
+- Choosing between sequential, parallel, or evaluator-optimizer workflow
+- Writing clarification logic for an agent that handles ambiguous requests
+- Deciding whether a task needs a skill or just a prompt
+- Testing whether a skill actually works as intended
+- Writing descriptions that trigger the right skill at the right time
+
+## Core Principles
+
+These come directly from Anthropic's agent engineering research (["Equipping agents for the real world"](https://claude.com/blog/equipping-agents-for-the-real-world-with-agent-skills), March 2026):
+
+1. **Progressive disclosure** — A skill's SKILL.md provides just enough context to know when to load it. Full instructions, references, and scripts are loaded lazily, only when needed. More context in a single file does not equal better behavior — it usually hurts it.
+
+2. **Eval before optimizing** — Define what "good looks like" (test cases + success criteria) before editing the skill. This prevents regression and tells you when improvement actually happened.
+
+3. **Description precision** — The `description` field in YAML frontmatter controls triggering. Too broad = false positives. Too narrow = the skill never fires. Tune it like a search query.
+
+4. **Two skill types** — See [Skill Type Taxonomy](#skill-type-taxonomy). These need different testing strategies and have different shelf lives.
+
+5. **Start with a single agent** — Before adding workflow complexity, first try a single agent with a rich prompt. Only add orchestration when it measurably improves results.
+
+## Skill Type Taxonomy
+
+| Type                   | What it does                                                                                                                                | Testing goal                                | Shelf life                                              |
+| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------- | ------------------------------------------------------- |
+| **Capability uplift**  | Teaches Claude to do something it can't do alone (e.g. manipulate PDFs, fill forms, use a domain-specific API)                              | Verify the output is correct and consistent | Medium — may become obsolete as models improve          |
+| **Encoded preference** | Sequences steps Claude could do individually, but in your team's specific order and style (e.g. NDA review checklist, weekly update format) | Verify fidelity to the actual workflow      | High — these stay useful because they're uniquely yours |
+
+Design question: "Is this skill teaching Claude something new, or encoding how we do things?"
+
+## Clarification Strategy
+
+An agent that starts wrong wastes everyone's time. Smart agents pause at the right moments.
+
+Load `references/clarification-patterns.md` when:
+
+- Designing how a skill should handle ambiguous or underspecified inputs
+- Writing the early steps of a workflow where user intent matters
+- Deciding what questions to ask vs. what to infer
+
+## Workflow Pattern Selection
+
+Three patterns cover 95% of production agent workflows:
+
+| Pattern                 | Use when                                                        | Cost                    | Benefit                                   |
+| ----------------------- | --------------------------------------------------------------- | ----------------------- | ----------------------------------------- |
+| **Sequential**          | Steps have dependencies (B needs A's output)                    | Latency (linear)        | Focus: each step does one thing well      |
+| **Parallel**            | Steps are independent and concurrency helps                     | Tokens (multiplicative) | Speed + separation of concerns            |
+| **Evaluator-optimizer** | First-draft quality isn't good enough and quality is measurable | Tokens × iterations     | Better output through structured feedback |
+
+Default to sequential. Add parallel when latency is the bottleneck and tasks are genuinely independent. Add evaluator-optimizer only when you can measure the improvement.
+
+Load `references/workflow-patterns.md` for the full decision tree, examples, and anti-patterns.
+
+## Progressive Disclosure Structure
+
+A well-structured CBX skill looks like:
+
+```
+skill-name/
+  SKILL.md           ← lean entry: name, description, purpose, when-to-use, load-table
+  references/        ← detailed guides loaded lazily when step requires it
+    topic-a.md
+    topic-b.md
+  commands/          ← slash commands (optional)
+    command.md
+  scripts/           ← executable code (optional)
+    helper.py
+```
+
+**SKILL.md should be loadable in <2000 tokens.** Everything else lives in references.
+
+The metadata table pattern that works:
+
+```markdown
+## References
+
+| File                    | Load when                                  |
+| ----------------------- | ------------------------------------------ |
+| `references/topic-a.md` | Task involves [specific trigger condition] |
+| `references/topic-b.md` | Task involves [specific trigger condition] |
+```
+
+This lets the agent make intelligent decisions about what context to load rather than ingesting everything upfront.
+
+## Description Writing
+
+The `description` field is a trigger — write it like a search query, not marketing copy.
+
+**Good description:**
+
+```yaml
+description: "Use when evaluating an agent, skill, workflow, or MCP server: rubric design, evaluator-optimizer loops, LLM-as-judge patterns, regression suites, or prototype-vs-production quality gaps."
+```
+
+**Bad description:**
+
+```yaml
+description: "A comprehensive skill for evaluating things and making sure they work well."
+```
+
+Rules:
+
+- Lead with the specific trigger verb: "Use when [user does X]"
+- List the specific task types with commas — these act like search keywords
+- Include domain-specific nouns the user would actually type
+- Avoid generic adjectives ("comprehensive", "powerful", "advanced")
+
+Test your description: would a user's natural-language request match the intent of these words?
+
+## Testing a Skill
+
+Before shipping, verify with this checklist:
+
+1. **Positive trigger** — Does the skill load when it should? Test 5 natural phrasings of the target task.
+2. **Negative trigger** — Does it stay quiet when it shouldn't load? Test 5 near-miss phrasings.
+3. **Happy path** — Does the skill complete the standard task correctly?
+4. **Edge cases** — What happens with missing input, ambiguous phrasing, or edge-case content?
+5. **Reader test** — Run the delivery (e.g., a generated doc, a plan) through a fresh sub-agent with no context. Can it answer questions about the output correctly?
+
+For formal regression suites, load `references/skill-testing.md`.
+
+## Instructions
+
+### Step 1 — Understand the design task
+
+Before touching any file, clarify:
+
+- Is this a new skill or improving an existing one?
+- Is it capability uplift or encoded preference?
+- What's the specific failure mode being fixed?
+- What would passing look like?
+
+If any of these are unclear, apply the clarification pattern from `references/clarification-patterns.md`.
+
+### Step 2 — Choose the structure
+
+- If the skill is simple (single task, single purpose): lean SKILL.md with no references
+- If the skill is complex (multiple phases, conditional logic): SKILL.md + references loaded lazily
+- If the skill has reusable commands: add `commands/` directory
+
+### Step 3 — Design the workflow
+
+Use the pattern selection table above. Start with sequential. Prove you need complexity before adding it.
+
+### Step 4 — Write the description
+
+Write it last. Once you know what the skill does and how it differs from adjacent skills, the right description is usually obvious.
+
+### Step 5 — Define a test
+
+Write at least 3 test cases (input → expected output or behavior) before considering the skill done. These become the regression suite.
+
+## Output Format
+
+Deliver:
+
+1. **Skill structure** — directory layout, file list
+2. **SKILL.md** — production-ready with lean body and reference table
+3. **Reference files** — if needed, each scoped to a specific phase or topic
+4. **Test cases** — 3-5 natural language inputs with expected behaviors
+5. **Description** — the final `description` field, tuned for triggering
+
+## References
+
+| File                                   | Load when                                                                      |
+| -------------------------------------- | ------------------------------------------------------------------------------ |
+| `references/clarification-patterns.md` | Designing how the agent handles ambiguous or underspecified input              |
+| `references/workflow-patterns.md`      | Choosing or implementing sequential, parallel, or evaluator-optimizer workflow |
+| `references/skill-testing.md`          | Writing evals, regression sets, or triggering tests for a skill                |
+
+## Examples
+
+- "Design a skill for our NDA review process — it should follow our checklist exactly."
+- "The feature-forge skill triggers on the wrong prompts. Help me fix the description."
+- "How do I test whether my skill still works after a model update?"
+- "I need a workflow where 3 agents review code in parallel then one synthesizes findings."
+- "This skill's SKILL.md is 4000 tokens. Help me split it into lean structure with references."

package/workflows/skills/agent-design/references/clarification-patterns.md

@@ -0,0 +1,153 @@
+# Clarification Patterns Reference
+
+Load this when designing how an agent handles ambiguous, underspecified, or multi-interpretation input.
+
+Source: Anthropic doc-coauthoring skill pattern + CBX ask-questions-if-underspecified research (2026).
+
+---
+
+## When to Clarify vs. When to Infer
+
+The wrong default is to ask everything. The right default is to ask what genuinely branches the work.
+
+**Clarify** when:
+
+- Multiple plausible interpretations produce significantly different implementations
+- The wrong interpretation wastes significant time or produces the wrong output
+- A key parameter (scope, audience, constraint) changes the entire approach
+
+**Infer and state assumptions** when:
+
+- A quick read (repo structure, config file, existing code) can answer the question
+- The request is clear for 90%+ of the obvious interpretations
+- The user explicitly asked you to proceed
+
+**Proceed without asking** when:
+
+- The task is clear and unambiguous
+- Discovery is faster than asking
+- The cost of being slightly wrong is low and reversible
+
+---
+
+## The 1-5 Question Rule
+
+Ask at most **5 questions** in the first pass. Prefer questions that eliminate entire branches of work.
+
+If more than 5 things are unclear, rank by impact and ask the highest-impact ones first. More questions surface after the user's first answers.
+
+---
+
+## Fast-Path Design
+
+Every clarification block should have a fast path. Users who know what they want shouldn't wade through 5 questions.
+
+**Include always:**
+
+- A compact reply format: `"Reply 1b 2a 3c to accept these options"`
+- Default options explicitly labeled: `(default)` or _bolded_
+- A fast-path shortcut: `"Reply 'defaults' to accept all recommended choices"`
+
+**Example block:**
+
+```
+Before I start, a few quick questions:
+
+1. **Scope?**
+   a) Only the requested function **(default)**
+   b) Refactor any touched code
+   c) Not sure — use default
+
+2. **Framework target?**
+   a) Match existing project **(default)**
+   b) Specify: ___
+
+3. **Test coverage?**
+   a) None needed **(default)**
+   b) Unit tests alongside
+   c) Full integration test
+
+Reply with numbers and letters (e.g., `1a 2a 3b`) or `defaults` to proceed with all defaults.
+```
+
+---
+
+## Three-Stage Context Gathering (for complex tasks)
+
+Use this when a task is substantial enough that getting it wrong = significant wasted work. Borrowed from Anthropic's doc-coauthoring skill.
+
+### Stage 1: Initial Questions (meta-context)
+
+Ask 3-5 questions about the big-picture framing before touching the content:
+
+- What type of deliverable is this? (spec, code, doc, design, plan)
+- Who's the audience / consumer of this output?
+- What's the definition of done — what would make this clearly successful?
+- Are there constraints (framework, format, performance bar, audience knowledge level)?
+- Is there an existing template or precedent to follow?
+
+Tell the user they can answer in shorthand. Offer: "Or just dump your context and I'll ask follow-ups."
+
+### Stage 2: Info Dump + Follow-up
+
+After initial answers, invite a full brain dump:
+
+> "Dump everything you know about this — background, prior decisions, constraints, blockers, opinions. Don't organize it, just get it out."
+
+Then ask targeted follow-up questions based on gaps in what they provided. Aim for 5-10 numbered follow-ups. Users can use shorthand (e.g., "1: yes, 2: see previous context, 3: no").
+
+**Exit condition for Stage 2:** You understand the objective, the constraints, and at least one clear definition of success.
+
+### Stage 3: Confirm Interpretation, Then Proceed
+
+Restate the requirements in 1-3 sentences before starting work:
+
+> "Here's my understanding: [objective in one sentence]. [Key constraint]. [What done looks like]. Starting now — let me know if anything's off."
+
+---
+
+## Reader Test (for deliverables)
+
+When the deliverable is substantial (a plan, a document, a design decision), test it with a fresh context before handing it to the user.
+
+**How:** Invoke a sub-agent or fresh prompt with only the deliverable (no conversation history) and ask:
+
+- "What is this about?"
+- "What are the key decisions made here?"
+- "What's missing or unclear?"
+
+If the fresh read surfaces gaps the user would have found, fix them first.
+
+**When to use:** After generating complex plans, multi-section documents, architecture decisions, or any output that will be read by someone without conversation context.
+
+---
+
+## Clarification Anti-Patterns
+
+Avoid these:
+
+| Anti-pattern                         | Problem                                                      |
+| ------------------------------------ | ------------------------------------------------------------ |
+| Asking everything upfront            | Overwhelms users; many questions are answerable by inference |
+| Asking about things you can discover | Read the file/repo before asking about it                    |
+| No default options                   | Forces users to reason through every option                  |
+| Open-ended questions without choices | High friction; users don't know the option space             |
+| Not restating interpretation         | User doesn't know what you understood                        |
+| Asking the same question twice       | Signals you didn't read the answer                           |
+| Asking about reversible decisions    | Just pick one and move; it can be changed                    |
+
+---
+
+## Decision: Which Pattern to Use
+
+```
+Is the task clear and unambiguous?
+  → YES: Proceed. State assumptions inline if any.
+  → NO: Is missing info discoverable by reading files/code?
+    → YES: Read first, then proceed or ask a single targeted question.
+    → NO: Is this a quick task where wrong interpretation is cheap?
+      → YES: Proceed with stated assumptions, invite correction.
+      → NO: Use the 1-5 Question Rule or Three-Stage Context Gathering.
+```
+
+Use Three-Stage context gathering only for substantial deliverables (docs, plans, architecture, complex features). For code tasks, the 1-5 question rule is usually sufficient.