aw-ecc 1.4.31 → 1.4.47

package/skills/aw-adk/references/artifact-wiring.md

@@ -0,0 +1,162 @@
+# Artifact Wiring
+
+How CASRE artifacts (Commands, Agents, Skills, Rules, Evals) reference each other.
+
+## Relationship Graph
+
+```
+Commands
+  │
+  ├──references──► Agents (agent roster table with phase assignments)
+  │                  │
+  │                  ├──references──► Skills (skills: frontmatter field)
+  │                  │                  │
+  │                  │                  └──contains──► References (references/ subdirectory)
+  │                  │
+  │                  └──tested-by──► Evals (target: frontmatter)
+  │
+  ├──tested-by──► Evals (target: frontmatter)
+  │
+  └──governed-by──► Rules
+
+Rules
+  │
+  ├──links-to──► Skills (skill link dimension)
+  │
+  └──tested-by──► Evals (target: frontmatter)
+
+Skills
+  │
+  └──tested-by──► Evals (target: frontmatter)
+
+Evals
+  │
+  └──tested-by──► Evals (meta-evals, target: frontmatter)
+```
+
+## Wiring Patterns
+
+### Commands to Agents
+
+Commands define which agents participate and in which phase via an **agent roster table** in the command body.
+
+```markdown
+## Agent Roster
+
+| Phase | Agent | Role |
+|-------|-------|------|
+| 1 - Research | planner | Create implementation plan |
+| 2 - Build | tdd-guide | Drive test-first development |
+| 3 - Review | code-reviewer | Review changes |
+| 3 - Review | security-reviewer | Security audit |
+```
+
+**Validation rules:**
+- Every agent referenced in the roster must have a corresponding `agents/<slug>.md` file.
+- Phase numbers must be sequential starting from 1.
+- Each phase should have at least one agent assigned.
+
+### Agents to Skills
+
+Agents declare their skill dependencies in the `skills:` frontmatter field.
+
+```yaml
+---
+name: planner
+type: agent
+skills:
+  - aw-adk
+  - incremental-implementation
+---
+```
+
+**Validation rules:**
+- Every slug in `skills:` must resolve to a `skills/<slug>/SKILL.md` file.
+- Skills are loaded in declaration order; first skill's instructions take precedence on conflict.
+- An agent without any skills is valid but should be flagged as a warning.
+
+### Evals to Parent Artifact
+
+Evals declare their target via the `target:` frontmatter field, using `<type>/<slug>` format.
+
+```yaml
+---
+target: skill/aw-adk
+type: eval
+---
+```
+
+**Validation rules:**
+- The `target:` value must resolve to an existing artifact.
+- Valid target prefixes: `skill/`, `agent/`, `command/`, `rule/`.
+- Meta-evals use `eval/` as the target prefix.
+- Each artifact should have at least 2 evals targeting it.
+
+### Rules to Skills
+
+Rules reference related skills via a **skill link dimension** -- a markdown link or frontmatter field pointing to the skill that provides implementation guidance for the rule.
+
+```markdown
+## References
+
+- Implement using [aw-adk](../../skills/aw-adk/SKILL.md) skill patterns
+```
+
+**Validation rules:**
+- Skill links should resolve to existing skill files.
+- Rules without skill links are valid (not all rules map to a skill).
+
+### Skills to References
+
+Skills contain a `references/` subdirectory with supporting markdown files linked from the skill body.
+
+```
+skills/aw-adk/
+  SKILL.md
+  references/
+    schemas.md
+    rubric-meta-eval.md
+    eval-placement-guide.md
+```
+
+**Validation rules:**
+- Every file in `references/` should be linked from `SKILL.md` or from another reference file.
+- Orphaned reference files (not linked from anywhere) should be flagged as warnings.
+- Reference files must be markdown (`.md`).
+
+## Cross-Artifact Dependency Patterns
+
+### Upward Dependencies (child references parent)
+
+- Evals reference their parent artifact via `target:`
+- This is the primary traceability mechanism.
+
+### Downward Dependencies (parent references child)
+
+- Commands reference agents via roster tables.
+- Agents reference skills via `skills:` frontmatter.
+- Skills reference documents via `references/` links.
+
+### Lateral Dependencies (peer references)
+
+- Rules reference skills for implementation guidance.
+- Skills may reference other skills' reference documents.
+
+## Validation Summary
+
+| Relationship | Source Field | Target Resolution | Required? |
+|---|---|---|---|
+| Command -> Agent | Agent roster table | `agents/<slug>.md` | Yes |
+| Agent -> Skill | `skills:` frontmatter | `skills/<slug>/SKILL.md` | No (warn if empty) |
+| Eval -> Parent | `target:` frontmatter | `<type>/<slug>` path | Yes |
+| Rule -> Skill | Markdown link | `skills/<slug>/SKILL.md` | No |
+| Skill -> Reference | Markdown link | `references/<file>.md` | No (warn if orphaned) |
+
+## Integrity Checks
+
+Run these checks before merging any CASRE artifact:
+
+1. **Forward resolution:** Every reference from artifact A to artifact B resolves to an existing file.
+2. **Eval coverage:** Every skill, agent, and command has at least 2 evals with matching `target:` values.
+3. **No orphans:** Reference files are linked from at least one parent. Evals have valid targets.
+4. **No cycles:** The dependency graph is a DAG. Commands sit at the top; evals and references sit at the leaves.

package/skills/aw-adk/references/cross-ide-mapping.md

@@ -0,0 +1,71 @@
+# Cross-IDE Mapping
+
+How AW registry artifacts manifest in different IDE environments after `aw init` and `aw pull`.
+
+## IDE Path Mapping
+
+| Artifact Type | Registry Path | Claude Code | Cursor | Codex |
+|---|---|---|---|---|
+| Skill | `.aw/.aw_registry/.../skills/<slug>/SKILL.md` | `.claude/skills/<slug>/SKILL.md` | `.cursor/rules/<slug>.mdc` | `.codex/skills/<slug>/` |
+| Agent | `.aw/.aw_registry/.../agents/<slug>.md` | `.claude/agents/<slug>.md` | `.cursor/rules/<slug>.mdc` | `.codex/agents/<slug>.md` |
+| Command | `.aw/.aw_registry/.../commands/<slug>.md` | `.claude/commands/<slug>.md` | N/A | N/A |
+| Rule | `.aw/.aw_rules/platform/<domain>/references/<slug>.md` | `.claude/rules/<domain>/<slug>.md` | `.cursor/rules/<slug>.mdc` | `.codex/rules/<slug>.md` |
+
+## How `aw init` Works
+
+1. Creates `.claude/`, `.cursor/`, `.codex/` directories if missing
+2. Installs core modules from `manifests/install-modules.json`
+3. Copies hooks configuration to appropriate locations
+4. Creates `skills-lock.json` to track installed versions
+
+## How `aw pull` Works
+
+1. Reads `skills-lock.json` for current state
+2. Fetches latest from `.aw/.aw_registry/` (platform-docs or local)
+3. Diffs against installed versions (SHA256 integrity check)
+4. Copies updated artifacts to IDE-local paths
+5. Updates `skills-lock.json`
+
+## `skills-lock.json` Format
+
+```json
+{
+  "version": 1,
+  "skills": {
+    "platform-core-aw-adk": {
+      "source": ".aw/.aw_registry/platform/core/skills/aw-adk/SKILL.md",
+      "integrity": "sha256-abc123...",
+      "installed_at": "2026-04-22T10:00:00Z",
+      "ide_paths": {
+        "claude": ".claude/skills/aw-adk/SKILL.md",
+        "cursor": ".cursor/rules/aw-adk.mdc"
+      }
+    }
+  }
+}
+```
+
+## Cursor-Specific Notes
+
+Cursor uses `.mdc` (Markdown with Context) files. The conversion from `.md` to `.mdc`:
+- Frontmatter is preserved as YAML
+- Body content is wrapped in Cursor's context format
+- `trigger` field maps to Cursor's "when" activation rules
+
+## Codex-Specific Notes
+
+Codex uses a flat directory structure under `.codex/`. Each artifact is a directory containing the artifact file plus any bundled resources.
+
+## What to Tell the User
+
+After creating any artifact, show them where it will appear:
+
+```
+Your new agent 'payments-processor' will be available at:
+
+  Claude Code: .claude/agents/payments-processor.md
+  Cursor:      .cursor/rules/payments-processor.mdc
+  Codex:       .codex/agents/payments-processor.md
+
+Run `aw pull` to sync from the registry to your IDE.
+```

package/skills/aw-adk/references/eval-placement-guide.md

@@ -0,0 +1,183 @@
+# Eval Placement Guide
+
+Evals live next to the artifacts they test. This document defines where eval files go and why.
+
+## Why Colocated > Centralized
+
+**Proximity to artifact.** When an eval lives in the same directory tree as the skill, agent, or command it tests, you see the eval every time you touch the artifact. Changes to the artifact naturally prompt eval updates.
+
+**Discoverability.** A developer exploring a skill directory finds its evals without searching a separate `evals/` monolith. New contributors understand what "good" looks like by reading colocated evals.
+
+**Ownership clarity.** The person who owns the artifact owns its evals. No ambiguity about who maintains a centralized eval that tests something in a different team's directory.
+
+**Refactor safety.** When an artifact moves or gets renamed, colocated evals move with it. Centralized evals require separate updates and are often forgotten, leading to orphaned or broken evals.
+
+## Directory Structure by Artifact Type
+
+### Skills
+
+Evals live inside the skill directory in an `evals/` subdirectory.
+
+```
+skills/
+  <slug>/
+    skill.md
+    references/
+    evals/
+      eval-<purpose>.md
+      eval-<purpose>.md
+```
+
+Example:
+```
+skills/
+  aw-adk/
+    skill.md
+    references/
+    evals/
+      eval-create-happy-path.md
+      eval-create-missing-fields.md
+      eval-score-minimal.md
+```
+
+### Agents
+
+Evals live in a sibling `evals/` directory scoped by agent slug.
+
+```
+agents/
+  <slug>.md
+  evals/
+    <slug>/
+      eval-<purpose>.md
+      eval-<purpose>.md
+```
+
+Example:
+```
+agents/
+  planner.md
+  code-reviewer.md
+  evals/
+    planner/
+      eval-plan-happy-path.md
+      eval-plan-ambiguous-input.md
+    code-reviewer/
+      eval-review-security-issue.md
+```
+
+### Commands
+
+Evals live in a sibling `evals/` directory scoped by command slug.
+
+```
+commands/
+  <slug>.md
+  evals/
+    <slug>/
+      eval-<purpose>.md
+      eval-<purpose>.md
+```
+
+Example:
+```
+commands/
+  aw-build.md
+  evals/
+    aw-build/
+      eval-build-happy-path.md
+      eval-build-missing-config.md
+```
+
+### Rules
+
+Evals live either within `.aw/.aw_rules/` references or in a dedicated `rules/evals/` directory.
+
+```
+# Option A: Inside .aw/.aw_rules references
+.aw/
+  .aw_rules/
+    platform/
+      <domain>/
+        references/
+          eval-<purpose>.md
+
+# Option B: Dedicated rules eval directory
+rules/
+  evals/
+    <slug>/
+      eval-<purpose>.md
+```
+
+### Meta-Evals (Evals of Evals)
+
+Evals that test the eval system itself live in a nested `evals/evals/` directory.
+
+```
+evals/
+  evals/
+    eval-<purpose>.md
+```
+
+## Naming Convention
+
+All eval files follow: `eval-<purpose>.md`
+
+The `<purpose>` segment describes what the eval tests in lowercase kebab-case.
+
+| Pattern | Example | Tests |
+|---------|---------|-------|
+| `eval-<action>-happy-path` | `eval-create-happy-path.md` | Standard successful execution |
+| `eval-<action>-<failure>` | `eval-create-missing-fields.md` | Specific failure scenario |
+| `eval-<action>-<edge>` | `eval-score-minimal.md` | Edge case or boundary condition |
+| `eval-<action>-adversarial` | `eval-create-adversarial.md` | Adversarial or malicious input |
+
+## Minimum Eval Count
+
+Every artifact requires at least **2 evals**:
+
+1. **Happy path** -- the artifact works correctly with valid, representative input.
+2. **Failure scenario** -- the artifact handles invalid input, missing data, or error conditions gracefully.
+
+For critical-path artifacts (commands users invoke directly, agents that orchestrate workflows), target **4+ evals**:
+
+1. Happy path
+2. Failure / error handling
+3. Edge case (boundary values, minimal input, maximum input)
+4. Adversarial (conflicting instructions, unexpected formats)
+
+## Eval File Structure
+
+Each eval file should contain:
+
+```markdown
+---
+target: <artifact-type>/<slug>
+type: eval
+purpose: <brief description>
+---
+
+# Eval: <Title>
+
+## Scenario
+<Description of the test scenario and input>
+
+## Expected Behavior
+<What the artifact should produce or do>
+
+## Grader
+<How to determine pass/fail -- deterministic checks preferred>
+
+## Pass Criteria
+<Explicit, binary pass/fail conditions>
+```
+
+## Validation Checklist
+
+Before merging an artifact, verify:
+
+- [ ] At least 2 eval files exist in the correct directory
+- [ ] Eval files follow `eval-<purpose>.md` naming
+- [ ] Each eval has a `target:` frontmatter referencing the parent artifact
+- [ ] At least one eval covers a failure scenario
+- [ ] Eval graders are specific enough to fail on wrong output (see [rubric-meta-eval.md](rubric-meta-eval.md))

package/skills/aw-adk/references/external-resources.md

@@ -0,0 +1,75 @@
+# External Resources for CASRE Authoring
+
+Curated references for writing high-quality Commands, Agents, Skills, Rules, and Evals.
+
+## Resources
+
+### Anthropic: Skill Best Practices
+
+**URL:** <https://platform.claude.com/docs/en/agents-and-tools/agent-skills/best-practices>
+
+Key takeaways: Structure matters more than length -- a well-organized 200-line skill outperforms a rambling 2000-line one. Front-load success criteria and constraints before procedural steps. Use concrete examples of good and bad output rather than abstract descriptions.
+
+### Anthropic: Equipping Agents with Agent Skills
+
+**URL:** <https://www.anthropic.com/engineering/equipping-agents-for-the-real-world-with-agent-skills>
+
+Key takeaways: Skills should encode domain expertise that the model lacks, not restate what it already knows. The most effective skills combine declarative knowledge (what good looks like) with procedural guardrails (what to avoid). Skills are most valuable when they reduce variance across runs -- the same input should produce consistently shaped output.
+
+### Anthropic: Demystifying Evals for AI Agents
+
+**URL:** <https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents>
+
+Key takeaways: Build evals bottom-up -- start with the smallest testable unit and compose upward. Prefer deterministic graders (exact match, regex, structured checks) over model-based graders wherever possible. When model-based grading is necessary, constrain the grader with explicit rubrics and examples of pass/fail. Eval quality directly determines your ability to iterate on agent behavior.
+
+### OpenAI: Eval Skills
+
+**URL:** <https://developers.openai.com/blog/eval-skills>
+
+Key takeaways: Evals should test behavior, not implementation. Define success criteria before writing the eval -- if you cannot state what "pass" looks like in concrete terms, the eval is not ready. Use multiple eval types (unit, integration, end-to-end) to cover different failure modes. Track eval results over time to detect regressions early.
+
+### Promptfoo: Agent Eval Patterns
+
+**URL:** <https://www.promptfoo.dev/docs/integrations/agent-skill/>
+
+Key takeaways: Separate the eval scenario (input + context) from the grader (how to judge output). This separation enables reuse -- the same grader can apply across multiple scenarios, and scenarios can be graded by different methods. Parameterize scenarios to generate coverage from templates rather than writing each case by hand.
+
+### O'Reilly: How to Write a Good Spec for AI Agents
+
+**URL:** <https://www.oreilly.com/radar/how-to-write-a-good-spec-for-ai-agents/>
+
+Key takeaways: A good spec defines the boundaries of acceptable output, not a single correct answer. Include examples of outputs that are wrong in subtle ways -- these teach the agent (and the eval grader) what to reject. Specs should be testable: every requirement should map to at least one eval scenario.
+
+### OpenAI: Evaluation Best Practices
+
+**URL:** <https://developers.openai.com/api/docs/guides/evaluation-best-practices>
+
+Key takeaways: Start with the simplest eval that provides signal and add complexity only when needed. Use a mix of automated and human evaluation, but automate first. Track baseline performance before making changes so you can measure improvement. Small, frequent eval runs catch regressions faster than large, infrequent ones.
+
+## Key Principles for CASRE Authoring
+
+These principles recur across the resources above. Apply them when writing any CASRE artifact.
+
+### Structure > Length
+
+A concise, well-organized artifact outperforms a verbose one. Use headings, tables, and lists to make content scannable. Front-load the most important information.
+
+### Success Criteria First
+
+Define what "done" and "good" look like before writing implementation details. For skills, state the expected output shape. For evals, state pass/fail criteria. For commands, state the end state.
+
+### Bottom-Up Eval Design
+
+Start with the smallest testable behavior. Write evals for individual skills before writing evals for agents that compose those skills. Compose simple evals into integration evals rather than writing monolithic end-to-end evals first.
+
+### Deterministic > Model-Based Graders
+
+Use exact match, regex, JSON schema validation, or structured checks whenever the output format allows. Reserve model-based grading for genuinely subjective or creative dimensions. When using model-based graders, provide explicit rubrics with scored examples.
+
+### Concrete Examples Over Abstract Descriptions
+
+Show what good output looks like. Show what bad output looks like. Examples reduce ambiguity more effectively than prose descriptions of quality. Include at least one positive and one negative example in skills and eval graders.
+
+### Testable Requirements
+
+Every requirement in a skill, rule, or command should map to at least one eval scenario. If a requirement cannot be tested, it is either too vague (rewrite it) or aspirational (move it to a "nice to have" section).

package/skills/aw-adk/references/getting-started.md

@@ -0,0 +1,66 @@
+---
+name: getting-started
+description: Quickstart guide for creating CASRE artifacts with the ADK
+---
+
+# Getting Started with the ADK
+
+## Your First Artifact in 5 Steps
+
+1. **Say what you want.** Use natural language — the ADK classifies the type for you.
+2. **Answer the interview.** The ADK asks targeted questions based on the artifact type.
+3. **Review the scaffold.** The ADK creates the file at the correct registry path.
+4. **Check the score.** The ADK scores your artifact against the type-specific rubric.
+5. **Run the evals.** The ADK creates 2+ evals and validates them.
+
+## Example Prompts by Type
+
+### Agent
+> Create an agent for code review automation in the platform/review namespace. It should analyze PR diffs for security issues, performance regressions, and style violations. Tools: Read, Grep, Glob, Bash. Model: sonnet.
+
+### Skill
+> Create a skill for MongoDB aggregation patterns in the platform/data namespace. Cover $lookup, $unwind, $group, pagination, and index-aware pipeline design.
+
+### Command
+> Create a command for incident response in the platform/infra namespace. Phases: triage → investigate → mitigate → postmortem. Human checkpoint before mitigation.
+
+### Rule
+> Create a rule called no-direct-db-connection for the backend domain. All database access must go through @platform-core/\* packages. Severity: MUST. File patterns: \*.service.ts, \*.repository.ts.
+
+### Eval
+> Create evals for the existing code-reviewer agent at .aw/.aw_registry/platform/review/agents/code-reviewer.md. One happy path, one where the PR has no issues but the agent flags false positives.
+
+## Common Intent Phrases
+
+These phrases trigger the ADK via the `using-aw-skills` router:
+
+- "create an agent/skill/command/rule/eval"
+- "score my agent/skill"
+- "audit all agents in platform/services"
+- "fix the lint errors on this skill"
+- "improve the payments-processor agent"
+- "delete the old migration command"
+
+## What Happens Under the Hood
+
+Every create follows the same 14-step pipeline:
+
+```
+CLASSIFY → INTERVIEW → RESOLVE PATH → SCAFFOLD → CHECKPOINT →
+LINT → SCORE → EVAL GATE (2+) → TEST RUNS → ITERATE →
+DESCRIPTION OPT → CROSS-IDE → REGISTRY UPDATES → SYNC
+```
+
+No steps are optional. Rules, agents, commands, skills, and evals all go through the full flow.
+
+## Deleting Artifacts
+
+The ADK also handles safe deletion with reverse reference scanning:
+
+```
+/aw:adk agent delete my-agent
+```
+
+The delete flow: LOCATE → INVENTORY → REVERSE REFERENCE SCAN → CONFIRM → DELETE → REGISTRY CLEANUP → SYNC
+
+It finds everything that points to the artifact (commands referencing an agent, agents referencing a skill) and offers to clean up those references too — no phantom dependencies left behind.