@sentry/warden 0.21.0 → 0.23.0

package/src/internal-skills/skill-writer/references/synthesis-path.md

@@ -0,0 +1,119 @@
+# Synthesis Path
+
+Use this path when creating or materially changing a skill.
+
+## Output Style
+
+- Keep synthesis notes terse.
+- Prefer tables, status lists, and gap lists over narrative summaries.
+- Record decisions as `adopted`, `rejected`, or `deferred`.
+
+## Step 0: Classify
+
+Record:
+
+1. skill class
+2. primary execution shape
+3. secondary shapes, if any
+4. why simpler shapes were not enough
+
+For `integration-documentation`, cover:
+
+1. API surface and behavior contracts
+2. config/runtime options
+3. downstream use cases
+4. issues/failure modes with workarounds
+5. version or migration variance
+
+## Step 1: Collect Sources
+
+Collect from:
+
+1. Agent Skills spec and best practices
+2. similar in-repo skills
+3. upstream implementations and orchestration docs
+4. domain or library docs
+5. repo conventions and validators
+6. tests, fixtures, changelogs, and issue or PR history
+7. commit history and blame for regressions or edge cases
+8. prior `SPEC.md`, `SOURCES.md`, `EVAL.md`, and `references/evidence/`
+
+If the shape uses provider-specific mechanics, include current provider docs.
+
+## Baseline Source Pack For Skill-Authoring
+
+Require at minimum:
+
+1. local `skill-writer` runtime files
+2. Agent Skills spec and repo conventions
+3. provider docs for any provider-specific mechanic being recommended
+
+## Step 1.5: Load Example Profiles
+
+Load only the profiles you need from `references/examples/`.
+
+## Step 1.6: Expand Coverage
+
+Run targeted passes for:
+
+| Pass | Retrieve |
+|------|----------|
+| core behavior | happy path and main workflow |
+| edge behavior | failures, retries, permissions, cleanup |
+| negative behavior | false positives, reviewer concerns, bad outputs |
+| repair patterns | fixes and corrected outputs |
+| version variance | platform or release differences |
+| shape mechanics | routing, delegation, loop stops, hook constraints |
+
+Extra retrieval for advanced shapes:
+
+1. route or delegation criteria
+2. worker or handoff contracts
+3. loop stopping rules
+4. provider-specific lifecycle or security constraints
+
+## Step 2: Capture Provenance
+
+For each source, record:
+
+- source URL or path
+- trust tier
+- confidence
+- contribution
+- usage constraints
+
+Store provenance in `SOURCES.md`, not long runtime prose.
+
+## Step 3: Synthesize Decisions
+
+Map each major decision to source evidence, including:
+
+- class choice
+- shape choice
+- provider-specific mechanics
+- deferred gaps
+
+## Step 4: Enforce Depth Gates
+
+All of these must pass:
+
+1. no missing high-impact coverage dimensions
+2. partial dimensions have explicit next retrieval actions
+3. authoring or generator skills include transformed examples
+4. selected profile requirements are satisfied
+5. coverage passes are reflected in the coverage matrix
+6. stopping rationale is explicit
+7. supporting refs stay focused and directly discoverable from `SKILL.md`
+8. `SPEC.md` exists or is updated when the contract changed
+9. advanced mechanics include required contracts and justification
+10. provider-specific mechanics include portability notes
+
+## Required Output
+
+- synthesis summary
+- source inventory in `SOURCES.md`
+- decisions and rationale
+- coverage matrix
+- gaps and next retrieval actions
+- selected class and shape
+- `SPEC.md` update summary when applicable

package/src/internal-skills/skill-writer/references/workflow-mechanics/evaluator-loops.md

@@ -0,0 +1,25 @@
+# Evaluator Loops
+
+Use this workflow when a first draft benefits from critique and revision against a clear rubric.
+
+## Choose this workflow when
+
+- the skill can score output against explicit criteria
+- revision improves quality materially
+- the loop has a bounded stopping rule
+
+## Required contract
+
+1. Rubric or acceptance criteria.
+2. Maximum loop count or plateau rule.
+3. What evidence or diffs must be preserved between rounds.
+
+## Example
+
+```markdown
+1. Produce a first draft
+2. Score it against the rubric
+3. Turn failures into concrete edits
+4. Revise
+5. Stop when the rubric passes or the loop plateaus
+```

package/src/internal-skills/skill-writer/references/workflow-mechanics/orchestrator-workers.md

@@ -0,0 +1,25 @@
+# Orchestrator-Workers
+
+Use this workflow when the subtasks are not known ahead of time and must be discovered from the input.
+
+## Choose this workflow when
+
+- the orchestrator must discover work units dynamically
+- each unit can be delegated to a stable worker contract
+- a final synthesis step can merge the results
+
+## Required contract
+
+1. Worker-task schema.
+2. Worker output schema.
+3. Expansion limit or stopping rule.
+4. Final synthesis rule.
+
+## Example
+
+```markdown
+1. Inspect the request and identify work units dynamically
+2. Assign each work unit to a worker path
+3. Collect worker summaries in a fixed schema
+4. Synthesize the result
+```

package/src/internal-skills/skill-writer/references/workflow-mechanics/parallel-workflows.md

@@ -0,0 +1,21 @@
+# Parallel Workflows
+
+Use this workflow when independent subtasks can run side-by-side, or when multiple judgments improve confidence.
+
+## Choose this workflow when
+
+- work units are independent
+- latency or coverage improves with parallel execution
+- aggregation can be described clearly
+
+## Common forms
+
+1. `sectioning`: split a task into independent work units.
+2. `voting`: run multiple judgments and aggregate them.
+
+## Required contract
+
+1. Unit of parallel work.
+2. Merge or vote rule.
+3. Conflict handling.
+4. Cost or latency cap.

package/src/internal-skills/skill-writer/references/workflow-mechanics/plan-validate-execute.md

@@ -0,0 +1,26 @@
+# Plan-Validate-Execute
+
+Use this workflow when a destructive or high-stakes action should be preceded by a machine-checkable plan.
+
+## Choose this workflow when
+
+- validating the plan is safer than validating the final action
+- the task changes many files or records
+- rollback would be expensive
+
+## Required contract
+
+1. The plan artifact and schema.
+2. The validation step and source of truth.
+3. Rework rules when validation fails.
+4. Final execution and verification steps.
+
+## Example
+
+```markdown
+1. Analyze the input and generate `changes.json`
+2. Validate the plan against source of truth
+3. Revise until validation passes
+4. Execute the plan
+5. Verify the result
+```

package/src/internal-skills/skill-writer/references/workflow-mechanics/prompt-chaining.md

@@ -0,0 +1,24 @@
+# Prompt Chaining
+
+Use this workflow when the task should move through fixed ordered steps and each step makes the next one easier.
+
+## Choose this workflow when
+
+- the decomposition is known in advance
+- step order matters
+- validation between steps improves quality
+
+## Required contract
+
+1. Step order.
+2. Inputs and outputs for each step.
+3. Validation or gate points between steps.
+
+## Example
+
+```markdown
+1. Summarize the current state
+2. Propose the target state
+3. Validate the proposal against constraints
+4. Write the final document
+```

package/src/internal-skills/skill-writer/references/workflow-mechanics/routing-workflows.md

@@ -0,0 +1,28 @@
+# Routing Workflows
+
+Use this workflow when the incoming request must be classified and sent to different downstream prompts, references, scripts, or tools.
+
+## Choose this workflow when
+
+- distinct request classes benefit from specialized downstream handling
+- one broad prompt would create conflicts or false positives
+- misroutes can be detected and recovered
+
+## Required contract
+
+1. Route-selection criteria.
+2. Default route or clarification fallback.
+3. Misroute recovery.
+4. Downstream contract for each route.
+
+## Example
+
+```markdown
+1. Classify the request:
+
+   **Billing question?** -> Load `references/billing.md`
+   **Refund request?** -> Run `scripts/refund_intake.py`
+   **Technical bug?** -> Load `references/triage.md`
+
+2. If classification is uncertain, ask one clarification question before continuing.
+```

package/src/internal-skills/skill-writer/references/workflow-mechanics/validation-loops.md

@@ -0,0 +1,26 @@
+# Validation Loops
+
+Use this workflow when a validator can reliably catch mistakes before the skill should claim completion.
+
+## Choose this workflow when
+
+- a script, schema, test, or parser can catch important failures
+- the agent should fix issues immediately after each change
+- the main risk is silent drift or invalid output
+
+## Required contract
+
+1. The validator or check to run.
+2. When it runs in the workflow.
+3. What to fix before retrying.
+4. What counts as a passing state.
+
+## Example
+
+```markdown
+1. Make the change
+2. Validate immediately
+3. If validation fails, fix the issue
+4. Re-run validation
+5. Only proceed when validation passes
+```

package/src/internal-skills/skill-writer/scripts/quick_validate.py

@@ -0,0 +1,158 @@
+# /// script
+# requires-python = ">=3.12"
+# dependencies = ["pyyaml"]
+# ///
+"""
+Quick validation script for Agent Skills.
+
+Validates only mechanical SKILL.md frontmatter and identity requirements.
+Qualitative content, reference depth, routing, and prose structure belong to
+human/agent review, not this script.
+
+Usage:
+    uv run quick_validate.py <skill_directory> [--skill-class <class>] [--strict-depth]
+
+Returns exit code 0 on success, 1 on failure. Outputs JSON with validation results.
+"""
+
+import argparse
+import json
+import re
+import sys
+from pathlib import Path
+
+import yaml
+
+MAX_NAME_LENGTH = 64
+MAX_DESCRIPTION_LENGTH = 1024
+
+SKILL_CLASSES = {
+    "auto",
+    "workflow-process",
+    "integration-documentation",
+    "security-review",
+    "skill-authoring",
+    "generic",
+}
+
+
+def parse_args(argv: list[str]) -> argparse.Namespace:
+    parser = argparse.ArgumentParser(
+        description="Validate agent skill frontmatter and identity fields.",
+    )
+    parser.add_argument("skill_directory")
+    parser.add_argument("--skill-class", choices=sorted(SKILL_CLASSES), default="auto")
+    parser.add_argument("--strict-depth", action="store_true")
+    return parser.parse_args(argv)
+
+
+def validate_skill(
+    skill_path: Path,
+    selected_skill_class: str = "auto",
+    strict_depth: bool = False,
+) -> tuple[bool, list[str], list[str], str]:
+    """Validate a skill directory. Returns (valid, errors, warnings, resolved_skill_class)."""
+    # Keep the flag for CLI compatibility; depth/content checks belong to review.
+    _ = strict_depth
+    errors: list[str] = []
+    warnings: list[str] = []
+
+    # Check SKILL.md exists.
+    skill_md = skill_path / "SKILL.md"
+    if not skill_md.exists():
+        return False, ["SKILL.md not found"], [], "generic"
+
+    content = skill_md.read_text()
+
+    # Check frontmatter exists and is first.
+    if not content.startswith("---"):
+        errors.append("No YAML frontmatter found (file must start with ---)")
+        return False, errors, warnings, "generic"
+
+    match = re.match(r"^---\n(.*?)\n---", content, re.DOTALL)
+    if not match:
+        errors.append("Invalid frontmatter format (missing closing ---)")
+        return False, errors, warnings, "generic"
+
+    # Parse frontmatter.
+    frontmatter_text = match.group(1)
+    try:
+        frontmatter = yaml.safe_load(frontmatter_text)
+        if not isinstance(frontmatter, dict):
+            errors.append("Frontmatter must be a YAML mapping")
+            return False, errors, warnings, "generic"
+    except yaml.YAMLError as exc:
+        errors.append(f"Invalid YAML in frontmatter: {exc}")
+        return False, errors, warnings, "generic"
+
+    # Validate frontmatter keys without hardcoding provider-specific optional fields.
+    invalid_keys = [key for key in frontmatter.keys() if not isinstance(key, str) or not key.strip()]
+    if invalid_keys:
+        errors.append("Frontmatter keys must be non-empty strings")
+
+    # Validate name.
+    if "name" not in frontmatter:
+        errors.append("Missing required field: name")
+    else:
+        name = frontmatter["name"]
+        if not isinstance(name, str):
+            errors.append(f"name must be a string, got {type(name).__name__}")
+        else:
+            name = name.strip()
+            if not name:
+                errors.append("name must not be empty")
+            elif len(name) > MAX_NAME_LENGTH:
+                errors.append(f"name is too long ({len(name)} chars, max {MAX_NAME_LENGTH})")
+            elif not re.match(r"^[a-z0-9-]+$", name):
+                errors.append(f"name '{name}' must contain only lowercase letters, digits, and hyphens")
+            elif name.startswith("-") or name.endswith("-"):
+                errors.append(f"name '{name}' must not start or end with a hyphen")
+            elif "--" in name:
+                errors.append(f"name '{name}' must not contain consecutive hyphens")
+            elif name != skill_path.name:
+                errors.append(f"name '{name}' does not match directory name '{skill_path.name}'")
+
+    # Validate description.
+    if "description" not in frontmatter:
+        errors.append("Missing required field: description")
+    else:
+        description = frontmatter["description"]
+        if not isinstance(description, str):
+            errors.append(f"description must be a string, got {type(description).__name__}")
+        else:
+            description = description.strip()
+            if not description:
+                errors.append("description must not be empty")
+            elif len(description) > MAX_DESCRIPTION_LENGTH:
+                errors.append(f"description is too long ({len(description)} chars, max {MAX_DESCRIPTION_LENGTH})")
+
+    resolved_skill_class = selected_skill_class if selected_skill_class != "auto" else "generic"
+
+    return len(errors) == 0, errors, warnings, resolved_skill_class
+
+
+def main() -> None:
+    args = parse_args(sys.argv[1:])
+    skill_path = Path(args.skill_directory).resolve()
+    if not skill_path.is_dir():
+        print(json.dumps({"valid": False, "errors": [f"Not a directory: {skill_path}"]}))
+        sys.exit(1)
+
+    valid, errors, warnings, resolved_skill_class = validate_skill(
+        skill_path,
+        selected_skill_class=args.skill_class,
+        strict_depth=args.strict_depth,
+    )
+    result = {
+        "valid": valid,
+        "skill_class": resolved_skill_class,
+        "strict_depth": args.strict_depth,
+        "errors": errors,
+        "warnings": warnings,
+    }
+    print(json.dumps(result, indent=2))
+    sys.exit(0 if valid else 1)
+
+
+if __name__ == "__main__":
+    main()

package/src/internal-skills/skill-writer/scripts/quick_validate_test.py

@@ -0,0 +1,79 @@
+import importlib.util
+import tempfile
+import unittest
+from pathlib import Path
+
+
+SCRIPT_PATH = Path(__file__).with_name("quick_validate.py")
+SPEC = importlib.util.spec_from_file_location("quick_validate", SCRIPT_PATH)
+assert SPEC is not None
+quick_validate = importlib.util.module_from_spec(SPEC)
+assert SPEC.loader is not None
+SPEC.loader.exec_module(quick_validate)
+
+
+class QuickValidateTest(unittest.TestCase):
+    def test_ignores_markdown_content_and_supporting_files(self) -> None:
+        with tempfile.TemporaryDirectory() as temp_dir:
+            skill_path = Path(temp_dir) / "example-skill"
+            skill_path.mkdir()
+            (skill_path / "SKILL.md").write_text(
+                """---
+name: example-skill
+description: I can review <example> content.
+---
+
+This intentionally references missing content that review may care about:
+
+- references/missing.md
+- scripts/missing.py
+- SOURCES.md
+
+/Users/example/private/path should not be script-validated.
+""",
+                encoding="utf-8",
+            )
+            (skill_path / "SPEC.md").write_text(
+                """# Example Skill Spec
+
+## Evaluation
+
+Run the local checks for this skill.
+""",
+                encoding="utf-8",
+            )
+
+            valid, errors, warnings, _skill_class = quick_validate.validate_skill(
+                skill_path,
+                selected_skill_class="integration-documentation",
+                strict_depth=True,
+            )
+
+        self.assertTrue(valid)
+        self.assertEqual(errors, [])
+        self.assertEqual(warnings, [])
+
+    def test_rejects_invalid_frontmatter_yaml(self) -> None:
+        with tempfile.TemporaryDirectory() as temp_dir:
+            skill_path = Path(temp_dir) / "example-skill"
+            skill_path.mkdir()
+            (skill_path / "SKILL.md").write_text(
+                """---
+name: [example-skill
+description: Use when reviewing example code.
+---
+
+Review example code.
+""",
+                encoding="utf-8",
+            )
+
+            valid, errors, warnings, _skill_class = quick_validate.validate_skill(skill_path)
+
+        self.assertFalse(valid)
+        self.assertEqual(warnings, [])
+        self.assertTrue(any("Invalid YAML in frontmatter" in message for message in errors), errors)
+
+
+if __name__ == "__main__":
+    unittest.main()