@sentry/warden 0.21.0 → 0.23.0

package/src/internal-skills/skill-writer/references/description-optimization.md

@@ -0,0 +1,28 @@
+# Description Optimization
+
+Use this path to improve skill triggering quality and reduce false matches.
+
+## Trigger quality loop
+
+1. Draft a description with realistic user language and concrete trigger phrases.
+2. Build two query sets:
+- should-trigger queries
+- should-not-trigger queries
+3. Evaluate the current description against both sets.
+4. Edit description wording to improve precision/recall.
+5. Repeat until false positives and false negatives are reduced to acceptable levels.
+
+## Authoring rules
+
+1. Keep the description in third person.
+2. Include what the skill does and when to use it.
+3. Avoid implementation details that do not help triggering.
+4. Avoid provider-specific phrasing unless the skill is intentionally provider-specific.
+5. For provider-agnostic skills, avoid naming Claude, Codex, or any provider in ways that would narrow portability expectations.
+
+## Required output
+
+- Final description text
+- should-trigger query set
+- should-not-trigger query set
+- Summary of edits made to improve trigger behavior

package/src/internal-skills/skill-writer/references/design-principles.md

@@ -0,0 +1,74 @@
+# Skill Design Principles
+
+Use this guide to keep skill instructions dense, scannable, and worth their token cost.
+
+## Core Rule
+
+- Every line should help the agent decide, do, or verify something.
+- Prefer tables, checklists, templates, and input/output examples over explanatory prose.
+- Keep rationale to one short sentence unless the agent is likely to make the wrong choice without it.
+
+## Add Vs Cut
+
+| Keep | Cut |
+|------|-----|
+| project-specific conventions | generic background the agent already knows |
+| non-obvious gotchas | motivational filler |
+| exact commands, schemas, and templates | repeated restatements of the same rule |
+| branch logic and defaults | long essays where a table would work |
+| one strong example | multiple weak examples saying the same thing |
+
+## Match Structure To Fragility
+
+| Fragility | Preferred structure | Avoid |
+|-----------|---------------------|-------|
+| high | exact steps, strict templates, validation gates | open-ended guidance |
+| medium | short checklist plus examples | long rationale-heavy prose |
+| low | brief goals and constraints | overspecified playbooks |
+
+## Preferred Instruction Shapes
+
+| Need | Preferred shape |
+|------|-----------------|
+| choose a path | decision table |
+| do a repeatable task | numbered checklist |
+| enforce output structure | template or schema |
+| show style or tone | input/output examples |
+| diagnose failures | symptom/cause/fix matrix |
+| communicate exact facts | compact reference table |
+
+## Description Rules
+
+- Keep `description` in third person.
+- Put trigger language in `description`, not the body.
+- Front-load what the skill does and when to use it.
+- Do not spend description space on internals unless they improve triggering.
+
+## Runtime Writing Rules
+
+- Use imperative voice.
+- State one default path before mentioning alternatives.
+- Use one term per concept; do not rotate synonyms.
+- Put universal rules in `SKILL.md`; put optional depth in routed refs.
+- If a section is mostly explanation, cut it or replace it with a denser structure.
+
+## Reference Rules
+
+- Reference filenames should predict their contents.
+- Each reference should answer one lookup question.
+- Subfolders are acceptable only when they make the lookup path clearer.
+- Every bundled reference should have a direct "open when..." entry in `SKILL.md`.
+- Do not create catch-all files for notes, context, or mixed patterns.
+
+## Independence And Portability
+
+- Do not require another skill by name at runtime.
+- Use skill-root-relative paths by default.
+- Reuse established repo-specific path variables only when the repo already standardizes on them.
+- Label provider-specific mechanics explicitly and add portability notes when they matter.
+
+## Long Files
+
+- Keep `SKILL.md` short enough to scan as a router.
+- For references over 100 lines, add `## Contents`.
+- If a reference grows because it mixes multiple lookup needs, split it.

package/src/internal-skills/skill-writer/references/evaluation-path.md

@@ -0,0 +1,73 @@
+# Evaluation Path
+
+Load this only when the user asks for evaluation, the change is high-risk, or you need to verify a non-obvious architectural choice.
+
+## Default approach (lightweight, guidance-only)
+
+If you do run evaluation, start here:
+
+1. Define representative prompts for the target skill task.
+2. Compare observed behavior before/after edits in concise notes.
+3. For material skill changes, judge the architectural choice as well as the prose:
+   - class selection: pass/fail
+   - execution shape selection: pass/fail
+   - reference routing clarity: pass/fail
+   - advanced mechanics justification: pass/fail
+   - portability/fallback notes: pass/fail
+4. Mark outcomes as improved, unchanged, or regressed.
+5. Record unresolved weaknesses and next steps.
+
+For `integration-documentation` and `skill-authoring` skills, include a concise depth rubric:
+
+1. API surface coverage: pass/fail.
+2. Known issues/workarounds coverage: pass/fail.
+3. Common use-case coverage: pass/fail.
+4. Gap handling quality (explicit next retrieval actions for partials): pass/fail.
+
+## Deeper eval playbook (optional)
+
+Use this only when:
+
+1. The user requests rigorous evals.
+2. The skill is high-risk or high-cost if wrong.
+3. You need regression-tracking over time.
+
+Suggested workflow:
+
+1. Build a prompt set with positives, implicit triggers, negatives, and wrong-shape temptations.
+2. Capture deterministic run traces (for example `codex exec --json`).
+3. Apply machine-checkable rubric/schema checks (for example `--output-schema` where applicable).
+4. Compare baseline vs updated behavior and report deltas.
+
+For advanced shapes, add targeted checks:
+
+1. routers: should-route and should-not-route prompts
+2. evaluator loops: stop-condition and acceptance checks
+3. subagent-fork skills: task-oriented prompt vs passive-guidance negative
+4. hook-backed skills: fallback behavior and security-note presence
+
+## Optional quantitative benchmark
+
+Run only when explicitly requested or when objective scoring is practical.
+
+1. Define baseline (without skill guidance).
+2. Define with-skill run.
+3. Use the same prompt set and scoring rubric for both.
+4. Report deltas and confidence in the result.
+
+Do not block completion on deeper evals unless the user asks for them.
+
+## Canonical eval prompts
+
+Keep reusable, copy/paste eval prompts in `../EVAL.md`.
+Use those prompts when you need a repeatable depth check against `skill-writer`.
+
+## Agent-agnostic requirement
+
+Keep evaluation instructions tool-agnostic so they work in both Codex and Claude environments.
+
+## Required Output
+
+- Qualitative evaluation summary
+- Deeper eval or quantitative summary (optional, if run)
+- Final acceptance decision and residual risks

package/src/internal-skills/skill-writer/references/examples/documentation-skill.md

@@ -0,0 +1,70 @@
+# Case Study: Documentation Skill Synthesis
+
+## Scenario
+
+Goal: create a skill that helps an agent answer and author code for a library without repeatedly re-reading upstream docs.
+
+## Input collection approach
+
+This case used breadth-first source collection and only stopped when new retrieval yielded mostly duplicates:
+
+1. Official docs landing pages and navigation trees.
+2. All API/class/module reference pages.
+3. Configuration and environment reference pages.
+4. Official examples/tutorials.
+5. Troubleshooting/error catalog pages.
+6. Migration/deprecation/changelog pages.
+7. Upstream repo README plus canonical examples.
+8. In-repo usage of the library (`rg` on imports and key APIs).
+
+## Coverage matrix used
+
+Required dimensions tracked during synthesis:
+
+1. Setup and installation.
+2. Core primitives and API surface.
+3. Configuration and runtime options.
+4. Normal usage patterns.
+5. Edge cases and failure handling.
+6. Version-specific differences.
+7. Migration and deprecation guidance.
+8. Instructional templates/examples for direct reuse.
+
+## Synthesized artifacts produced
+
+The resulting skill references included:
+
+1. Happy-path implementation template.
+2. Production-safe variant with defensive defaults.
+3. Anti-pattern and corrected implementation.
+4. Intent-to-reference routing guide (which section to load for which user request).
+5. Gap log with explicit next retrieval steps.
+
+## Source-to-decision trace (sample)
+
+1. Source class: migration/changelog docs.
+   Decision: add a version-compatibility checklist section to the skill.
+   Why: multiple API signatures existed across versions; without this, answers were inconsistent.
+2. Source class: troubleshooting/error catalog.
+   Decision: add an error-to-fix lookup table in references.
+   Why: user prompts often start from failures, not idealized setup.
+3. Source class: in-repo usage scan (`rg`).
+   Decision: prioritize examples matching local project patterns.
+   Why: produced outputs became directly usable with fewer edits.
+
+## Concrete artifacts (sample)
+
+1. Prompt and output skeleton:
+   Prompt: "Configure <library> client for retries and auth in production."
+   Output: a production-safe template with retry/backoff, timeout defaults, and auth placeholders.
+2. Anti-pattern transformation:
+   Before: single inline config with no timeout/error handling.
+   After: structured config with explicit timeout, retry policy, and failure handling notes.
+3. Reference routing snippet:
+   If request mentions "migration" -> load migration/changelog reference first, then API reference.
+
+## What made this high quality
+
+1. Input retrieval was exhaustive across all doc classes, not just top pages.
+2. The skill shipped transformed examples, not citation-only notes.
+3. Coverage and gaps were explicit, so iteration could continue safely.

package/src/internal-skills/skill-writer/references/examples/evaluator-loop-skill.md

@@ -0,0 +1,38 @@
+# Case Study: Evaluator-Loop Skill Synthesis
+
+## Scenario
+
+Goal: create a skill where one pass is not enough and quality improves through critique and revision.
+
+## Input collection approach
+
+This case collected:
+
+1. examples of first-draft failures
+2. existing review rubrics or acceptance criteria
+3. evidence that critique materially improves output
+4. stopping rules to avoid endless loops
+
+## Coverage matrix used
+
+Required dimensions tracked during synthesis:
+
+1. rubric quality
+2. feedback-to-edit mapping
+3. loop stop conditions
+4. evidence retention between rounds
+
+## Synthesized artifacts produced
+
+The resulting skill references included:
+
+1. first-draft to critique to revision example
+2. rubric reference
+3. max-loop or plateau rule
+4. acceptance checklist
+
+## What made this high quality
+
+1. the rubric was explicit
+2. the loop had a clear stopping rule
+3. the revision process produced measurable improvement rather than vague repetition

package/src/internal-skills/skill-writer/references/examples/hook-backed-skill.md

@@ -0,0 +1,39 @@
+# Case Study: Hook-Backed Skill Synthesis
+
+## Scenario
+
+Goal: create a skill that enforces a deterministic check at a specific lifecycle or tool boundary.
+
+## Input collection approach
+
+This case collected:
+
+1. official hook lifecycle and schema docs
+2. security guidance for shell-executed hooks
+3. examples of narrow matchers vs over-broad hooks
+4. fallback behavior for environments without hooks
+
+## Coverage matrix used
+
+Required dimensions tracked during synthesis:
+
+1. event and matcher scope
+2. decision behavior
+3. security boundaries
+4. fallback behavior
+5. portability constraints
+
+## Synthesized artifacts produced
+
+The resulting skill references included:
+
+1. hook configuration example
+2. safety checklist
+3. fallback path without hooks
+4. anti-pattern showing over-broad or unsafe hook scope
+
+## What made this high quality
+
+1. the hook scope was narrow and auditable
+2. security assumptions were explicit
+3. the skill still described what to do when hooks were unavailable

package/src/internal-skills/skill-writer/references/examples/router-skill.md

@@ -0,0 +1,41 @@
+# Case Study: Router Skill Synthesis
+
+## Scenario
+
+Goal: create a skill that classifies requests into distinct downstream paths without overloading one prompt.
+
+## Input collection approach
+
+This case collected:
+
+1. examples of request categories
+2. known ambiguous cases
+3. downstream resources for each route
+4. historical misroutes and their fixes
+
+Collection stopped only after route criteria and fallback behavior were explicit.
+
+## Coverage matrix used
+
+Required dimensions tracked during synthesis:
+
+1. route categories and triggers
+2. ambiguous or overlapping cases
+3. downstream ownership per route
+4. default/fallback path
+5. misroute recovery behavior
+
+## Synthesized artifacts produced
+
+The resulting skill references included:
+
+1. route-selection table
+2. one reference or script per route
+3. ambiguous-case examples
+4. fallback rule for unclear input
+
+## What made this high quality
+
+1. the route table was explicit
+2. every route had one clear downstream owner
+3. the skill knew what to do when classification was uncertain

package/src/internal-skills/skill-writer/references/examples/security-review-skill.md

@@ -0,0 +1,69 @@
+# Case Study: Security Review Skill Synthesis
+
+## Scenario
+
+Goal: build a skill that finds real vulnerabilities while minimizing false positives.
+
+## Input collection approach
+
+This case required balanced collection across offensive and defensive material:
+
+1. Canonical standards and cheat sheets.
+2. Framework/language-specific secure coding docs.
+3. Real-world exploit writeups and postmortems.
+4. Fixed vulnerability diffs and secure rewrites.
+5. Benign patterns often misclassified as vulnerabilities.
+6. Existing in-repo security skills and review heuristics.
+
+Collection continued until each vulnerability class had both exploit and mitigation evidence.
+
+## Coverage matrix used
+
+Required dimensions tracked during synthesis:
+
+1. Vulnerability class definitions and prerequisites.
+2. Exploitable dataflow examples.
+3. False-positive controls.
+4. Severity/confidence calibration.
+5. Concrete remediation patterns.
+6. Framework-specific caveats and exceptions.
+
+## Synthesized artifacts produced
+
+The resulting skill references included:
+
+1. True-positive case with exploitation path.
+2. False-positive case with proof of safety.
+3. Fix/remediation case with corrected code pattern.
+4. Severity and confidence decision rubric.
+5. Evidence checklist to prevent pattern-only claims.
+
+## Source-to-decision trace (sample)
+
+1. Source class: exploit writeups.
+   Decision: require attacker-controlled input path in every high-confidence finding.
+   Why: removed pattern-only false alarms.
+2. Source class: benign counterexamples.
+   Decision: add explicit safe-pattern checks before reporting.
+   Why: reduced repeated false positives on sanitized data paths.
+3. Source class: fixed vulnerability diffs.
+   Decision: include remediation examples as patch-shaped guidance.
+   Why: improved downstream fix quality and speed.
+
+## Concrete artifacts (sample)
+
+1. True-positive case:
+   Input pattern: untrusted data reaches shell/API call without escaping.
+   Output: finding includes source, sink, exploit path, and minimal patch recommendation.
+2. False-positive case:
+   Input pattern: potentially dangerous API with validated allowlist and strict escaping.
+   Output: no vulnerability finding; include reason for non-reporting.
+3. Remediation case:
+   Before: dynamic query construction from user input.
+   After: parameterized query plus validation guard.
+
+## What made this high quality
+
+1. It was trained on both attacks and safe counterexamples.
+2. Findings required evidence of exploitability, not keyword matching.
+3. Remediation guidance was concrete and immediately applicable.

package/src/internal-skills/skill-writer/references/examples/subagent-fork-skill.md

@@ -0,0 +1,39 @@
+# Case Study: Subagent-Fork Skill Synthesis
+
+## Scenario
+
+Goal: create a skill that runs a self-contained task in isolated context and returns a concise summary.
+
+## Input collection approach
+
+This case collected:
+
+1. official provider docs for skill execution in forked context
+2. examples of self-contained tasks that benefit from isolation
+3. failure cases where passive guidance was incorrectly put into subagents
+4. summary expectations for returning results to the main thread
+
+## Coverage matrix used
+
+Required dimensions tracked during synthesis:
+
+1. why isolation helps
+2. task prompt clarity
+3. output/summary contract
+4. tool/model assumptions
+5. portability constraints
+
+## Synthesized artifacts produced
+
+The resulting skill references included:
+
+1. actionable task body
+2. expected summary schema
+3. portability note
+4. anti-pattern showing passive guidance that should stay inline
+
+## What made this high quality
+
+1. the skill body was a task, not a convention list
+2. the isolation benefit was concrete
+3. the result expected back in the main thread was explicit

package/src/internal-skills/skill-writer/references/examples/workflow-process-skill.md

@@ -0,0 +1,65 @@
+# Case Study: Workflow/Process Skill Synthesis
+
+## Scenario
+
+Goal: create a skill for repeatable operational workflows (for example PR prep, CI triage, branching, settings audit).
+
+## Input collection approach
+
+This case collected process truth from all authoritative locations:
+
+1. Official tool docs and syntax references.
+2. Repository workflow conventions and policy docs.
+3. Existing local skills with adjacent process logic.
+4. CI logs, failure patterns, and known operational pitfalls.
+5. Positive and negative historical examples from prior runs.
+
+Collection stopped only after failure and recovery paths were well represented.
+
+## Coverage matrix used
+
+Required dimensions tracked during synthesis:
+
+1. Preconditions and required context.
+2. Ordered execution flow.
+3. Safety/permission boundaries.
+4. Expected outputs and acceptance checks.
+5. Failure handling and retry behavior.
+6. Escalation and handoff behavior.
+
+## Synthesized artifacts produced
+
+The resulting skill references included:
+
+1. Happy-path execution transcript.
+2. Guarded variant with stricter safety constraints.
+3. Failure-recovery transcript for a critical broken step.
+4. Output template for deterministic reporting.
+5. Changelog rules for iterative improvement from examples.
+
+## Source-to-decision trace (sample)
+
+1. Source class: repo policy docs.
+   Decision: add explicit precondition checks before running side-effecting steps.
+   Why: prevented invalid execution in partially configured environments.
+2. Source class: CI failure logs.
+   Decision: add a mandatory failure triage branch with retry vs escalate criteria.
+   Why: reduced dead-end loops during workflow execution.
+3. Source class: historical positive/negative examples.
+   Decision: standardize output format for easier review and iteration.
+   Why: made regressions and improvements comparable across runs.
+
+## Concrete artifacts (sample)
+
+1. Happy-path transcript snippet:
+   Preconditions pass -> execute steps 1..N -> emit structured summary with status per step.
+2. Failure-recovery transcript snippet:
+   Step fails -> classify transient/permanent -> retry once or escalate with captured evidence.
+3. Deterministic report template:
+   Sections: Preconditions, Actions Taken, Validation Results, Failures/Recoveries, Next Actions.
+
+## What made this high quality
+
+1. The workflow was executable without rediscovering steps.
+2. Non-happy paths were first-class, not afterthoughts.
+3. Outputs were structured for consistent review and iteration.

package/src/internal-skills/skill-writer/references/execution-shapes.md

@@ -0,0 +1,68 @@
+# Execution Shapes
+
+Use this guide to choose the runtime shape of a skill before you decide its files.
+Default rule: choose the simplest adequate shape, then add complexity only when it clearly improves outcomes.
+Once you pick a shape, load only the concrete leaf references it needs.
+
+## Defaulting To The Simplest Shape
+
+Start from these questions, in order:
+
+1. Can one coherent set of instructions handle most requests?
+   If yes, prefer `inline-guidance`.
+2. Is the main complexity optional knowledge rather than control flow?
+   If yes, prefer `reference-backed-expert`.
+3. Is the hard part data extraction, validation, or repeatable automation?
+   If yes, prefer `script-backed-workflow`.
+4. Does the user usually invoke the skill with explicit parameters?
+   If yes, add `argument-driven`.
+5. Only then consider routing, worker delegation, evaluator loops, subagent execution, hooks, or templates.
+
+Do not jump to advanced mechanics because they sound powerful.
+
+## Shape Decision Table
+
+| Shape | Use when | Open next | Portability notes |
+|-------|----------|-----------|-------------------|
+| `inline-guidance` | one coherent policy, checklist, or procedure is enough | `references/artifact-layouts/inline-skill-layout.md` | most portable default |
+| `reference-backed-expert` | optional deep knowledge is the main complexity | `references/artifact-layouts/reference-backed-skill-layout.md` | portable if file references stay relative |
+| `script-backed-workflow` | repeated parsing, validation, APIs, or transformations are fragile in plain shell | `references/artifact-layouts/script-backed-skill-layout.md` | portable if dependencies are explicit |
+| `argument-driven` | the skill is usually invoked with issue numbers, paths, targets, or modes | `references/artifact-layouts/argument-driven-skill-layout.md` | often provider-specific beyond basic manual invocation |
+| `router` | distinct categories need different downstream prompts, tools, or references | `references/workflow-mechanics/routing-workflows.md` | portable if routing stays in prompt logic |
+| `parallelization` | independent subtasks or multiple votes improve speed or confidence | `references/workflow-mechanics/parallel-workflows.md` | often implemented with tools or agents |
+| `orchestrator-workers` | the number or type of subtasks is discovered at runtime | `references/workflow-mechanics/orchestrator-workers.md` | usually higher-latency and provider-sensitive |
+| `evaluator-optimizer` | critique-and-revise loops improve output quality materially | `references/workflow-mechanics/evaluator-loops.md` | portable in concept; costly if overused |
+| `subagent-fork` | the skill needs isolated context, tools, or model defaults | `references/claude-code/subagent-fork-skills.md` | Claude Code-specific |
+| `hook-backed` | deterministic enforcement is required beyond prompt guidance | `references/claude-code/hook-backed-skills.md` | highly provider-specific and security-sensitive |
+| `asset-template` | reusable templates, schemas, or static artifacts carry most of the value | `references/artifact-layouts/asset-template-skill-layout.md` | portable if assets are generic files |
+
+If the chosen shape also uses explicit arguments, Claude-specific frontmatter, or shell preprocessing, load the matching file from `references/claude-code/`.
+
+## Secondary Workflow Mechanics
+
+These are not usually primary execution shapes, but they often refine one:
+
+- fixed ordered steps -> `references/workflow-mechanics/prompt-chaining.md`
+- validate-fix-repeat loops -> `references/workflow-mechanics/validation-loops.md`
+- plan-before-execute flows -> `references/workflow-mechanics/plan-validate-execute.md`
+
+## Hybrid Shapes
+
+Use a hybrid only when one primary shape is insufficient.
+
+1. Declare one primary shape.
+2. Add only the minimum secondary shapes needed.
+3. Keep each secondary shape scoped to one concrete need.
+4. Avoid stacking multiple advanced shapes without a clear base path.
+
+## Advanced-Shape Hard Stops
+
+Do not finalize a skill when any of these are true:
+
+1. The chosen shape is implied but not named.
+2. A simpler shape was not considered.
+3. A router has no fallback or default path.
+4. An evaluator loop has no stopping rule.
+5. A subagent-fork skill contains only passive guidance.
+6. A hook-backed skill lacks a security note or fallback behavior.
+7. Provider-specific mechanics are used without portability notes.