vinta-ai-workflows 0.1.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +830 -0
- package/LICENSE +21 -0
- package/README.md +547 -0
- package/package.json +27 -0
- package/schemas/README.md +84 -0
- package/schemas/mcp-preflight-cache.v1.schema.json +50 -0
- package/schemas/prs-context-comments.v1.schema.json +51 -0
- package/schemas/prs-context-frontmatter.v1.schema.json +77 -0
- package/schemas/sub-agent.v1.schema.json +105 -0
- package/schemas/vinta-ai-workflows-config.v1.schema.json +283 -0
- package/skills/vinta-analyze-codebase/SKILL.md +325 -0
- package/skills/vinta-bootstrap-ai-tools/SKILL.md +375 -0
- package/skills/vinta-bootstrap-ai-tools/resources/stacks/README.md +50 -0
- package/skills/vinta-bootstrap-ai-tools/resources/stacks/django/notes.md +52 -0
- package/skills/vinta-bootstrap-ai-tools/resources/stacks/medplum/notes.md +45 -0
- package/skills/vinta-bootstrap-ai-tools/resources/stacks/nextjs-app-router/notes.md +35 -0
- package/skills/vinta-bootstrap-ai-tools/resources/stacks/python-package/notes.md +31 -0
- package/skills/vinta-bootstrap-ai-tools/resources/stacks/typescript-monorepo/notes.md +32 -0
- package/skills/vinta-derive-skills/SKILL.md +200 -0
- package/skills/vinta-derive-skills/resources/amend-plan-template.md +355 -0
- package/skills/vinta-derive-skills/resources/foundation-skills/add-one-off-script/SKILL.md +292 -0
- package/skills/vinta-derive-skills/resources/foundation-skills/add-one-off-script/resources/one_off_script_base.py +602 -0
- package/skills/vinta-derive-skills/resources/foundation-skills/add-one-off-script/resources/one_off_script_base.ts +624 -0
- package/skills/vinta-derive-skills/resources/foundation-skills/create-qa-use-cases/SKILL.md +232 -0
- package/skills/vinta-derive-skills/resources/foundation-skills/create-spec/SKILL.md +235 -0
- package/skills/vinta-derive-skills/resources/foundation-skills/open-pr-from-context/SKILL.md +78 -0
- package/skills/vinta-derive-skills/resources/foundation-skills/open-pr-from-context/scripts/open-pr.sh +357 -0
- package/skills/vinta-derive-skills/resources/foundation-skills/plan-feature/SKILL.md +418 -0
- package/skills/vinta-derive-skills/resources/implement-plan-template.md +375 -0
- package/skills/vinta-derive-skills/resources/prs-context-template.md +91 -0
- package/skills/vinta-derive-skills/resources/systematic-debugging-mcp-tools.md +125 -0
- package/skills/vinta-derive-skills/resources/systematic-debugging-template.md +167 -0
- package/skills/vinta-derive-subagents/SKILL.md +192 -0
- package/skills/vinta-install-ai-tools-setup/SKILL.md +138 -0
- package/skills/vinta-install-ai-tools-setup/resources/setup-ai-tools.mjs +392 -0
- package/skills/vinta-migrate-plans-specs/SKILL.md +229 -0
- package/skills/vinta-sync-ai-tools/SKILL.md +184 -0
- package/skills/vinta-update-project-skills/SKILL.md +178 -0
- package/skills/vinta-write-agents-md/SKILL.md +192 -0
- package/vinta-ai-workflows.mjs +316 -0
|
@@ -0,0 +1,375 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: implement-plan
|
|
3
|
+
description: Execute a phased implementation plan from `{{PLAN_DIR}}/` in {{PROJECT_NAME}} by orchestrating one subagent per phase (using whatever model the plan suggests and the runtime supports), pushing one stacked branch per phase to {{CODE_HOST}}, and tracking progress. Use when the user says "implement the plan", "execute plan X", "start implementation", "run phase N of plan Y", "implement {feature} plan", or asks to drive a `*_IMPLEMENTATION_PLAN.md` file phase-by-phase. NOT for one-off changes, single-file edits, or work that doesn't have an existing plan. {{PR_POLICY_DESCRIPTION}}
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Implement Plan
|
|
7
|
+
|
|
8
|
+
Drive a phased plan in [`{{PLAN_DIR}}/`]({{PLAN_DIR}}/) to completion: spawn one subagent per phase (whichever model plan recommends + runtime can run), run lint / typecheck / unit / e2e (where applicable), push one stacked {{CODE_HOST}} branch per phase, keep a progress tracking file as context handoff between phases. Harness-agnostic — claude-code, OpenAI Codex, Google's runtime, or any framework with a "spawn subagent with model + prompt" primitive.
|
|
9
|
+
|
|
10
|
+
Execution counterpart to [plan-feature](../plan-feature/SKILL.md). Plan = contract; this skill = build pipeline.
|
|
11
|
+
|
|
12
|
+
## Working assumptions
|
|
13
|
+
|
|
14
|
+
- Repo: {{PROJECT_NAME}} ({{STACK_SUMMARY}}). Conventions: [AGENTS.md](AGENTS.md).
|
|
15
|
+
- Plan files: [`{{PLAN_DIR}}/YYYY-MM-DD-FEATURE_NAME_IMPLEMENTATION_PLAN.md`]({{PLAN_DIR}}/).
|
|
16
|
+
- Lint: `{{LINT_CMD}}`. Format: `{{FORMAT_CMD}}`.
|
|
17
|
+
- Type / build gate: `{{BUILD_CMD}}`{{TYPECHECK_NOTE}}.
|
|
18
|
+
- Unit / integration tests: `{{TEST_CMD}}` (everything){{SCOPED_TEST_NOTE}}.
|
|
19
|
+
{{E2E_BLOCK}}
|
|
20
|
+
{{STACK_SPECIFIC_DEPLOY_BLOCK}}
|
|
21
|
+
- Code host: **{{CODE_HOST}}**. {{PR_POLICY_BLOCK}}
|
|
22
|
+
- {{COAUTHOR_POLICY_BLOCK}}
|
|
23
|
+
|
|
24
|
+
## Step 0 — Locate + parse plan
|
|
25
|
+
|
|
26
|
+
Parse once, reuse for every phase:
|
|
27
|
+
|
|
28
|
+
1. **Identify plan file.** Ask user which plan (path or feature name). Feature name: `ls {{PLAN_DIR}}/` + grep; confirm before proceeding.
|
|
29
|
+
2. **Extract structured fields**, in order:
|
|
30
|
+
- **Feature name** + **plan id** — derived from filename's `FEATURE_NAME` portion only: strip `YYYY-MM-DD-` prefix + `_IMPLEMENTATION_PLAN.md` suffix. Kebab variant for branch names.
|
|
31
|
+
- **Goals + Non-goals** section — verbatim, used in every phase prompt.
|
|
32
|
+
- **Guiding Decisions** section — verbatim. Pay attention to: feature flag (key, scope, default, flip-on criterion), storage shape, tenant scoping, API contract decisions.
|
|
33
|
+
- **Data Model Changes** section — keep full body; later phases reference earlier subsections.
|
|
34
|
+
- **Phased Rollout** section — parse into phase records: `{ id, title, goal, body, spec_use_case, suggested_model_tier, reusable_skills, has_e2e, acceptance, is_cross_repo, is_flag_removal }`.
|
|
35
|
+
- **Risk & Rollout Notes**, **Open Questions**, **Touch List** sections — keep available; include in phase prompts only when relevant.
|
|
36
|
+
3. **Classify each phase**: `is_cross_repo`, `is_flag_removal` — orchestrator does NOT auto-execute these.
|
|
37
|
+
4. **Ask the user two opt-in questions** via `AskUserQuestion`. Both default to off; record both answers in tracking under `run_options`:
|
|
38
|
+
|
|
39
|
+
a. **Pause between phases?** *"Do you want me to pause and wait for confirmation after each phase, before starting the next one? Lets you review the diff / branch / PR / tracking summary before moving on."* Options: `Auto-flow (default) — keep going phase to phase`, `Pause between phases — wait for go after each one`.
|
|
40
|
+
|
|
41
|
+
b. **Draft inline review comments per phase?** *"On top of the standard PR description, do you want me to scan each phase's diff and add 3–10 inline comments calling out non-obvious decisions (subtle invariants, feature-flag short-circuits, cross-phase coupling, upstream-contract naming)? Off by default — say yes when reviewers will appreciate annotated diffs."* Options: `Yes — include inline comments`, `No — PR description only`.
|
|
42
|
+
|
|
43
|
+
PR opening itself is **not** asked here — it's governed by the project's PR creation policy captured at bootstrap (see `{{PR_POLICY_BLOCK}}` above). When that policy = "agents create PRs", the [Open PR via context file](#1f-open-pr-via-context-file) step always opens the PR via [open-pr.sh](../foundation-skills/open-pr-from-context/scripts/open-pr.sh) regardless of the comment opt-in.
|
|
44
|
+
|
|
45
|
+
5. **Confirm with user before starting.** Show plan path, phase list (id + title + tier + cross-repo/flag-removal flags + e2e flag), phases this skill will execute vs defer, branch naming pattern (default: `plan/{plan-id-kebab}/phase-{phase-id}`), captured `run_options.pause_between_phases` + `run_options.generate_inline_comments`{{PR_REMINDER_LINE}}.
|
|
46
|
+
|
|
47
|
+
Wait for "go". After that, the per-phase pause behavior follows `run_options.pause_between_phases`. Inline-comment drafting follows `run_options.generate_inline_comments`.
|
|
48
|
+
|
|
49
|
+
## Step 1 — Per-phase loop
|
|
50
|
+
|
|
51
|
+
For each phase that's `not is_cross_repo and not is_flag_removal`, in plan order:
|
|
52
|
+
|
|
53
|
+
### 1a. Prepare agent prompt (token-efficient)
|
|
54
|
+
|
|
55
|
+
Compose with **only what the agent needs**:
|
|
56
|
+
|
|
57
|
+
```
|
|
58
|
+
You are implementing {phase.id}: {phase.title} of plan {plan.id}.
|
|
59
|
+
|
|
60
|
+
## Repo
|
|
61
|
+
{{PROJECT_NAME}} ({{STACK_SUMMARY}}).
|
|
62
|
+
|
|
63
|
+
## Read first
|
|
64
|
+
1. AGENTS.md — repo conventions.
|
|
65
|
+
2. {{PLAN_DIR}}/{plan-filename}, the **Goals + Non-goals**, **Guiding Decisions**, **Data Model Changes** sections and YOUR phase body inside **Phased Rollout**.
|
|
66
|
+
|
|
67
|
+
## Plan-level decisions (from Goals + Non-goals + Guiding Decisions)
|
|
68
|
+
{Goals + Non-goals verbatim}
|
|
69
|
+
{Guiding Decisions table verbatim}
|
|
70
|
+
{If feature flag declared:}
|
|
71
|
+
Feature flag: `{flag-key}` — scope `{per-tenant|per-request}`, default `{false|true}`.
|
|
72
|
+
Wire reads + writes per the plan's **Guiding Decisions** entry. Off-flag path = byte-for-byte pre-feature behavior.
|
|
73
|
+
|
|
74
|
+
## What was already implemented in prior phases
|
|
75
|
+
{Tracking file "Completed Phases" section. First executed phase: "Nothing yet — this is the first phase."}
|
|
76
|
+
|
|
77
|
+
## Your tasks (Phase {id} only)
|
|
78
|
+
{phase.body verbatim, including Goal / Spec use-case / Feature flag / Changes / Tests / Acceptance lines}
|
|
79
|
+
|
|
80
|
+
## Reusable skills you SHOULD invoke
|
|
81
|
+
{phase.reusable_skills — for each, instruct the agent to first read ai-tools/skills/{name}/SKILL.md, then follow that pattern.}
|
|
82
|
+
|
|
83
|
+
Project skills available: {{PROJECT_SKILLS_LIST}}
|
|
84
|
+
|
|
85
|
+
## Working instructions
|
|
86
|
+
1. Read existing code paths your changes touch — do not write before reading.
|
|
87
|
+
2. Implement using Read/Edit/Write. Match existing patterns.
|
|
88
|
+
3. **Inner loop — fast iteration.** Scoped to files/apps you touched:
|
|
89
|
+
a. `{{LINT_CMD}}` until clean.
|
|
90
|
+
b. {{NEW_TEST_CMD_PATTERN}} for new tests individually.
|
|
91
|
+
c. Scoped suite: {{SCOPED_TEST_PATTERN}}.
|
|
92
|
+
4. Iterate 2–3 until **new tests pass individually** and the scoped suite is green. Do **not** advance to step 5 with red scoped tests.
|
|
93
|
+
5. **Outer gate — full local verification, only after step 4 is green.** All MUST pass before staging:
|
|
94
|
+
a. **Type / build:** `{{BUILD_CMD}}`.
|
|
95
|
+
b. **Full test suite:** `{{TEST_CMD}}`.
|
|
96
|
+
{{E2E_OUTER_GATE_LINE}}
|
|
97
|
+
6. Outer gate fails → return step 2 (fix regression), re-run inner loop, then 5a/5b/5c. **Never** commit, push, or proceed while any gate is red.
|
|
98
|
+
7. Stage right files (NEVER `git add -A` — {{ANTI_GIT_ADD_ALL_REASON}}). Stage explicitly: `git add {{STAGE_PATTERN}}`.
|
|
99
|
+
8. Commit with the repo's style — look at `git log -10 --oneline` first. {{COMMIT_STYLE_LINE}}.
|
|
100
|
+
9. {{COAUTHOR_INSTRUCTION_LINE}}
|
|
101
|
+
10. {{PUSH_INSTRUCTION_LINE}}
|
|
102
|
+
|
|
103
|
+
## Required output (single final report)
|
|
104
|
+
- Status: SUCCESS or FAILURE (and why).
|
|
105
|
+
- Files created/modified (paths only).
|
|
106
|
+
- 5–15 line summary of what you implemented and key decisions.
|
|
107
|
+
{{E2E_REPORT_FIELD}}
|
|
108
|
+
- Deviations from the plan body and reasoning.
|
|
109
|
+
- Anything you couldn't do (with explanation).
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
**Don't** dump the full plan into every prompt. Tracking summaries replace prior phases as context. Always include the **Goals + Non-goals** and **Guiding Decisions** sections plus the relevant **Data Model Changes** subsection — load-bearing decisions; phases reach back frequently.
|
|
113
|
+
|
|
114
|
+
### 1b. Pick model from plan's per-phase suggestion
|
|
115
|
+
|
|
116
|
+
**Plan owns model selection — this skill does not re-derive tiers, doesn't assume vendor.** Each phase carries `**Suggested AI model**:` listing one model per vendor.
|
|
117
|
+
|
|
118
|
+
Pick:
|
|
119
|
+
|
|
120
|
+
1. Read line, parse out **all** vendor suggestions.
|
|
121
|
+
2. **Filter to what's actually available in the runtime.** Different harnesses expose different sets.
|
|
122
|
+
3. From surviving suggestions, **pick the cheapest / fastest** the runner can use.
|
|
123
|
+
4. Translate the chosen model to whatever form the runner's spawning tool expects.
|
|
124
|
+
5. Phase suggestion straddles tiers → pick the higher-tier suggestion.
|
|
125
|
+
6. Line missing / malformed → **ask the user**. Don't silently re-derive tier.
|
|
126
|
+
|
|
127
|
+
**Retry escalation (no user prompt):** picked model fails on a clear capability gap → step **one tier up** + retry once. After Tier 4 fails, STOP. Update tracking with `❌`, post the agent's report to the user, ask how to proceed.
|
|
128
|
+
|
|
129
|
+
Record the **model actually used** + the **plan's suggested tier** in tracking.
|
|
130
|
+
|
|
131
|
+
### 1c. Spawn subagent
|
|
132
|
+
|
|
133
|
+
Use whatever agent-spawning primitive the runtime exposes. Pass:
|
|
134
|
+
|
|
135
|
+
- Descriptive label (e.g. `"{plan.id} {phase.id}: {phase.title}"`).
|
|
136
|
+
- Model from the [Pick model from plan's per-phase suggestion](#1b-pick-model-from-plans-per-phase-suggestion) step, translated.
|
|
137
|
+
- Phase prompt from the [Prepare agent prompt](#1a-prepare-agent-prompt-token-efficient) step.
|
|
138
|
+
- The right **agent type**.
|
|
139
|
+
|
|
140
|
+
**Agent type per phase.** Project agents in [`ai-tools/agents/`](ai-tools/agents/) (exposed to claude-code via `.claude/agents` symlink):
|
|
141
|
+
|
|
142
|
+
{{AGENT_DISPATCH_TABLE}}
|
|
143
|
+
|
|
144
|
+
Phase combines shapes → agent type stays `implementer`, prompt lists every relevant SKILL.md. Agent type changes only when a stack-specialist's risk is the primary one.
|
|
145
|
+
|
|
146
|
+
**Avoid bouncing the same phase between multiple agents.** Wanting to "hand off" mid-phase → the plan should have split into sub-phases instead.
|
|
147
|
+
|
|
148
|
+
### 1d. Thorough review
|
|
149
|
+
|
|
150
|
+
Three layers, all required, in order. The orchestrator never edits — every issue surfaces as a fix-up subagent task.
|
|
151
|
+
|
|
152
|
+
#### Layer 1 — Mechanical checks
|
|
153
|
+
|
|
154
|
+
1. `git status` + `git diff --stat`: confirm file list matches the agent's report.
|
|
155
|
+
2. **Read the full diff** for every changed file using `git diff`. Spot-checking is not enough.
|
|
156
|
+
3. **Verify the outer gate** ran + green. Look in the report for explicit confirmation that `{{BUILD_CMD}}` AND `{{TEST_CMD}}` were executed + passed{{E2E_LAYER1_NOTE}}. Vague confirmation → **re-run yourself**.
|
|
157
|
+
4. **Scope creep**: file touched outside expected surface area? Unrelated formatting churn? Surface it.
|
|
158
|
+
5. **No-secrets scan**: `git diff` for `password|secret|token|api_key|AKIA|BEGIN [A-Z]+ KEY`.
|
|
159
|
+
{{COAUTHOR_LAYER1_CHECK}}
|
|
160
|
+
|
|
161
|
+
#### Layer 2 — Plan compliance walkthrough
|
|
162
|
+
|
|
163
|
+
Open phase body alongside diff and walk:
|
|
164
|
+
|
|
165
|
+
1. **Every numbered "Changes" item implemented.**
|
|
166
|
+
2. **Every "Tests" entry materialized**, with assertions actually exercising the called-out behavior.
|
|
167
|
+
3. **Acceptance line satisfiable** by the diff.
|
|
168
|
+
4. **Repo conventions** from AGENTS.md.
|
|
169
|
+
5. **Reusable-skill compliance.**
|
|
170
|
+
{{E2E_LAYER2_CHECK}}
|
|
171
|
+
6. **Feature-flag wiring** if the plan's **Guiding Decisions** declared a flag — flag-OFF byte-for-byte pre-feature behavior, ≥1 test asserts.
|
|
172
|
+
7. **Cross-phase consistency** with prior tracking summaries.
|
|
173
|
+
|
|
174
|
+
#### Layer 3 — Independent reviewer subagent
|
|
175
|
+
|
|
176
|
+
After Layers 1–2 pass, spawn a **separate** subagent (different session, no implementation context) using the project's `reviewer` agent type ([ai-tools/agents/reviewer.md](ai-tools/agents/reviewer.md)). Read-only by design.
|
|
177
|
+
|
|
178
|
+
Reviewer prompt template — see the reviewer agent's body for the standard form. Triage findings:
|
|
179
|
+
- **BLOCKER**: must fix before the **{{BRANCH_PUSH_HEADING}}** step below.
|
|
180
|
+
- **SHOULD-FIX**: fix in-phase if cheap; else follow-up issue + tracking note.
|
|
181
|
+
- **NIT**: ignore unless trivially cheap.
|
|
182
|
+
|
|
183
|
+
Reviewer finds nothing on a >300-LoC multi-file phase → suspicious. Read once more.
|
|
184
|
+
|
|
185
|
+
#### Fix loop
|
|
186
|
+
|
|
187
|
+
1. Spawn a **new** subagent — project's `fixer` agent type ([ai-tools/agents/fixer.md](ai-tools/agents/fixer.md)). Fix prompt quotes the finding verbatim.
|
|
188
|
+
2. The `fixer`'s system prompt mandates re-running the inner loop + outer gate.
|
|
189
|
+
3. After fixer returns, redo Layer 1 in full + the affected portion of Layer 2.
|
|
190
|
+
4. Loop until Layers 1, 2, 3 are clean.
|
|
191
|
+
|
|
192
|
+
### 1e. {{BRANCH_PUSH_HEADING}}
|
|
193
|
+
|
|
194
|
+
Branch naming: `plan/{plan-id-kebab}/phase-{phase.id}`.
|
|
195
|
+
|
|
196
|
+
**First executed phase** (branches from `{{DEFAULT_BRANCH}}`):
|
|
197
|
+
```bash
|
|
198
|
+
git checkout {{DEFAULT_BRANCH}}
|
|
199
|
+
git pull --ff-only
|
|
200
|
+
git checkout -b plan/{plan-id-kebab}/phase-{phase.id}
|
|
201
|
+
# subagent's commits land on this branch
|
|
202
|
+
git push -u origin plan/{plan-id-kebab}/phase-{phase.id}
|
|
203
|
+
```
|
|
204
|
+
|
|
205
|
+
**Subsequent phases** (stacked on the previous phase's branch):
|
|
206
|
+
```bash
|
|
207
|
+
git checkout plan/{plan-id-kebab}/phase-{prev.id}
|
|
208
|
+
git checkout -b plan/{plan-id-kebab}/phase-{phase.id}
|
|
209
|
+
git push -u origin plan/{plan-id-kebab}/phase-{phase.id}
|
|
210
|
+
```
|
|
211
|
+
|
|
212
|
+
PR opening lives in the [Open PR via context file](#1f-open-pr-via-context-file) step below (single flow — context file + `open-pr.sh`). Subagents never open PRs themselves; the orchestrator does, after review passes.
|
|
213
|
+
|
|
214
|
+
### 1f. Open PR via context file
|
|
215
|
+
|
|
216
|
+
This is the **only** PR-creation path. PRs always go through a `.vinta-ai-workflows/prs-context/{feature-kebab}/phase-{phase.id}.md` file + the bundled [open-pr.sh](../foundation-skills/open-pr-from-context/scripts/open-pr.sh) script — even when inline comments are not requested. The file is the durable record; the script is the publisher.
|
|
217
|
+
|
|
218
|
+
Two project-level signals decide the actual behavior:
|
|
219
|
+
|
|
220
|
+
| `{{PR_POLICY_BLOCK}}` policy | `run_options.generate_inline_comments` | What this **Open PR via context file** step does |
|
|
221
|
+
|---|---|---|
|
|
222
|
+
| agents create PRs | false | Write minimal context file (`# Title`, `# Description`, empty `# Comments`). Run `open-pr.sh` → PR opened, no inline comments. |
|
|
223
|
+
| agents create PRs | true | Write full context file (title + description + 3–10 inline comments). Run `open-pr.sh` → PR opened, all comments posted. |
|
|
224
|
+
| branches only | false | **Skip this step entirely.** Human will open PR manually from the pushed branch. |
|
|
225
|
+
| branches only | true | Write full context file (durable record). **Don't run `open-pr.sh`.** Human can publish later from a CLI-equipped session via [open-pr-from-context](../foundation-skills/open-pr-from-context/SKILL.md). Surface this in the [Send brief update](#1h-send-brief-update-to-user) step. |
|
|
226
|
+
|
|
227
|
+
#### Steps
|
|
228
|
+
|
|
229
|
+
1. **Skip if neither column applies** (policy = branches only AND `generate_inline_comments = false`). Jump to the [Update tracking file](#1g-update-tracking-file) step.
|
|
230
|
+
|
|
231
|
+
2. **Honor existing PR / MR templates.** Read `project.pr_template_paths` from `.vinta-ai-workflows.yaml`. For each entry:
|
|
232
|
+
- **One template** → load it; the prs-context `# Description` body must follow that template's section structure verbatim. Fill each section with phase-specific content drawn from the plan's **Goals + Non-goals**, **Guiding Decisions**, and the phase body. Preserve any `<!-- HTML comments -->` placeholders; do not strip the template's checklists. Sections you can't fill from phase data → leave the template's placeholder/prompt untouched (don't fabricate).
|
|
233
|
+
- **Multiple templates** (`PULL_REQUEST_TEMPLATE/` directory) → ask once via `AskUserQuestion`: list each template + its filename, ask which to use for this run. Cache the choice in tracking under `run_options.pr_template_used` so subsequent phases of the same plan use the same one without re-asking.
|
|
234
|
+
- **Empty array** → free-form description. Default sections: `## Summary` (1–3 sentences), `## Plan reference` (link / phase id), `## Test plan` (commands the reviewer can run).
|
|
235
|
+
|
|
236
|
+
When the project's PR template includes a checkbox checklist (e.g. `- [ ] Tests added`, `- [ ] Docs updated`), tick the boxes the phase's diff actually satisfies and leave unsatisfied ones unticked — never auto-tick everything.
|
|
237
|
+
|
|
238
|
+
GitHub also honors `?template=<name>` in the PR-create URL when the project has a multi-template directory. `gh pr create --body-file` writes the body directly so the URL trick isn't needed; the body must match the chosen template's structure regardless.
|
|
239
|
+
|
|
240
|
+
3. **Pick comment targets** (only when `generate_inline_comments = true`). Read the phase diff via `git diff {{DEFAULT_BRANCH}}...HEAD` (or the previous phase branch for stacked phases). Select 3–10 spots that benefit from a one-paragraph context note — typically:
|
|
241
|
+
- A subtle invariant the diff relies on (cite the plan's **Goals + Non-goals** / **Guiding Decisions** entries — by name, never use `§` shorthand).
|
|
242
|
+
- A workaround for a known framework / library limitation.
|
|
243
|
+
- A naming choice driven by an upstream contract.
|
|
244
|
+
- The off-flag short-circuit when a feature flag is in **Guiding Decisions**.
|
|
245
|
+
- Why a seemingly-cleaner refactor wasn't made (out of scope per **Goals + Non-goals**).
|
|
246
|
+
- Cross-phase coupling (this hook is consumed by phase N+k).
|
|
247
|
+
|
|
248
|
+
Skip lint/format churn, boilerplate matching nearby files, standard patterns from AGENTS.md, and self-explanatory test names. **A clean phase produces few comments — that's fine. Don't pad.**
|
|
249
|
+
|
|
250
|
+
When `generate_inline_comments = false`: skip this step. The file's `# Comments` block stays empty.
|
|
251
|
+
|
|
252
|
+
4. **Write `.vinta-ai-workflows/prs-context/{feature-kebab}/phase-{phase.id}.md`** following [resources/prs-context-template.md](../prs-context-template.md). Frontmatter: `plan_id`, `feature_name`, `phase_id`, `phase_title`, `branch`, `base`, `created_at`, `status: pending`, empty `pr_url`. Body sections: `# Title` (single-line PR title), `# Description` (Markdown body — uses the project's PR template structure from step 2 when one exists), `# Comments` (YAML list of `{file, start_line, end_line?, side, body}` — empty list when comments are off).
|
|
253
|
+
|
|
254
|
+
5. **Confirm `.vinta-ai-workflows/prs-context/` is in `.gitignore`.** [vinta-install-ai-tools-setup](../../../vinta-install-ai-tools-setup/SKILL.md) runs the multi-vendor setup script which appends `.vinta-ai-workflows/prs-context/` on its first invocation. If an older bootstrap missed it, append it now.
|
|
255
|
+
|
|
256
|
+
6. **Run `open-pr.sh`** (only when policy = agents create PRs). Detect a usable CLI (`gh` for GitHub, `glab` for GitLab) plus the script's other deps (`yq`, `jq`):
|
|
257
|
+
|
|
258
|
+
```bash
|
|
259
|
+
bash ai-tools/skills/open-pr-from-context/scripts/open-pr.sh .vinta-ai-workflows/prs-context/{feature-kebab}/phase-{phase.id}.md
|
|
260
|
+
```
|
|
261
|
+
|
|
262
|
+
Script opens the PR (or detects an existing one), posts each inline comment, rewrites the file's frontmatter to `status: published` + populated `pr_url`, appends a publish log. Exit codes:
|
|
263
|
+
|
|
264
|
+
- `0` — PR up, all comments (if any) posted. Capture `pr_url` for the [Send brief update](#1h-send-brief-update-to-user) step.
|
|
265
|
+
- `1` — PR up, ≥1 comment failed. Surface the failed `(file:line)` list to the user; continue to the [Update tracking file](#1g-update-tracking-file) step.
|
|
266
|
+
- `2` — Hard failure (deps missing, branch not pushed, CLI unauthed, file invalid). Surface the script's stderr; treat the phase as having no PR. The file stays `status: pending` so the user can re-run after fixing the gap.
|
|
267
|
+
|
|
268
|
+
When policy = "branches only": **don't run the script.** File stays `status: pending`.
|
|
269
|
+
|
|
270
|
+
7. **Skill wrapper** — [open-pr-from-context](../foundation-skills/open-pr-from-context/SKILL.md) is available for ad-hoc invocation (after the run, on a different machine, etc.). The orchestrator can call the script directly here; the skill is for humans.
|
|
271
|
+
|
|
272
|
+
### 1g. Update tracking file
|
|
273
|
+
|
|
274
|
+
Tracking lives at `{{PLAN_DIR}}/TRACKING_{plan-id}.md`. Commit on the **current** phase's branch — deletion in Step 3.
|
|
275
|
+
|
|
276
|
+
Schema: feature-name, plan path, started/last-updated dates, optional feature-flag info, completed-phases (with status, model, branch, base, e2e+screenshots if any, 5–15 line summary), current phase, remaining phases, deferred phases.
|
|
277
|
+
|
|
278
|
+
The orchestrator writes this from the git diff + the agent's summary — not from the agent's narration.
|
|
279
|
+
|
|
280
|
+
### 1h. Send brief update to user
|
|
281
|
+
|
|
282
|
+
One short paragraph: phase N done, branch pushed{{PR_LINK_NOTE}}, what got built, and — when the [Open PR via context file](#1f-open-pr-via-context-file) step ran — the PR-context file path with its `status` (`published` + URL when `open-pr.sh` opened the PR; `pending` when the script wasn't run because PR policy = branches only or deps were missing). When `status: pending`, mention how to publish later (`bash ai-tools/skills/open-pr-from-context/scripts/open-pr.sh <path>`). Moving to phase N+1. No long retrospective — tracking file is the durable record.
|
|
283
|
+
|
|
284
|
+
### 1i. Per-phase pause gate (opt-in)
|
|
285
|
+
|
|
286
|
+
`run_options.pause_between_phases = false` (default) → **immediately spawn the next phase**. Do not wait.
|
|
287
|
+
|
|
288
|
+
`run_options.pause_between_phases = true` → ask the user via `AskUserQuestion`:
|
|
289
|
+
|
|
290
|
+
- `Continue — start phase N+1`
|
|
291
|
+
- `Pause — stop here, I'll resume later by re-invoking the skill` (orchestrator exits cleanly; tracking file already records progress so the next invocation resumes mid-plan per the "Re-running mid-plan" section).
|
|
292
|
+
- `Stop — abort the plan run` (orchestrator stops; user decides next steps manually).
|
|
293
|
+
|
|
294
|
+
Wait for the answer. Don't spawn anything in the meantime. The pause is the user's review window — they may inspect the diff, the branch, the PR-context file, or the tracking file before agreeing to continue.
|
|
295
|
+
|
|
296
|
+
## Cross-repo phases
|
|
297
|
+
|
|
298
|
+
Phase in another repo:
|
|
299
|
+
1. **Do not implement.**
|
|
300
|
+
2. Mark in tracking under "Deferred Phases".
|
|
301
|
+
3. Continue to the next in-repo phase. Don't block on cross-repo work.
|
|
302
|
+
|
|
303
|
+
## Flag-removal phase (always out of scope)
|
|
304
|
+
|
|
305
|
+
Plan declared a flag → last phase is `Phase N — Remove the {flag-key} feature flag`. This skill **never** executes that phase. Flag removal is gated on real-world soak signal + is the exclusive responsibility of a dedicated flag-removal skill (separate skill).
|
|
306
|
+
|
|
307
|
+
What this skill does instead:
|
|
308
|
+
1. Identify the phase during Step 0; always exclude.
|
|
309
|
+
2. Mark in tracking as deferred.
|
|
310
|
+
3. End the run with a `/schedule` offer pointing at the dedicated flag-removal skill.
|
|
311
|
+
4. Refuse + redirect if user asks this skill to remove the flag.
|
|
312
|
+
|
|
313
|
+
## Re-running mid-plan
|
|
314
|
+
|
|
315
|
+
User invokes the skill against a partially-done plan:
|
|
316
|
+
|
|
317
|
+
1. Read `{{PLAN_DIR}}/TRACKING_{plan-id}.md` if present.
|
|
318
|
+
2. `git branch -a | grep plan/{plan-id-kebab}` to detect already-pushed phase branches.
|
|
319
|
+
3. Cross-reference with the plan's phase list.
|
|
320
|
+
4. Confirm resumption point with the user.
|
|
321
|
+
|
|
322
|
+
## Step 2 — Final report
|
|
323
|
+
|
|
324
|
+
After all executable phases complete:
|
|
325
|
+
|
|
326
|
+
1. **Delete `TRACKING_{plan-id}.md`** on the last phase's branch. Commit. The plan file stays.
|
|
327
|
+
2. Send the user a final summary: branches pushed (with bases, in stack order); for UI-flow phases — list of `pr-screenshots/` files (if applicable); deferred phases (cross-repo + flag-removal); next steps for the human.
|
|
328
|
+
{{FINAL_REPORT_PR_NOTE}}
|
|
329
|
+
3. Flag-removal phase deferred → end with `/schedule` offer for the dedicated flag-removal skill.
|
|
330
|
+
|
|
331
|
+
## Important rules
|
|
332
|
+
|
|
333
|
+
- **Read AGENTS.md** in every phase prompt.
|
|
334
|
+
- **Stage explicitly.** No `git add -A`.
|
|
335
|
+
- **Subagents work in fresh sessions.** Each phase = a new subagent. Tracking + plan files = the context handoff.
|
|
336
|
+
- **Orchestrator owns git topology.** Subagents commit but never branch, push, {{PR_RULE_TAIL}}.
|
|
337
|
+
{{COAUTHOR_RULE_LINE}}
|
|
338
|
+
- **Trust the plan's per-phase model suggestion.**
|
|
339
|
+
- **Don't re-implement what a project skill encodes.**
|
|
340
|
+
{{UI_E2E_RULE_LINE}}
|
|
341
|
+
- **Two-tier verification, in order, every phase.** Inner scoped, outer full repo.
|
|
342
|
+
- **Three-layer review, every phase, no exceptions.**
|
|
343
|
+
- **Orchestrator never edits code.**
|
|
344
|
+
- **Feature flags = gates, not toggles for tests.**
|
|
345
|
+
- **Never remove a feature flag from this skill.**
|
|
346
|
+
- **Stop on Tier-4 failure.**
|
|
347
|
+
- **Honor opt-in flags.** `run_options.pause_between_phases` controls the [Per-phase pause gate](#1i-per-phase-pause-gate-opt-in); `run_options.generate_inline_comments` controls whether the [Open PR via context file](#1f-open-pr-via-context-file) step drafts inline comments (always writes the file when that step runs at all — empty comments when off).
|
|
348
|
+
- **PR-context file + `open-pr.sh` is the only PR-creation path.** No raw `gh pr create` / `glab mr create` calls outside the bundled script. The file is durable; the script is the publisher.
|
|
349
|
+
- **[Open PR via context file](#1f-open-pr-via-context-file) gating** = combination of project PR policy ({{PR_POLICY_BLOCK}}) and `generate_inline_comments`. See the matrix in that step. Skip it entirely only when policy = branches only AND comments = off.
|
|
350
|
+
- **Never use `§N` shorthand to point at sections** — neither in this skill body nor in any rendered file (tracking, prs-context, branch description). Always use the section's full name with a markdown link when possible. `§N` references are hard to read for humans and brittle when section numbering shifts.
|
|
351
|
+
|
|
352
|
+
## Quick checklist (orchestrator, per phase)
|
|
353
|
+
|
|
354
|
+
- [ ] Plan parsed; structured fields cached.
|
|
355
|
+
- [ ] Cross-repo + flag-removal phases identified + deferred.
|
|
356
|
+
- [ ] Current phase: prompt composed with **Goals + Non-goals** + **Guiding Decisions** + relevant **Data Model Changes** subsection + tracking summaries + this phase's body.
|
|
357
|
+
- [ ] Model picked from `**Suggested AI model**:` line (cheapest available); plan tier recorded.
|
|
358
|
+
- [ ] Subagent spawned, report received.
|
|
359
|
+
- [ ] Inner loop green: scoped lint + new tests individually + scoped suite.
|
|
360
|
+
- [ ] **Outer gate green:** `{{BUILD_CMD}}` AND `{{TEST_CMD}}`{{E2E_OUTER_GATE_CHECKLIST}} both passed.
|
|
361
|
+
- [ ] Layer 1 review: full diff read; no scope creep; no secrets; outer gate confirmed{{COAUTHOR_CHECKLIST_NOTE}}.
|
|
362
|
+
- [ ] Layer 2 review: every "Changes" ticked; every "Tests" materialized; acceptance line satisfiable; conventions, reusable skills, e2e + screenshot compliance (if applicable), flag wiring all checked.
|
|
363
|
+
- [ ] Layer 3 review: adversarial review run; BLOCKERs fixed; SHOULD-FIX either fixed or noted.
|
|
364
|
+
- [ ] After any fix-up: Layers 1 + 2 + outer gate re-run.
|
|
365
|
+
- [ ] Stacked branch created; pushed.{{PR_CHECKLIST_NOTE}}
|
|
366
|
+
- [ ] **Open PR via context file** decision applied per matrix (PR policy + `generate_inline_comments`):
|
|
367
|
+
- [ ] PR-context file written when at least one of policy=create / comments=true holds.
|
|
368
|
+
- [ ] `open-pr.sh` run when policy=create AND deps available; PR URL captured.
|
|
369
|
+
- [ ] Per-comment failures (exit 1) surfaced with `(file:line)` list.
|
|
370
|
+
- [ ] Hard failure (exit 2) surfaced; file left `status: pending`.
|
|
371
|
+
- [ ] `TRACKING_{plan-id}.md` updated.
|
|
372
|
+
- [ ] One-paragraph user update sent (PR URL or pending-file path included).
|
|
373
|
+
- [ ] If `run_options.pause_between_phases = true`: prompted user (`Continue` / `Pause` / `Stop`); honored answer.
|
|
374
|
+
- [ ] If `run_options.pause_between_phases = false`: next phase started immediately.
|
|
375
|
+
- [ ] On final phase: tracking file deleted; final summary lists branches{{FINAL_CHECKLIST_PR_NOTE}}; any `status: pending` PR-context files listed with publish command; `/schedule` offer for flag-removal if applicable.
|
|
@@ -0,0 +1,91 @@
|
|
|
1
|
+
---
|
|
2
|
+
# yaml-language-server: $schema=./node_modules/vinta-ai-workflows/schemas/prs-context-frontmatter.v1.schema.json
|
|
3
|
+
schema_version: 1 # see schemas/prs-context-frontmatter.v1.schema.json
|
|
4
|
+
plan_id: <plan-id> # filename feature portion, kebab-case
|
|
5
|
+
feature_name: <FEATURE_NAME> # UPPERCASE_WITH_UNDERSCORES, matches plan/spec
|
|
6
|
+
phase_id: <phase-id> # e.g. "1", "4a"
|
|
7
|
+
phase_title: <phase title> # verbatim from the plan's Phased Rollout section
|
|
8
|
+
branch: plan/<feature-kebab>/phase-<id> # branch the PR opens from
|
|
9
|
+
base: <main | plan/<feature-kebab>/phase-<prev-id>> # target branch for the PR
|
|
10
|
+
created_at: <ISO 8601 timestamp>
|
|
11
|
+
status: pending # `pending` until published; `published` after CLI run
|
|
12
|
+
pr_url: # set by open-pr-from-context after publishing
|
|
13
|
+
---
|
|
14
|
+
|
|
15
|
+
# Title
|
|
16
|
+
|
|
17
|
+
<single-line PR title — follow project commit style; keep ≤72 chars>
|
|
18
|
+
|
|
19
|
+
# Description
|
|
20
|
+
|
|
21
|
+
<Markdown body. **Body shape is dictated by `project.pr_template_paths` in
|
|
22
|
+
`.vinta-ai-workflows.yaml`:**
|
|
23
|
+
|
|
24
|
+
- **One template path** → follow that template's section structure verbatim.
|
|
25
|
+
Fill each section with phase-specific content. Preserve every
|
|
26
|
+
`<!-- HTML comment -->` placeholder. Tick checklist items the diff actually
|
|
27
|
+
satisfies; leave the rest unticked. Don't strip sections you can't fill —
|
|
28
|
+
leave the template's prompt untouched.
|
|
29
|
+
- **Multiple template paths** (project has a `PULL_REQUEST_TEMPLATE/` directory
|
|
30
|
+
on GitHub or `merge_request_templates/` on GitLab) → the orchestrator picks
|
|
31
|
+
one (cached in tracking under `run_options.pr_template_used`) and uses its
|
|
32
|
+
structure here.
|
|
33
|
+
- **Empty array** (no project template) → free-form. Default sections:
|
|
34
|
+
- `## Summary` (1–3 sentences).
|
|
35
|
+
- `## Plan reference` (link to `{{PLAN_DIR}}/<plan-file>` + phase id).
|
|
36
|
+
- `## Test plan` (commands the reviewer can run).
|
|
37
|
+
|
|
38
|
+
In every case, cover:
|
|
39
|
+
|
|
40
|
+
- Phase goal in one line.
|
|
41
|
+
- Decisions that aren't obvious from the diff (cite the plan's **Goals + Non-goals** / **Guiding Decisions** entries — by name, never with `§` shorthand).
|
|
42
|
+
- Feature-flag behavior (off-flag = pre-feature, per the plan's **Guiding Decisions** entry).
|
|
43
|
+
- Anything reviewers will ask about that the diff doesn't answer.
|
|
44
|
+
|
|
45
|
+
Don't restate every diff — that's what `git diff` is for.>
|
|
46
|
+
|
|
47
|
+
# Comments
|
|
48
|
+
|
|
49
|
+
The agent picks **non-obvious** spots in the diff that benefit from a one-paragraph
|
|
50
|
+
context note — typically 3–10 per phase. Skip everything that's already obvious
|
|
51
|
+
from the diff itself or from AGENTS.md conventions.
|
|
52
|
+
|
|
53
|
+
YAML inside this fence is validated against
|
|
54
|
+
[`schemas/prs-context-comments.v1.schema.json`](../../../../schemas/prs-context-comments.v1.schema.json).
|
|
55
|
+
|
|
56
|
+
```yaml
|
|
57
|
+
# yaml-language-server: $schema=../../../../node_modules/vinta-ai-workflows/schemas/prs-context-comments.v1.schema.json
|
|
58
|
+
- file: <relative path from repo root>
|
|
59
|
+
start_line: <line number on the new side>
|
|
60
|
+
end_line: <optional; omit for single-line>
|
|
61
|
+
side: RIGHT # RIGHT = new code (default). LEFT = pre-change code (rare).
|
|
62
|
+
body: |
|
|
63
|
+
<1–3 lines. Why this code looks the way it does. Reference plan/spec section
|
|
64
|
+
when the decision lives there. Don't restate the code — explain the constraint.>
|
|
65
|
+
|
|
66
|
+
- file: <relative path>
|
|
67
|
+
start_line: <number>
|
|
68
|
+
body: |
|
|
69
|
+
<next comment...>
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
## What counts as "non-obvious"
|
|
73
|
+
|
|
74
|
+
- A subtle invariant the diff relies on (e.g. "this query is intentionally not
|
|
75
|
+
inside the tenant filter — see the **Guiding Decisions** row on tenant scoping").
|
|
76
|
+
- A workaround for a known framework bug or library limitation.
|
|
77
|
+
- A naming choice driven by an upstream contract (don't rename, will break X).
|
|
78
|
+
- The off-flag short-circuit when a feature flag is in **Guiding Decisions**.
|
|
79
|
+
- Why a seemingly-cleaner refactor wasn't made (out of scope per **Goals + Non-goals**).
|
|
80
|
+
- Cross-phase coupling (this hook will be consumed by phase 3).
|
|
81
|
+
|
|
82
|
+
## Never use `§N` shorthand
|
|
83
|
+
|
|
84
|
+
Plan sections have **names** (`Goals + Non-goals`, `Guiding Decisions`, `Data Model Changes`, `Phased Rollout`, `Risk & Rollout Notes`, `Open Questions`, `Touch List`). Use the names. `§1`, `§2.3`, `§5` etc. force the reader to flip back to the plan and count headings — that's friction. Title + section name is unambiguous and survives renumbering.
|
|
85
|
+
|
|
86
|
+
## What does NOT need a comment
|
|
87
|
+
|
|
88
|
+
- Lint / format changes.
|
|
89
|
+
- Boilerplate matching nearby files.
|
|
90
|
+
- Standard patterns documented in AGENTS.md.
|
|
91
|
+
- Test bodies whose names already describe the assertion.
|
|
@@ -0,0 +1,125 @@
|
|
|
1
|
+
# Observability evidence categories (MCP-agnostic)
|
|
2
|
+
|
|
3
|
+
Reference consumed by [vinta-derive-skills](../SKILL.md) when rendering [systematic-debugging-template.md](systematic-debugging-template.md). The systematic-debugging skill needs evidence from the platform before any hypothesis, but the **specific MCP tool names, function signatures, and parameters change too fast to hardcode**. This file describes the *categories of evidence* the skill demands plus the runtime discovery flow that finds the right calls in whatever MCP servers the project has connected today.
|
|
4
|
+
|
|
5
|
+
## Render contract
|
|
6
|
+
|
|
7
|
+
The renderer of [systematic-debugging-template.md](systematic-debugging-template.md) substitutes:
|
|
8
|
+
|
|
9
|
+
- `{{OBSERVABILITY_MCP_LIST}}` — comma-separated list of the **MCP server identifiers** the project marked as available in `skills.systematic-debugging.observability_mcp_servers` of `.vinta-ai-workflows.yaml`. Free-form strings — whatever the project calls them (e.g. `sentry`, `datadog`, `our-internal-traces`, `grafana-prod`). Falls back to `none configured` when the array is empty.
|
|
10
|
+
- `{{OBSERVABILITY_MCP_BLOCK}}` — the verbatim block below ("Phase 0 evidence categories"), prefixed with one paragraph naming the configured servers and instructing the agent to discover the right calls inside them at runtime. When the array is empty, the block is replaced with the no-tools fallback at the end of this file.
|
|
11
|
+
|
|
12
|
+
The renderer **does not** inline tool-specific function names. The agent runs Phase 0 by listing MCP tools at runtime and picking the ones that match each evidence category.
|
|
13
|
+
|
|
14
|
+
## Phase 0 evidence categories
|
|
15
|
+
|
|
16
|
+
The block rendered into the SKILL.md is exactly this:
|
|
17
|
+
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
### How to discover the right MCP calls
|
|
21
|
+
|
|
22
|
+
The cached preflight in the rendered SKILL.md decides which servers to introspect this run (cache-hit servers skip listing; freshly-preflighted servers get listed). For each server cleared by preflight, list every tool it exposes and group them by the evidence categories below — match on tool description and parameter names, not on remembered names from past sessions. MCP servers rename tools and add capabilities frequently; **trust the live tool list, not training data**.
|
|
23
|
+
|
|
24
|
+
If a server claims to cover a category but no listed tool matches the description, ask the user before falling back to "no evidence available" — the tool may exist under a name the matcher missed.
|
|
25
|
+
|
|
26
|
+
If a tool call fails mid-session with auth / connection / transport errors, mark the server `dirty` in `.vinta-ai-workflows/cache.yaml` so the next debug run re-runs preflight on it. Do not re-preflight inside the same session — keep moving with the evidence already gathered.
|
|
27
|
+
|
|
28
|
+
### Categories of evidence the skill requires
|
|
29
|
+
|
|
30
|
+
For each category: what evidence is needed, what to extract, and what shape of MCP tool typically provides it. Skip a category only when no configured server can supply it; record the skip in the Phase 0 evidence note (*"no metric source available"*).
|
|
31
|
+
|
|
32
|
+
#### 1. Error tracking — fingerprints, stack traces, occurrence counts
|
|
33
|
+
|
|
34
|
+
**Why:** identifies the exact failure signature, how many users / tenants / regions hit it, when it started.
|
|
35
|
+
|
|
36
|
+
**Look for tools that:** search or list errors / issues / events; fetch a single error's full payload (stack trace, breadcrumbs, request context, release tag, environment); group by fingerprint; resolve / mute / link an error.
|
|
37
|
+
|
|
38
|
+
**Extract into the evidence note:**
|
|
39
|
+
- The error's stable id and permalink.
|
|
40
|
+
- First-seen and last-seen timestamps in UTC.
|
|
41
|
+
- Affected scope: environment, release / build, tenant, route, user count.
|
|
42
|
+
- Correlated identifiers (trace id, request id, session id) so the next category can cross-reference.
|
|
43
|
+
|
|
44
|
+
#### 2. Distributed tracing — span chains, upstream / downstream calls
|
|
45
|
+
|
|
46
|
+
**Why:** shows the request path across services, which span actually failed, what was upstream of it.
|
|
47
|
+
|
|
48
|
+
**Look for tools that:** fetch a trace by id; query traces by service + time window; surface the slow / failing span in a trace; list dependencies of a service.
|
|
49
|
+
|
|
50
|
+
**Extract into the evidence note:**
|
|
51
|
+
- The failing span's name, service, and the parent chain leading to it.
|
|
52
|
+
- Whether the failure originates in our code or a downstream dependency.
|
|
53
|
+
- Latency on each span vs. its baseline (helps tell "slow downstream" from "broken downstream").
|
|
54
|
+
|
|
55
|
+
#### 3. Logs — raw lines around the incident window
|
|
56
|
+
|
|
57
|
+
**Why:** error tracking shows the exception, but logs show what the process did *before* and *after*.
|
|
58
|
+
|
|
59
|
+
**Look for tools that:** filter / search logs by time window + service + free-text or structured query; tail logs around a specific request id or trace id; list log streams for a resource.
|
|
60
|
+
|
|
61
|
+
**Extract into the evidence note:**
|
|
62
|
+
- The log lines before the error (what state the process was in).
|
|
63
|
+
- Log lines after the error (did the process recover, restart, crash?).
|
|
64
|
+
- Any other error or warning in the same window from the same service — often a precursor.
|
|
65
|
+
|
|
66
|
+
#### 4. Metrics — RED / USE / saturation signals
|
|
67
|
+
|
|
68
|
+
**Why:** confirms whether the bug is a one-off, a steady leak, or a step-change tied to a deploy.
|
|
69
|
+
|
|
70
|
+
**Look for tools that:** query a metric over a time range; list metrics for a service / resource; describe a dashboard or panel.
|
|
71
|
+
|
|
72
|
+
**Extract into the evidence note:**
|
|
73
|
+
- Error rate, request rate, p50 / p95 / p99 latency on the affected operation, around first-seen and over the prior equivalent window.
|
|
74
|
+
- Saturation signals where applicable: queue depth, CPU, memory, connection pool, db locks.
|
|
75
|
+
- Whether the anomaly is a spike, a step-change, or a slow drift.
|
|
76
|
+
|
|
77
|
+
#### 5. Alerts / monitors / SLO burn
|
|
78
|
+
|
|
79
|
+
**Why:** the platform may already have flagged the incident; if it didn't, the skill should leave a note for whoever maintains the alerts.
|
|
80
|
+
|
|
81
|
+
**Look for tools that:** list firing alerts in a window; describe an alert / monitor's current state; show SLO burn rate.
|
|
82
|
+
|
|
83
|
+
**Extract into the evidence note:**
|
|
84
|
+
- Any alert that fired or recovered in the incident window.
|
|
85
|
+
- Any alert that *should have* fired but didn't (gap in coverage to flag in the PR description).
|
|
86
|
+
- SLO burn rate if the affected operation has one.
|
|
87
|
+
|
|
88
|
+
#### 6. Deploys / releases / config changes
|
|
89
|
+
|
|
90
|
+
**Why:** "what changed" is usually the answer. The platform's deploy timeline beats `git log` because it shows what's actually running.
|
|
91
|
+
|
|
92
|
+
**Look for tools that:** list recent deploys for a project / service; describe a deploy's commit sha, author, and timestamp; list feature-flag flips; show config-store revisions.
|
|
93
|
+
|
|
94
|
+
**Extract into the evidence note:**
|
|
95
|
+
- The closest deploy / release / flag flip / config push before first-seen.
|
|
96
|
+
- Whether the incident first-seen aligns within minutes of that change.
|
|
97
|
+
- The commit sha or change id so the implementer can `git show` it during Phase 1.
|
|
98
|
+
|
|
99
|
+
#### 7. Dashboards — pre-built views the team already trusts
|
|
100
|
+
|
|
101
|
+
**Why:** SREs and product engineers have curated panels that the systematic-debugging agent should not try to recreate from scratch.
|
|
102
|
+
|
|
103
|
+
**Look for tools that:** search dashboards by title / tag; render a dashboard URL pinned to a time window; list panels in a dashboard.
|
|
104
|
+
|
|
105
|
+
**Extract into the evidence note:**
|
|
106
|
+
- URL of the most relevant dashboard, pinned to the incident window.
|
|
107
|
+
- The panel that visualises the anomaly (so the human reviewer can verify by eye).
|
|
108
|
+
|
|
109
|
+
### How to use the evidence
|
|
110
|
+
|
|
111
|
+
Once the categories above are filled in (or explicitly skipped), the Phase 0 evidence note is complete. Carry trace ids and request ids forward — the same id often unlocks deeper queries in a tool the agent already used. Do not move to Phase 1 until the cause's *blast radius and timeline* are written down: which scope is affected, when it started, what changed in the same window.
|
|
112
|
+
|
|
113
|
+
### When no MCP observability servers are configured
|
|
114
|
+
|
|
115
|
+
Replace the entire block above with this single paragraph:
|
|
116
|
+
|
|
117
|
+
> No observability MCP server is wired up for this project. Phase 0 collapses to "read the local logs available to the developer and reproduce the failure deterministically." Write *"no platform evidence available"* at the top of the Phase 0 note so reviewers know the evidence floor was local-only. If the bug is production-only, stop and ask the user to wire up an error-tracking or log MCP server before continuing — production-only bugs without telemetry are a guess factory.
|
|
118
|
+
|
|
119
|
+
---
|
|
120
|
+
|
|
121
|
+
## Renderer notes
|
|
122
|
+
|
|
123
|
+
- The block is emitted verbatim — there is no per-server templating. Tool discovery is a runtime activity, not a generation-time activity.
|
|
124
|
+
- `skills.systematic-debugging.observability_mcp_servers` is a free-form list of identifiers the user recognises; the renderer only joins them into `{{OBSERVABILITY_MCP_LIST}}`. It is allowed to be empty (the no-tools fallback above renders) but the bootstrap interview should warn the user that an empty list defeats the skill's purpose.
|
|
125
|
+
- If the user adds a new MCP server later, [vinta-sync-ai-tools](../../vinta-sync-ai-tools/SKILL.md) just appends it to the list and re-renders — no new catalogue entry needed.
|