@yemi33/minions 0.1.2070 → 0.1.2072
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dashboard/js/qa.js +358 -0
- package/dashboard/js/state.js +2 -1
- package/dashboard/pages/qa.html +72 -0
- package/dashboard/styles.css +102 -0
- package/dashboard.js +410 -6
- package/docs/qa-runbook-lifecycle.md +232 -0
- package/engine/cleanup.js +4 -1
- package/engine/comment-classifier.js +8 -1
- package/engine/cooldown.js +6 -2
- package/engine/gh-comment.js +74 -3
- package/engine/gh-token.js +7 -9
- package/engine/lifecycle.js +100 -0
- package/engine/pipeline.js +9 -1
- package/engine/playbook.js +39 -0
- package/engine/qa-runners/maestro.js +152 -0
- package/engine/qa-runners/playwright.js +149 -0
- package/engine/qa-runners.js +323 -0
- package/engine/qa-sessions.js +1008 -0
- package/engine/shared.js +71 -12
- package/engine.js +140 -0
- package/package.json +1 -1
- package/playbooks/qa-session-draft.md +158 -0
- package/playbooks/qa-session-execute.md +165 -0
- package/playbooks/qa-session-setup.md +154 -0
- package/prompts/cc-system.md +43 -0
- package/routing.md +3 -0
|
@@ -0,0 +1,154 @@
|
|
|
1
|
+
---
|
|
2
|
+
requiresProjectContext: true
|
|
3
|
+
---
|
|
4
|
+
|
|
5
|
+
# Playbook: QA Session SETUP
|
|
6
|
+
|
|
7
|
+
You are {{agent_name}}, the {{agent_role}} on the {{project_name}} project.
|
|
8
|
+
TEAM ROOT: {{team_root}}
|
|
9
|
+
|
|
10
|
+
## Your Task
|
|
11
|
+
|
|
12
|
+
QA Session **SETUP** phase for session **{{session_id}}** (work item {{item_id}}).
|
|
13
|
+
|
|
14
|
+
A user asked Minions to QA the following target and flows:
|
|
15
|
+
|
|
16
|
+
- **Session id:** `{{session_id}}`
|
|
17
|
+
- **Target kind:** `{{target_kind}}`
|
|
18
|
+
- **Target PR id:** `{{target_pr_id}}`
|
|
19
|
+
- **Target branch:** `{{target_branch}}`
|
|
20
|
+
- **Target commit SHA:** `{{target_sha}}`
|
|
21
|
+
- **Target worktree (kind=current):** `{{target_worktree}}`
|
|
22
|
+
- **Raw target JSON:** `{{target_json}}`
|
|
23
|
+
- **Flows (natural language):** {{flows_raw}}
|
|
24
|
+
- **Runner hint (optional explicit runner):** `{{runner_hint}}`
|
|
25
|
+
- **Capture:** `{{capture}}`
|
|
26
|
+
- **Mode:** `{{session_mode}}`
|
|
27
|
+
|
|
28
|
+
{{additional_context}}
|
|
29
|
+
|
|
30
|
+
## What "qa-session-setup" means
|
|
31
|
+
|
|
32
|
+
A `qa-session-setup` task is the **first** of three chained work items the
|
|
33
|
+
engine dispatches for each QA Session (SETUP → DRAFT → EXECUTE). Your job is
|
|
34
|
+
to make the target runnable so the DRAFT and EXECUTE agents can drive a real
|
|
35
|
+
live instance:
|
|
36
|
+
|
|
37
|
+
1. **Resolve the target** (`{{target_kind}}`) into a checked-out worktree.
|
|
38
|
+
- `pr` → check out the PR's head branch (`{{target_pr_id}}`).
|
|
39
|
+
- `branch` → check out branch `{{target_branch}}`.
|
|
40
|
+
- `commit` → detach at `{{target_sha}}`.
|
|
41
|
+
- `current` → reuse the existing worktree at `{{target_worktree}}` (or
|
|
42
|
+
`MINIONS_AGENT_CWD` if `{{target_worktree}}` is empty).
|
|
43
|
+
2. **Inspect the codebase** to find a single "dev-up" command. Look in this
|
|
44
|
+
order: `package.json` `scripts.dev|start|serve`, top-level `Procfile`,
|
|
45
|
+
project README "Run locally / Getting Started" section, a `Makefile` `dev`
|
|
46
|
+
target, or a docker-compose service that exposes an HTTP port. Pick the
|
|
47
|
+
smallest command that brings the app up and binds to a TCP port.
|
|
48
|
+
3. **Write the managed-spawn sidecar** to
|
|
49
|
+
`agents/{{agent_id}}/managed-spawn.json` (relative to the Minions root)
|
|
50
|
+
with **exactly one** spec named **`{{managed_spawn_name}}`**. Use the JSON
|
|
51
|
+
shape the `managed_spawn` section below documents. The engine ingests this
|
|
52
|
+
sidecar on your exit and gates the next phase on the first healthcheck.
|
|
53
|
+
|
|
54
|
+
## Hard requirements on the sidecar
|
|
55
|
+
|
|
56
|
+
The engine validates the sidecar through `evaluateManagedSpawnAcceptance`.
|
|
57
|
+
Anything that fails validation flips your dispatch to FAILED with
|
|
58
|
+
`failure_class: 'invalid-managed-spawn'` and the QA Session transitions to
|
|
59
|
+
`failed` automatically. Specifically:
|
|
60
|
+
|
|
61
|
+
- `specs[0].name` MUST equal `{{managed_spawn_name}}` (exact match). The
|
|
62
|
+
engine joins the spawn back to its owning session by this convention.
|
|
63
|
+
- `specs[0].healthcheck` MUST be present and verifiable. Prefer HTTP
|
|
64
|
+
(`type: 'http'`) with a real URL and `expect_status` set; fall back to
|
|
65
|
+
`type: 'command'` only when the app has no HTTP surface (e.g. a CLI worker).
|
|
66
|
+
- `specs[0].cmd` MUST be on the engine's allowlist (`node`, `bun`, `npm`,
|
|
67
|
+
`npx`, `pnpm`, `yarn`, `python`, `docker`, `adb`, `gradle`, `gradlew`,
|
|
68
|
+
`mvn`, `pwsh`, `powershell`, `bash`, `sh`, `curl`, `git`, …). If the
|
|
69
|
+
project requires a non-allowlisted binary, **stop and report a setup
|
|
70
|
+
failure** (see below) — do NOT try to work around it.
|
|
71
|
+
- `specs[0].cwd` MUST be an absolute path inside the resolved worktree.
|
|
72
|
+
- Pick a free port and put it in both `ports[]` and the healthcheck URL.
|
|
73
|
+
|
|
74
|
+
The `managed_spawn` block injected later in this prompt has the full schema
|
|
75
|
+
and the executable allowlist enumerated. Read it before writing the sidecar.
|
|
76
|
+
|
|
77
|
+
## No PR, no commit, no test code yet
|
|
78
|
+
|
|
79
|
+
SETUP only resolves the target and writes the managed-spawn spec. **Do not**:
|
|
80
|
+
|
|
81
|
+
- write any test code — that belongs to the DRAFT phase
|
|
82
|
+
- commit, push, or open a PR — sessions are tracked via the session record,
|
|
83
|
+
not a merged PR
|
|
84
|
+
- modify project source — the dev-up command should run the project as-is
|
|
85
|
+
- start the app yourself (`bun run dev` in a detached process) — the engine
|
|
86
|
+
spawns the spec for you after you exit
|
|
87
|
+
|
|
88
|
+
If the project genuinely will not run without a code change (missing
|
|
89
|
+
dependency wiring, hard-coded prod URL, etc.), stop and report a setup
|
|
90
|
+
failure so the human can decide whether to patch it themselves.
|
|
91
|
+
|
|
92
|
+
## Failure path (REQUIRED)
|
|
93
|
+
|
|
94
|
+
If you cannot resolve the target, cannot find a dev-up command, hit a
|
|
95
|
+
non-allowlisted binary, or otherwise produce a sidecar that
|
|
96
|
+
`evaluateManagedSpawnAcceptance` would reject, **do not write a malformed
|
|
97
|
+
sidecar**. Instead, write your completion report with:
|
|
98
|
+
|
|
99
|
+
```json
|
|
100
|
+
{
|
|
101
|
+
"status": "failed",
|
|
102
|
+
"summary": "<one-line human-readable explanation of what blocked SETUP>",
|
|
103
|
+
"failure_class": "qa-session-setup-failed",
|
|
104
|
+
"retryable": false,
|
|
105
|
+
"needs_rerun": false,
|
|
106
|
+
"nonce": "<value of MINIONS_COMPLETION_NONCE env var>",
|
|
107
|
+
"artifacts": []
|
|
108
|
+
}
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
The `engine/qa-sessions.js#handleSetupComplete` hook reads `failure_class`
|
|
112
|
+
and the summary, transitions the session to `failed`, and surfaces the
|
|
113
|
+
explanation in the dashboard session card so the human knows exactly why
|
|
114
|
+
SETUP gave up.
|
|
115
|
+
|
|
116
|
+
Examples of legitimate failure summaries:
|
|
117
|
+
|
|
118
|
+
- `"Project has no detectable dev-up command — no package.json scripts.dev, Procfile, or Makefile dev target."`
|
|
119
|
+
- `"Required binary 'cargo' is not on the engine's managed-spawn allowlist."`
|
|
120
|
+
- `"Target PR #1234 head branch could not be checked out: fatal: reference is not a tree."`
|
|
121
|
+
- `"Detected dev-up command but it requires a database connection string we have no way to provide in CI."`
|
|
122
|
+
|
|
123
|
+
## Working directory
|
|
124
|
+
|
|
125
|
+
```bash
|
|
126
|
+
# PowerShell
|
|
127
|
+
echo $env:MINIONS_AGENT_CWD
|
|
128
|
+
pwd
|
|
129
|
+
|
|
130
|
+
# bash/zsh
|
|
131
|
+
echo "$MINIONS_AGENT_CWD"
|
|
132
|
+
pwd
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
`MINIONS_AGENT_CWD` is the engine-resolved worktree root. Prefer it over
|
|
136
|
+
`pwd` for any cwd-sensitive command.
|
|
137
|
+
|
|
138
|
+
## Findings
|
|
139
|
+
|
|
140
|
+
Write findings to `{{team_root}}/notes/inbox/{{agent_id}}-{{item_id}}-{{date}}.md`
|
|
141
|
+
only after successful completion. Include:
|
|
142
|
+
|
|
143
|
+
- Session id + target summary
|
|
144
|
+
- Dev-up command chosen and where it was discovered (file:line)
|
|
145
|
+
- Managed-spawn name, healthcheck shape, port
|
|
146
|
+
- Notes for future setup runs on the same target (flaky startup, env-vars
|
|
147
|
+
needed, port collisions)
|
|
148
|
+
|
|
149
|
+
## Constraints
|
|
150
|
+
|
|
151
|
+
- Do not modify production code unless explicitly asked.
|
|
152
|
+
- Do not remove worktrees; the engine handles cleanup automatically.
|
|
153
|
+
- The sidecar is the deliverable — without it, the session is stuck in
|
|
154
|
+
`spawning` until the SETUP WI times out.
|
package/prompts/cc-system.md
CHANGED
|
@@ -152,6 +152,49 @@ curl -s http://localhost:{{dashboard_port}}/api/work-items
|
|
|
152
152
|
curl -s http://localhost:{{dashboard_port}}/api/status
|
|
153
153
|
```
|
|
154
154
|
|
|
155
|
+
## QA Sessions — natural-language → POST /api/qa/session
|
|
156
|
+
|
|
157
|
+
When the user describes a UI/E2E flow they want validated against a *live, running* app instance — "QA the login flow on PR #1234", "smoke test the homepage", "test the checkout flow on the `develop` branch", "validate the signup journey on my current worktree", etc. — dispatch a QA Session via `POST /api/qa/session` instead of opening an `implement` or `test` work item. The session pipeline boots the dev-up command as a managed-spawn, drafts a runner-native test (Playwright or Maestro) from the natural-language flows, then executes it against the live target and captures the requested artifacts.
|
|
158
|
+
|
|
159
|
+
**When to use this endpoint:**
|
|
160
|
+
|
|
161
|
+
- The user wants a behavioural / end-to-end check ("does the login form actually work", "make sure the cart adds items"). Use `/api/qa/session`.
|
|
162
|
+
- The user wants a code change, unit/integration tests in the repo, or an investigation. Keep using `/api/work-items` with `type: "fix"`, `"implement"`, `"test"`, `"explore"`.
|
|
163
|
+
- The user explicitly says "QA …", "smoke test …", "test the … flow", "validate the … journey", or names a concrete UI walkthrough against a live app. Use `/api/qa/session`.
|
|
164
|
+
|
|
165
|
+
**Body shape (mirrors `engine/qa-sessions.js#validateSpec`):**
|
|
166
|
+
|
|
167
|
+
- `target` — REQUIRED object describing what to QA against. `target.kind` is one of:
|
|
168
|
+
- `pr` — needs `target.prId` (e.g. `"github:yemi33/minions#2911"` or `"ado:office/iss/constellation#5215493"`).
|
|
169
|
+
- `branch` — needs `target.branch` (e.g. `"develop"`).
|
|
170
|
+
- `commit` — needs `target.sha` (full 40-char SHA).
|
|
171
|
+
- `current` — uses the user's current worktree. `target.worktree` is optional; the SETUP agent falls back to `MINIONS_AGENT_CWD` when empty.
|
|
172
|
+
- `flowsRaw` — REQUIRED natural-language description of the steps to test (≤4000 chars). Paste the user's own description verbatim where possible; the runner adapter translates it into the runner-native script.
|
|
173
|
+
- `mode` — `"confirm"` (default — pauses at `awaiting-approval` so the user can review the drafted test before EXECUTE fires) or `"auto"` (chains straight from DRAFT to EXECUTE). Pick `"confirm"` unless the user said "just run it" / "no review needed" / "auto".
|
|
174
|
+
- `capture` — optional `{ video?: bool, screenshots?: bool, logs?: bool }`. Default is everything false. Set what the user asked for.
|
|
175
|
+
- `runner` — optional kebab-case name to force a specific runner (`"playwright"`, `"maestro"`, or a plugin). Omit to let the engine auto-detect (Maestro wins when the project has `.maestro/`; Playwright is the safe default).
|
|
176
|
+
- `project` — REQUIRED when multiple projects are configured (mirrors `/api/work-items`). Omit for the central path.
|
|
177
|
+
|
|
178
|
+
**Worked example — PR target, confirm mode (default):**
|
|
179
|
+
```
|
|
180
|
+
curl -s -X POST http://localhost:{{dashboard_port}}/api/qa/session \
|
|
181
|
+
-H 'Content-Type: application/json' \
|
|
182
|
+
-H 'X-CC-Turn-Id: {{cc_turn_id}}' \
|
|
183
|
+
-d '{"target":{"kind":"pr","prId":"github:yemi33/MyApp#1234"},"flowsRaw":"Open the homepage, click Login, enter test@example.com / hunter2, and verify the dashboard renders with the user'\''s name in the header.","mode":"confirm","capture":{"screenshots":true,"logs":true},"project":"MyApp"}'
|
|
184
|
+
```
|
|
185
|
+
|
|
186
|
+
**Worked example — current worktree, auto mode, video capture:**
|
|
187
|
+
```
|
|
188
|
+
curl -s -X POST http://localhost:{{dashboard_port}}/api/qa/session \
|
|
189
|
+
-H 'Content-Type: application/json' \
|
|
190
|
+
-H 'X-CC-Turn-Id: {{cc_turn_id}}' \
|
|
191
|
+
-d '{"target":{"kind":"current"},"flowsRaw":"Add three items to the cart, go to checkout, complete the payment form with the Stripe test card, and verify the success page.","mode":"auto","capture":{"video":true,"screenshots":true},"project":"MyApp"}'
|
|
192
|
+
```
|
|
193
|
+
|
|
194
|
+
**Response:** `{ sessionId, state: "spawning", setupWorkItemId, managedSpawnName }`. Tell the user the session id so they can watch it at `/qa` and steer it via the `/approve` (run the drafted test), `/edit` (re-draft with feedback), `/dismiss` (accept the draft without running), `/cancel` (give up), or `/kill` (cancel + tear down the managed-spawn) endpoints listed in `GET /api/routes`.
|
|
195
|
+
|
|
196
|
+
**Do not also dispatch a `/api/work-items` `implement` or `test` for the same QA request.** The QA Session pipeline owns its own SETUP → DRAFT → EXECUTE work items end-to-end; firing a parallel work-item is the same double-dispatch class that the "Never both" rule above forbids. If the user asks for both a QA pass AND a code change, do them as two separate, sequential calls — QA Session for the behavioural check, work-item for the fix.
|
|
197
|
+
|
|
155
198
|
**Required fields per endpoint** — the server returns `{ error: "..." }` if missing. Common cases:
|
|
156
199
|
- `POST /api/work-items`: `title` REQUIRED. `description` recommended. `project` REQUIRED when multiple projects are configured (server returns the list of known names if you guess wrong). `type` defaults to `implement`; valid values: `fix`, `implement`, `implement:large`, `setup`, `explore`, `ask`, `review`, `test`, `verify`. Agent hint via `agent` (string) or `agents` (array).
|
|
157
200
|
- Exempt from the `project` requirement (these run rootless or via central paths): `ask`, `explore`, `plan`, `plan-to-prd`, `meeting`. (`docs` is intentionally NOT exempt — it's write-capable and lands in `WORKTREE_REQUIRING_TYPES`, so it needs a real project worktree. For minions-repo docs work, pass `project: "minions"` explicitly.) `setup` is also in the project-required set — it operates inside a real project worktree but produces no PR. Every other type needs a project worktree, so the server rejects project-less creates with `400 { error, knownProjects }` when ≠1 project is configured.
|
package/routing.md
CHANGED
|
@@ -22,6 +22,9 @@ How the engine decides who handles what. Parsed by engine.js — keep the table
|
|
|
22
22
|
| docs | lambert | _any_ |
|
|
23
23
|
| setup | dallas | _any_ |
|
|
24
24
|
| qa-validate | dallas | ralph |
|
|
25
|
+
| qa-session-setup | dallas | ralph |
|
|
26
|
+
| qa-session-draft | dallas | ralph |
|
|
27
|
+
| qa-session-execute | dallas | ralph |
|
|
25
28
|
|
|
26
29
|
Notes:
|
|
27
30
|
- `_author_` means route to the PR author
|