@iamu-dev/iamu 0.8.0-beta.0 → 0.8.0-beta.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +4 -2
- package/dist/bin.js +13590 -11724
- package/dist/commands/advise.d.ts +4 -2
- package/dist/commands/agent-commands.d.ts +22 -0
- package/dist/commands/browse-actions.d.ts +1 -0
- package/dist/commands/browse-driver.d.ts +30 -6
- package/dist/commands/browse.d.ts +58 -2
- package/dist/commands/cloud-push.d.ts +3 -3
- package/dist/commands/complete.d.ts +21 -0
- package/dist/commands/completion.d.ts +2 -0
- package/dist/commands/context.d.ts +19 -0
- package/dist/commands/dev-script.d.ts +14 -0
- package/dist/commands/doctor.d.ts +6 -0
- package/dist/commands/edit-entity.d.ts +5 -3
- package/dist/commands/evaluate.d.ts +10 -3
- package/dist/commands/finding-stability.d.ts +1 -0
- package/dist/commands/findings.d.ts +9 -0
- package/dist/commands/init.d.ts +14 -5
- package/dist/commands/instructions-walk.d.ts +8 -0
- package/dist/commands/instructions.d.ts +2 -11
- package/dist/commands/list.d.ts +3 -0
- package/dist/commands/match.d.ts +8 -5
- package/dist/commands/new-entity.d.ts +2 -0
- package/dist/commands/new-finding.d.ts +2 -0
- package/dist/commands/persona-today.d.ts +20 -0
- package/dist/commands/preflight.d.ts +4 -2
- package/dist/commands/pull.d.ts +1 -1
- package/dist/commands/set-entity.d.ts +15 -3
- package/dist/commands/skill-command.d.ts +13 -0
- package/dist/commands/step.d.ts +4 -0
- package/dist/commands/summary.d.ts +2 -0
- package/dist/commands/triage.d.ts +15 -0
- package/dist/index.js +13571 -11715
- package/dist/templates/prompts/complete.md +2 -3
- package/dist/templates/prompts/designer.md +10 -4
- package/dist/templates/prompts/persona.md +2 -1
- package/dist/templates/prompts/walk.md +26 -12
- package/dist/templates/skills/iamu-design.md +20 -15
- package/dist/templates/skills/iamu-findings.md +6 -6
- package/dist/templates/skills/iamu-new.md +9 -9
- package/dist/templates/skills/iamu-run.md +66 -42
- package/dist/templates/skills/iamu-setup.md +155 -76
- package/dist/templates/skills/iamu-stage.md +29 -23
- package/dist/templates/skills/iamu-status.md +2 -0
- package/dist/templates.d.ts +1 -1
- package/package.json +3 -3
- package/dist/templates/prompts/evaluate.md +0 -64
|
@@ -78,70 +78,108 @@ No local codebase detected. To seed context, provide one of:
|
|
|
78
78
|
|
|
79
79
|
### 3. Step 1/3: Establish the Safety Fence (`iamu/testability.md`)
|
|
80
80
|
|
|
81
|
-
|
|
81
|
+
Never dump the target URL, dev command, login mechanism, and untestable boundaries into a single "take-it-or-leave-it" blob. Walk through them **one focused question at a time**, using what you found in the codebase to pre-fill the recommended choice and persisting each answer immediately with `iamu set testability`.
|
|
82
82
|
|
|
83
|
-
|
|
84
|
-
|
|
83
|
+
#### Step 1a: Target URL & Dev Command (1 question)
|
|
84
|
+
|
|
85
|
+
Before asking, check whether a server is already responding on the detected port (`curl -I` or `iamu doctor --target <url> --json`). If this is a monorepo with multiple runnable apps (`apps/*`, `packages/*`), list each detected app and its URL as selectable options.
|
|
86
|
+
|
|
87
|
+
Output the detected target briefly to chat first, then invoke `ask_question` in the same turn:
|
|
85
88
|
|
|
86
89
|
```text
|
|
87
|
-
Step
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
Detected Environment:
|
|
91
|
-
Target URL: <target_url, e.g. http://localhost:5173>
|
|
92
|
-
Dev Command: <dev_command, e.g. ./dev or pnpm dev>
|
|
93
|
-
Safe Fixtures: <detected seed scripts or mock handlers, e.g. organization_realistic_seed.sql, MSW>
|
|
94
|
-
DO NOT WALK: <detected external services, e.g. Arbetsmiljöverket external submission portal (anmalarbetsskada.se), Real SMS/Email gateways, Real OIDC Keycloak>
|
|
90
|
+
Step 1a: Target URL & Dev Server
|
|
91
|
+
Detected URL: <target_url, e.g. http://localhost:5173> (<currently running | not running>)
|
|
92
|
+
Dev Command: <dev_command, e.g. pnpm dev>
|
|
95
93
|
```
|
|
96
94
|
|
|
97
|
-
**Interactive Decision Gate:**
|
|
98
|
-
Prompt the user to verify the safety boundary:
|
|
99
95
|
- If interactive user-prompt tools exist (`ask_question`, `ask_followup_question`), invoke it:
|
|
100
96
|
- **Question:**
|
|
101
97
|
"""
|
|
102
|
-
Step
|
|
98
|
+
Step 1a: Where should Iamu run browser tests against your app?
|
|
99
|
+
|
|
100
|
+
Detected URL: <target_url> (<currently running | not running>)
|
|
101
|
+
Dev Command: <dev_command>
|
|
102
|
+
"""
|
|
103
|
+
- **Options:**
|
|
104
|
+
1. "(Recommended) Use <target_url> (start with `<dev_command>`)"
|
|
105
|
+
2. "<Second detected workspace app URL + dev command, if monorepo; otherwise omit>"
|
|
106
|
+
3. "Use a different target URL or port"
|
|
107
|
+
4. "Change the dev startup command"
|
|
108
|
+
- Otherwise (text-only/CLI):
|
|
109
|
+
Ask: `Where should Iamu test your app? Detected <target_url> (start command: <dev_command>). Reply to confirm, or give the right URL / dev command.`
|
|
110
|
+
|
|
111
|
+
**If the user picks Option 3 or Option 4 (or writes in a correction):**
|
|
112
|
+
- Ask for the exact URL (e.g. `http://localhost:4000` or a staging URL) and/or dev command, and wait for their reply before proceeding.
|
|
113
|
+
|
|
114
|
+
**Persist immediately:**
|
|
115
|
+
```bash
|
|
116
|
+
iamu set testability \
|
|
117
|
+
--target "<confirmed_target_url>" \
|
|
118
|
+
--dev-command "<confirmed_dev_command>" \
|
|
119
|
+
--json
|
|
120
|
+
```
|
|
103
121
|
|
|
104
|
-
|
|
105
|
-
Dev Command: <dev_command>
|
|
106
|
-
Safe Fixtures: <detected seed scripts or mock handlers>
|
|
122
|
+
#### Step 1b: How Tests Log In / Switch Users (1 question)
|
|
107
123
|
|
|
108
|
-
|
|
109
|
-
<detected external services, real SMS/email gateways, external submission portals>
|
|
124
|
+
Next, ask **specifically** how the browser walker should log in or switch roles during tests. Never guess and save `--auth-strategy` or `--access` silently.
|
|
110
125
|
|
|
111
|
-
|
|
126
|
+
Output what you found in the codebase first, then invoke `ask_question` in the same turn:
|
|
127
|
+
|
|
128
|
+
```text
|
|
129
|
+
Step 1b: Test Login & User Switching
|
|
130
|
+
Detected Auth: <What code inspection found, e.g. "Dev switcher via localStorage.sf_dev_persona", "Seeded test accounts in prisma/seed.ts (coordinator@site-a.test)", "Login form at /login (no seeded account found)", or "Public routes — no auth guard detected">
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
- If interactive user-prompt tools exist (`ask_question`, `ask_followup_question`), invoke it:
|
|
134
|
+
- **Question:**
|
|
135
|
+
"""
|
|
136
|
+
Step 1b: How should Iamu log in or switch users when running tests?
|
|
137
|
+
|
|
138
|
+
Detected in codebase: <concise summary of detected auth guard / switcher / seed accounts>
|
|
112
139
|
"""
|
|
113
140
|
- **Options:**
|
|
114
|
-
1. "
|
|
115
|
-
2. "
|
|
116
|
-
3. "
|
|
141
|
+
1. "(Recommended) <Specific mechanism detected in code, e.g. 'Use seeded test account (coordinator@site-a.test)' or 'Use dev persona switcher' or 'No login needed (public app)'>"
|
|
142
|
+
2. "Interactive login — open Chrome and let me log in manually before the test starts"
|
|
143
|
+
3. "Use specific test credentials or login instructions (I'll provide them)"
|
|
144
|
+
4. "No login needed (test public pages only)"
|
|
117
145
|
- Otherwise (text-only/CLI):
|
|
118
|
-
|
|
119
|
-
Target URL: <target_url>
|
|
120
|
-
Dev Command: <dev_command>
|
|
121
|
-
Safe Fixtures: <detected seed scripts or mock handlers>
|
|
122
|
-
DO NOT WALK: <detected external services>
|
|
146
|
+
Ask: `How should Iamu log in for tests? Detected: <summary>. Reply to confirm, say "interactive" to log in manually in Chrome, or tell me how to sign in.`
|
|
123
147
|
|
|
124
|
-
|
|
125
|
-
|
|
148
|
+
**If the user picks Option 3 (or wants to specify how login works):**
|
|
149
|
+
- Ask: *"Which test account, login URL, or dev bypass should the walker use? (For passwords or tokens, name the environment variable or dev seed rather than pasting production secrets.)"*
|
|
150
|
+
- Wait for their answer before proceeding.
|
|
126
151
|
|
|
127
|
-
**
|
|
128
|
-
Persist the safety fence using `iamu set testability` (never invoke host file creation or edit tools):
|
|
152
|
+
**Persist immediately:**
|
|
129
153
|
```bash
|
|
130
154
|
iamu set testability \
|
|
131
|
-
--target "<target_url>" \
|
|
132
|
-
--dev-command "<dev_command>" \
|
|
133
155
|
--auth-strategy "<dev_switcher | interactive | test_accounts | public>" \
|
|
134
|
-
[--access "<
|
|
156
|
+
[--access "<confirmed login or user-switching instructions>"] \
|
|
157
|
+
--json
|
|
158
|
+
```
|
|
159
|
+
|
|
160
|
+
#### Step 1c: Untestable Boundaries (only when external side-effects exist)
|
|
161
|
+
|
|
162
|
+
If code inspection found un-mocked external services or side-effect boundaries (e.g., Stripe checkout, real SMS/email gateways, external government submission portals, live production OAuth), ask one short question to confirm the `DO NOT WALK` fence:
|
|
163
|
+
- Show the detected external boundaries and ask whether to block the walker from touching them (or add others).
|
|
164
|
+
- If no external side-effect integrations exist, do not ask an extra question.
|
|
165
|
+
|
|
166
|
+
Persist the complete testability summary body via `iamu set testability` (never invoke host file creation or edit tools):
|
|
167
|
+
```bash
|
|
168
|
+
iamu set testability \
|
|
169
|
+
--target "<confirmed_target_url>" \
|
|
170
|
+
--dev-command "<confirmed_dev_command>" \
|
|
171
|
+
--auth-strategy "<confirmed_auth_strategy>" \
|
|
172
|
+
[--access "<confirmed_access_mechanism>"] \
|
|
135
173
|
[--harnesses "<detected harness 1>, <detected harness 2>"] \
|
|
136
174
|
[--untestable-boundaries "<external dependency 1>, <external dependency 2>"] \
|
|
137
175
|
--body "# Project Testability & Environment
|
|
138
176
|
|
|
139
177
|
## Target & Run Instructions
|
|
140
|
-
- Command: <
|
|
141
|
-
- Base URL: <
|
|
178
|
+
- Command: <confirmed_dev_command>
|
|
179
|
+
- Base URL: <confirmed_target_url>
|
|
142
180
|
|
|
143
181
|
## Auth & Access Boundary
|
|
144
|
-
- <
|
|
182
|
+
- <confirmed_access_mechanism>
|
|
145
183
|
|
|
146
184
|
## Reachable Surfaces
|
|
147
185
|
- <List discovered routes/entry points>
|
|
@@ -162,6 +200,8 @@ Explain the purpose of project context: traditional tests only assert button cli
|
|
|
162
200
|
|
|
163
201
|
Present the discovered domain and project description. Do NOT introduce or ask the user to approve personas or roles at this stage — personas are cast later when designing specific test journeys.
|
|
164
202
|
|
|
203
|
+
**Treat codebase evidence as a hypothesis, not user approval.** Scan the repo so the user never has to repeat obvious facts, but if the codebase leaves a material gap (who the real primary users are, what high-stakes workflow matters most, or what real-world constraints apply), call out that gap explicitly in your question rather than inventing domain stakes.
|
|
204
|
+
|
|
165
205
|
**Describe the product, not the environment you explored.** Persona drafting and the advisor read these fields and the description as the truth about the product. Write them as the product its people use in their real work. Leave out everything about how this copy runs, because Step 1 already recorded it in `testability.md`:
|
|
166
206
|
- **Data and build:** seeded, mocked, fixture, sample or fake data; demo, prototype or staging status; where data is stored; frontend or backend architecture; dev servers.
|
|
167
207
|
- **Seed numbers:** headcounts, named teams, stores or sites, and record counts taken from seed or fixture files describe the test data, not the customers. Say who the product is for in general terms ("small and mid-sized companies"). Rules and policies the code enforces (a notice period, an approval threshold) are product behaviour; keep them.
|
|
@@ -202,7 +242,7 @@ Host interactive tools (`ask_question`, `ask_followup_question`) render a modal
|
|
|
202
242
|
Confirm project domain & description?
|
|
203
243
|
"""
|
|
204
244
|
- **Options:**
|
|
205
|
-
1. "
|
|
245
|
+
1. "(Recommended) Project description is accurate — proceed to design first test"
|
|
206
246
|
2. "Save project description only (skip scenario creation for now)"
|
|
207
247
|
3. "Adjust domain or description"
|
|
208
248
|
- Otherwise (text-only/CLI):
|
|
@@ -215,10 +255,12 @@ Host interactive tools (`ask_question`, `ask_followup_question`) render a modal
|
|
|
215
255
|
Project Description:
|
|
216
256
|
<1 cohesive paragraph>
|
|
217
257
|
|
|
218
|
-
|
|
219
|
-
|
|
258
|
+
Reply to confirm this project description, say "save only" to skip scenario creation, or tell me what to adjust.
|
|
259
|
+
|
|
260
|
+
**If the user chooses Option 3 (Adjust domain or description):**
|
|
261
|
+
- Ask what is inaccurate or missing, revise the domain fields and paragraph accordingly, and confirm before saving.
|
|
220
262
|
|
|
221
|
-
**Action on confirmation (Option 1):**
|
|
263
|
+
**Action on confirmation (Option 1 or after adjustment):**
|
|
222
264
|
Persist the domain context using `iamu set context` (never invoke host file creation or edit tools). Put structured fields in frontmatter and the project description in the markdown body below the `---` via `--body`:
|
|
223
265
|
```bash
|
|
224
266
|
iamu set context \
|
|
@@ -229,18 +271,31 @@ iamu set context \
|
|
|
229
271
|
```
|
|
230
272
|
|
|
231
273
|
*If user chooses Option 2 (Save context only):* run `iamu set context ...` as above, and exit cleanly:
|
|
232
|
-
"✓ Project context initialized. When you're ready to create your first test journey, run
|
|
274
|
+
"✓ Project context initialized. When you're ready to create your first test journey, run the `iamu-new` skill."
|
|
233
275
|
|
|
234
276
|
---
|
|
235
277
|
|
|
236
278
|
### 4b. Design docs (quick; the user may skip it, you may not)
|
|
237
279
|
|
|
238
|
-
The team's design docs are part of the product's context, like the domain
|
|
280
|
+
The team's design docs are part of the product's context, like the domain—and **Iamu gets significantly sharper when it has them**. Without a design doc (`DESIGN.md`), Iamu's designer can only judge screens by generic UI craft; with a recorded `DESIGN.md`, every walkthrough checks your actual token roles, component contracts, form/loading/empty states, and contrast rules, links persona friction to broken rules, and proposes grounded `Before → After` edits when a rule hurts a user.
|
|
281
|
+
|
|
282
|
+
Iamu reads design docs wherever they are and never edits them on its own. Run `iamu design detect --json` and always show the user its result, even when every candidate looks unrelated. Never drop a candidate yourself: one in a spike, sandbox or example folder is listed with that note, unticked, and the user decides.
|
|
239
283
|
|
|
240
284
|
- **Found docs** (a `DESIGN.md` in any format, design-system, brand, voice or interaction docs, design sections of `CLAUDE.md` or `AGENTS.md` shown as `CLAUDE.md#<heading>`, a `PRODUCT.md`, token files, a component library): list them once, one line each (kind, path). Show a candidate with a `note` (anything in a sample, export, fixture or spike folder) with its note, unticked. Ask once whether to record them; the user may drop some or add others. On yes, `iamu design sources <paths…>` (in a monorepo, `--applies-to` with the URL prefix or the app's folder each doc covers).
|
|
241
285
|
- **A legacy `iamu/design.md`** (`iamu design status --json` shows `legacy.doc` and no `legacy.migrated_at`): fold it into the same single question. Run `iamu design migrate --json` first, and offer three answers next to the files: **move it** to a plain `DESIGN.md` the team owns (`iamu design migrate --move-to DESIGN.md --apply`), **keep** `iamu/design.md` as it is (`iamu design migrate --apply`, which records the choice so it is not offered again), or **not now** (run nothing; it is offered again next time). Only an explicit "keep" runs `--apply` without a move.
|
|
242
|
-
- **Nothing found:** say so in one line: "No design docs found. The designer reviews each run by general craft until you record some (`iamu-design`)."
|
|
243
|
-
- **No written design doc** (detection found only tokens or components, or nothing, and no design doc is recorded): after the question above, or in its place when there was nothing to record,
|
|
286
|
+
- **Nothing found:** say so in one line: "No design docs found. The designer reviews each run by general craft until you record some (`iamu-design`) — strongly recommended so runs check your own tokens, components, and rules."
|
|
287
|
+
- **No written design doc** (detection found only tokens or components, or nothing, and no design doc is recorded): after the question above, or in its place when there was nothing to record, explain in one sentence why a design doc makes Iamu much stronger (*"With a `DESIGN.md`, Iamu's designer checks your own tokens, components, states, and rules on every run instead of only general craft"*), and ask one question the user may skip:
|
|
288
|
+
- If interactive user-prompt tools exist (`ask_question`, `ask_followup_question`), invoke it:
|
|
289
|
+
- **Question:** "There is no `DESIGN.md` yet. Iamu gets much sharper with one (every run checks your own tokens, components, states, and rules instead of generic craft). Start one from your code now?"
|
|
290
|
+
- **Options:**
|
|
291
|
+
1. "(Recommended) Yes — start `DESIGN.md` from my code now (auto-extracts tokens, components, states & formats, then asks 2–3 quick intent questions)"
|
|
292
|
+
2. "Not now — finish setup first (keep reminding me to run `iamu-design`)"
|
|
293
|
+
3. "No — review by general craft only and stop offering `DESIGN.md`"
|
|
294
|
+
- Otherwise (text-only/CLI):
|
|
295
|
+
"There is no design doc yet, and Iamu gets much sharper with one (checking your own tokens, components, and rules on every run). Shall I start one from your code? I auto-extract your tokens, components, states, formats and contrast clashes from code, then ask 2–3 quick questions on feel and clash rules. **Yes** (recommended) / **not now** (keep recommending `iamu-design`) / **no**."
|
|
296
|
+
- **Yes:** follow the `iamu-design` skill, mode E, then carry on with setup.
|
|
297
|
+
- **Not now:** run nothing; it stays recommended at the end of setup, in `iamu status`, and after runs until a design doc is recorded.
|
|
298
|
+
- **No:** run `iamu design template --decline`, so runs stop offering it.
|
|
244
299
|
|
|
245
300
|
Do not assess design docs here, and never write one without the user filling it in (mode E). Never block setup on this step.
|
|
246
301
|
|
|
@@ -250,16 +305,16 @@ Do not assess design docs here, and never write one without the user filling it
|
|
|
250
305
|
|
|
251
306
|
Explain the journey model: an Iamu test binds a **Persona** (with habits, pressures, and impatience) to a **Situation** (a realistic high-stakes problem) with an **Evaluation Checkpoint** (a concrete binary question for the AI judge).
|
|
252
307
|
|
|
253
|
-
|
|
308
|
+
Split this into two quick questions: first agree on **who and what situation** to test (Step 3a), then confirm **which route and account** the walk starts with (Step 3b).
|
|
254
309
|
|
|
255
|
-
|
|
256
|
-
You MUST output the baseline test proposal to the chat stream FIRST. In the SAME turn, invoke `ask_question` with the full journey spec embedded in the `question` argument:
|
|
310
|
+
#### Step 3a: Persona & Situation (Baseline Test Proposal)
|
|
257
311
|
|
|
258
|
-
|
|
259
|
-
Step 3/3: Co-Design Your First Test Journey
|
|
260
|
-
Why this matters: Validates whether a real human with specific operational pressures can complete a critical task without getting blocked or frustrated.
|
|
312
|
+
Identify the top 2–3 high-stakes user workflows in the codebase. Lead with the strongest one in full detail as the recommended `Baseline Test Proposal`, and offer the other 1–2 discovered workflows as direct selectable options in the prompt so the user can switch flows in one click if they want to start elsewhere.
|
|
261
313
|
|
|
262
|
-
|
|
314
|
+
Output the proposed journey to the chat stream first, then invoke `ask_question` in the same turn:
|
|
315
|
+
|
|
316
|
+
```text
|
|
317
|
+
Step 3a: Baseline Test Proposal
|
|
263
318
|
Capability: <Capability Name> (<What staff can do in this area>)
|
|
264
319
|
Persona: <Name> (<Role / Title>)
|
|
265
320
|
• Reality: <Working context, physical habits, tenure, pressures>
|
|
@@ -267,40 +322,64 @@ Baseline Test Proposal (WYSIWGW Spec-Sheet):
|
|
|
267
322
|
Situation: <Outline Title>
|
|
268
323
|
• Tension: <Core problem-space tension in X vs Y format — no UI button names>
|
|
269
324
|
• Evaluation: <Concrete binary yes/partial/no question for evaluator>
|
|
270
|
-
Walk Surface: Entry: `<entry_path>` | Target: `<target_url>`
|
|
271
|
-
Auth Precondition: <Detected auth mechanism, e.g. "Dev persona switcher (localStorage.sf_dev_persona = 'anna')" or "Public">
|
|
272
325
|
Walker Brief: "<2–3 sentence first-person test brief that will be written verbatim to disk. States immediate intent and constraints before opening tool. Zero UI terminology.>"
|
|
273
326
|
```
|
|
274
327
|
|
|
275
|
-
**CRITICAL: Embed Journey Spec in Interactive Prompt Modal:**
|
|
276
328
|
- If interactive user-prompt tools exist (`ask_question`, `ask_followup_question`), invoke it:
|
|
277
329
|
- **Question:**
|
|
278
330
|
"""
|
|
279
|
-
Step
|
|
331
|
+
Step 3a: Confirm First Test Scenario
|
|
280
332
|
|
|
281
|
-
|
|
282
|
-
|
|
283
|
-
|
|
284
|
-
|
|
285
|
-
Evaluation Check: <Concrete binary yes/partial/no question for evaluator>
|
|
286
|
-
Walk Surface: Entry: `<entry_path>` | Target: `<target_url>`
|
|
287
|
-
Auth Precondition: <Detected auth mechanism>
|
|
333
|
+
Persona: <Name> (<Role / Title>) — Friction: <Diagnostic friction trigger>
|
|
334
|
+
Situation: <Outline Title> (<Capability Name>)
|
|
335
|
+
Core Tension: <Core problem-space tension in X vs Y format>
|
|
336
|
+
Evaluation Check: <Concrete binary yes/partial/no question for evaluator>
|
|
288
337
|
|
|
289
338
|
WALKER BRIEF:
|
|
290
339
|
"<2–3 sentence first-person test brief>"
|
|
291
340
|
|
|
292
|
-
|
|
341
|
+
Test this persona and situation first?
|
|
293
342
|
"""
|
|
294
343
|
- **Options:**
|
|
295
|
-
1. "
|
|
296
|
-
2. "
|
|
297
|
-
3. "
|
|
298
|
-
4. "Skip scenario for now"
|
|
344
|
+
1. "(Recommended) Yes — test <Name> in '<Outline Title>'"
|
|
345
|
+
2. "Test another flow instead: <Candidate Flow 2 — Role & Situation from codebase>"
|
|
346
|
+
3. "Test a different user role or workflow (I'll describe it)"
|
|
347
|
+
4. "Skip scenario creation for now"
|
|
299
348
|
- Otherwise (text-only/CLI):
|
|
300
|
-
|
|
301
|
-
|
|
349
|
+
Ask whether to use `<Name>` and `<Outline Title>`, pick `<Candidate Flow 2>`, or describe another role/situation to test first.
|
|
350
|
+
|
|
351
|
+
**If Option 2 or Option 3 is chosen:** draft the persona and situation for the chosen flow and confirm before continuing to Step 3b.
|
|
352
|
+
|
|
353
|
+
#### Step 3b: Confirm Starting Route & Persona Login (`entry_path` & `persona.access`)
|
|
354
|
+
|
|
355
|
+
Now verify the exact landing route and user account for `<Name>` (using the login method established in Step 1b):
|
|
356
|
+
|
|
357
|
+
```text
|
|
358
|
+
Step 3b: Walk Surface & Persona Login
|
|
359
|
+
Walk Surface: Entry: `<entry_path>` | Target: `<confirmed_target_url>`
|
|
360
|
+
Auth Precondition: <Specific account/role for <Name> using Step 1b's auth strategy, e.g. "coordinator@site-a.test" or "Interactive manual login" or "Public">
|
|
361
|
+
```
|
|
362
|
+
|
|
363
|
+
- If interactive user-prompt tools exist (`ask_question`, `ask_followup_question`), invoke it:
|
|
364
|
+
- **Question:**
|
|
365
|
+
"""
|
|
366
|
+
Step 3b: Confirm Starting Route & Account for <Name>
|
|
367
|
+
|
|
368
|
+
Starting URL: <confirmed_target_url><entry_path> (entry path: `<entry_path>`)
|
|
369
|
+
Auth Precondition: <Specific account/role for <Name>, or "Interactive login" / "Public">
|
|
370
|
+
|
|
371
|
+
Where should <Name>'s walkthrough start, and as which user?
|
|
372
|
+
"""
|
|
373
|
+
- **Options:**
|
|
374
|
+
1. "(Recommended) Start at `<entry_path>` (<confirmed_target_url><entry_path>) as <persona account or auth mode>"
|
|
375
|
+
2. "Start on a different route (e.g. `/` or `/login`)"
|
|
376
|
+
3. "Use a different account or role for <Name>"
|
|
377
|
+
- Otherwise (text-only/CLI):
|
|
378
|
+
Ask: `Start <Name> at <confirmed_target_url><entry_path> as <account/auth>? Reply to confirm, or give the entry path / account to use.`
|
|
379
|
+
|
|
380
|
+
**If Option 2 or Option 3 is chosen:** ask for the entry path (must start with `/`, e.g. `/dashboard`) or the account/role identifier for `<Name>`, and use their answer below.
|
|
302
381
|
|
|
303
|
-
**
|
|
382
|
+
**Scaffold & Persist Entities:**
|
|
304
383
|
1. **Scaffold Primary Capability** (if not already existing):
|
|
305
384
|
```bash
|
|
306
385
|
iamu new capability --name "<Capability Name>" --description "<What staff can do in this area>" --json
|
|
@@ -311,11 +390,11 @@ Baseline Test Proposal (WYSIWGW Spec-Sheet):
|
|
|
311
390
|
```
|
|
312
391
|
3. **Scaffold Persona**:
|
|
313
392
|
```bash
|
|
314
|
-
iamu new persona --name "<Name>" --role "<Role>" --disposition "<Disposition>" --body "<2-4 sentence narrative detailing technical habits, patience level, and specific friction triggers>" [--access "<account-or-role>"] --json
|
|
393
|
+
iamu new persona --name "<Name>" --role "<Role>" --disposition "<Disposition>" --body "<2-4 sentence narrative detailing technical habits, patience level, and specific friction triggers>\n\n## Today\n<how they do this job today without the product, and what that costs or gives them>" [--access "<confirmed-account-or-role>"] --json
|
|
315
394
|
```
|
|
316
395
|
4. **Scaffold Bound Scenario**:
|
|
317
396
|
```bash
|
|
318
|
-
iamu new scenario --name "<persona-slug>--<outline-slug>" --outline "<outline-slug>" --persona "<persona-slug>" --entry "<
|
|
397
|
+
iamu new scenario --name "<persona-slug>--<outline-slug>" --outline "<outline-slug>" --persona "<persona-slug>" --entry "<confirmed_entry_path>" --body "<Persona's first-person test brief describing what they intend to achieve>" --json
|
|
319
398
|
```
|
|
320
399
|
5. **Validate all entities**:
|
|
321
400
|
```bash
|
|
@@ -343,7 +422,7 @@ Render the output:
|
|
|
343
422
|
iamu/
|
|
344
423
|
├── testability.md [safety fence active: <target_url>]
|
|
345
424
|
├── context.yml [domain: <domain>]
|
|
346
|
-
├── design/sources.json [<N> design docs] (
|
|
425
|
+
├── design/sources.json [<N> design docs] (when recorded; if no DESIGN.md yet and not declined: [no DESIGN.md yet — run iamu-design to unlock project rule reviews])
|
|
347
426
|
├── personas/
|
|
348
427
|
│ └── <persona-slug>.md
|
|
349
428
|
├── outlines/
|
|
@@ -357,9 +436,9 @@ Next step:
|
|
|
357
436
|
iamu-run <persona-slug>--<outline-slug>
|
|
358
437
|
|
|
359
438
|
Other commands when you're ready:
|
|
439
|
+
• iamu-design Create or record DESIGN.md from your code (strongly recommended if not set yet: makes every run check your own tokens, components, and rules instead of general craft)
|
|
360
440
|
• iamu-new Draft a scenario for a different product flow
|
|
361
441
|
• iamu-propose Brainstorm high-risk workflows and blind spots
|
|
362
|
-
• iamu-design Record the project's design docs, review a page, decide proposed edits
|
|
363
442
|
• iamu doctor Diagnose test environment and target health
|
|
364
443
|
<If cloud_linked: true>
|
|
365
444
|
• iamu sync Sync completed runs and findings to Iamu Cloud
|
|
@@ -68,22 +68,41 @@ Inspect the project to understand what the system under test already supports:
|
|
|
68
68
|
4. **Running Dev Server:**
|
|
69
69
|
- Verify the target URL (e.g. `http://localhost:3000`). If a local server is running, check what port it uses.
|
|
70
70
|
|
|
71
|
-
### 3. Audit Gaps &
|
|
71
|
+
### 3. Audit Gaps & Confirm Target, Entry & Login
|
|
72
72
|
|
|
73
73
|
Present an honest, concise audit of what the scenario needs vs. what the system currently provides:
|
|
74
74
|
|
|
75
75
|
```markdown
|
|
76
76
|
### Staging Audit for `<scenario-slug>`
|
|
77
77
|
|
|
78
|
-
- **
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
- **
|
|
82
|
-
- **World State:** 5 open incidents, 1 serious and unassigned.
|
|
83
|
-
- Current DB / Seed: Default seed only creates 0 incidents, or empty demo state.
|
|
78
|
+
- **Target URL:** `http://localhost:3000` (`pnpm dev`, from `iamu/testability.md`)
|
|
79
|
+
- **Entry Route:** `/incidents` (verified in `src/app/(dashboard)/incidents/page.tsx`)
|
|
80
|
+
- **Persona Login:** Maria Gomez (Shift Coordinator) → `coordinator@site-a.test` (`dev_switcher` / `test_accounts` / `interactive` / `public`)
|
|
81
|
+
- **World State:** 5 open incidents, 1 serious and unassigned (Current DB / Seed: 0 incidents)
|
|
84
82
|
```
|
|
85
83
|
|
|
86
|
-
|
|
84
|
+
#### Step 3a: Confirm Target URL, Entry Route & Persona Login
|
|
85
|
+
Never bury the Target URL, Entry Route, or Login mechanism inside a world-state prompt where the user cannot easily override them. Before asking about database seeds, confirm where the walk starts and how it signs in (one focused question):
|
|
86
|
+
|
|
87
|
+
- If your environment provides an interactive user-prompt tool (e.g. `ask_question`, `ask_followup_question`), invoke it:
|
|
88
|
+
- **Question:** "Start URL & Login for `<scenario-slug>`: Should Iamu start at `<target_url><entry_route>` and sign in via `<detected_access_or_strategy>`?"
|
|
89
|
+
- **Options:**
|
|
90
|
+
1. "[Recommended] Confirm `<target_url><entry_route>` & `<detected_access_or_strategy>`"
|
|
91
|
+
2. "Change Target URL or Entry Route — tell me the URL or `/path`"
|
|
92
|
+
3. "Change how tests log in as `<persona.name>` (e.g. interactive Chrome login, different test account, or dev switcher)"
|
|
93
|
+
4. "No login needed (public / unauthenticated flow)"
|
|
94
|
+
- Otherwise, state the recommended Target URL, Entry Route, and Login mechanism and ask the user to confirm or override any of them.
|
|
95
|
+
|
|
96
|
+
Persist any Target URL, Entry Route, or Login updates immediately:
|
|
97
|
+
```bash
|
|
98
|
+
iamu set testability --target <url> --auth-strategy <strategy> --access "<project-auth-mechanism>"
|
|
99
|
+
iamu set scenario <slug> --entry "/<route>"
|
|
100
|
+
iamu set persona <persona-slug> --access "<account-or-role-key>"
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
#### Step 3b: Present the Strategic Fork (World State & Seeds)
|
|
104
|
+
|
|
105
|
+
Now present the **Strategic Fork** for the scenario's world state. Do not be an unyielding bureaucrat. Give the user clear options:
|
|
87
106
|
|
|
88
107
|
#### Option A: Build Testability (The High-Integrity Path)
|
|
89
108
|
*Recommend this when a clean seed script or preset is easy to add.*
|
|
@@ -106,23 +125,10 @@ Now present the **Strategic Fork** to the user. Do not be an unyielding bureaucr
|
|
|
106
125
|
- Mark `--state-description "No particular data needed"`.
|
|
107
126
|
|
|
108
127
|
**Dual-Surface Staging Gate (Sequencing Rule):**
|
|
109
|
-
You MUST output the staging options to the chat stream FIRST. In the SAME turn, invoke `ask_question` with the
|
|
128
|
+
You MUST output the staging options to the chat stream FIRST. In the SAME turn, invoke `ask_question` with the world-state options:
|
|
110
129
|
|
|
111
130
|
- If your environment provides an interactive user-prompt tool (e.g. `ask_question`, `ask_followup_question`), invoke it:
|
|
112
|
-
- **Question:**
|
|
113
|
-
"""
|
|
114
|
-
Staging Plan for '<scenario-slug>':
|
|
115
|
-
|
|
116
|
-
Entry Route: <route>
|
|
117
|
-
Required World State: <Description of required world in domain words>
|
|
118
|
-
|
|
119
|
-
Staging Options:
|
|
120
|
-
• Option A (Testability): <concrete seed or mock fixture proposal>
|
|
121
|
-
• Option B (Semantic Squinting): <squinting rule formula>
|
|
122
|
-
• Option C (Zero Setup): <why flow is public or self-contained>
|
|
123
|
-
|
|
124
|
-
How should we stage '<scenario-slug>' for walkthrough?
|
|
125
|
-
"""
|
|
131
|
+
- **Question:** "World State for '<scenario-slug>' (<Required World State in domain words>): How should we stage the data?"
|
|
126
132
|
- **Options:**
|
|
127
133
|
1. "[Recommended] Option A: Build Testability (<brief seed proposal>)"
|
|
128
134
|
2. "Option B: Semantic Squinting (test immediately against current dev database)"
|
|
@@ -29,6 +29,7 @@ When the user asks for project status, lead with an executive tally banner, foll
|
|
|
29
29
|
|
|
30
30
|
```
|
|
31
31
|
Project Status: <P> Personas · <S> Scenarios (<R> Tested, <U> Untested) · <F> Open Findings · <C> Capabilities
|
|
32
|
+
Design Docs: <recorded design doc paths, or "none (designer reviews by general craft only — run iamu-design so runs check your own design rules)">
|
|
32
33
|
Product Map: <T> Surfaces (<R> Reached · <U> Unreached) [if explore map exists]
|
|
33
34
|
```
|
|
34
35
|
|
|
@@ -68,6 +69,7 @@ Conclude with exact copy-pasteable commands:
|
|
|
68
69
|
```
|
|
69
70
|
Next actions:
|
|
70
71
|
iamu-run <scenario-slug> # Run a walkthrough for this scenario
|
|
72
|
+
iamu-design # Record or create DESIGN.md so runs check your own design rules
|
|
71
73
|
iamu-propose # Propose missing personas or scenarios
|
|
72
74
|
iamu-new # Draft a new scenario or persona
|
|
73
75
|
iamu-findings # Triage open findings
|
package/dist/templates.d.ts
CHANGED
|
@@ -9,7 +9,7 @@ export declare function getTemplatesRoot(): string;
|
|
|
9
9
|
export declare function getTemplateDir(subdir: "skills" | "prompts"): string;
|
|
10
10
|
export declare function getTemplateFile(subdir: "skills" | "prompts", filename: string): string;
|
|
11
11
|
/** Prompt templates `iamu init` installs and `iamu update` refreshes in `iamu/.prompts/`. */
|
|
12
|
-
export declare const PROMPT_FILES: readonly ["walk.md", "persona.md", "outline.md", "combine.md", "complete.md", "
|
|
12
|
+
export declare const PROMPT_FILES: readonly ["walk.md", "persona.md", "outline.md", "combine.md", "complete.md", "advise.md", "design-draft.md", "design-review.md", "design-review-match.md", "designer.md"];
|
|
13
13
|
/**
|
|
14
14
|
* A prompt as the project has it: its `iamu/.prompts/<name>` copy (which
|
|
15
15
|
* a team may tune), else the bundled template.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@iamu-dev/iamu",
|
|
3
|
-
"version": "0.8.0-beta.
|
|
3
|
+
"version": "0.8.0-beta.2",
|
|
4
4
|
"type": "module",
|
|
5
5
|
"description": "Test your product as realistic personas — scenario-driven development CLI",
|
|
6
6
|
"license": "MIT",
|
|
@@ -59,8 +59,8 @@
|
|
|
59
59
|
"typescript": "^5.8.0",
|
|
60
60
|
"vitest": "^4.1.11",
|
|
61
61
|
"@iamu-dev/local-store": "0.1.0",
|
|
62
|
-
"@iamu/
|
|
63
|
-
"@iamu
|
|
62
|
+
"@iamu-dev/protocol": "0.1.0",
|
|
63
|
+
"@iamu/tsconfig": "0.1.0"
|
|
64
64
|
},
|
|
65
65
|
"scripts": {
|
|
66
66
|
"build": "tsup && tsc -b --force tsconfig.build.json --emitDeclarationOnly && node scripts/copy-assets.mjs",
|
|
@@ -1,64 +0,0 @@
|
|
|
1
|
-
# Evaluation methodology
|
|
2
|
-
|
|
3
|
-
You are the EVALUATOR of a walkthrough. A different agent (the walker) drove this walkthrough as a specific persona. That walker was DELIBERATELY NOT shown the human-test questions below — your job is to read the recorded evidence and answer the questions yourself, without confirmation bias.
|
|
4
|
-
|
|
5
|
-
Your output is the verdict on whether the product earns the scenario's "Human Test" — would this persona come back and use it voluntarily?
|
|
6
|
-
|
|
7
|
-
## Rules
|
|
8
|
-
|
|
9
|
-
- **Ground every verdict in evidence.** Cite specific step IDs and finding IDs. Quote specific phrases from `ui_fact`, `consequence`, or `persona_reasoning` when they're load-bearing. A verdict without grounding is rejected.
|
|
10
|
-
- **Justifications must be falsifiable.** A reader with the same evidence must be able to verify what you cited.
|
|
11
|
-
- **Disagree with the walker if the evidence supports it.** The walker's summaries are inputs, not verdicts. If a finding contradicts a walker claim, your verdict reflects the finding.
|
|
12
|
-
|
|
13
|
-
## Verdict calibration
|
|
14
|
-
|
|
15
|
-
- **`no`** — evidence uniformly negative on the critical-path task. Don't manufacture nuance.
|
|
16
|
-
- **`yes`** — evidence shows the persona completed the task without measurable friction.
|
|
17
|
-
- **`partial`** — genuinely different outcomes on different surfaces. NOT "mixed but ultimately negative" — that's `no`.
|
|
18
|
-
- **`not_applicable`** — cannot determine from available evidence. Use instead of guessing.
|
|
19
|
-
|
|
20
|
-
## Input
|
|
21
|
-
|
|
22
|
-
Read `iamu/.runs/<run-id>/canonical.json` for:
|
|
23
|
-
- `steps[]` — what the walker observed (persona_reasoning, actions, screenshots)
|
|
24
|
-
- `findings[]` — what helped, hindered, was missing, or was incorrect
|
|
25
|
-
- `human_tests[]` — the questions to answer (verdict is currently null)
|
|
26
|
-
|
|
27
|
-
Read the outline file for the full human_test questions if needed.
|
|
28
|
-
|
|
29
|
-
## Output
|
|
30
|
-
|
|
31
|
-
Write the evaluation results to a temporary file, then use it to update canonical.json.
|
|
32
|
-
|
|
33
|
-
Write `/tmp/iamu-eval-<run-id>.json`:
|
|
34
|
-
|
|
35
|
-
```json
|
|
36
|
-
{
|
|
37
|
-
"verdicts": [
|
|
38
|
-
{
|
|
39
|
-
"human_test_id": "<id>",
|
|
40
|
-
"verdict": "yes|partial|no|not_applicable",
|
|
41
|
-
"reasoning": "Steps 3-5 show... Finding X demonstrates...",
|
|
42
|
-
"grounding_step_ids": ["step-3", "step-4", "step-5"],
|
|
43
|
-
"grounding_finding_ids": ["<finding-uuid>"]
|
|
44
|
-
}
|
|
45
|
-
],
|
|
46
|
-
"overall": {
|
|
47
|
-
"question": "Would <persona> come back?",
|
|
48
|
-
"answer": "<one paragraph grounded in specific observations>",
|
|
49
|
-
"reasoning": "<supporting argument>"
|
|
50
|
-
},
|
|
51
|
-
"provenance": {
|
|
52
|
-
"judge_mode": "subagent",
|
|
53
|
-
"model": "<model that ran this evaluation>",
|
|
54
|
-
"prompt_id": "judge",
|
|
55
|
-
"prompt_version": "1.0.0"
|
|
56
|
-
}
|
|
57
|
-
}
|
|
58
|
-
```
|
|
59
|
-
|
|
60
|
-
Then update `canonical.json`:
|
|
61
|
-
- Set each `human_tests[].verdict`, `.reasoning`, `.grounding_step_ids`, `.grounding_finding_ids`
|
|
62
|
-
- Set `human_test_overall.answer` and `.reasoning`
|
|
63
|
-
- Set `provenance.judge_prompt` and `provenance.judge_mode`
|
|
64
|
-
- Regenerate `report.md` with verdicts filled in
|