@iamu-dev/iamu 0.8.0-beta.1 → 0.8.0-beta.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +36 -6
- package/dist/bin.js +13900 -12243
- package/dist/commands/advise.d.ts +4 -2
- package/dist/commands/agent-commands.d.ts +22 -0
- package/dist/commands/browse-driver.d.ts +5 -1
- package/dist/commands/browse.d.ts +37 -1
- package/dist/commands/cloud-push.d.ts +3 -3
- package/dist/commands/complete.d.ts +21 -0
- package/dist/commands/completion.d.ts +2 -0
- package/dist/commands/context.d.ts +19 -0
- package/dist/commands/dev-script.d.ts +14 -0
- package/dist/commands/doctor.d.ts +6 -0
- package/dist/commands/edit-entity.d.ts +2 -0
- package/dist/commands/evaluate.d.ts +10 -3
- package/dist/commands/finding-stability.d.ts +1 -0
- package/dist/commands/findings.d.ts +9 -0
- package/dist/commands/init.d.ts +11 -2
- package/dist/commands/instructions-walk.d.ts +2 -0
- package/dist/commands/instructions.d.ts +2 -11
- package/dist/commands/list.d.ts +3 -0
- package/dist/commands/match.d.ts +8 -5
- package/dist/commands/new-entity.d.ts +2 -0
- package/dist/commands/new-finding.d.ts +2 -0
- package/dist/commands/persona-today.d.ts +20 -0
- package/dist/commands/post-walk.d.ts +18 -0
- package/dist/commands/pull.d.ts +1 -1
- package/dist/commands/set-entity.d.ts +15 -3
- package/dist/commands/skill-command.d.ts +13 -0
- package/dist/commands/step.d.ts +2 -0
- package/dist/commands/summary.d.ts +2 -0
- package/dist/commands/triage.d.ts +19 -2
- package/dist/index.js +13878 -12231
- package/dist/templates/prompts/complete.md +2 -3
- package/dist/templates/prompts/design-review.md +1 -1
- package/dist/templates/prompts/designer.md +14 -7
- package/dist/templates/prompts/persona.md +2 -1
- package/dist/templates/prompts/walk.md +26 -8
- package/dist/templates/skills/iamu-findings.md +17 -28
- package/dist/templates/skills/iamu-new.md +9 -9
- package/dist/templates/skills/iamu-run.md +65 -42
- package/dist/templates/skills/iamu-setup.md +2 -2
- package/dist/templates/skills/iamu-stage.md +1 -1
- package/dist/templates.d.ts +1 -1
- package/package.json +3 -3
- package/dist/templates/prompts/evaluate.md +0 -64
|
@@ -18,6 +18,8 @@ This skill resolves what scenario to run following a clear **Target Scenario Res
|
|
|
18
18
|
|
|
19
19
|
**`--critic`:** when `$ARGUMENTS` contains `--critic`, take it out before resolving the scenario. This run is one critic pass in a builder ↔ critic loop (a gauntlet loop) the user orchestrates: the builder needs the verdict and the findings fast, not fixes or a design review. Skip the advisor (C2) entirely and skip the designer's review (C3): after A, record `iamu designer-record <run-id> --skipped "critic mode"`. Run A, B, C, D (and E in linked mode) as usual. In step 5, present the glance without the Advisor and Designer lines, and do not offer **directions** or **designer**. Per invocation only, like `--no-ux`. When the loop looks converged (every human test passes, or no open major or critical findings), add one line: `Loop looks converged: run once without --critic for the advisor's and designer's review.`
|
|
20
20
|
|
|
21
|
+
**`--fresh`:** when `$ARGUMENTS` contains `--fresh`, take it out before resolving the scenario. Clear the app's storage after `iamu browse start` (step 4) so the walk starts from clean storage, not from what an earlier walk left behind. `iamu browse` only; per invocation, like `--no-ux`.
|
|
22
|
+
|
|
21
23
|
$ARGUMENTS
|
|
22
24
|
|
|
23
25
|
## Prerequisites
|
|
@@ -38,7 +40,7 @@ When more than one qualifies, prefer in this order:
|
|
|
38
40
|
|
|
39
41
|
1. **One attached to the user's own running Chrome** (a Chrome DevTools-style MCP or a Chrome extension bridge). The user can watch the walk live, the session and dev state they already have is present, and console and network output are available to tell a tooling glitch from a product bug.
|
|
40
42
|
2. **One that launches its own browser** (a Playwright-style MCP). Works, but starts from a blank profile and the user cannot see it happen.
|
|
41
|
-
3. **`iamu browse`** — the CLI's own shell-driven browser, when no tool above can save a screenshot to a file (many hosts' browser tools return screenshots to the model only, and every step then records `screenshot: null`). It drives the installed Chrome in a visible window: `iamu browse start --run <run-id>` after each `start-run` (step 4), then `goto`, `snapshot` (ARIA roles and names), `click <role> "<name>"`, `click-text`, `fill <role> "<name>" "<text>" [--submit]`, `press`, `scroll`, `back`, each with `--shot <
|
|
43
|
+
3. **`iamu browse`** — the CLI's own shell-driven browser, when no tool above can save a screenshot to a file (many hosts' browser tools return screenshots to the model only, and every step then records `screenshot: null`). It drives the installed Chrome in a visible window: `iamu browse start --run <run-id>` after each `start-run` (step 4), then `goto`, `snapshot` (ARIA roles and names; `--indexed` numbers the interactive elements `[#N]`), `click-index <N>`, `fill-index <N> "<text>" [--submit]`, `click <role> "<name>"`, `click-text`, `fill <role> "<name>" "<text>" [--submit]`, `select <role> "<name>" "<option>"`, `press`, `scroll`, `back`, each with `--shot <path>`; `iamu browse stop` when done. Start it yourself before dispatching the walker, never leave that to the walker; `iamu instructions walk --surface browse` (the default) writes the command list into the walker prompt.
|
|
42
44
|
|
|
43
45
|
Tell the user in one line which one you picked and why. If the project's own agent docs name a preferred browser tool or constrain where screenshots may be written, that wins over the order above.
|
|
44
46
|
|
|
@@ -134,18 +136,12 @@ If the resolved scenario has `persona: null`, prompt to bind before executing:
|
|
|
134
136
|
"This scenario is a draft with no persona bound. I recommend binding `<Existing Persona 1>` (<Role>). Proceeding with this persona — name another persona or type 'new' to propose candidates."
|
|
135
137
|
|
|
136
138
|
1. Read `iamu/personas/` directory or run `iamu list personas`; present as pickable list + "new" option.
|
|
137
|
-
2. Picked existing → bind that persona
|
|
138
|
-
3. Picked "new" → generate 3 persona candidates grounded in the situation narrative and project context → user picks one + refines →
|
|
139
|
-
4. After binding the persona, generate a 3
|
|
140
|
-
|
|
141
|
-
```
|
|
142
|
-
|
|
143
|
-
id: <uuid>
|
|
144
|
-
slug: maria-gomez--busy-shift-handoff
|
|
145
|
-
outline: busy-shift-handoff
|
|
146
|
-
persona: maria-gomez
|
|
147
|
-
---
|
|
148
|
-
I've got five open incidents and my relief is already here...
|
|
139
|
+
2. Picked existing → bind that persona and write the scenario narrative using `iamu set scenario <slug> --persona <persona-slug> --body "<narrative>"`.
|
|
140
|
+
3. Picked "new" → generate 3 persona candidates grounded in the situation narrative and project context → user picks one + refines → create the persona via `iamu new persona --name "<Name>" --role "<Role>" --disposition "<Disposition>" --body "<background and ## Today>"` → bind.
|
|
141
|
+
4. After binding the persona, generate a 3–5 sentence first-person brief and persist both the bound persona and the scenario markdown body via `iamu set`:
|
|
142
|
+
|
|
143
|
+
```bash
|
|
144
|
+
iamu set scenario <slug> --persona <persona-slug> --body "I've got five open incidents and my relief is already here..."
|
|
149
145
|
```
|
|
150
146
|
|
|
151
147
|
After bind + brief, the scenario is ready to execute. Continue to step 3.
|
|
@@ -155,7 +151,7 @@ After bind + brief, the scenario is ready to execute. Continue to step 3.
|
|
|
155
151
|
**Web** — determine and confirm the target URL:
|
|
156
152
|
1. If the user passed `--target <url>` (or a URL in `$ARGUMENTS`), persist it with `iamu set testability --target <url>` (and clear any stale scenario override with `iamu set scenario <slug> --no-target`) and use that.
|
|
157
153
|
2. Otherwise, check the scenario's `target_url`, then check `iamu/testability.md` for `target_url` and `status`.
|
|
158
|
-
3. **Unverified or missing target gate:** If `iamu/testability.md` is missing, has no `target_url`, or has `status: draft` (an unverified guess from `iamu init`), do NOT blindly trust the guess. Ask one focused question to confirm the Target URL and Dev Command before probing or booting:
|
|
154
|
+
3. **Unverified or missing target gate:** If `iamu/testability.md` is missing, has no `target_url`, or has `status: draft` (an unverified guess from `iamu init`), do NOT blindly trust the guess. (`status: configured` means a person set the URL but it was not answering then — keep it, and let the preflight check below find out whether it answers now.) Ask one focused question to confirm the Target URL and Dev Command before probing or booting:
|
|
159
155
|
- If your environment provides an interactive user-prompt tool (e.g. `ask_question`, `ask_followup_question`), invoke it:
|
|
160
156
|
- Question: "Target URL & Dev Command: Where does this app run locally?"
|
|
161
157
|
- Options:
|
|
@@ -163,7 +159,7 @@ After bind + brief, the scenario is ready to execute. Continue to step 3.
|
|
|
163
159
|
2. "Same command (`<dev_command>`), different port or URL — tell me the URL"
|
|
164
160
|
3. "Use a staging or remote URL — tell me the URL"
|
|
165
161
|
- Otherwise, state the detected URL and command and ask the user to confirm or provide the right URL.
|
|
166
|
-
- Persist the confirmed target immediately
|
|
162
|
+
- Persist the confirmed target immediately. `iamu/testability.md` becomes `status: verified` when the URL answers, else `status: configured`:
|
|
167
163
|
```bash
|
|
168
164
|
iamu set testability --target <url> --dev-command "<dev_command>"
|
|
169
165
|
```
|
|
@@ -236,10 +232,10 @@ Never silently guess login credentials or auth strategy without confirming with
|
|
|
236
232
|
3. "Interactive login in Chrome (SSO / 2FA / manual sign-in)"
|
|
237
233
|
4. "No login needed (public / unauthenticated)"
|
|
238
234
|
- Otherwise, state the recommended login mechanism and ask the user to confirm or specify how to log in.
|
|
239
|
-
3. Persist what the user confirmed: the project mechanism into `iamu/testability.md` via `iamu set testability --auth-strategy <strategy> --access "<mechanism>"`, and the persona's own account/role key with `iamu set persona <slug> --access "<key>"`. Passwords and tokens never go in either — name where they live (an env var, a vault entry) instead.
|
|
235
|
+
3. Persist what the user confirmed: the project mechanism into `iamu/testability.md` via `iamu set testability --auth-strategy <strategy> --access "<mechanism>"` (where `<strategy>` is one of `dev_switcher`, `test_accounts`, `interactive`, `public`), and the persona's own account/role key with `iamu set persona <slug> --access "<key>"`. Passwords and tokens never go in either — name where they live (an env var, a vault entry) instead.
|
|
240
236
|
|
|
241
|
-
**Interactive Auth Handshake (for `
|
|
242
|
-
If `auth_strategy` is `
|
|
237
|
+
**Interactive Auth Handshake (for `interactive`, enterprise auth, SSO, 2FA, BankID):**
|
|
238
|
+
If `auth_strategy` is `interactive` or the product is behind enterprise login where automated dev persona switching is not configured:
|
|
243
239
|
1. Open the browser to the target URL (`iamu browse start --run <run-id>` or browser tool).
|
|
244
240
|
2. Prompt the human user:
|
|
245
241
|
> 👉 **Please log in as `<persona.role>` (`<persona.name>`) in the opened browser window.** Tell me once you reach the dashboard (e.g. "I'm logged in") to hand control to the walker...
|
|
@@ -275,7 +271,7 @@ iamu browse start --run <run-id>
|
|
|
275
271
|
|
|
276
272
|
It sizes the page as the device the persona's tags name (`mobile`/`phone`, `tablet`/`ipad`, `desktop`/`laptop`; `start-run` prints it as `device`) and resizes a browser left at another persona's size.
|
|
277
273
|
|
|
278
|
-
**Fresh walk
|
|
274
|
+
**Fresh walk (`--fresh`, `iamu browse` only).** When `--fresh` was given, or when running iterative passes against a browser left open (e.g. with `--critic`), clear the origin's storage after `browse start` so each walk starts from clean app storage:
|
|
279
275
|
|
|
280
276
|
```bash
|
|
281
277
|
iamu browse clear-storage --run <run-id>
|
|
@@ -304,13 +300,13 @@ A fresh subagent is NOT blind by default. On most hosts it still loads the proje
|
|
|
304
300
|
- **Child Session Suppression:** The walker must never execute parent workflow routines (`iamu init`, `iamu triage`, `iamu new`). All steps, findings, and summaries are recorded atomically via `iamu browse step`, `iamu new-finding`, and `iamu summary`.
|
|
305
301
|
|
|
306
302
|
**Model tiering & subagent dispatch:**
|
|
307
|
-
- **Walker (Browser Navigation):** High turn count,
|
|
308
|
-
- **Matcher:** Short-text classification (same problem or not) into strict JSON that `match-record` validates. Use the **fastest** tier (`haiku` in Claude Code, `flash_lite` in Antigravity, else the
|
|
309
|
-
- **Judge:** Evaluates the human tests from the walk's recorded steps and findings (`judge_prompt.md`, text-only reading comprehension). Use the **
|
|
303
|
+
- **Walker (Browser Navigation):** High turn count, and a judgement at every screen about what fits the persona's role. Use the **mid** tier (`sonnet` in Claude Code, `flash` in Antigravity) by default: on the role-fit benchmark in Claude Code it found the same problems as `haiku` about 1.7× faster, at a higher price per token. In `--critic` loops, where cost per walk matters more than the time of one walk, the **fastest** tier (`haiku` in Claude Code) is an option.
|
|
304
|
+
- **Matcher:** Short-text classification (same problem or not) into strict JSON that `match-record` validates. Use the **fastest** tier (`haiku` in Claude Code, `flash_lite` in Antigravity, else the mid tier).
|
|
305
|
+
- **Judge:** Evaluates the human tests from the walk's recorded steps and findings (`judge_prompt.md`, text-only reading comprehension). Use the **mid** tier (`sonnet` in Claude Code, `flash` in Antigravity).
|
|
310
306
|
|
|
311
307
|
```
|
|
312
308
|
# In Claude Code:
|
|
313
|
-
Agent({ description: "Walk <scenario-slug> as <persona.name>", model: "
|
|
309
|
+
Agent({ description: "Walk <scenario-slug> as <persona.name>", model: "sonnet", prompt: "<walker_prompt, verbatim>" })
|
|
314
310
|
|
|
315
311
|
# In Antigravity / generic subagent hosts:
|
|
316
312
|
invoke_subagent(Role="Persona Walker", Model="flash", Prompt="<walker_prompt, verbatim>")
|
|
@@ -389,12 +385,21 @@ iamu summary <run-id> --worked "What worked 1" --worked "What worked 2" --questi
|
|
|
389
385
|
|
|
390
386
|
**Post-walk pipeline (MANDATORY — do NOT skip any step):**
|
|
391
387
|
|
|
392
|
-
After the walker returns, you MUST complete ALL of the following before presenting findings to the user. Do NOT present findings early. Do NOT ask the user anything. Execute A, then B and C in parallel, then C2
|
|
388
|
+
After the walker returns, you MUST complete ALL of the following before presenting findings to the user. Do NOT present findings early. Do NOT ask the user anything. Execute A, then B and C in parallel, then C2; the designer's review (C3) starts right after A and runs beside B, C and C2. D and E wait until every record (B, C, C2, C3) is in, so the report you read and the run you publish are complete.
|
|
393
389
|
|
|
394
390
|
> **CRITICAL EXECUTION RULE (Zero Host File Tools):**
|
|
395
|
-
> Never invoke host edit or write tools (`write_to_file`, `create_file`, `replace_file_content`, etc.) on any files in `iamu/` (including run artifacts, steps, findings, outlines, personas, scenarios, or guidance). All files in `iamu/` must be generated and modified exclusively via `iamu` CLI commands (`iamu new`, `iamu set`, `iamu step`, `iamu new-finding`, `iamu evaluate-record`, `iamu match-record`, etc.). If an entity needs a field updated (e.g. adding a capability to an outline), run `iamu set <type> <slug> ...` instead of editing files by hand. Record subagent JSON outputs
|
|
391
|
+
> Never invoke host edit or write tools (`write_to_file`, `create_file`, `replace_file_content`, etc.) on any files in `iamu/` (including run artifacts, steps, findings, outlines, personas, scenarios, or guidance). All files in `iamu/` must be generated and modified exclusively via `iamu` CLI commands (`iamu new`, `iamu set`, `iamu step`, `iamu new-finding`, `iamu evaluate-record`, `iamu match-record`, `iamu post-walk`, etc.). If an entity needs a field updated (e.g. adding a capability to an outline), run `iamu set <type> <slug> ...` instead of editing files by hand. Record subagent JSON outputs via stdin heredoc (`--file - <<'EOF' ... EOF`), `--file <path>`, or `--data '<json>'`.
|
|
392
|
+
|
|
393
|
+
**Fast path — prepare all post-walk prompts in one hop:**
|
|
394
|
+
You can complete the run locally AND generate `match_prompt.md`, `judge_prompt.md`, `advise_prompt.md`, and `designer_prompt.md` in a single command:
|
|
395
|
+
|
|
396
|
+
```bash
|
|
397
|
+
iamu post-walk <run-id> --json
|
|
398
|
+
```
|
|
399
|
+
|
|
400
|
+
`iamu post-walk <run-id> --json` returns the `needed` status, `prompt_file_abs`, and `answer_file_abs` for `match`, `evaluate`, `advise`, and `designer` in one JSON object (pass `--critic` or `--no-ux` when applicable). If subagent JSON outputs are later written to their `answer_file_abs` paths in `<run-dir>/`, running `iamu post-walk <run-id> --json` again records all of them in lock-safe order automatically.
|
|
396
401
|
|
|
397
|
-
**A. Assemble canonical + report (local).** Use `--no-push` so post-processing stays fast, local, and synchronous:
|
|
402
|
+
**A. Assemble canonical + report (local).** Use `--no-push` so post-processing stays fast, local, and synchronous (already done if you ran `iamu post-walk <run-id> --json`):
|
|
398
403
|
|
|
399
404
|
```bash
|
|
400
405
|
iamu complete <run-id> --no-prompt --no-push --json
|
|
@@ -404,48 +409,59 @@ This reads all steps, findings, and entity files, assembles `canonical.json`, ge
|
|
|
404
409
|
|
|
405
410
|
**If the result has `blocked`**, the walker could not start the walk as it was set up (`iamu summary --blocked`): it could not become the persona, or the world the situation needs was not there. That is a setup problem, not a product finding. Skip B and C — a blocked run is never compared with earlier runs and never counts as a re-run. Tell the user in one line what was missing (`blocked.reason`), then run the `iamu-stage` skill with that reason as the starting point: fix the scenario's `state`, the persona's access, or the seed, and offer to run the scenario again. Findings the walker recorded before it stopped stay as they are. E still runs in linked mode; the report says the run was blocked.
|
|
406
411
|
|
|
407
|
-
**B and C run in parallel.** The matcher (B) and the judge (C) answer different questions from different inputs, and the judge never reads the matcher's output. Run `iamu match` and `iamu evaluate`
|
|
412
|
+
**B and C run in parallel.** The matcher (B) and the judge (C) answer different questions from different inputs, and the judge never reads the matcher's output. Run `iamu post-walk <run-id> --json` (or `iamu match` and `iamu evaluate`), then dispatch the needed subagents at the same time. Record their answers one after the other as they arrive (`match-record` and `evaluate-record` both rewrite the report; never run them at the same time). Keep them separate: the judge must not see earlier runs' findings, so its verdict cannot drift toward "it was fixed, so it passes".
|
|
408
413
|
|
|
409
414
|
**How to dispatch the matcher and the judge.** Do not paste the prompt into the dispatch; give the absolute path. Their whole prompt is: `Read <absolute path to match_prompt.md / judge_prompt.md> and answer exactly as it says. Everything you need is in that file. Do not open any other file, run any command or search any code. Reply with the JSON only.` Both are pure reading tasks; a subagent that goes looking for context (the repo, the CLI's source, other runs) costs minutes and can see things it must not.
|
|
410
415
|
|
|
411
|
-
**
|
|
416
|
+
**Avoid zsh quote-escaping failures when recording JSON:** Subagent outputs are 5–15 KB of JSON that frequently contain single quotes (`"Something's off"`) and escaped double quotes (`"\"Hall rules\""`). Prefer piping via a single-quoted heredoc (`--file - <<'EOF'`) or `--file <path>` so the shell never interprets quotes inside the JSON.
|
|
417
|
+
|
|
418
|
+
**B. Match findings to earlier runs.** Get the matcher bundle (if not already prepared via `iamu post-walk`):
|
|
412
419
|
|
|
413
420
|
```bash
|
|
414
421
|
iamu match <run-id> --json
|
|
415
422
|
```
|
|
416
423
|
|
|
417
|
-
If `needed` is `false` (no earlier findings for this scenario, or none this run), there is no matcher subagent. Otherwise, read the prompt from `iamu/.runs/<run-id>/match_prompt.md` directly and dispatch a matcher subagent with it as its only context (use the **fastest** model tier: `haiku` in Claude Code, `Model="flash_lite"` in Antigravity). Record its JSON answer via `--
|
|
424
|
+
If `needed` is `false` (no earlier findings for this scenario, or none this run), there is no matcher subagent. Otherwise, read the prompt from `iamu/.runs/<run-id>/match_prompt.md` directly and dispatch a matcher subagent with it as its only context (use the **fastest** model tier: `haiku` in Claude Code, `Model="flash_lite"` in Antigravity). Record its JSON answer via `--file - <<'EOF'` (or `--data`):
|
|
418
425
|
|
|
419
426
|
```bash
|
|
420
|
-
iamu match-record <run-id> --
|
|
427
|
+
iamu match-record <run-id> --file - <<'EOF'
|
|
428
|
+
<raw_matches_json>
|
|
429
|
+
EOF
|
|
430
|
+
# or: iamu match-record <run-id> --data '<raw_matches_json>'
|
|
421
431
|
```
|
|
422
432
|
|
|
423
433
|
This links recurrences to their earlier canonical finding, marks earlier findings the run's steps show fixed (only with cited steps), and re-derives the delta and report. If the matcher's output is rejected, show the error to the matcher once and retry; if it fails again, continue without it — the exact-text delta from A stands.
|
|
424
434
|
|
|
425
|
-
**C. Evaluate human tests.** Get the evaluation prompt:
|
|
435
|
+
**C. Evaluate human tests.** Get the evaluation prompt (if not already prepared via `iamu post-walk`):
|
|
426
436
|
|
|
427
437
|
```bash
|
|
428
438
|
iamu evaluate <run-id> --json
|
|
429
439
|
```
|
|
430
440
|
|
|
431
|
-
Read the evaluation prompt from `iamu/.runs/<run-id>/judge_prompt.md` directly. Dispatch the evaluator as a subagent with that prompt, at the same time as the matcher (use the **
|
|
441
|
+
Read the evaluation prompt from `iamu/.runs/<run-id>/judge_prompt.md` directly. Dispatch the evaluator as a subagent with that prompt, at the same time as the matcher (use the **mid** model tier: `sonnet` in Claude Code, `Model="flash"` in Antigravity). After the evaluator returns verdicts, record them via `--file - <<'EOF'` (or `--data`). Like every record command it stays local and never prompts; E publishes:
|
|
432
442
|
|
|
433
443
|
```bash
|
|
434
|
-
iamu evaluate-record <run-id> --
|
|
444
|
+
iamu evaluate-record <run-id> --file - <<'EOF'
|
|
445
|
+
<raw_verdicts_json>
|
|
446
|
+
EOF
|
|
447
|
+
# or: iamu evaluate-record <run-id> --data '<raw_verdicts_json>'
|
|
435
448
|
```
|
|
436
449
|
|
|
437
450
|
This links findings to human tests (`human_test_id`) and updates the report with judge verdicts.
|
|
438
451
|
|
|
439
|
-
**C2. Advise: fixes and directions.** Skip in `--critic` mode. Otherwise get the advisor prompt:
|
|
452
|
+
**C2. Advise: fixes and directions.** Skip in `--critic` mode. Otherwise get the advisor prompt (if not already prepared via `iamu post-walk`):
|
|
440
453
|
|
|
441
454
|
```bash
|
|
442
455
|
iamu advise <run-id> --json
|
|
443
456
|
```
|
|
444
457
|
|
|
445
|
-
If `needed` is `false` (blocked run, nothing open to advise, or already advised), skip to D. Otherwise read the prompt from `iamu/.runs/<run-id>/advise_prompt.md` and dispatch an advisor subagent with it as its only context (use the **thorough** model tier). Unlike the walker, the advisor reads the project's code (read-only) and the screenshots: it classifies each finding local or systemic, writes the fix from the persona's need, looks at how leading systems in the domain handle the moment, and proposes directions for the flow with their predicted reach on other personas. Record its JSON answer via `--data
|
|
458
|
+
If `needed` is `false` (blocked run, nothing open to advise, or already advised), skip to D. Otherwise read the prompt from `iamu/.runs/<run-id>/advise_prompt.md` and dispatch an advisor subagent with it as its only context (use the **thorough** model tier). Unlike the walker, the advisor reads the project's code (read-only) and the screenshots: it classifies each finding local or systemic, writes the fix from the persona's need, looks at how leading systems in the domain handle the moment, and proposes directions for the flow with their predicted reach on other personas. Record its JSON answer via `--file - <<'EOF'` (or `--data`):
|
|
446
459
|
|
|
447
460
|
```bash
|
|
448
|
-
iamu advise-record <run-id> --
|
|
461
|
+
iamu advise-record <run-id> --file - <<'EOF'
|
|
462
|
+
<raw_advice_json>
|
|
463
|
+
EOF
|
|
464
|
+
# or: iamu advise-record <run-id> --data '<raw_advice_json>'
|
|
449
465
|
```
|
|
450
466
|
|
|
451
467
|
Each entry is checked on its own. Entries that come back rejected (an invented file or symbol, a path in the prose, a direction with too little evidence) stay unadvised and the rest is kept; a run is advised once, so do not record again. Mention rejected entries in step 5. Keep the result's `directions` and `rerun` for step 5.
|
|
@@ -458,15 +474,22 @@ The judge (C) and the advisor (C2) are independent, so their subagents may run i
|
|
|
458
474
|
iamu designer <run-id> --json
|
|
459
475
|
```
|
|
460
476
|
|
|
461
|
-
If `needed` is `false` (`reason` is `blocked`, `no_screens` or `already_recorded`), skip it. Otherwise dispatch a fresh subagent that can look at images (the **thorough** model tier) with `iamu/.runs/<run-id>/designer_prompt.md` as its only context, alongside the matcher (B), the judge (C) and the advisor (C2) where the host allows.
|
|
477
|
+
If `needed` is `false` (`reason` is `blocked`, `no_screens` or `already_recorded`), skip it. Otherwise dispatch a fresh subagent that can look at images (the **thorough** model tier) with `iamu/.runs/<run-id>/designer_prompt.md` as its only context, alongside the matcher (B), the judge (C) and the advisor (C2) where the host allows. The subagent MUST open and look at the key screenshots the prompt lists (their paths are `key_screenshots` in the result) with its own file-viewing tool, judging visual hierarchy, layout, contrast and typography against the design docs the prompt names, else general craft. What it finds are designer findings: judgement, never the persona's experience, and never counted with the persona's findings or in the verdict.
|
|
462
478
|
|
|
463
|
-
|
|
479
|
+
The advisor (C2) does not wait for the designer, but D and E do: when its answer arrives, record it:
|
|
464
480
|
|
|
465
481
|
```bash
|
|
466
|
-
iamu designer-record <run-id> --
|
|
482
|
+
iamu designer-record <run-id> --file - <<'EOF'
|
|
483
|
+
<raw_designer_json>
|
|
484
|
+
EOF
|
|
485
|
+
# or: iamu designer-record <run-id> --data '<raw_designer_json>'
|
|
467
486
|
```
|
|
468
487
|
|
|
469
|
-
|
|
488
|
+
Record commands run one at a time: `match-record`, `evaluate-record` and `advise-record` keep their order, and `designer-record` goes between them when its answer arrives (the CLI also holds each run's record lock, so one that starts while another records waits for it). It re-derives the report and never touches the persona's finding files. If the subagent fails or its answer is rejected, retry once, then continue without it and record `iamu designer-record <run-id> --skipped "the designer failed"`. If `designer-record` says `rederive_failed`, run the `fix` it prints.
|
|
489
|
+
|
|
490
|
+
**Never fabricate a review.** Where the host can dispatch a subagent that looks at images, the review is that subagent's. Only on a host that cannot dispatch one, and only when you can open and look at the screenshots yourself, answer `designer_prompt.md` yourself and record it with `iamu designer-record <run-id> --by-orchestrator --file - <<'EOF'` (or `iamu designer-record <run-id> --by-orchestrator --data '<raw_designer_json>'`): the report then says it was written by the orchestrator, which is less independent. If neither is possible, record `iamu designer-record <run-id> --skipped "no subagent can look at images"`. `--no-ux` and `--critic` record their own reasons (see the top of this skill).
|
|
491
|
+
|
|
492
|
+
Keep the result's `proposals`, and `offer_template` from `iamu designer`, for step 5.
|
|
470
493
|
|
|
471
494
|
**D. Read the completed canonical.** Read `iamu/.runs/<run-id>/canonical.json` for the final data including the verification delta (if a prior run exists) and `analysis.directions` (if the advisor ran). Use this for step 5 presentation.
|
|
472
495
|
|
|
@@ -486,7 +509,7 @@ It prints the URL only if the run is on the cloud; otherwise it exits 1 and says
|
|
|
486
509
|
|
|
487
510
|
In linked-no-credentials or local-only mode, skip E.
|
|
488
511
|
|
|
489
|
-
Only after A→E (
|
|
512
|
+
Only after A→E (including C2 Advisor and C3 Designer) are complete, proceed to step 5: every review is recorded before E publishes and before you present the results.
|
|
490
513
|
|
|
491
514
|
### 4a. Walk methodology summary
|
|
492
515
|
|
|
@@ -528,7 +551,7 @@ Then, one short line each, only the ones that apply. These are counts; the detai
|
|
|
528
551
|
- Advisor: `Advisor: <n> directions (absorb <n> findings)`, plus `· <n> notes` for `summary.observations` and `· <n> entries rejected` when the advise record (C2) rejected entries.
|
|
529
552
|
- Design docs: `Design docs: <N> proposed edits — decided after the findings` (`design_proposals` from C2, `proposals` from C3). They wait for a person; never apply them yourself.
|
|
530
553
|
- ⚠ Test setup (`summary.setupNotes`): `⚠ Test setup: <n> doubts about the test, not the product. A finding resting on one may not be real.` Never fold this one away.
|
|
531
|
-
- Sync (E): `⚠ Sync: <n> findings changed by a teammate too` when the result has `conflicts`: name them and give the resolution (`iamu pull --force
|
|
554
|
+
- Sync (E): `⚠ Sync: <n> findings changed by a teammate too` when the result has `conflicts`: name them and give the resolution (`iamu pull --force <slug>` to take theirs for those findings after copying your edits aside, or `iamu sync --keep-mine` to keep yours; `--force` without a slug replaces all local files). When it has `pending_deletes`: `<n> deletions wait for you: check them, then run iamu sync --confirm-deletes`. Never run either yourself.
|
|
532
555
|
- Design doc offer: only when the designer's result (C3) had `offer_template: true`: `No design doc yet (designer judged by general craft only — Iamu gets much sharper when it checks your own tokens, components, and rules): say "start it" (or run iamu-design) and I'll draft one from your code with you, or "no" and I won't offer again.` On "start it", after triage, follow the `iamu-design` skill, mode E. On "no", run `iamu design template --decline`.
|
|
533
556
|
|
|
534
557
|
Then the link to the full walkthrough, which shows the persona's reasoning beside each screenshot:
|
|
@@ -271,7 +271,7 @@ iamu set context \
|
|
|
271
271
|
```
|
|
272
272
|
|
|
273
273
|
*If user chooses Option 2 (Save context only):* run `iamu set context ...` as above, and exit cleanly:
|
|
274
|
-
"✓ Project context initialized. When you're ready to create your first test journey, run
|
|
274
|
+
"✓ Project context initialized. When you're ready to create your first test journey, run the `iamu-new` skill."
|
|
275
275
|
|
|
276
276
|
---
|
|
277
277
|
|
|
@@ -390,7 +390,7 @@ Step 3b: Walk Surface & Persona Login
|
|
|
390
390
|
```
|
|
391
391
|
3. **Scaffold Persona**:
|
|
392
392
|
```bash
|
|
393
|
-
iamu new persona --name "<Name>" --role "<Role>" --disposition "<Disposition>" --body "<2-4 sentence narrative detailing technical habits, patience level, and specific friction triggers>" [--access "<confirmed-account-or-role>"] --json
|
|
393
|
+
iamu new persona --name "<Name>" --role "<Role>" --disposition "<Disposition>" --body "<2-4 sentence narrative detailing technical habits, patience level, and specific friction triggers>\n\n## Today\n<how they do this job today without the product, and what that costs or gives them>" [--access "<confirmed-account-or-role>"] --json
|
|
394
394
|
```
|
|
395
395
|
4. **Scaffold Bound Scenario**:
|
|
396
396
|
```bash
|
|
@@ -77,7 +77,7 @@ Present an honest, concise audit of what the scenario needs vs. what the system
|
|
|
77
77
|
|
|
78
78
|
- **Target URL:** `http://localhost:3000` (`pnpm dev`, from `iamu/testability.md`)
|
|
79
79
|
- **Entry Route:** `/incidents` (verified in `src/app/(dashboard)/incidents/page.tsx`)
|
|
80
|
-
- **Persona Login:** Maria Gomez (Shift Coordinator) → `coordinator@site-a.test` (`
|
|
80
|
+
- **Persona Login:** Maria Gomez (Shift Coordinator) → `coordinator@site-a.test` (`dev_switcher` / `test_accounts` / `interactive` / `public`)
|
|
81
81
|
- **World State:** 5 open incidents, 1 serious and unassigned (Current DB / Seed: 0 incidents)
|
|
82
82
|
```
|
|
83
83
|
|
package/dist/templates.d.ts
CHANGED
|
@@ -9,7 +9,7 @@ export declare function getTemplatesRoot(): string;
|
|
|
9
9
|
export declare function getTemplateDir(subdir: "skills" | "prompts"): string;
|
|
10
10
|
export declare function getTemplateFile(subdir: "skills" | "prompts", filename: string): string;
|
|
11
11
|
/** Prompt templates `iamu init` installs and `iamu update` refreshes in `iamu/.prompts/`. */
|
|
12
|
-
export declare const PROMPT_FILES: readonly ["walk.md", "persona.md", "outline.md", "combine.md", "complete.md", "
|
|
12
|
+
export declare const PROMPT_FILES: readonly ["walk.md", "persona.md", "outline.md", "combine.md", "complete.md", "advise.md", "design-draft.md", "design-review.md", "design-review-match.md", "designer.md"];
|
|
13
13
|
/**
|
|
14
14
|
* A prompt as the project has it: its `iamu/.prompts/<name>` copy (which
|
|
15
15
|
* a team may tune), else the bundled template.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@iamu-dev/iamu",
|
|
3
|
-
"version": "0.8.0-beta.
|
|
3
|
+
"version": "0.8.0-beta.3",
|
|
4
4
|
"type": "module",
|
|
5
5
|
"description": "Test your product as realistic personas — scenario-driven development CLI",
|
|
6
6
|
"license": "MIT",
|
|
@@ -59,8 +59,8 @@
|
|
|
59
59
|
"typescript": "^5.8.0",
|
|
60
60
|
"vitest": "^4.1.11",
|
|
61
61
|
"@iamu-dev/local-store": "0.1.0",
|
|
62
|
-
"@iamu
|
|
63
|
-
"@iamu/
|
|
62
|
+
"@iamu/tsconfig": "0.1.0",
|
|
63
|
+
"@iamu-dev/protocol": "0.1.0"
|
|
64
64
|
},
|
|
65
65
|
"scripts": {
|
|
66
66
|
"build": "tsup && tsc -b --force tsconfig.build.json --emitDeclarationOnly && node scripts/copy-assets.mjs",
|
|
@@ -1,64 +0,0 @@
|
|
|
1
|
-
# Evaluation methodology
|
|
2
|
-
|
|
3
|
-
You are the EVALUATOR of a walkthrough. A different agent (the walker) drove this walkthrough as a specific persona. That walker was DELIBERATELY NOT shown the human-test questions below — your job is to read the recorded evidence and answer the questions yourself, without confirmation bias.
|
|
4
|
-
|
|
5
|
-
Your output is the verdict on whether the product earns the scenario's "Human Test" — would this persona come back and use it voluntarily?
|
|
6
|
-
|
|
7
|
-
## Rules
|
|
8
|
-
|
|
9
|
-
- **Ground every verdict in evidence.** Cite specific step IDs and finding IDs. Quote specific phrases from `ui_fact`, `consequence`, or `persona_reasoning` when they're load-bearing. A verdict without grounding is rejected.
|
|
10
|
-
- **Justifications must be falsifiable.** A reader with the same evidence must be able to verify what you cited.
|
|
11
|
-
- **Disagree with the walker if the evidence supports it.** The walker's summaries are inputs, not verdicts. If a finding contradicts a walker claim, your verdict reflects the finding.
|
|
12
|
-
|
|
13
|
-
## Verdict calibration
|
|
14
|
-
|
|
15
|
-
- **`no`** — evidence uniformly negative on the critical-path task. Don't manufacture nuance.
|
|
16
|
-
- **`yes`** — evidence shows the persona completed the task without measurable friction.
|
|
17
|
-
- **`partial`** — genuinely different outcomes on different surfaces. NOT "mixed but ultimately negative" — that's `no`.
|
|
18
|
-
- **`not_applicable`** — cannot determine from available evidence. Use instead of guessing.
|
|
19
|
-
|
|
20
|
-
## Input
|
|
21
|
-
|
|
22
|
-
Read `iamu/.runs/<run-id>/canonical.json` for:
|
|
23
|
-
- `steps[]` — what the walker observed (persona_reasoning, actions, screenshots)
|
|
24
|
-
- `findings[]` — what helped, hindered, was missing, or was incorrect
|
|
25
|
-
- `human_tests[]` — the questions to answer (verdict is currently null)
|
|
26
|
-
|
|
27
|
-
Read the outline file for the full human_test questions if needed.
|
|
28
|
-
|
|
29
|
-
## Output
|
|
30
|
-
|
|
31
|
-
Write the evaluation results to a temporary file, then use it to update canonical.json.
|
|
32
|
-
|
|
33
|
-
Write `/tmp/iamu-eval-<run-id>.json`:
|
|
34
|
-
|
|
35
|
-
```json
|
|
36
|
-
{
|
|
37
|
-
"verdicts": [
|
|
38
|
-
{
|
|
39
|
-
"human_test_id": "<id>",
|
|
40
|
-
"verdict": "yes|partial|no|not_applicable",
|
|
41
|
-
"reasoning": "Steps 3-5 show... Finding X demonstrates...",
|
|
42
|
-
"grounding_step_ids": ["step-3", "step-4", "step-5"],
|
|
43
|
-
"grounding_finding_ids": ["<finding-uuid>"]
|
|
44
|
-
}
|
|
45
|
-
],
|
|
46
|
-
"overall": {
|
|
47
|
-
"question": "Would <persona> come back?",
|
|
48
|
-
"answer": "<one paragraph grounded in specific observations>",
|
|
49
|
-
"reasoning": "<supporting argument>"
|
|
50
|
-
},
|
|
51
|
-
"provenance": {
|
|
52
|
-
"judge_mode": "subagent",
|
|
53
|
-
"model": "<model that ran this evaluation>",
|
|
54
|
-
"prompt_id": "judge",
|
|
55
|
-
"prompt_version": "1.0.0"
|
|
56
|
-
}
|
|
57
|
-
}
|
|
58
|
-
```
|
|
59
|
-
|
|
60
|
-
Then update `canonical.json`:
|
|
61
|
-
- Set each `human_tests[].verdict`, `.reasoning`, `.grounding_step_ids`, `.grounding_finding_ids`
|
|
62
|
-
- Set `human_test_overall.answer` and `.reasoning`
|
|
63
|
-
- Set `provenance.judge_prompt` and `provenance.judge_mode`
|
|
64
|
-
- Regenerate `report.md` with verdicts filled in
|