@iamu-dev/iamu 0.2.0-beta.5 → 0.2.0-beta.7

package/dist/templates/skills/iamu-run.md

@@ -127,19 +127,30 @@ Record it as a target string the run can carry, for example `sim://<device-name>
 
 **Then verify the starting state before spawning anyone.** Launch the app and take ONE screenshot confirming the product is where the scenario assumes it starts. Wait a few seconds after launch first: a screen rendered before its state lands shows the wrong thing, and verifying too early makes the verification itself lie — in the direction of "the harness is broken", which costs a debugging detour or, worse, waves a real seeding bug through.
 
-**Web: check the product is ready, not just up.** A server that answers is not a product that has its data. Before spawning the walker, confirm the thing the scenario depends on actually exists — the list the persona will open is populated, the seeded records are there, background setup (seeding, migrations, indexing) has finished. Use the project's documented readiness signal if it has one; otherwise look at one screen or one API response yourself. A walk started too early reports "the feature is missing" as a critical finding, and nothing in the walker's view distinguishes that from a real defect.
+**Expected State & Semantic Squinting — verify environment before walking.** A scenario testing triage, review, or audits requires a populated, realistic system.
+1. **Expected State:** If the scenario defines `state.description`, verify the landing screen matches what the persona expects to see (e.g. populated register, open incidents). If the screen is completely blank and the persona cannot self-seed, stop early and report the run as blocked by missing preconditions rather than logging false product bugs.
+2. **Data Squinting & Approximation:** If the exact record described in the narrative is absent (e.g. no "damaged conveyor belt" hazard), the walker must follow `state.approximation` — "squint" at the real data, pick the closest plausible record (e.g. an active mechanical or safety hazard), and proceed with the evaluation/delegation workflow.
+   - **Squinting Guardrail:** Squinting is bounded to semantically related items within the same domain workflow. It does NOT permit drifting to unrelated modules (e.g. switching from risk assessment to user profile settings). If no plausible record exists, declare the run blocked.
+3. **Empty State Fallback:**
+   - If the database is empty and the persona role has creation rights (e.g. Manager or Reporter), the persona may create the minimal viable record in-character to proceed with the core test.
+   - If the persona role is read-only or subordinate (cannot self-seed), stop and report the run as blocked by preconditions.
 
 **Access — how the walker becomes this persona.** A walk as a forklift driver that runs with an administrator's session reports nothing true about what the driver can see or reach. Access is recorded in two places, and `iamu instructions walk` (step 4) returns both as `access`:
 
 - `access.project` — from `iamu/context.yml` `access`: HOW switching user is done in this product (a dev persona switcher, a storage key, seeded test accounts, a login form). Written once per project.
 - `access.persona` — from the persona file's `access`: WHICH user this persona is (an account name, a role, a switcher key).
 
-If both are present, use them as they are. If either is missing, fill the gap now and SAVE it so no later run has to ask:
+**Interactive Auth Handshake (for enterprise auth, SSO, 2FA, BankID):**
+If the product is behind enterprise login where automated dev persona switching is not configured:
+1. Open the browser to the target URL (`iamu browse start --run <run-id>` or browser tool).
+2. Prompt the human user:
+   > 👉 **Please log in as `<persona.role>` (`<persona.name>`) in the opened browser window.** Press Enter once you reach the dashboard to hand control to the walker...
+3. Wait for the user to confirm, then let the walker drive the authenticated session.
 
-1. Check the project's own agent docs (`CLAUDE.md`, `AGENTS.md`, anything they point to for the dev environment) for a documented way to switch user or role.
-2. Map the persona to the closest account or role the product actually has. If nothing fits, say so — a persona the product cannot represent is itself worth telling the user.
-3. If the docs are silent, ask the user once. "No login, everyone is the same user" is a valid answer.
-4. Write what you learned: the mechanism into `iamu/context.yml` as `access:`, the persona's own key into `iamu/personas/<slug>.md` frontmatter as `access:`. Passwords and tokens never go in either — name where they live (an env var, a vault entry) instead.
+If automated switching exists:
+1. Check the project's own agent docs (`CLAUDE.md`, `AGENTS.md`) for a documented way to switch user or role.
+2. Map the persona to the closest account or role the product actually has. If nothing fits, say so.
+3. Write what you learned: the mechanism into `iamu/context.yml` as `access:`, the persona's own key into `iamu/personas/<slug>.md` frontmatter as `access:`. Passwords and tokens never go in either — name where they live (an env var, a vault entry) instead.
 
 Carry the result into the walking prompt as the `## Access` section: the exact steps to become the persona, with the instruction to do them BEFORE the first recorded step.
 
@@ -185,12 +196,16 @@ The result's `walker_prompt` is the walker's entire prompt. Pass it verbatim —
 
 A fresh subagent is NOT blind by default. On most hosts it still loads the project's agent instructions and can read files in the repo, so it may know the domain model, the route table, and why a screen was built the way it was — none of which the persona knows. The methodology's "Stay outside the product" rule is what keeps it honest; it is part of `walker_prompt` — never remove or soften it. If the host lets you restrict a subagent's tools, give the walker only the driving surface, file write/edit, and a shell for `iamu` commands.
 
+**Model tiering & subagent dispatch:**
+- **Walker (Browser Navigation):** High turn count, mechanical tool-use. Use the **fast** model tier (`flash` or `haiku`) by default to cut walk time by ~60% and conserve session quota.
+- **Matcher & Judge:** Broad synthesis and semantic comparison. Use the **thorough** frontier tier (`pro`, `sonnet`, or active model) to prevent severity inflation.
+
 ```
 # In Claude Code:
-Agent({ description: "Walk <scenario-slug> as <persona.name>", prompt: "<walker_prompt, verbatim>" })
+Agent({ description: "Walk <scenario-slug> as <persona.name>", model: "haiku", prompt: "<walker_prompt, verbatim>" })
 
 # In Antigravity / generic subagent hosts:
-invoke_subagent(Role="Persona Walker", Prompt="<walker_prompt, verbatim>")
+invoke_subagent(Role="Persona Walker", Model="flash", Prompt="<walker_prompt, verbatim>")
 ```
 
 The subagent inherits the driving-surface tools and drives the product independently. It does not see this conversation.
@@ -202,7 +217,7 @@ The subagent inherits the driving-surface tools and drives the product independe
 - **Screenshot every 3-4 taps at most.** Batching more desyncs the moment an answer opens a screen she did not expect.
 - **Coordinates are device points, origin top-left.** A scaled screenshot must be mapped back to that space before tapping.
 
-**If subagent dispatch is not supported by your host editor** (e.g. standard Cursor without subagent tools), fall back to walking in the same context — but strictly adhere to the persona behavioral constraints in `walker_prompt`.
+**If subagent dispatch or model-switching is not supported by your host editor** (e.g. standard Cursor / Windsurf single-session mode), fall back to walking directly in the current session (inheriting the active model) — but strictly adhere to the persona behavioral constraints in `walker_prompt`.
 
 After the walker returns, continue to the post-walk pipeline below.
 
@@ -269,7 +284,7 @@ This reads all steps, findings, and entity files, assembles `canonical.json`, ge
 iamu match <run-id> --json
 ```
 
-If `needed` is `false` (no earlier findings for this scenario, or none this run), skip to C. Otherwise, read the prompt from `iamu/.runs/<run-id>/match_prompt.md` directly and dispatch a matcher subagent with it as its only context. Write its JSON answer to `<matches.json>` and record it:
+If `needed` is `false` (no earlier findings for this scenario, or none this run), skip to C. Otherwise, read the prompt from `iamu/.runs/<run-id>/match_prompt.md` directly and dispatch a matcher subagent with it as its only context (use the **thorough** model tier, e.g. `Model="pro"` in Antigravity, to accurately evaluate semantic equivalence). Write its JSON answer to `<matches.json>` and record it:
 
 ```bash
 iamu match-record <run-id> --file <matches.json>
@@ -283,7 +298,7 @@ This links recurrences to their earlier canonical finding, marks earlier finding
 iamu evaluate <run-id> --json
 ```
 
-Read the evaluation prompt from `iamu/.runs/<run-id>/judge_prompt.md` directly. Dispatch the evaluator as a subagent with that prompt. After the evaluator returns verdicts, record them locally with `--no-push`:
+Read the evaluation prompt from `iamu/.runs/<run-id>/judge_prompt.md` directly. Dispatch the evaluator as a subagent with that prompt (use the **thorough** model tier, e.g. `Model="pro"` in Antigravity, to prevent severity inflation). After the evaluator returns verdicts, record them locally with `--no-push`:
 
 ```bash
 iamu evaluate-record <run-id> --file <verdicts.json> --no-prompt --no-push

package/dist/templates/skills/iamu-setup.md

@@ -34,7 +34,7 @@ Check whether `iamu/config.json` already exists:
 
 ### 2. Understand the project
 
-Explore the codebase automatically — do NOT ask the user what files to read. Build a **project-context summary** (3–6 sentences) the downstream skills will use as grounding. Read the following files when present:
+Explore the codebase automatically — do NOT ask the user what files to read. Build a **substantive project description** that serves as the foundation for all downstream analysis (personas, scenarios, and walkthroughs). Read the following files when present:
 
 - `README.md`, `CLAUDE.md`, `agents.md`
 - Package manifests — `package.json`, `Cargo.toml`, `pyproject.toml`, `go.mod`
@@ -42,114 +42,131 @@ Explore the codebase automatically — do NOT ask the user what files to read. B
 - `docs/`, OpenAPI / JSON / GraphQL schemas, `src/`-style source directories
 - `.env.example`
 
-Extract these key dimensions:
+**Quality Standard for Project Context (Knowledgeable Observer):**
+Write as an objective domain expert describing the business and its users, NOT marketing jargon, cookie notices, or tech stack. Do NOT hedge with "appears to be" or "seems to".
 
-1. **Domain & Purpose** — what this product does, core workflows, and problem space (mandatory).
-2. **Jurisdiction & Regulations (`governed_by`)** — country/region and specific compliance frameworks (AFS 2023:2, GDPR, HIPAA, ISO 27001, etc.). **Strictly optional:** only extract if the repo explicitly references legal or compliance standards. Do NOT invent regulations for standard apps.
-3. **Key Actors & Roles** — the main user archetypes and roles discovered in the codebase (auth models, routes, permissions).
-4. **Dev Access & Auth** — how personas switch or log in during local dev/test (e.g. dev persona switcher, seeded accounts, storage key, mock headers). Essential for walkthrough automation.
-5. **Key Terminology (`glossary`)** — 2–4 domain-specific terms or acronyms found in the codebase.
-6. **Primary Personas** — determine the 1–2 most critical user archetypes that stress-test this product (e.g. End User / Frontline vs. Admin / Compliance, or First-Timer vs. Power User). Propose:
-   - Name and Role
-   - Disposition / perspective (patience level, tech literacy, primary friction triggers)
+The description MUST capture four substantive dimensions:
+1. **What the product fundamentally IS** — core workflow, product type, and problem space.
+2. **Who actually uses it and what they come here to do** — enumerate the real user journeys (e.g. reporting incidents, triaging risks, delegating actions, auditing).
+3. **What makes THIS product specific** — geography, jurisdiction, regulatory mandates, or business model.
+4. **Operational characteristics** — how users physically and organizationally encounter the software (e.g. deskless frontline mobile workers in high-hazard environments, strict statutory completion deadlines, immutable audit trails).
 
-Do NOT collect code architecture (agent reads the codebase directly — tech stack doesn't affect walkthroughs).
+*Example of target substance:*
+> "IA-systemet is Sweden's national work environment and safety incident management platform, utilized by thousands of Swedish enterprises to comply with statutory systematic work environment requirements (AML / AFS). It enables frontline employees to quickly log near-misses (tillbud), workplace accidents (olycksfall), and risk observations from desktop or mobile devices in the field. Core user journeys include real-time hazard reporting, line manager root-cause investigations, safety committee review, and action plan delegation with statutory compliance tracking."
 
-**If there is no code (greenfield, design phase):** ask the user: what does this project do, who are the users, what domain, and are there any specific regulations?
+Synthesize the project into a cohesive, substantive narrative (1–2 paragraphs):
+- What the product fundamentally is, its primary domain, and who uses it.
+- Key user journeys and operational characteristics (e.g. mobile floor use, audit deadlines).
+- Market, geography, and specific compliance frameworks (only if evident).
 
-**Present the proposal with clean, structured Markdown for confirmation:**
+Do NOT collect code architecture (agent reads the codebase directly — tech stack doesn't affect walkthroughs).
 
-```markdown
-### 📋 Discovered Project Context
+**If there is no code (greenfield, design phase, or test sandbox):** do NOT interrogate the user with a multi-part survey or ask for dev auth plumbing. Ask for a single entry point:
+> No local codebase detected. To seed context, provide one of:
+> - A target URL (staging, production, or docs — e.g., `https://app.example.com`)
+> - A one-sentence description of what this product does
+> - Or type `"sandbox"` to seed a pre-configured B2B SaaS demo
 
-- **Domain & Purpose:** <Concise description of the product and problem space>
-- **Jurisdiction & Rules:** <Country/region and compliance frameworks, or "None identified">
-- **Key Roles Identified:** <List of detected roles and their responsibilities>
-- **Dev Access & Auth:** <Detected mechanism to switch/assume personas in dev, or "Standard login">
-- **Domain Terminology:** <2–4 key terms/acronyms found in the codebase, if any>
+**Autonomous Enrichment:**
+- **URL provided:** Fetch or search public marketing/docs for that site. Automatically extract the substantive narrative, core user journeys, and key actors. If industry regulations are obvious (e.g., healthcare/HIPAA, Swedish workplace safety/AML), include them in the narrative.
+- **One-sentence summary provided:** Immediately expand into a full substantive narrative using domain knowledge.
+- **"sandbox" provided:** Seed a standard B2B SaaS narrative with Admin and End-User roles.
+- **Do NOT ask upfront about auth mechanics (`access`) or compliance frameworks (`governed_by`).** Infer if obvious, otherwise leave them blank.
 
----
+**Present ONLY the narrative description for user confirmation (Do NOT include metadata bullets or personas):**
 
-### 🎭 Proposed Primary Personas
-
-#### 1. <Name> — *<Role / Title>*
-- **Context & Usage:** <Where and how they use the app (e.g., mobile on the floor vs. desktop in office)>
-- **Disposition:** <Patience level, technical confidence, key motivations>
-- **Friction Triggers:** <Specific interactions that cause friction, drop-off, or failure>
+```markdown
+### 📋 Discovered Project Context
 
-#### 2. <Name> — *<Role / Title>*
-- **Context & Usage:** <Where and how they use the app>
-- **Disposition:** <Patience level, technical confidence, key motivations>
-- **Friction Triggers:** <Specific interactions that cause friction, drop-off, or failure>
+<1–2 cohesive paragraphs written as a knowledgeable observer: what the product fundamentally is, who actually uses it, key user journeys, operational reality (e.g. mobile floor use, audit deadlines), and regulatory/market context. No marketing fluff, no hedging, no tech stack descriptions, and no bullet points.>
 
 ---
 
 **Next step:**
-Reply **"yes"** to save this context to `iamu/context.yml` and scaffold these personas, or tell me what to adjust (e.g., change roles, edit domain, or add/skip personas).
+Reply **"yes"** to save this context to `iamu/context.yml`, or tell me what to adjust.
 ```
 
-Wait for the user to confirm or correct before writing. The user only needs to verify domain and choose whether to accept, edit, suggest alternative personas, or defer persona creation.
+Wait for the user to confirm or correct before writing. The user only needs to verify the narrative description and choose whether to accept or edit.
 
-### 3. Capture domain context and scaffold personas
+### 3. Capture domain context and seed personas (Two-Phase Confirmation)
 
-Persist the step-2 summary as structured fields so every scenario draft and walkthrough loads the same grounding. Downstream skills auto-read this file.
-
-1. Check existing state by reading `iamu/context.yml` directly.
-
-2. **Branch on what you find:**
-
-   - **Empty / not present** → synthesize structured fields from step 2 and write `iamu/context.yml` with your file-editing tool (not a shell heredoc — it avoids permission prompts).
-   - **Present but thin** (missing domain) → show what's missing and offer to fill it. User picks **refine** or **replace**.
-   - **Present and complete** (domain set) → show summary and ask "Use as-is, refine, or replace?" Default is **skip**.
-
-3. **Write `iamu/context.yml`**:
+#### Phase A: Persist Context
+Once the user confirms the narrative description:
+1. Derive the structured frontmatter fields (`domain`, `actors`, `jurisdiction`, `governed_by`) from the confirmed text, and write `iamu/context.yml` with your file-editing tool. Put structured fields in frontmatter and the substantive narrative in the markdown body below the `---`:
 
    ```yaml
    ---
    schema_version: 1
-   domain: e-commerce storefront
-   # jurisdiction: EU / US          # optional
-   # governed_by:                    # strictly optional - omit if not regulated
-   #   - name: GDPR
+   domain: workplace safety management
+   # jurisdiction: Sweden / EU          # optional
+   # governed_by:                       # strictly optional - omit if not regulated
+   #   - name: AFS 2023:2
    #     authority_level: regulation
    actors:
      - name: Store Admin
        role: Manages catalog, inventory, and order fulfillment
      - name: First-Time Buyer
        role: Explores products, adds to cart, and checks out
-   glossary:
-     - term: SKU
-       definition: Stock Keeping Unit
    access: >-
      No login in dev. In the page, set localStorage "dev_persona" to the
      persona's access value and reload.
    ---
+
+   <The substantive 1–2 paragraph project description and core user journeys>
    ```
 
    Rules:
    - `domain` — one line. Mandatory.
+   - Body markdown (below `---`) — substantive description and user journeys. Downstream skills read this as extended grounding.
    - `jurisdiction` — optional. Only if evident or relevant.
-   - `governed_by[]` — strictly optional. Order by authority (legislation > regulation > standard > company_policy > guideline > informational). Specific names, not categories. Omit entirely if no regulations apply.
+   - `governed_by[]` — strictly optional. Order by authority. Omit entirely if no regulations apply.
    - `actors[]` — all identified roles and their responsibilities.
-   - `glossary[]` — domain terms only. Thin is better than padded with generic terms.
-   - `access` — optional. Only what the docs actually say; leave it out rather than guess. `iamu-run` fills it in on the first walk if it is missing.
+   - `access` — optional. Only what the docs actually say; leave it out rather than guess.
    - Do NOT add `industry` or `architecture` — these don't affect walkthroughs.
 
-4. **Scaffold confirmed personas (unless `skip-personas` selected):**
+#### Phase B: Persona Casting (Opt-In & Separate Confirmation)
+Immediately after saving `iamu/context.yml`, ask the user if they want to seed baseline personas:
 
-   Use the CLI to scaffold valid persona entities with schema-compliant UUIDs, slugs, frontmatter, and narrative body in a single command:
+> Context saved to `iamu/context.yml`.
+> 
+> Would you like to seed 1–2 baseline personas to your casting panel now?
+> Reply **"yes"** to propose personas, or **"skip"** to create scenarios directly via `iamu-new`.
+
+**If the user chooses "yes":**
+Propose 1–2 primary personas grounded in the confirmed actors and operational characteristics:
+
+```markdown
+### 🎭 Proposed Primary Personas
+
+#### 1. <Name> — *<Role / Title>*
+- **Context & Usage:** <Where and how they use the app (e.g. mobile on the floor vs. desktop in office)>
+- **Disposition:** <Patience level, technical confidence, key motivations>
+- **Friction Triggers:** <Specific interactions that cause friction, drop-off, or failure>
+
+#### 2. <Name> — *<Role / Title>*
+- **Context & Usage:** <Where and how they use the app>
+- **Disposition:** <Patience level, technical confidence, key motivations>
+- **Friction Triggers:** <Specific interactions that cause friction, drop-off, or failure>
+
+---
+
+**Next step:**
+Reply **"yes"** to scaffold these personas into `iamu/personas/`, or tell me what to adjust.
+```
+
+Wait for the user to confirm or adjust. Once confirmed:
+1. Scaffold personas using the CLI:
    ```bash
    iamu new persona --name "<Name>" --role "<Role>" --disposition "<Disposition>" --body "<2-4 sentence narrative detailing technical habits, patience level, and specific friction triggers>" [--access "<account-or-role>"] --json
    ```
-
-   If `iamu/context.yml` specifies an `access` mechanism, pass `--access "<role>"` matching an existing product account/role.
-
-   Once all confirmed personas are created, run validation once:
+2. Validate once:
    ```bash
    iamu validate --all
    ```
+3. Confirm: "Context saved and personas seeded in iamu/personas/. Run `iamu-new` to draft your first scenario."
 
-5. Confirm: "Context saved and personas seeded in iamu/personas/. Run `iamu-new` to draft your first scenario." (or "Context saved. Run `iamu-new` to draft your first scenario." if personas were skipped).
+**If the user chooses "skip":**
+Confirm: "Context saved. Run `iamu-new` to draft your first scenario."
 
 ### 4. Offer cloud link (optional)
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@iamu-dev/iamu",
-  "version": "0.2.0-beta.5",
+  "version": "0.2.0-beta.7",
   "type": "module",
   "description": "Test your product as realistic personas — scenario-driven development CLI",
   "license": "UNLICENSED",
@@ -54,12 +54,13 @@
   "devDependencies": {
     "@types/node": "^22.0.0",
     "@vitest/coverage-v8": "^4.1.11",
+    "google-auth-library": "^11.1.0",
     "tsup": "^8.5.1",
     "typescript": "^5.8.0",
     "vitest": "^4.1.11",
     "@iamu-dev/local-store": "0.1.0",
-    "@iamu/tsconfig": "0.1.0",
-    "@iamu-dev/protocol": "0.1.0"
+    "@iamu-dev/protocol": "0.1.0",
+    "@iamu/tsconfig": "0.1.0"
   },
   "scripts": {
     "build": "tsup && tsc -b tsconfig.build.json --emitDeclarationOnly && node scripts/copy-assets.mjs",
@@ -67,6 +68,8 @@
     "typecheck": "tsc --noEmit",
     "test": "vitest run",
     "test:coverage": "vitest run --coverage",
+    "eval:walkthroughs": "vitest run --config vitest.benchmark.config.ts",
+    "eval:gemini": "node dist/eval/gemini-eval-cli.js",
     "pack:dry-run": "npm pack --dry-run"
   }
 }