ai-fob 1.3.0 → 1.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/assets/agents/build-validator-agent.md +4 -0
- package/assets/commands/build-phase-V2.md +27 -0
- package/assets/commands/create-highlevel-plan-phases.md +4 -1
- package/assets/skills/testing-and-validation/SKILL.md +16 -0
- package/assets/skills/workflow-repair/reports/2026-03-28_mobile-viewport-testing.md +136 -0
- package/manifest.json +1 -1
- package/package.json +1 -1
|
@@ -26,6 +26,10 @@ You are a rigorous build validator. Your job is to run validation checks provide
|
|
|
26
26
|
|
|
27
27
|
Run every validation check. Report exactly what you observe. If a check fails, describe the failure precisely -- what was expected vs. what actually happened. Do not attempt to fix the code or suggest fixes. Your report goes back to the orchestrator who will route failures to a builder agent for correction.
|
|
28
28
|
|
|
29
|
+
## Browser Tool Constraint
|
|
30
|
+
|
|
31
|
+
NEVER use the macOS `open` command to open URLs in a browser. ALWAYS use `agent-browser open <url>` for all browser-based checks. The `open` command launches Safari, which cannot be automated, snapshotted, or device-emulated. All browser interactions MUST go through `agent-browser` (Chromium via Playwright).
|
|
32
|
+
|
|
29
33
|
## Checks
|
|
30
34
|
|
|
31
35
|
The calling prompt provides a numbered list of checks to run. Execute every check listed -- no more, no fewer. For each check:
|
|
@@ -927,6 +927,12 @@ Store the total check count as `BUILD_CHECK_COUNT`. Store the count of HL criter
|
|
|
927
927
|
|
|
928
928
|
Determine if browser checks exist: scan the assembled check list for any check that mentions "browser", "agent-browser", "navigate", "page", "UI", or "localhost". Store as `HAS_BROWSER_CHECKS` (true/false).
|
|
929
929
|
|
|
930
|
+
If `HAS_BROWSER_CHECKS` is true: Read the Mobile Test Devices section from the testing-and-validation skill. Extract the Primary Device and Secondary Device values. Store as `MOBILE_PRIMARY_DEVICE` and `MOBILE_SECONDARY_DEVICE`. Determine `HAS_MOBILE_CHECKS`:
|
|
931
|
+
- If `MOBILE_PRIMARY_DEVICE` is "NONE": set `HAS_MOBILE_CHECKS = false`.
|
|
932
|
+
- Otherwise: set `HAS_MOBILE_CHECKS = true`.
|
|
933
|
+
|
|
934
|
+
If `HAS_BROWSER_CHECKS` is false: set `HAS_MOBILE_CHECKS = false`.
|
|
935
|
+
|
|
930
936
|
#### 5b. Start Dev Server (if needed)
|
|
931
937
|
|
|
932
938
|
If `HAS_BROWSER_CHECKS` is true:
|
|
@@ -997,6 +1003,27 @@ Read the build report(s) for context on what was built:
|
|
|
997
1003
|
If Username is "REPLACE_WITH_TEST_USERNAME":
|
|
998
1004
|
"Test credentials have NOT been configured in the testing-and-validation skill. The placeholder values have not been replaced. Browser checks that require authentication MUST be marked as BLOCKED with reason: 'Test credentials not configured in testing-and-validation skill -- user must replace placeholder values.'"}
|
|
999
1005
|
|
|
1006
|
+
## Mobile Device Testing
|
|
1007
|
+
{Read the Mobile Test Devices section from the testing-and-validation skill. Extract the Primary Device and Secondary Device values.
|
|
1008
|
+
|
|
1009
|
+
If HAS_MOBILE_CHECKS is true (Primary Device is NOT "NONE"):
|
|
1010
|
+
"Mobile viewport testing is configured for this project. After completing each browser check at the default desktop viewport, you MUST repeat the visual/layout portions of that check at the mobile viewport.
|
|
1011
|
+
|
|
1012
|
+
Mobile testing procedure (use the agent-browser skill):
|
|
1013
|
+
1. Complete the browser check at the default desktop viewport first
|
|
1014
|
+
2. Set the mobile device: `agent-browser set device \"{MOBILE_PRIMARY_DEVICE}\"`
|
|
1015
|
+
3. Reload the page: `agent-browser reload`
|
|
1016
|
+
4. Take a snapshot to verify layout at mobile viewport: `agent-browser snapshot -i`
|
|
1017
|
+
5. Take a screenshot for visual evidence: `agent-browser screenshot`
|
|
1018
|
+
6. Verify the page renders correctly at the mobile viewport -- no overlapping elements, no horizontal scrolling, no truncated content, no inaccessible interactive elements
|
|
1019
|
+
7. Reset to desktop viewport when done: `agent-browser set viewport 1920 1080`{If MOBILE_SECONDARY_DEVICE is not "NONE":
|
|
1020
|
+
8. Repeat steps 2-7 with the secondary device: `agent-browser set device \"{MOBILE_SECONDARY_DEVICE}\"`}
|
|
1021
|
+
|
|
1022
|
+
For each browser check, report desktop and mobile results separately. If a check passes at desktop but fails at mobile, the overall check result is FAIL. Include the device name in the findings (e.g., 'FAIL at iPhone 12 Pro: navigation menu overlaps content')."
|
|
1023
|
+
|
|
1024
|
+
If HAS_MOBILE_CHECKS is false:
|
|
1025
|
+
"Mobile viewport testing is not configured (Primary Device is NONE in the testing-and-validation skill). All browser checks run at the default desktop viewport only."}
|
|
1026
|
+
|
|
1000
1027
|
## Validation Checks ({BUILD_CHECK_COUNT} -- run ALL of these)
|
|
1001
1028
|
|
|
1002
1029
|
{The assembled numbered check list from step 5a -- paste the full list verbatim, including the [HL] prefixed checks at the end}
|
|
@@ -93,7 +93,8 @@ The user provided a feature document from a reverse-engineering analysis. Use it
|
|
|
93
93
|
- **Name**: short descriptive name
|
|
94
94
|
- **Goal**: 1-2 sentences on what this phase achieves and why it comes at this position
|
|
95
95
|
- **Dependencies**: which prior phase(s) must complete first (or "None" for the first)
|
|
96
|
-
- **Success criteria**: concrete, verifiable statements (can be user stories, anti-stories, API checks, state checks, UI checks, build checks)
|
|
96
|
+
- **Success criteria**: concrete, verifiable statements (can be user stories, anti-stories, API checks, state checks, UI checks, build checks, mobile viewport checks)
|
|
97
|
+
- For phases with UI work, ask: "Does this need to work on mobile viewports?" If yes, include mobile-specific success criteria (e.g., "Navigation menu is usable on iPhone 12 Pro viewport")
|
|
97
98
|
Target 3-5 phases. Present the suggested phases to the user:
|
|
98
99
|
```
|
|
99
100
|
Based on the feature document, I suggest this phase breakdown:
|
|
@@ -155,7 +156,9 @@ The user provided a feature document from a reverse-engineering analysis. Use it
|
|
|
155
156
|
- State/infrastructure checks: "Database tables exist and are accessible"
|
|
156
157
|
- UI checks: "Login form renders with email and password fields"
|
|
157
158
|
- Build/tooling checks: "`bun dev` starts without errors"
|
|
159
|
+
- Mobile viewport checks: "Navigation is usable at iPhone 12 Pro viewport"
|
|
158
160
|
- For phases with user-facing behavior, success criteria MUST include both user stories (what users CAN do) and anti-stories (what users CANNOT do)
|
|
161
|
+
- For phases with UI work, ask the user: "Does this need to work on mobile viewports?" If yes, include mobile-specific success criteria. Mobile viewport testing is configured in the testing-and-validation skill.
|
|
159
162
|
- Each criterion must be testable -- a future validator should be able to determine PASS/FAIL
|
|
160
163
|
- The last phase should typically be "Integration & Polish"
|
|
161
164
|
8. Do NOT proceed to Phase 2 until you and the user explicitly agree on the task description, user stories, anti-stories, phase breakdown, and detailed specifications (if any were provided).
|
|
@@ -58,6 +58,22 @@ Credentials for automated browser-based authentication. These are used by valida
|
|
|
58
58
|
3. The Post-Login URL is the page the browser should land on after successful login -- used to confirm authentication succeeded
|
|
59
59
|
4. If your app does not require authentication, set Username to `NONE` -- validators will skip the authentication step
|
|
60
60
|
|
|
61
|
+
## Mobile Test Devices
|
|
62
|
+
|
|
63
|
+
Device configurations for mobile viewport testing via Chrome DevTools device emulation (Playwright). These are used by validator agents to run browser checks at mobile viewports in addition to desktop. If your application has responsive layouts, configure a device here so automated tests verify mobile rendering.
|
|
64
|
+
|
|
65
|
+
| Setting | Value |
|
|
66
|
+
|---------|-------|
|
|
67
|
+
| Primary Device | `iPhone 12 Pro` |
|
|
68
|
+
| Secondary Device | `NONE` |
|
|
69
|
+
|
|
70
|
+
**Setup instructions:**
|
|
71
|
+
1. The Primary Device is the Chrome DevTools device name used for mobile viewport checks (e.g., `iPhone 12 Pro`, `Pixel 5`, `iPad Air`)
|
|
72
|
+
2. The Secondary Device is an optional second device for additional viewport coverage. Set to `NONE` to skip.
|
|
73
|
+
3. Device emulation is set via `agent-browser set device "{device name}"` before navigating to a page
|
|
74
|
+
4. Reset to desktop after mobile checks with `agent-browser set viewport 1920 1080`
|
|
75
|
+
5. If your app does not need mobile testing, set Primary Device to `NONE` -- validators will skip mobile viewport checks
|
|
76
|
+
|
|
61
77
|
## Front-End Testing
|
|
62
78
|
|
|
63
79
|
For visual and interactive front-end testing, use the `agent-browser` skill.
|
|
@@ -0,0 +1,136 @@
|
|
|
1
|
+
---
|
|
2
|
+
type: diagnosis-report
|
|
3
|
+
workflow: repair-workflow
|
|
4
|
+
target-asset: .claude/skills/testing-and-validation/SKILL.md, .claude/commands/build-phase-V2.md, .claude/commands/create-highlevel-plan-phases.md, .claude/agents/build-validator-agent.md
|
|
5
|
+
asset-type: skill, command, command, agent
|
|
6
|
+
date: 2026-03-28
|
|
7
|
+
status: validated
|
|
8
|
+
failure-patterns: [Implicit Assumptions, Missing Guardrails]
|
|
9
|
+
repair-strategies: [Explicit Instruction, Constraint Injection]
|
|
10
|
+
root-cause-layer: Command-Layer
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
# Diagnosis Report: Mobile viewport testing absent from build validation pipeline
|
|
14
|
+
|
|
15
|
+
## Problem Summary
|
|
16
|
+
|
|
17
|
+
Mobile viewport validation never happened during build-phase-V2 browser checks -- all checks ran at the default desktop viewport only, missing responsive layout issues on smaller screens. Additionally, the build validator agent occasionally used the macOS `open` command (launching Safari) instead of `agent-browser open` (Chromium), creating unpredictable, unautomatable browser behavior. The root cause spans the full prompt chain: mobile testing intent was never captured during planning, no device configuration existed in the testing skill, and the build command never instructed the validator to set a mobile viewport.
|
|
18
|
+
|
|
19
|
+
## Prompt Chain Trace
|
|
20
|
+
|
|
21
|
+
### Command Markdown (Chain A: Planning)
|
|
22
|
+
- **File**: `.claude/commands/create-highlevel-plan-phases.md`
|
|
23
|
+
- **Finding**: ABSENT. Zero mentions of mobile, viewport, device, or responsive anywhere in the 477-line file. User story and success criteria sections never prompted for mobile testing needs. Because mobile intent was never captured, no mobile success criteria could flow downstream.
|
|
24
|
+
|
|
25
|
+
### Command Markdown (Chain B: Validation)
|
|
26
|
+
- **File**: `.claude/commands/build-phase-V2.md`
|
|
27
|
+
- **Finding**: ABSENT. Browser check assembly (Step 5a) and validator delegation prompt (Step 5c) had no viewport/device instructions. The Test Credentials pattern existed and worked but was not replicated for device configuration.
|
|
28
|
+
|
|
29
|
+
### Delegation Prompt Text
|
|
30
|
+
- **Agent spawned**: build-validator-agent (via Step 5c of build-phase-V2)
|
|
31
|
+
- **Finding**: ABSENT. The delegation prompt included dev server info and test credentials but no viewport/device configuration. Without instructions, the validator ran all browser checks at the default Chromium viewport.
|
|
32
|
+
|
|
33
|
+
### Agent System Prompt
|
|
34
|
+
- **File**: `.claude/agents/build-validator-agent.md`
|
|
35
|
+
- **Finding**: ABSENT for mobile. The browser verification workflow (line 37) described navigate → snapshot → interact → re-snapshot but never mentioned setting a viewport. Also ABSENT for Safari guardrail: the agent had unrestricted `Bash` access with no prohibition on macOS `open`.
|
|
36
|
+
|
|
37
|
+
### Skill Content
|
|
38
|
+
- **Skills examined**: testing-and-validation, agent-browser
|
|
39
|
+
- **Finding**: testing-and-validation INCOMPLETE -- mentioned "responsiveness" at line 71 but had no device config table. agent-browser PRESENT but UNUSED -- `set device` and `set viewport` commands were fully documented in references/commands.md but never invoked by any workflow.
|
|
40
|
+
|
|
41
|
+
### Agent Behavior
|
|
42
|
+
- **Observed**: Validator ran all browser checks at default desktop viewport. Occasionally used macOS `open` (Safari) instead of `agent-browser open` (Chromium).
|
|
43
|
+
- **Expected**: Validator should also check at mobile viewport(s) when configured. Should always use `agent-browser open` for automatable, consistent browser testing.
|
|
44
|
+
|
|
45
|
+
## Failure Pattern Classification
|
|
46
|
+
|
|
47
|
+
### Primary Pattern: Implicit Assumptions
|
|
48
|
+
- **Evidence**: The entire prompt chain implicitly assumed desktop-only browser testing was sufficient. The capability existed (`agent-browser set device`), the need existed (mobile layout issues), but no file in the chain connected them. The assumption was invisible because each file looked correct in isolation.
|
|
49
|
+
- **Prompt chain link**: All links -- the assumption propagated through the entire chain from planning (no mobile question) through config (no device table) to validation (no viewport instructions).
|
|
50
|
+
|
|
51
|
+
### Contributing Pattern: Missing Guardrails
|
|
52
|
+
- **Evidence**: build-validator-agent had unrestricted `Bash` in its tools and no explicit prohibition on macOS `open`. The agent-browser skill's `allowed-tools: Bash(agent-browser:*)` was additive, not restrictive. The soft instruction at line 37 to use `agent-browser open` was insufficient.
|
|
53
|
+
- **Prompt chain link**: Agent system prompt (build-validator-agent.md) -- missing negative constraint.
|
|
54
|
+
|
|
55
|
+
## Root Cause Analysis
|
|
56
|
+
|
|
57
|
+
- **Root cause layer**: Command-Layer (primary), Skill-Layer (contributing)
|
|
58
|
+
- **Root cause description**: Both commands (create-highlevel-plan-phases and build-phase-V2) lacked workflow steps to establish and execute mobile testing. The planning command never asked about mobile viewports, so no mobile success criteria entered the plan. The build command never read device config or injected viewport instructions, so the validator had no reason to test at mobile dimensions. The testing-and-validation skill lacked a device configuration table, so even if the commands wanted to read config, there was nothing to read.
|
|
59
|
+
- **Why shallow fixes failed**: N/A -- first repair attempt. However, fixing only build-phase-V2 would have been insufficient because mobile intent must originate at the planning level to flow into success criteria.
|
|
60
|
+
|
|
61
|
+
## Impact Analysis
|
|
62
|
+
|
|
63
|
+
- **Target assets**: `.claude/skills/testing-and-validation/SKILL.md`, `.claude/commands/build-phase-V2.md`, `.claude/commands/create-highlevel-plan-phases.md`, `.claude/agents/build-validator-agent.md`
|
|
64
|
+
- **Blast radius**: Contained
|
|
65
|
+
- **Affected consumers**: build-validator-agent (gains constraint + mobile config), setup-project.md (not modified -- new section not in its scope), plan-validator-agent (will see plans with mobile criteria -- no behavior change)
|
|
66
|
+
- **High-risk aspects**: None. All changes are additive. Projects with Primary Device = `NONE` behave identically to before.
|
|
67
|
+
|
|
68
|
+
## Repair Applied
|
|
69
|
+
|
|
70
|
+
### Fix Specification
|
|
71
|
+
- **Repair strategy**: Explicit Instruction
|
|
72
|
+
- **Additional strategies**: Constraint Injection
|
|
73
|
+
- **What was changed**: Added mobile device config table to testing-and-validation skill. Added mobile device detection and conditional testing instructions to build-phase-V2. Added mobile testing awareness prompts to create-highlevel-plan-phases (both paths). Added NEVER/ALWAYS Safari guardrail to build-validator-agent.
|
|
74
|
+
- **File(s) modified**:
|
|
75
|
+
- `.claude/skills/testing-and-validation/SKILL.md`
|
|
76
|
+
- `.claude/commands/build-phase-V2.md`
|
|
77
|
+
- `.claude/commands/create-highlevel-plan-phases.md`
|
|
78
|
+
- `.claude/agents/build-validator-agent.md`
|
|
79
|
+
|
|
80
|
+
### Fix Details
|
|
81
|
+
|
|
82
|
+
6 changes across 4 files. Most significant change shown below.
|
|
83
|
+
|
|
84
|
+
**Before** (build-phase-V2.md, after Test Credentials block):
|
|
85
|
+
|
|
86
|
+
```
|
|
87
|
+
If Username is "REPLACE_WITH_TEST_USERNAME":
|
|
88
|
+
"Test credentials have NOT been configured..."}
|
|
89
|
+
|
|
90
|
+
## Validation Checks ({BUILD_CHECK_COUNT} -- run ALL of these)
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
**After** (build-phase-V2.md, Mobile Device Testing block inserted):
|
|
94
|
+
|
|
95
|
+
```
|
|
96
|
+
If Username is "REPLACE_WITH_TEST_USERNAME":
|
|
97
|
+
"Test credentials have NOT been configured..."}
|
|
98
|
+
|
|
99
|
+
## Mobile Device Testing
|
|
100
|
+
{Read the Mobile Test Devices section from the testing-and-validation skill.
|
|
101
|
+
|
|
102
|
+
If HAS_MOBILE_CHECKS is true (Primary Device is NOT "NONE"):
|
|
103
|
+
"Mobile viewport testing is configured. After completing each browser check at
|
|
104
|
+
desktop, you MUST repeat visual/layout checks at the mobile viewport.
|
|
105
|
+
|
|
106
|
+
Mobile testing procedure:
|
|
107
|
+
1. Complete desktop check first
|
|
108
|
+
2. Set mobile device: `agent-browser set device "{MOBILE_PRIMARY_DEVICE}"`
|
|
109
|
+
3. Reload: `agent-browser reload`
|
|
110
|
+
4. Snapshot: `agent-browser snapshot -i`
|
|
111
|
+
5. Screenshot: `agent-browser screenshot`
|
|
112
|
+
6. Verify: no overlapping elements, no horizontal scrolling, no truncated content
|
|
113
|
+
7. Reset to desktop: `agent-browser set viewport 1920 1080`
|
|
114
|
+
|
|
115
|
+
If a check passes at desktop but fails at mobile, overall result is FAIL."
|
|
116
|
+
|
|
117
|
+
If HAS_MOBILE_CHECKS is false:
|
|
118
|
+
"Mobile viewport testing is not configured. Desktop viewport only."}
|
|
119
|
+
|
|
120
|
+
## Validation Checks ({BUILD_CHECK_COUNT} -- run ALL of these)
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
## Validation Results
|
|
124
|
+
|
|
125
|
+
- **Fix addresses root cause**: Yes -- mobile testing is now explicitly wired through all 4 prompt chain links (planning → config → validation → agent constraint)
|
|
126
|
+
- **No new failure patterns introduced**: Yes -- the `set device reset` issue caught during pre-implementation validation was corrected to `set viewport 1920 1080` before implementation
|
|
127
|
+
- **Impact analysis clear**: Yes -- all consumers verified, no regressions found
|
|
128
|
+
- **Test result**: Pass -- post-implementation validation passed 4/4 checks
|
|
129
|
+
|
|
130
|
+
## Prevention Recommendations
|
|
131
|
+
|
|
132
|
+
1. **Capability-to-instruction audit**: When a skill documents a capability (like `agent-browser set device`), trace every consuming command/agent and verify there is an explicit instruction to use it when relevant. Undocumented capabilities are dead weight.
|
|
133
|
+
|
|
134
|
+
2. **Config table completeness check**: The testing-and-validation skill is the config hub for validators. New testing dimensions (mobile viewports, accessibility, performance budgets) should follow the established pattern: config table in skill → conditional read in build-phase-V2 Step 5a → conditional injection in Step 5c.
|
|
135
|
+
|
|
136
|
+
3. **Negative constraint review for unrestricted Bash agents**: Any agent with `tools: Bash` should have explicit NEVER constraints for known-bad alternatives to intended tools (e.g., macOS `open` vs `agent-browser open`).
|
package/manifest.json
CHANGED