cbrowser 7.4.19 → 7.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +28 -6
- package/dist/mcp-server-remote.js +3 -3
- package/dist/mcp-server-remote.js.map +1 -1
- package/dist/mcp-server.js +1 -1
- package/dist/mcp-server.js.map +1 -1
- package/examples/ci-cd/README.md +34 -0
- package/examples/ci-cd/github-actions.yml +57 -0
- package/examples/ci-cd/gitlab-ci.yml +63 -0
- package/examples/journeys/signup-flow.json +35 -0
- package/examples/natural-language-tests/README.md +77 -0
- package/examples/natural-language-tests/auth-flow-suite.txt +47 -0
- package/examples/natural-language-tests/e-commerce-suite.txt +96 -0
- package/examples/personas/accessibility-tester.json +53 -0
- package/examples/workflows/accessibility-audit.md +151 -0
- package/examples/workflows/chaos-resilience-testing.md +151 -0
- package/examples/workflows/e2e-login-checkout.md +106 -0
- package/examples/workflows/persona-comparison-report.md +153 -0
- package/examples/workflows/visual-regression-ci.md +123 -0
- package/package.json +1 -1
|
@@ -0,0 +1,77 @@
|
|
|
1
|
+
# Natural Language Test Suites
|
|
2
|
+
|
|
3
|
+
CBrowser supports writing browser tests in plain, human-readable text files. No code, no selectors, no framework boilerplate -- just describe what the test should do and CBrowser handles the rest.
|
|
4
|
+
|
|
5
|
+
## Available Commands
|
|
6
|
+
|
|
7
|
+
| Command | Description | Example |
|
|
8
|
+
|---|---|---|
|
|
9
|
+
| `# Test: <name>` | Start a new test case | `# Test: Login Flow` |
|
|
10
|
+
| `go to <url>` | Navigate to a URL | `go to https://example.com` |
|
|
11
|
+
| `click "<text>"` | Click an element by visible text | `click "Sign In"` |
|
|
12
|
+
| `click the <description>` | Click an element by description | `click the hamburger menu icon` |
|
|
13
|
+
| `fill "<field>" with "<value>"` | Fill a form field | `fill "Email" with "user@test.com"` |
|
|
14
|
+
| `fill <field> with "<value>"` | Fill a form field (no quotes on field) | `fill Email with "user@test.com"` |
|
|
15
|
+
| `type "<text>" in <field>` | Type into a specific field | `type "hello" in search box` |
|
|
16
|
+
| `verify page contains "<text>"` | Assert text is visible on page | `verify page contains "Welcome"` |
|
|
17
|
+
| `verify url contains "<path>"` | Assert current URL includes path | `verify url contains "/dashboard"` |
|
|
18
|
+
| `take screenshot` | Capture the current page state | `take screenshot` |
|
|
19
|
+
| `scroll down` | Scroll down the page | `scroll down` |
|
|
20
|
+
| `scroll up` | Scroll up the page | `scroll up` |
|
|
21
|
+
| `scroll down N times` | Scroll down multiple times | `scroll down 3 times` |
|
|
22
|
+
| `wait <N> seconds` | Pause execution | `wait 2 seconds` |
|
|
23
|
+
| `@viewport mobile` | Set viewport to mobile size | `@viewport mobile` |
|
|
24
|
+
| `@viewport tablet` | Set viewport to tablet size | `@viewport tablet` |
|
|
25
|
+
| `@viewport desktop` | Set viewport to desktop size | `@viewport desktop` |
|
|
26
|
+
|
|
27
|
+
## Running Tests
|
|
28
|
+
|
|
29
|
+
Run a test suite file:
|
|
30
|
+
|
|
31
|
+
```bash
|
|
32
|
+
npx cbrowser test-suite path/to/suite.txt
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
### Flags
|
|
36
|
+
|
|
37
|
+
| Flag | Description |
|
|
38
|
+
|---|---|
|
|
39
|
+
| `--dry-run` | Parse and validate the test file without executing any browser actions. Useful for checking syntax. |
|
|
40
|
+
| `--fuzzy-match` | Use fuzzy matching for click targets and text verification. Handles minor text differences gracefully. |
|
|
41
|
+
| `--step-through` | Pause after each step and wait for Enter to continue. Great for debugging. |
|
|
42
|
+
| `--verbose` | Print detailed logs for each step including timing, selectors found, and actions taken. |
|
|
43
|
+
| `--html` | Generate an HTML report with screenshots and pass/fail status for each step. |
|
|
44
|
+
|
|
45
|
+
### Examples
|
|
46
|
+
|
|
47
|
+
```bash
|
|
48
|
+
# Validate syntax without running
|
|
49
|
+
npx cbrowser test-suite auth-flow-suite.txt --dry-run
|
|
50
|
+
|
|
51
|
+
# Run with detailed output
|
|
52
|
+
npx cbrowser test-suite e-commerce-suite.txt --verbose
|
|
53
|
+
|
|
54
|
+
# Step through interactively for debugging
|
|
55
|
+
npx cbrowser test-suite auth-flow-suite.txt --step-through
|
|
56
|
+
|
|
57
|
+
# Generate an HTML report
|
|
58
|
+
npx cbrowser test-suite e-commerce-suite.txt --html --verbose
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
## Tips for Writing Robust Tests
|
|
62
|
+
|
|
63
|
+
1. **Use visible text for clicks.** Prefer `click "Sign In"` over trying to describe elements by position. CBrowser matches against visible text content first.
|
|
64
|
+
|
|
65
|
+
2. **Add waits after navigation.** Pages need time to load. A `wait 2 seconds` after `go to` or after clicks that trigger navigation prevents flaky failures.
|
|
66
|
+
|
|
67
|
+
3. **Take screenshots at key moments.** Screenshots at critical checkpoints make debugging failures much easier. Place them after verifications.
|
|
68
|
+
|
|
69
|
+
4. **Verify before acting.** Use `verify page contains` to confirm the page is in the expected state before clicking or filling forms. This catches navigation issues early.
|
|
70
|
+
|
|
71
|
+
5. **Keep tests focused.** Each `# Test:` block should test one specific flow. Smaller tests are easier to debug and maintain.
|
|
72
|
+
|
|
73
|
+
6. **Use `@viewport` for responsive testing.** Set the viewport at the start of a test to validate mobile or tablet layouts. The viewport persists for the entire test case.
|
|
74
|
+
|
|
75
|
+
7. **Use `--fuzzy-match` for dynamic content.** If text on the page varies slightly between runs (timestamps, counts), fuzzy matching prevents false failures.
|
|
76
|
+
|
|
77
|
+
8. **Use `--dry-run` to validate new tests.** Before running a new suite against a live site, dry-run it to catch typos and syntax issues.
|
|
@@ -0,0 +1,47 @@
|
|
|
1
|
+
# Test: Login with Valid Credentials
|
|
2
|
+
|
|
3
|
+
go to https://app.example.com/login
|
|
4
|
+
verify page contains "Sign In"
|
|
5
|
+
take screenshot
|
|
6
|
+
fill "Email" with "user@example.com"
|
|
7
|
+
fill "Password" with "SecurePass123!"
|
|
8
|
+
click "Sign In"
|
|
9
|
+
wait 3 seconds
|
|
10
|
+
verify page contains "Dashboard"
|
|
11
|
+
verify url contains "/dashboard"
|
|
12
|
+
take screenshot
|
|
13
|
+
|
|
14
|
+
# Test: Login with Invalid Credentials
|
|
15
|
+
|
|
16
|
+
go to https://app.example.com/login
|
|
17
|
+
verify page contains "Sign In"
|
|
18
|
+
fill "Email" with "user@example.com"
|
|
19
|
+
fill "Password" with "WrongPassword"
|
|
20
|
+
click "Sign In"
|
|
21
|
+
wait 2 seconds
|
|
22
|
+
verify page contains "Invalid email or password"
|
|
23
|
+
take screenshot
|
|
24
|
+
verify url contains "/login"
|
|
25
|
+
|
|
26
|
+
fill "Email" with "nonexistent@example.com"
|
|
27
|
+
fill "Password" with "AnyPassword1"
|
|
28
|
+
click "Sign In"
|
|
29
|
+
wait 2 seconds
|
|
30
|
+
verify page contains "Invalid email or password"
|
|
31
|
+
take screenshot
|
|
32
|
+
|
|
33
|
+
# Test: Password Reset Flow
|
|
34
|
+
|
|
35
|
+
go to https://app.example.com/login
|
|
36
|
+
verify page contains "Sign In"
|
|
37
|
+
click "Forgot password?"
|
|
38
|
+
wait 2 seconds
|
|
39
|
+
verify page contains "Reset Password"
|
|
40
|
+
verify url contains "/reset-password"
|
|
41
|
+
take screenshot
|
|
42
|
+
fill "Email" with "user@example.com"
|
|
43
|
+
click "Send Reset Link"
|
|
44
|
+
wait 3 seconds
|
|
45
|
+
verify page contains "Check your email"
|
|
46
|
+
verify page contains "reset link"
|
|
47
|
+
take screenshot
|
|
@@ -0,0 +1,96 @@
|
|
|
1
|
+
# Test: Guest Checkout Flow
|
|
2
|
+
|
|
3
|
+
go to https://shop.example.com
|
|
4
|
+
verify page contains "Welcome"
|
|
5
|
+
click "Shop All"
|
|
6
|
+
wait 2 seconds
|
|
7
|
+
scroll down 3 times
|
|
8
|
+
click "Classic Cotton T-Shirt"
|
|
9
|
+
verify page contains "Add to Cart"
|
|
10
|
+
take screenshot
|
|
11
|
+
click "Add to Cart"
|
|
12
|
+
verify page contains "Added to cart"
|
|
13
|
+
click "Cart"
|
|
14
|
+
verify page contains "Classic Cotton T-Shirt"
|
|
15
|
+
click "Proceed to Checkout"
|
|
16
|
+
verify page contains "Checkout"
|
|
17
|
+
click "Continue as Guest"
|
|
18
|
+
fill "Email" with "guest@example.com"
|
|
19
|
+
fill "First Name" with "Jordan"
|
|
20
|
+
fill "Last Name" with "Lee"
|
|
21
|
+
fill "Address" with "742 Evergreen Terrace"
|
|
22
|
+
fill "City" with "Springfield"
|
|
23
|
+
fill "Zip Code" with "62704"
|
|
24
|
+
click "Continue to Shipping"
|
|
25
|
+
verify page contains "Shipping Method"
|
|
26
|
+
click "Standard Shipping"
|
|
27
|
+
click "Continue to Payment"
|
|
28
|
+
verify page contains "Payment"
|
|
29
|
+
take screenshot
|
|
30
|
+
fill "Card Number" with "4111111111111111"
|
|
31
|
+
fill "Expiration" with "12/28"
|
|
32
|
+
fill "CVV" with "123"
|
|
33
|
+
click "Place Order"
|
|
34
|
+
wait 3 seconds
|
|
35
|
+
verify page contains "Order Confirmed"
|
|
36
|
+
verify page contains "Thank you"
|
|
37
|
+
take screenshot
|
|
38
|
+
|
|
39
|
+
# Test: Search and Filter Products
|
|
40
|
+
|
|
41
|
+
go to https://shop.example.com
|
|
42
|
+
fill "Search" with "running shoes"
|
|
43
|
+
click "Search"
|
|
44
|
+
wait 2 seconds
|
|
45
|
+
verify page contains "running shoes"
|
|
46
|
+
take screenshot
|
|
47
|
+
click "Price: Low to High"
|
|
48
|
+
wait 1 seconds
|
|
49
|
+
click "Size"
|
|
50
|
+
click "10"
|
|
51
|
+
click "Color"
|
|
52
|
+
click "Black"
|
|
53
|
+
click "Apply Filters"
|
|
54
|
+
wait 2 seconds
|
|
55
|
+
verify page contains "results"
|
|
56
|
+
scroll down 2 times
|
|
57
|
+
take screenshot
|
|
58
|
+
click the first product in the results
|
|
59
|
+
verify page contains "Add to Cart"
|
|
60
|
+
verify page contains "running"
|
|
61
|
+
verify url contains "/product/"
|
|
62
|
+
take screenshot
|
|
63
|
+
|
|
64
|
+
# Test: Mobile Responsive Checkout
|
|
65
|
+
|
|
66
|
+
@viewport mobile
|
|
67
|
+
go to https://shop.example.com
|
|
68
|
+
verify page contains "Welcome"
|
|
69
|
+
take screenshot
|
|
70
|
+
click the hamburger menu icon
|
|
71
|
+
wait 1 seconds
|
|
72
|
+
click "Shop All"
|
|
73
|
+
wait 2 seconds
|
|
74
|
+
scroll down 2 times
|
|
75
|
+
click the first product card
|
|
76
|
+
verify page contains "Add to Cart"
|
|
77
|
+
click "Add to Cart"
|
|
78
|
+
verify page contains "Added to cart"
|
|
79
|
+
take screenshot
|
|
80
|
+
click the cart icon
|
|
81
|
+
verify page contains "Cart"
|
|
82
|
+
click "Checkout"
|
|
83
|
+
verify page contains "Checkout"
|
|
84
|
+
fill "Email" with "mobile@example.com"
|
|
85
|
+
fill "First Name" with "Alex"
|
|
86
|
+
fill "Last Name" with "Rivera"
|
|
87
|
+
fill "Address" with "100 Main Street"
|
|
88
|
+
fill "City" with "Portland"
|
|
89
|
+
fill "Zip Code" with "97201"
|
|
90
|
+
scroll down
|
|
91
|
+
click "Continue to Shipping"
|
|
92
|
+
wait 2 seconds
|
|
93
|
+
click "Standard Shipping"
|
|
94
|
+
click "Continue to Payment"
|
|
95
|
+
take screenshot
|
|
96
|
+
verify page contains "Payment"
|
|
@@ -0,0 +1,53 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "accessibility-tester",
|
|
3
|
+
"description": "User with visual impairments who relies on screen magnification and keyboard navigation",
|
|
4
|
+
"demographics": {
|
|
5
|
+
"age_range": "40-65",
|
|
6
|
+
"tech_level": "intermediate",
|
|
7
|
+
"device": "desktop"
|
|
8
|
+
},
|
|
9
|
+
"behaviors": {
|
|
10
|
+
"methodical": true,
|
|
11
|
+
"checks_edge_cases": true,
|
|
12
|
+
"documents_issues": false,
|
|
13
|
+
"follows_test_plans": false,
|
|
14
|
+
"uses_keyboard_navigation": true,
|
|
15
|
+
"avoids_mouse": true,
|
|
16
|
+
"uses_screen_magnification": true,
|
|
17
|
+
"prefers_high_contrast": true
|
|
18
|
+
},
|
|
19
|
+
"humanBehavior": {
|
|
20
|
+
"timing": {
|
|
21
|
+
"reactionTime": { "min": 800, "max": 2000 },
|
|
22
|
+
"clickDelay": { "min": 400, "max": 900 },
|
|
23
|
+
"typeSpeed": { "min": 120, "max": 250 },
|
|
24
|
+
"readingSpeed": 150,
|
|
25
|
+
"scrollPauseTime": { "min": 800, "max": 2000 }
|
|
26
|
+
},
|
|
27
|
+
"errors": {
|
|
28
|
+
"misClickRate": 0.15,
|
|
29
|
+
"doubleClickAccidental": 0.08,
|
|
30
|
+
"typoRate": 0.10,
|
|
31
|
+
"backtrackRate": 0.25
|
|
32
|
+
},
|
|
33
|
+
"mouse": {
|
|
34
|
+
"curvature": 0.6,
|
|
35
|
+
"jitter": 12,
|
|
36
|
+
"overshoot": 0.20,
|
|
37
|
+
"speed": "slow"
|
|
38
|
+
},
|
|
39
|
+
"attention": {
|
|
40
|
+
"pattern": "focused",
|
|
41
|
+
"scrollBehavior": "incremental",
|
|
42
|
+
"focusAreas": ["text", "cta"],
|
|
43
|
+
"distractionRate": 0.15
|
|
44
|
+
}
|
|
45
|
+
},
|
|
46
|
+
"context": {
|
|
47
|
+
"viewport": [1920, 1080],
|
|
48
|
+
"zoom": 200,
|
|
49
|
+
"assistiveTech": ["screen-magnifier", "keyboard-only"],
|
|
50
|
+
"preferredNavigation": "keyboard",
|
|
51
|
+
"colorScheme": "high-contrast"
|
|
52
|
+
}
|
|
53
|
+
}
|
|
@@ -0,0 +1,151 @@
|
|
|
1
|
+
# Accessibility Audit with Bug Hunting and Persona Testing
|
|
2
|
+
|
|
3
|
+
Find accessibility issues, get actionable fix recommendations, and validate usability across different user personas with assistive technology needs.
|
|
4
|
+
|
|
5
|
+
## Prerequisites
|
|
6
|
+
|
|
7
|
+
- Node.js 18+
|
|
8
|
+
- CBrowser installed: `npm install -g cbrowser`
|
|
9
|
+
- Target application at `https://app.example.com`
|
|
10
|
+
|
|
11
|
+
## Step 1: Hunt for Accessibility Bugs
|
|
12
|
+
|
|
13
|
+
Scan pages for WCAG violations, usability issues, and assistive technology problems.
|
|
14
|
+
|
|
15
|
+
```bash
|
|
16
|
+
npx cbrowser hunt "https://app.example.com"
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
### Sample Bug Report Output
|
|
20
|
+
|
|
21
|
+
```
|
|
22
|
+
Bug Hunt Report: https://app.example.com
|
|
23
|
+
Scan completed: 2026-02-03T10:15:00Z
|
|
24
|
+
Pages scanned: 5
|
|
25
|
+
|
|
26
|
+
=== Critical (3 issues) ===
|
|
27
|
+
|
|
28
|
+
BUG-001: Missing form labels
|
|
29
|
+
Page: /signup
|
|
30
|
+
Element: <input type="email" placeholder="Email">
|
|
31
|
+
WCAG: 1.3.1 (Info and Relationships), 4.1.2 (Name, Role, Value)
|
|
32
|
+
Impact: Screen readers cannot identify the purpose of this field
|
|
33
|
+
Recommendation: Add <label for="email">Email address</label> or aria-label
|
|
34
|
+
|
|
35
|
+
BUG-002: Insufficient color contrast
|
|
36
|
+
Page: /dashboard
|
|
37
|
+
Element: .status-text (gray #999 on white #FFF)
|
|
38
|
+
WCAG: 1.4.3 (Contrast Minimum)
|
|
39
|
+
Impact: Text unreadable for users with low vision
|
|
40
|
+
Contrast ratio: 2.85:1 (minimum required: 4.5:1)
|
|
41
|
+
Recommendation: Darken text to at least #767676 for 4.54:1 ratio
|
|
42
|
+
|
|
43
|
+
BUG-003: No skip navigation link
|
|
44
|
+
Page: / (all pages)
|
|
45
|
+
WCAG: 2.4.1 (Bypass Blocks)
|
|
46
|
+
Impact: Keyboard users must tab through 24 nav items on every page
|
|
47
|
+
Recommendation: Add a visually hidden "Skip to main content" link as the first focusable element
|
|
48
|
+
|
|
49
|
+
=== Moderate (5 issues) ===
|
|
50
|
+
|
|
51
|
+
BUG-004: Images missing alt text (3 instances on /products)
|
|
52
|
+
BUG-005: Focus order breaks in modal dialogs (/settings)
|
|
53
|
+
BUG-006: Timeout with no warning on /checkout (30s session)
|
|
54
|
+
BUG-007: Error messages not announced to screen readers (/signup)
|
|
55
|
+
BUG-008: Touch targets below 44x44px on mobile nav
|
|
56
|
+
|
|
57
|
+
Summary: 3 critical, 5 moderate, 4 informational
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
## Step 2: Persona-Based Testing
|
|
61
|
+
|
|
62
|
+
Test the application from the perspective of users with different needs.
|
|
63
|
+
|
|
64
|
+
```bash
|
|
65
|
+
# Test as a screen reader user
|
|
66
|
+
npx cbrowser test-persona "https://app.example.com" --persona screen-reader-user \
|
|
67
|
+
--goal "Sign up for an account and navigate to the dashboard"
|
|
68
|
+
|
|
69
|
+
# Test as an elderly user
|
|
70
|
+
npx cbrowser test-persona "https://app.example.com" --persona elderly-user \
|
|
71
|
+
--goal "Find a product and add it to the cart"
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
### Sample Persona Output (Screen Reader User)
|
|
75
|
+
|
|
76
|
+
```
|
|
77
|
+
Persona Test: screen-reader-user
|
|
78
|
+
Goal: Sign up for an account and navigate to the dashboard
|
|
79
|
+
|
|
80
|
+
Step 1: Landing page
|
|
81
|
+
- Page title announced: "Welcome to App" [OK]
|
|
82
|
+
- Landmark regions found: header, nav, main, footer [OK]
|
|
83
|
+
- Heading hierarchy: h1 > h2 > h3 [OK]
|
|
84
|
+
|
|
85
|
+
Step 2: Navigate to signup
|
|
86
|
+
- Found "Sign Up" link via navigation landmark [OK]
|
|
87
|
+
- Link purpose clear from text alone [OK]
|
|
88
|
+
|
|
89
|
+
Step 3: Complete signup form
|
|
90
|
+
- Email field: NO LABEL ANNOUNCED [FAIL]
|
|
91
|
+
- Password field: label "Password" announced [OK]
|
|
92
|
+
- Submit button: "Create Account" announced [OK]
|
|
93
|
+
- Form errors: NOT announced on submission [FAIL]
|
|
94
|
+
|
|
95
|
+
Step 4: Dashboard
|
|
96
|
+
- Page title changed: announced [OK]
|
|
97
|
+
- Dynamic content region: no live region set [FAIL]
|
|
98
|
+
|
|
99
|
+
Result: PARTIAL SUCCESS (6/9 checks passed)
|
|
100
|
+
Blockers: 3 issues would prevent independent task completion
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
## Step 3: Compare Personas
|
|
104
|
+
|
|
105
|
+
Run the same journey across multiple personas to surface patterns.
|
|
106
|
+
|
|
107
|
+
```bash
|
|
108
|
+
npx cbrowser compare-personas \
|
|
109
|
+
--personas elderly-user,power-user \
|
|
110
|
+
--start "https://app.example.com" \
|
|
111
|
+
--goal "Complete signup"
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
```
|
|
115
|
+
Persona Comparison: "Complete signup"
|
|
116
|
+
|
|
117
|
+
elderly-user power-user
|
|
118
|
+
Completed: Yes (with difficulty) Yes
|
|
119
|
+
Time to complete: 4m 12s 0m 48s
|
|
120
|
+
Errors encountered: 3 0
|
|
121
|
+
Friction points: 5 1
|
|
122
|
+
- Small text [x] [ ]
|
|
123
|
+
- Unclear errors [x] [ ]
|
|
124
|
+
- Tiny click targets [x] [ ]
|
|
125
|
+
- Complex password [x] [x]
|
|
126
|
+
- No text resize [x] [ ]
|
|
127
|
+
|
|
128
|
+
Top recommendation: Increase base font size and touch targets
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
## Combining Audits
|
|
132
|
+
|
|
133
|
+
Run a full accessibility workflow in one pass:
|
|
134
|
+
|
|
135
|
+
```bash
|
|
136
|
+
# Full audit pipeline
|
|
137
|
+
npx cbrowser hunt "https://app.example.com" --output a11y-report.json
|
|
138
|
+
npx cbrowser compare-personas \
|
|
139
|
+
--personas screen-reader-user,elderly-user,power-user \
|
|
140
|
+
--start "https://app.example.com" \
|
|
141
|
+
--goal "Complete signup" \
|
|
142
|
+
--html
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
## Next Steps
|
|
146
|
+
|
|
147
|
+
- Fix critical issues first: missing labels and contrast failures block the most users.
|
|
148
|
+
- Rerun `hunt` after fixes to verify remediation.
|
|
149
|
+
- Add persona tests to CI to prevent regressions.
|
|
150
|
+
- Expand to additional personas: `keyboard-only-user`, `cognitive-disability-user`, `low-vision-user`.
|
|
151
|
+
- Generate compliance reports with `--format wcag21-aa` for stakeholder review.
|
|
@@ -0,0 +1,151 @@
|
|
|
1
|
+
# Chaos Engineering for Web Applications
|
|
2
|
+
|
|
3
|
+
Inject real-world failures into your web application and verify it degrades gracefully. Simulate network outages, slow responses, missing elements, and server errors to build confidence in your app's resilience.
|
|
4
|
+
|
|
5
|
+
## Prerequisites
|
|
6
|
+
|
|
7
|
+
- Node.js 18+
|
|
8
|
+
- CBrowser installed: `npm install -g cbrowser`
|
|
9
|
+
- Target application at `https://app.example.com`
|
|
10
|
+
|
|
11
|
+
## Step 1: Run a Chaos Test
|
|
12
|
+
|
|
13
|
+
CBrowser injects failures while monitoring how the application responds.
|
|
14
|
+
|
|
15
|
+
```bash
|
|
16
|
+
npx cbrowser chaos "https://app.example.com/dashboard"
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
This runs a default suite of chaos scenarios: network interruptions, slow API responses, DOM element removal, and error injection.
|
|
20
|
+
|
|
21
|
+
## Step 2: Target Specific Chaos Scenarios
|
|
22
|
+
|
|
23
|
+
Focus on individual failure modes for deeper analysis.
|
|
24
|
+
|
|
25
|
+
```bash
|
|
26
|
+
# Simulate network failures (offline, intermittent, packet loss)
|
|
27
|
+
npx cbrowser chaos "https://app.example.com/dashboard" --scenario network-failure
|
|
28
|
+
|
|
29
|
+
# Simulate slow API responses (2s, 5s, 10s, 30s delays)
|
|
30
|
+
npx cbrowser chaos "https://app.example.com/dashboard" --scenario slow-responses
|
|
31
|
+
|
|
32
|
+
# Remove critical DOM elements mid-session
|
|
33
|
+
npx cbrowser chaos "https://app.example.com/dashboard" --scenario element-removal
|
|
34
|
+
|
|
35
|
+
# Inject HTTP 500 errors on API calls
|
|
36
|
+
npx cbrowser chaos "https://app.example.com/dashboard" --scenario server-errors
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
## Sample Chaos Test Output
|
|
40
|
+
|
|
41
|
+
```
|
|
42
|
+
Chaos Test Report: https://app.example.com/dashboard
|
|
43
|
+
Generated: 2026-02-03T16:00:00Z
|
|
44
|
+
Scenarios executed: 4
|
|
45
|
+
|
|
46
|
+
=== Network Failure ===
|
|
47
|
+
|
|
48
|
+
Test: Complete offline (network disabled for 10s)
|
|
49
|
+
App behavior: Loading spinner appeared after 3s [OK]
|
|
50
|
+
Error message: "You're offline. Changes will sync when reconnected." [OK]
|
|
51
|
+
Data loss: None (localStorage cache used) [OK]
|
|
52
|
+
Recovery: Auto-reconnected and synced within 2s [OK]
|
|
53
|
+
Result: PASS
|
|
54
|
+
|
|
55
|
+
Test: Intermittent connectivity (50% packet loss for 15s)
|
|
56
|
+
App behavior: Partial data loaded, retry indicators shown [OK]
|
|
57
|
+
Error message: None shown to user [WARN - silent failure]
|
|
58
|
+
Data loss: 2 API calls dropped without retry [FAIL]
|
|
59
|
+
Recovery: Required manual page refresh [FAIL]
|
|
60
|
+
Result: FAIL
|
|
61
|
+
Recommendation: Implement automatic retry with exponential backoff
|
|
62
|
+
|
|
63
|
+
=== Slow Responses ===
|
|
64
|
+
|
|
65
|
+
Test: API latency 5s
|
|
66
|
+
App behavior: Loading skeleton displayed [OK]
|
|
67
|
+
User feedback: Progress indicator visible [OK]
|
|
68
|
+
Timeout handling: No timeout applied [WARN]
|
|
69
|
+
Result: PASS
|
|
70
|
+
|
|
71
|
+
Test: API latency 30s
|
|
72
|
+
App behavior: Frozen UI, no loading indicator [FAIL]
|
|
73
|
+
User feedback: None for 30 seconds [FAIL]
|
|
74
|
+
Timeout handling: No timeout, user stuck [FAIL]
|
|
75
|
+
Result: FAIL
|
|
76
|
+
Recommendation: Set 10s timeout with retry option and loading state
|
|
77
|
+
|
|
78
|
+
=== Element Removal ===
|
|
79
|
+
|
|
80
|
+
Test: Navigation bar removed
|
|
81
|
+
App behavior: No JavaScript errors [OK]
|
|
82
|
+
Fallback: Breadcrumb links still functional [OK]
|
|
83
|
+
Result: PASS
|
|
84
|
+
|
|
85
|
+
Test: Data table removed mid-render
|
|
86
|
+
App behavior: Uncaught TypeError in console [FAIL]
|
|
87
|
+
Fallback: Blank screen, no error boundary [FAIL]
|
|
88
|
+
Result: FAIL
|
|
89
|
+
Recommendation: Add React error boundaries around data components
|
|
90
|
+
|
|
91
|
+
=== Server Errors (HTTP 500) ===
|
|
92
|
+
|
|
93
|
+
Test: Dashboard API returns 500
|
|
94
|
+
App behavior: "Something went wrong" message displayed [OK]
|
|
95
|
+
Retry option: "Try again" button present and functional [OK]
|
|
96
|
+
Logging: Error logged to monitoring service [OK]
|
|
97
|
+
Result: PASS
|
|
98
|
+
|
|
99
|
+
Test: Auth API returns 500
|
|
100
|
+
App behavior: Silent failure, dashboard shows stale data [FAIL]
|
|
101
|
+
User feedback: No indication of auth failure [FAIL]
|
|
102
|
+
Result: FAIL
|
|
103
|
+
Recommendation: Redirect to login with "Session expired" message
|
|
104
|
+
|
|
105
|
+
Summary: 4 passed, 4 failed, 2 warnings
|
|
106
|
+
Resilience score: 5.2/10
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
## Step 3: Chaos with Assertions
|
|
110
|
+
|
|
111
|
+
Combine chaos injection with explicit assertions to create repeatable resilience tests.
|
|
112
|
+
|
|
113
|
+
```bash
|
|
114
|
+
npx cbrowser chaos "https://app.example.com/dashboard" \
|
|
115
|
+
--scenario network-failure \
|
|
116
|
+
--assert "error message is visible within 5 seconds" \
|
|
117
|
+
--assert "no data is lost after reconnection" \
|
|
118
|
+
--assert "no uncaught JavaScript errors in console"
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
```
|
|
122
|
+
Chaos + Assertions: network-failure
|
|
123
|
+
|
|
124
|
+
[PASS] error message is visible within 5 seconds (shown at 3.1s)
|
|
125
|
+
[PASS] no data is lost after reconnection (all items verified)
|
|
126
|
+
[FAIL] no uncaught JavaScript errors in console
|
|
127
|
+
Found: TypeError: Cannot read properties of undefined (reading 'map')
|
|
128
|
+
Source: dashboard.js:142
|
|
129
|
+
|
|
130
|
+
Results: 2 passed, 1 failed
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
## CI Integration
|
|
134
|
+
|
|
135
|
+
Add resilience checks to your deployment pipeline:
|
|
136
|
+
|
|
137
|
+
```yaml
|
|
138
|
+
- name: Chaos Resilience Gate
|
|
139
|
+
run: |
|
|
140
|
+
npx cbrowser chaos "$STAGING_URL/dashboard" --scenario network-failure --ci
|
|
141
|
+
npx cbrowser chaos "$STAGING_URL/dashboard" --scenario slow-responses --ci
|
|
142
|
+
npx cbrowser chaos "$STAGING_URL/checkout" --scenario server-errors --ci
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
## Next Steps
|
|
146
|
+
|
|
147
|
+
- Start with `network-failure` and `server-errors` scenarios, as these are the most common production issues.
|
|
148
|
+
- Set a resilience score threshold in CI: `--min-score 7.0`.
|
|
149
|
+
- Combine chaos tests with visual regression to catch UI breakage during failures.
|
|
150
|
+
- Add custom scenarios for your domain: payment gateway timeouts, third-party script failures, CDN outages.
|
|
151
|
+
- Run chaos tests against staging before every production deploy.
|
|
@@ -0,0 +1,106 @@
|
|
|
1
|
+
# End-to-End Login + Checkout Flow
|
|
2
|
+
|
|
3
|
+
Automate a complete purchase journey from authentication through checkout using session persistence and natural language test assertions.
|
|
4
|
+
|
|
5
|
+
## Prerequisites
|
|
6
|
+
|
|
7
|
+
- Node.js 18+
|
|
8
|
+
- CBrowser installed: `npm install -g cbrowser`
|
|
9
|
+
- Target application running at `https://shop.example.com`
|
|
10
|
+
|
|
11
|
+
## Step 1: Authenticate and Save Session
|
|
12
|
+
|
|
13
|
+
Log in once and persist the authenticated session for reuse across tests.
|
|
14
|
+
|
|
15
|
+
```bash
|
|
16
|
+
# Navigate to the login page and authenticate
|
|
17
|
+
npx cbrowser navigate "https://shop.example.com/login"
|
|
18
|
+
|
|
19
|
+
# Fill credentials and submit (CBrowser uses AI to find form fields)
|
|
20
|
+
npx cbrowser act "Enter username 'testuser@example.com' and password 'secure123', then click Sign In"
|
|
21
|
+
|
|
22
|
+
# Save the authenticated session for later reuse
|
|
23
|
+
npx cbrowser session save logged-in
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
## Step 2: Define the Test Suite
|
|
27
|
+
|
|
28
|
+
Create a file called `checkout-flow.test.nl` with natural language test steps:
|
|
29
|
+
|
|
30
|
+
```text
|
|
31
|
+
suite: Checkout Flow
|
|
32
|
+
session: logged-in
|
|
33
|
+
|
|
34
|
+
test: Add item to cart
|
|
35
|
+
navigate to https://shop.example.com/products
|
|
36
|
+
click on the first "Add to Cart" button
|
|
37
|
+
assert the cart badge shows "1"
|
|
38
|
+
|
|
39
|
+
test: Update cart quantity
|
|
40
|
+
navigate to https://shop.example.com/cart
|
|
41
|
+
change the quantity of the first item to 2
|
|
42
|
+
assert the subtotal updates to reflect 2 items
|
|
43
|
+
|
|
44
|
+
test: Apply discount code
|
|
45
|
+
type "SAVE10" into the discount code field
|
|
46
|
+
click "Apply"
|
|
47
|
+
assert a discount line item appears showing 10% off
|
|
48
|
+
|
|
49
|
+
test: Complete checkout
|
|
50
|
+
click "Proceed to Checkout"
|
|
51
|
+
assert the shipping address form is visible
|
|
52
|
+
select "Standard Shipping"
|
|
53
|
+
click "Place Order"
|
|
54
|
+
assert the page shows "Order Confirmed"
|
|
55
|
+
assert an order number is displayed
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
## Step 3: Run the Test Suite
|
|
59
|
+
|
|
60
|
+
```bash
|
|
61
|
+
npx cbrowser test-suite checkout-flow.test.nl
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
## Expected Output
|
|
65
|
+
|
|
66
|
+
```
|
|
67
|
+
CBrowser Test Suite: Checkout Flow
|
|
68
|
+
Session: loaded "logged-in" (authenticated as testuser@example.com)
|
|
69
|
+
|
|
70
|
+
[PASS] Add item to cart (3.2s)
|
|
71
|
+
- Navigated to /products
|
|
72
|
+
- Clicked "Add to Cart" on "Wireless Headphones"
|
|
73
|
+
- Cart badge updated to "1"
|
|
74
|
+
|
|
75
|
+
[PASS] Update cart quantity (2.1s)
|
|
76
|
+
- Navigated to /cart
|
|
77
|
+
- Changed quantity from 1 to 2
|
|
78
|
+
- Subtotal updated: $49.98
|
|
79
|
+
|
|
80
|
+
[FAIL] Apply discount code (4.5s)
|
|
81
|
+
- Typed "SAVE10" into discount field
|
|
82
|
+
- Clicked "Apply"
|
|
83
|
+
- ASSERTION FAILED: Expected discount line item showing 10% off
|
|
84
|
+
Actual: Error message "Code expired" displayed
|
|
85
|
+
|
|
86
|
+
[PASS] Complete checkout (5.8s)
|
|
87
|
+
- Clicked "Proceed to Checkout"
|
|
88
|
+
- Shipping address form visible
|
|
89
|
+
- Selected "Standard Shipping"
|
|
90
|
+
- Clicked "Place Order"
|
|
91
|
+
- Order confirmed: #ORD-20260203-7842
|
|
92
|
+
|
|
93
|
+
Results: 3 passed, 1 failed (15.6s)
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
## Tips
|
|
97
|
+
|
|
98
|
+
- Sessions expire based on the target app's cookie lifetime. Re-run the save command if tests start failing on auth.
|
|
99
|
+
- Add `--headed` to watch the browser execute each step: `npx cbrowser test-suite checkout-flow.test.nl --headed`
|
|
100
|
+
- Combine with visual regression to catch UI changes during checkout.
|
|
101
|
+
|
|
102
|
+
## Next Steps
|
|
103
|
+
|
|
104
|
+
- Add edge-case tests: empty cart checkout, invalid payment, out-of-stock items.
|
|
105
|
+
- Integrate into CI with `npx cbrowser test-suite checkout-flow.test.nl --ci --reporter json`.
|
|
106
|
+
- Layer on persona testing to verify the flow works for different user types.
|