testdriverai 6.1.10 → 6.1.11-canary.5037571.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
|
@@ -54,6 +54,7 @@ jobs:
|
|
|
54
54
|
- name: Setup AWS Instance
|
|
55
55
|
id: aws-setup
|
|
56
56
|
run: |
|
|
57
|
+
chmod +x ./setup/aws/spawn-runner.sh
|
|
57
58
|
OUTPUT=$(./setup/aws/spawn-runner.sh | tee /dev/stderr) # Capture and display output
|
|
58
59
|
echo "$OUTPUT"
|
|
59
60
|
PUBLIC_IP=$(echo "$OUTPUT" | grep "PUBLIC_IP=" | cut -d'=' -f2)
|
|
@@ -68,9 +69,8 @@ jobs:
|
|
|
68
69
|
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
|
69
70
|
AWS_REGION: us-east-2
|
|
70
71
|
AWS_LAUNCH_TEMPLATE_ID: lt-00d02f31cfc602f27
|
|
71
|
-
AMI_ID: ami-
|
|
72
|
-
|
|
73
|
-
RESOLUTION_HEIGHT: 1080
|
|
72
|
+
AMI_ID: ami-055cd47506a2f39bb
|
|
73
|
+
RESOLUTION: 1920x1080
|
|
74
74
|
- name: Run TestDriver
|
|
75
75
|
run: node bin/testdriverai.js run testdriver/acceptance/${{ matrix.test }} --ip="${{ steps.aws-setup.outputs.public-ip }}" --junit=out.xml
|
|
76
76
|
env:
|
package/agent/index.js
CHANGED
|
@@ -43,9 +43,10 @@ The setup process involves three main steps:
|
|
|
43
43
|
- AWS account with permissions to run CloudFormation.
|
|
44
44
|
- [AWS CLI](https://aws.amazon.com/cli/) installed locally.
|
|
45
45
|
|
|
46
|
-
<Tip>
|
|
47
|
-
|
|
48
|
-
|
|
46
|
+
<Tip>
|
|
47
|
+
Be sure to run `aws configure` with your credentials
|
|
48
|
+
</Tip>
|
|
49
|
+
- Access to the TestDriver AMI (Golden Image is `ami-055cd47506a2f39bb`)\
|
|
49
50
|
[Contact us with your preferred AWS Region for access](https://form.typeform.com/to/UECf9rDx?typeform-source=testdriver.ai).
|
|
50
51
|
- A GitHub repository for committing your tests & workflow.
|
|
51
52
|
|
|
@@ -76,8 +77,7 @@ aws cloudformation deploy \
|
|
|
76
77
|
```
|
|
77
78
|
|
|
78
79
|
<Danger>
|
|
79
|
-
**Security**: Replace `AllowedIngressCidr=0.0.0.0/0` with your specific IP
|
|
80
|
-
ranges to lock down access to your VPC.
|
|
80
|
+
**Security**: Replace `AllowedIngressCidr=0.0.0.0/0` with your specific IP ranges to lock down access to your VPC.
|
|
81
81
|
</Danger>
|
|
82
82
|
|
|
83
83
|
### Get Launch Template ID
|
|
@@ -91,7 +91,9 @@ aws cloudformation describe-stacks \
|
|
|
91
91
|
--output text
|
|
92
92
|
```
|
|
93
93
|
|
|
94
|
-
<Tip
|
|
94
|
+
<Tip>
|
|
95
|
+
**Save this ID** – you'll need it for the next step.
|
|
96
|
+
</Tip>
|
|
95
97
|
|
|
96
98
|
## Step 2: Spawn a New TestDriver Runner
|
|
97
99
|
|
|
@@ -127,15 +129,12 @@ AWS_REGION=us-east-2
|
|
|
127
129
|
```
|
|
128
130
|
|
|
129
131
|
<Note>
|
|
130
|
-
**Instance Lifecycle**: Instances spawned by this script will continue running
|
|
131
|
-
until you manually terminate them. They are automatically tagged with
|
|
132
|
-
`Name=TestDriverRunner` and `Project=[your ProjectTag value]` for easy
|
|
133
|
-
identification in the AWS console.
|
|
132
|
+
**Instance Lifecycle**: Instances spawned by this script will continue running until you manually terminate them. They are automatically tagged with `Name=TestDriverRunner` and `Project=[your ProjectTag value]` for easy identification in the AWS console.
|
|
134
133
|
</Note>
|
|
135
134
|
|
|
136
135
|
#### Changing Resolution in Lifecycle Files
|
|
137
136
|
|
|
138
|
-
You can also change the resolution before running tests by adding an `exec` command in your `lifecycle/provision.yaml` file:
|
|
137
|
+
You can also change the resolution before running tests by adding an [`exec`](/commands/exec) command in your `lifecycle/provision.yaml` file:
|
|
139
138
|
|
|
140
139
|
```yaml lifecycle/provision.yaml
|
|
141
140
|
version: 6.0.0
|
|
@@ -273,12 +272,50 @@ Our TestDriver Golden Image (AMI) comes pre-configured with everything you need
|
|
|
273
272
|
|
|
274
273
|
### Modifying the AMI
|
|
275
274
|
|
|
276
|
-
|
|
275
|
+
<Danger>
|
|
276
|
+
**Change Default Password First**: The TestDriver AMI includes a default password for initial access. You MUST change this password before saving a new AMI. Never use the default password in production environments.
|
|
277
|
+
</Danger>
|
|
278
|
+
|
|
279
|
+
You can customize the AMI for your specific needs. Follow these steps carefully:
|
|
280
|
+
|
|
281
|
+
#### Step 1: Initial Access
|
|
282
|
+
|
|
283
|
+
Connect to your instance via RDP using the default credentials:
|
|
284
|
+
|
|
285
|
+
- **Username**: `testdriver`
|
|
286
|
+
- **Default Password**: `changemeABC123`
|
|
287
|
+
|
|
288
|
+
#### Step 2: Change the Password
|
|
289
|
+
|
|
290
|
+
Before doing anything else, change the default password:
|
|
291
|
+
|
|
292
|
+
1. Open PowerShell on the instance
|
|
293
|
+
2. Run the password rotation script:
|
|
294
|
+
```powershell
|
|
295
|
+
C:\testdriver\RotateLocalPasswords.ps1
|
|
296
|
+
```
|
|
297
|
+
3. **Save the new password securely** - you'll need it for all future access to this instance
|
|
298
|
+
|
|
299
|
+
<Note>
|
|
300
|
+
The password rotation script will generate a secure password for the `testdriver` account. Make sure to save this password in a secure location (password manager, secrets vault, etc.) before proceeding.
|
|
301
|
+
</Note>
|
|
302
|
+
|
|
303
|
+
#### Step 3: Make Your Customizations
|
|
304
|
+
|
|
305
|
+
Now that you've secured the instance with a new password:
|
|
306
|
+
|
|
307
|
+
1. **Make your changes** using the `testdriver` account with your new password
|
|
308
|
+
2. Install additional software as needed
|
|
309
|
+
3. Configure settings specific to your environment
|
|
310
|
+
|
|
311
|
+
<Tip>
|
|
312
|
+
**Need to make further customizations or debug issues?** RDP back into the instance using the `testdriver` account with the new password you set in Step 2. This gives you full desktop access to install software, modify configurations, or troubleshoot problems before saving your custom AMI.
|
|
313
|
+
</Tip>
|
|
314
|
+
|
|
315
|
+
#### Step 4: Save Your Custom AMI
|
|
277
316
|
|
|
278
|
-
1. **
|
|
279
|
-
2. **
|
|
280
|
-
3. **Create a new AMI** from your modified instance
|
|
281
|
-
4. **Update your workflow** to use the new AMI ID
|
|
317
|
+
1. **Create a new AMI** from your modified instance using the AWS console or CLI
|
|
318
|
+
2. **Update your workflow** to use the new AMI ID
|
|
282
319
|
|
|
283
320
|
### Amazon Image Builder
|
|
284
321
|
|
|
@@ -11,11 +11,10 @@ import GitignoreWarning from "/snippets/gitignore-warning.mdx";
|
|
|
11
11
|
The supported environment variables in TestDriver are:
|
|
12
12
|
|
|
13
13
|
<div className="env-vars-table">
|
|
14
|
-
| Variable
|
|
15
|
-
|
|
16
|
-
| TD_ANALYTICS
|
|
17
|
-
|
|
18
|
-
spawn VMs with TestDriver Pro. |
|
|
14
|
+
| Variable | Type | Description |
|
|
15
|
+
|:---------------:|:------------:|:-------------------------------------------------------------------------------:|
|
|
16
|
+
| `TD_ANALYTICS` | `boolean` | Send analytics to TestDriver servers. This helps provide feedback to inform our roadmap. |
|
|
17
|
+
| `TD_API_KEY` | `string` | Set this to spawn VMs with TestDriver Pro. |
|
|
19
18
|
</div>
|
|
20
19
|
<GitignoreWarning />
|
|
21
20
|
## Example
|
|
@@ -11,9 +11,12 @@ TestDriver operates a full desktop environment, so it can run any application.
|
|
|
11
11
|
|
|
12
12
|
<div className="comparison-table">
|
|
13
13
|
| Application | TestDriver | Playwright | Selenium |
|
|
14
|
-
|:-----------------:|:---------:|:-----------:|:--------:|
|
|
15
|
-
|
|
16
|
-
|
|
14
|
+
|:-----------------:|:---------:|:-----------:|:--------:|
|
|
15
|
+
| Web Apps | ✅ | ✅ | ✅ |
|
|
16
|
+
| Mobile Apps | ✅ | ✅ | ✅ |
|
|
17
|
+
| VS Code Plugins | ✅ | | |
|
|
18
|
+
| Desktop Apps | ✅ | | |
|
|
19
|
+
| Chrome Extensions | ✅ | | |
|
|
17
20
|
</div>
|
|
18
21
|
|
|
19
22
|
## Testing features
|
|
@@ -21,11 +24,16 @@ TestDriver operates a full desktop environment, so it can run any application.
|
|
|
21
24
|
TestDriver is AI first.
|
|
22
25
|
|
|
23
26
|
<div className="comparison-table">
|
|
24
|
-
| Feature
|
|
25
|
-
|
|
26
|
-
|
|
|
27
|
-
|
|
28
|
-
|
|
27
|
+
| Feature | TestDriver | Playwright | Selenium |
|
|
28
|
+
|:--------------------:|:----------:|:----------:|:--------:|
|
|
29
|
+
| Test Generation | ✅ | | |
|
|
30
|
+
| Adaptive Testing | ✅ | | |
|
|
31
|
+
| Visual Assertions | ✅ | | |
|
|
32
|
+
| Self Healing | ✅ | | |
|
|
33
|
+
| Application Switching| ✅ | | |
|
|
34
|
+
| GitHub Actions | ✅ | ✅ | |
|
|
35
|
+
| Team Dashboard | ✅ | | |
|
|
36
|
+
| Team Collaboration | ✅ | | |
|
|
29
37
|
</div>
|
|
30
38
|
|
|
31
39
|
## Test coverage
|
|
@@ -36,16 +44,16 @@ TestDriver has more coverage than selector-based frameworks.
|
|
|
36
44
|
| Feature | TestDriver | Playwright | Selenium |
|
|
37
45
|
|:------------------:|:----------:|:----------:|:--------:|
|
|
38
46
|
| Browser Viewport | ✅ | ✅ | ✅ |
|
|
39
|
-
| Browser App | ✅ | |
|
|
40
|
-
| Operating System | ✅ | |
|
|
41
|
-
| PDFs | ✅ | |
|
|
42
|
-
| File System | ✅ | |
|
|
43
|
-
| Push Notifications | ✅ | |
|
|
44
|
-
| Image Content | ✅ | |
|
|
45
|
-
| Video Content | ✅ | |
|
|
46
|
-
| `<iframe>` | ✅ | |
|
|
47
|
-
| `<canvas>` | ✅ | |
|
|
48
|
-
| `<video>` | ✅ | |
|
|
47
|
+
| Browser App | ✅ | | |
|
|
48
|
+
| Operating System | ✅ | | |
|
|
49
|
+
| PDFs | ✅ | | |
|
|
50
|
+
| File System | ✅ | | |
|
|
51
|
+
| Push Notifications | ✅ | | |
|
|
52
|
+
| Image Content | ✅ | | |
|
|
53
|
+
| Video Content | ✅ | | |
|
|
54
|
+
| `<iframe>` | ✅ | | |
|
|
55
|
+
| `<canvas>` | ✅ | | |
|
|
56
|
+
| `<video>` | ✅ | | |
|
|
49
57
|
</div>
|
|
50
58
|
|
|
51
59
|
## Debugging features
|
|
@@ -53,11 +61,15 @@ TestDriver has more coverage than selector-based frameworks.
|
|
|
53
61
|
Debugging features are powered by [Dashcam.io](https://dashcam.io).
|
|
54
62
|
|
|
55
63
|
<div className="comparison-table">
|
|
56
|
-
| Feature
|
|
57
|
-
|
|
58
|
-
|
|
|
59
|
-
|
|
|
60
|
-
|
|
64
|
+
| Feature | TestDriver | Playwright | Selenium |
|
|
65
|
+
|:--------------------:|:----------:|:----------:|:--------:|
|
|
66
|
+
| AI Summary | ✅ | | |
|
|
67
|
+
| Video Replay | ✅ | ✅ | |
|
|
68
|
+
| Browser Logs | ✅ | ✅ | |
|
|
69
|
+
| Desktop Logs | ✅ | | |
|
|
70
|
+
| Network Requests | ✅ | ✅ | |
|
|
71
|
+
| Team Dashboard | ✅ | | |
|
|
72
|
+
| Team Collaboration | ✅ | | |
|
|
61
73
|
</div>
|
|
62
74
|
|
|
63
75
|
## Web browser support
|
|
@@ -65,18 +77,25 @@ Debugging features are powered by [Dashcam.io](https://dashcam.io).
|
|
|
65
77
|
TestDriver is browser agnostic and supports any version of any browser.
|
|
66
78
|
|
|
67
79
|
<div className="comparison-table">
|
|
68
|
-
| Feature
|
|
69
|
-
|:--------:|:----------:|:----------:|:--------:|
|
|
70
|
-
|
|
71
|
-
|
|
80
|
+
| Feature | TestDriver | Playwright | Selenium |
|
|
81
|
+
|:--------:|:----------:|:----------:|:--------:|
|
|
82
|
+
| Chrome | ✅ | ✅ | ✅ |
|
|
83
|
+
| Firefox | ✅ | ✅ | ✅ |
|
|
84
|
+
| Webkit | ✅ | ✅ | ✅ |
|
|
85
|
+
| IE | ✅ | | ✅ |
|
|
86
|
+
| Edge | ✅ | ✅ | ✅ |
|
|
87
|
+
| Opera | ✅ | | ✅ |
|
|
88
|
+
| Safari | ✅ | | ✅ |
|
|
72
89
|
</div>
|
|
73
90
|
|
|
74
91
|
## Operating system support
|
|
75
92
|
|
|
76
|
-
TestDriver currently supports
|
|
93
|
+
TestDriver currently supports all the three major operating systems!
|
|
77
94
|
|
|
78
95
|
<div className="comparison-table">
|
|
79
|
-
| Feature
|
|
80
|
-
|:--------:|:----------:|:----------:|:--------:|
|
|
81
|
-
|
|
96
|
+
| Feature | TestDriver | Playwright | Selenium |
|
|
97
|
+
|:--------:|:----------:|:----------:|:--------:|
|
|
98
|
+
| Windows | ✅ | ✅ | ✅ |
|
|
99
|
+
| Mac | ✅ | ✅ | ✅ |
|
|
100
|
+
| Linux | ✅ | ✅ | ✅ |
|
|
82
101
|
</div>
|
package/package.json
CHANGED
|
@@ -168,8 +168,15 @@ Resources:
|
|
|
168
168
|
IpProtocol: tcp,
|
|
169
169
|
FromPort: 8765,
|
|
170
170
|
ToPort: 8765,
|
|
171
|
-
CidrIp:
|
|
172
|
-
Description: "pyautogui-cli WebSockets",
|
|
171
|
+
CidrIp: 35.171.123.200/32,
|
|
172
|
+
Description: "pyautogui-cli WebSockets - Static IP 1",
|
|
173
|
+
}
|
|
174
|
+
- {
|
|
175
|
+
IpProtocol: tcp,
|
|
176
|
+
FromPort: 8765,
|
|
177
|
+
ToPort: 8765,
|
|
178
|
+
CidrIp: 52.201.199.222/32,
|
|
179
|
+
Description: "pyautogui-cli WebSockets - Static IP 2",
|
|
173
180
|
}
|
|
174
181
|
- {
|
|
175
182
|
IpProtocol: tcp,
|