@testdriverai/agent 7.11.239-test → 7.11.241-canary
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/ai/skills/testdriver-agent/SKILL.md +174 -91
- package/ai/skills/testdriver-assert/SKILL.md +73 -269
- package/ai/skills/testdriver-aws-setup/SKILL.md +194 -312
- package/ai/skills/testdriver-bring-your-own-machine/SKILL.md +137 -204
- package/ai/skills/testdriver-cache/SKILL.md +142 -149
- package/ai/skills/testdriver-caching/SKILL.md +97 -87
- package/ai/skills/testdriver-captcha/SKILL.md +126 -111
- package/ai/skills/testdriver-ci-cd/SKILL.md +153 -542
- package/ai/skills/testdriver-click/SKILL.md +74 -229
- package/ai/skills/testdriver-client/SKILL.md +175 -229
- package/ai/skills/testdriver-customizing-devices/SKILL.md +72 -250
- package/ai/skills/testdriver-dashcam/SKILL.md +71 -404
- package/ai/skills/testdriver-debugging-with-screenshots/SKILL.md +103 -369
- package/ai/skills/testdriver-double-click/SKILL.md +46 -67
- package/ai/skills/testdriver-errors/SKILL.md +102 -176
- package/ai/skills/testdriver-events/SKILL.md +170 -327
- package/ai/skills/testdriver-exec/SKILL.md +92 -256
- package/ai/skills/testdriver-extract/SKILL.md +69 -170
- package/ai/skills/testdriver-find/SKILL.md +140 -736
- package/ai/skills/testdriver-focus-application/SKILL.md +69 -246
- package/ai/skills/testdriver-generating-tests/SKILL.md +102 -136
- package/ai/skills/testdriver-hosted/SKILL.md +79 -143
- package/ai/skills/testdriver-hover/SKILL.md +62 -235
- package/ai/skills/testdriver-locating-elements/SKILL.md +71 -366
- package/ai/skills/testdriver-making-assertions/SKILL.md +114 -72
- package/ai/skills/testdriver-mcp-setup/SKILL.md +79 -62
- package/ai/skills/testdriver-mouse-down/SKILL.md +46 -124
- package/ai/skills/testdriver-mouse-up/SKILL.md +45 -128
- package/ai/skills/testdriver-options/SKILL.md +73 -46
- package/ai/skills/testdriver-parse/SKILL.md +95 -184
- package/ai/skills/testdriver-performing-actions/SKILL.md +101 -91
- package/ai/skills/testdriver-press-keys/SKILL.md +59 -299
- package/ai/skills/testdriver-provision/SKILL.md +135 -179
- package/ai/skills/testdriver-quickstart/SKILL.md +24 -8
- package/ai/skills/testdriver-quickstart-claude/SKILL.md +84 -0
- package/ai/skills/testdriver-quickstart-github/SKILL.md +62 -32
- package/ai/skills/testdriver-quickstart-manual/SKILL.md +59 -32
- package/ai/skills/testdriver-quickstart-vscode/SKILL.md +96 -0
- package/ai/skills/testdriver-redraw/SKILL.md +100 -164
- package/ai/skills/testdriver-reusable-code/SKILL.md +95 -199
- package/ai/skills/testdriver-right-click/SKILL.md +45 -88
- package/ai/skills/testdriver-screenshot/SKILL.md +47 -201
- package/ai/skills/testdriver-screenshots/SKILL.md +86 -136
- package/ai/skills/testdriver-scroll/SKILL.md +71 -154
- package/ai/skills/testdriver-secrets/SKILL.md +94 -90
- package/ai/skills/testdriver-self-hosted/SKILL.md +82 -140
- package/ai/skills/testdriver-test-results-json/SKILL.md +165 -202
- package/ai/skills/testdriver-type/SKILL.md +56 -304
- package/ai/skills/testdriver-variables/SKILL.md +119 -80
- package/ai/skills/testdriver-wait/SKILL.md +31 -29
- package/ai/skills/testdriver-what-is-testdriver/SKILL.md +80 -32
- package/package.json +1 -1
|
@@ -1,237 +1,179 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: testdriver:aws-setup
|
|
3
|
-
description:
|
|
3
|
+
description: Run each test on a fresh Windows EC2 instance in your AWS account, set up with CloudFormation.
|
|
4
4
|
---
|
|
5
5
|
<!-- Generated from aws-setup.mdx. DO NOT EDIT. -->
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
import { EnterpriseBadge, EnterpriseCta } from '/snippets/enterprise.mdx';
|
|
8
8
|
|
|
9
|
-
|
|
10
|
-
graph LR
|
|
11
|
-
A[Vitest Test] --> B[setup-aws hook]
|
|
12
|
-
B --> C[Spawns EC2]
|
|
13
|
-
C --> D[Runs Test]
|
|
14
|
-
D --> E[Terminates EC2]
|
|
15
|
-
```
|
|
16
|
-
|
|
17
|
-
## How It Works
|
|
18
|
-
|
|
19
|
-
TestDriver manages the AWS EC2 instances for your tests automatically:
|
|
20
|
-
|
|
21
|
-
1. **Deploy CloudFormation**. Set up the infrastructure one time.
|
|
22
|
-
2. **Configure Vitest**. Add one line to your config.
|
|
23
|
-
3. **Run Tests**. The instances start automatically, run the tests, and stop.
|
|
24
|
-
|
|
25
|
-
That is all. You do not need to manage the instances by hand.
|
|
26
|
-
|
|
27
|
-
# Quickstart
|
|
28
|
-
|
|
29
|
-
<Steps>
|
|
30
|
-
<Step title="Deploy Infrastructure">
|
|
31
|
-
<Card
|
|
32
|
-
title="Launch CloudFormation Stack"
|
|
33
|
-
icon="aws"
|
|
34
|
-
href="https://console.aws.amazon.com/cloudformation/home#/stacks/create/review?templateURL=https://v7-cloudformation-template.s3.us-east-2.amazonaws.com/cloudformation.yaml"
|
|
35
|
-
horizontal
|
|
36
|
-
arrow
|
|
37
|
-
>
|
|
38
|
-
One-click AWS setup
|
|
39
|
-
</Card>
|
|
40
|
-
</Step>
|
|
41
|
-
|
|
42
|
-
<Step title="Add to Vitest Config">
|
|
43
|
-
```javascript vitest.config.mjs
|
|
44
|
-
setupFiles: ['testdriverai/vitest/setup', 'testdriverai/vitest/setup-aws']
|
|
45
|
-
```
|
|
46
|
-
</Step>
|
|
47
|
-
|
|
48
|
-
<Step title="Run Tests">
|
|
49
|
-
```bash
|
|
50
|
-
TD_OS=windows AWS_REGION=us-east-2 \
|
|
51
|
-
AWS_LAUNCH_TEMPLATE_ID=lt-xxx AMI_ID=ami-xxx \
|
|
52
|
-
vitest run
|
|
53
|
-
```
|
|
54
|
-
</Step>
|
|
55
|
-
</Steps>
|
|
9
|
+
Run your Windows tests on EC2 instances in your own AWS account. For each test, TestDriver:
|
|
56
10
|
|
|
57
|
-
|
|
11
|
+
1. Launches a fresh instance from the TestDriver AMI.
|
|
12
|
+
2. Installs and starts the runner on it over AWS Systems Manager (SSM).
|
|
13
|
+
3. Runs the test, then terminates the instance.
|
|
58
14
|
|
|
59
|
-
|
|
15
|
+
A CloudFormation stack creates the network, security group, IAM role, and launch template the instances need. You deploy it once.
|
|
60
16
|
|
|
61
|
-
|
|
62
|
-
2. **Configure Vitest** — Add `setup-aws` to automatically manage instance lifecycle
|
|
63
|
-
3. **Run Tests** — Set `TD_OS=windows` with AWS credentials and instances spawn/terminate automatically
|
|
17
|
+
<EnterpriseCta feature="Self-hosting on AWS" />
|
|
64
18
|
|
|
19
|
+
<Note>
|
|
20
|
+
Self-hosting is part of the Enterprise plan. See [Self-hosted](/self-hosted).
|
|
21
|
+
</Note>
|
|
65
22
|
|
|
66
|
-
##
|
|
23
|
+
## Before you begin
|
|
67
24
|
|
|
68
|
-
|
|
25
|
+
- **Access to the TestDriver AMI.** TestDriver shares it with your AWS account. [Contact TestDriver](https://testdriver.ai/enterprise) with your account ID and region to get the AMI ID.
|
|
26
|
+
- **The AWS CLI**, [installed](https://aws.amazon.com/cli/) and signed in, on every machine that runs tests (your laptop and your CI runners).
|
|
27
|
+
- **`jq`**, which the instance launch script uses to read AWS responses.
|
|
28
|
+
- **A TestDriver project** with your API key in `TD_API_KEY`. See [Set up a project](/quickstart-manual).
|
|
69
29
|
|
|
70
|
-
|
|
71
|
-
- [AWS CLI](https://aws.amazon.com/cli/) installed and configured (`aws configure`)
|
|
72
|
-
- Access to the TestDriver AMI — [Contact us](https://testdriver.ai/enterprise) with your AWS region
|
|
73
|
-
- A GitHub repository for your tests
|
|
30
|
+
## Step 1: Deploy the CloudFormation stack
|
|
74
31
|
|
|
75
|
-
|
|
76
|
-
The TestDriver Golden Image AMI ID is `ami-0504bf50fad62f312`. Contact us to get access in your preferred AWS region.
|
|
77
|
-
</Tip>
|
|
32
|
+
The template creates:
|
|
78
33
|
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
- Dedicated VPC with public subnet
|
|
84
|
-
- Security group with required port access
|
|
85
|
-
- IAM roles and instance profiles
|
|
86
|
-
- EC2 launch template for instance creation
|
|
34
|
+
- A VPC with a public subnet and internet gateway.
|
|
35
|
+
- A security group that allows RDP, VNC, and web traffic from the IP range you choose.
|
|
36
|
+
- An IAM role and instance profile with SSM access.
|
|
37
|
+
- An EC2 launch template, and optionally a key pair.
|
|
87
38
|
|
|
88
39
|
<Tabs>
|
|
89
|
-
<Tab title="
|
|
90
|
-
Click the button below to launch the CloudFormation stack in your AWS Console:
|
|
91
|
-
|
|
92
|
-
<Card
|
|
93
|
-
title="Launch Stack"
|
|
94
|
-
icon="aws"
|
|
95
|
-
href="https://console.aws.amazon.com/cloudformation/home#/stacks/create/review?templateURL=https://v7-cloudformation-template.s3.us-east-2.amazonaws.com/cloudformation.yaml"
|
|
96
|
-
horizontal
|
|
97
|
-
arrow
|
|
98
|
-
>
|
|
99
|
-
Deploy TestDriver infrastructure with one click
|
|
100
|
-
</Card>
|
|
40
|
+
<Tab title="AWS console">
|
|
101
41
|
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
- **AllowedIngressCidr**: Your IP range (e.g., `203.0.113.0/24`)
|
|
106
|
-
- **InstanceType**: `c5.xlarge` (recommended)
|
|
107
|
-
- **CreateKeyPair**: `true`
|
|
108
|
-
|
|
109
|
-
<Warning>
|
|
110
|
-
**Security**: Replace `AllowedIngressCidr` with your specific IP ranges to restrict VPC access. Avoid using `0.0.0.0/0` in production.
|
|
111
|
-
</Warning>
|
|
112
|
-
|
|
113
|
-
### Get Your Launch Template ID
|
|
42
|
+
<Card title="Launch the stack" icon="aws" href="https://console.aws.amazon.com/cloudformation/home#/stacks/create/review?templateURL=https://v7-cloudformation-template.s3.us-east-2.amazonaws.com/cloudformation.yaml" horizontal arrow>
|
|
43
|
+
Open the template in the CloudFormation console.
|
|
44
|
+
</Card>
|
|
114
45
|
|
|
115
|
-
|
|
46
|
+
Switch the console to the region you'll run tests in, fill in the parameters below, and create the stack.
|
|
116
47
|
|
|
117
|
-
|
|
48
|
+
When the stack finishes, open its **Outputs** tab and copy `LaunchTemplateId`.
|
|
118
49
|
|
|
119
|
-
<Tip>
|
|
120
|
-
**Save this ID** — you'll need it for spawning instances and CI configuration.
|
|
121
|
-
</Tip>
|
|
122
50
|
</Tab>
|
|
123
|
-
<Tab title="CLI">
|
|
124
|
-
|
|
51
|
+
<Tab title="AWS CLI">
|
|
52
|
+
|
|
53
|
+
The template ships with the SDK. From your project, deploy it with:
|
|
125
54
|
|
|
126
55
|
```bash
|
|
127
56
|
aws cloudformation deploy \
|
|
128
|
-
--
|
|
57
|
+
--region us-east-2 \
|
|
58
|
+
--template-file node_modules/testdriverai/setup/aws/cloudformation.yaml \
|
|
129
59
|
--stack-name testdriver-infrastructure \
|
|
130
60
|
--parameter-overrides \
|
|
131
|
-
|
|
132
|
-
AllowedIngressCidr=0.0.0.0/0 \
|
|
61
|
+
AllowedIngressCidr=203.0.113.0/24 \
|
|
133
62
|
InstanceType=c5.xlarge \
|
|
134
|
-
CreateKeyPair=
|
|
63
|
+
CreateKeyPair=yes \
|
|
135
64
|
--capabilities CAPABILITY_IAM
|
|
136
65
|
```
|
|
137
66
|
|
|
138
|
-
|
|
139
|
-
**Security**: Replace `AllowedIngressCidr=0.0.0.0/0` with your specific IP ranges to restrict VPC access.
|
|
140
|
-
</Warning>
|
|
141
|
-
|
|
142
|
-
### Get Your Launch Template ID
|
|
143
|
-
|
|
144
|
-
After deployment completes, retrieve the launch template ID:
|
|
67
|
+
Then get the launch template ID:
|
|
145
68
|
|
|
146
69
|
```bash
|
|
147
70
|
aws cloudformation describe-stacks \
|
|
71
|
+
--region us-east-2 \
|
|
148
72
|
--stack-name testdriver-infrastructure \
|
|
149
73
|
--query 'Stacks[0].Outputs[?OutputKey==`LaunchTemplateId`].OutputValue' \
|
|
150
74
|
--output text
|
|
151
75
|
```
|
|
152
76
|
|
|
153
|
-
<Tip>
|
|
154
|
-
**Save this ID** — you'll need it for spawning instances and CI configuration.
|
|
155
|
-
</Tip>
|
|
156
77
|
</Tab>
|
|
157
78
|
</Tabs>
|
|
158
79
|
|
|
159
|
-
|
|
80
|
+
### Stack parameters
|
|
81
|
+
|
|
82
|
+
| Parameter | Default | Description |
|
|
83
|
+
| --- | --- | --- |
|
|
84
|
+
| `AllowedIngressCidr` | `0.0.0.0/0` | The IP range allowed to reach the instances over RDP and VNC. Set it to your office or VPN range. |
|
|
85
|
+
| `InstanceType` | `c5.xlarge` | The EC2 instance type. Allowed: `c5.xlarge` to `c5.24xlarge`, `c5.metal`, or `i3.metal`. |
|
|
86
|
+
| `CreateKeyPair` | `yes` | `yes` creates a new key pair. `no` uses the key named in `ExistingKeyName`. |
|
|
87
|
+
| `ExistingKeyName` | | An existing EC2 key pair. Required when `CreateKeyPair` is `no`. |
|
|
88
|
+
| `ProjectTag` | `testdriver` | The `Project` tag added to every resource. |
|
|
89
|
+
| `NotificationEmail` | | An email address to notify when the stack finishes. Optional. |
|
|
90
|
+
|
|
91
|
+
<Warning>
|
|
92
|
+
The default `AllowedIngressCidr` of `0.0.0.0/0` opens RDP and VNC to the whole internet. Set it to your own IP range.
|
|
93
|
+
</Warning>
|
|
160
94
|
|
|
161
|
-
Add the AWS setup
|
|
95
|
+
## Step 2: Add the AWS setup file to Vitest
|
|
96
|
+
|
|
97
|
+
Add `testdriverai/vitest/setup-aws` to `setupFiles`. It launches an instance before each test that runs on Windows.
|
|
162
98
|
|
|
163
99
|
```javascript vitest.config.mjs
|
|
164
100
|
import { defineConfig } from 'vitest/config';
|
|
165
|
-
import { config } from 'dotenv';
|
|
166
101
|
import TestDriver from 'testdriverai/vitest';
|
|
167
102
|
|
|
168
|
-
config(); // Load .env file
|
|
169
|
-
|
|
170
103
|
export default defineConfig({
|
|
171
104
|
test: {
|
|
172
105
|
testTimeout: 900000,
|
|
173
106
|
hookTimeout: 900000,
|
|
174
|
-
|
|
175
|
-
reporters: [
|
|
176
|
-
'default',
|
|
177
|
-
TestDriver(),
|
|
178
|
-
['junit', { outputFile: 'test-report.junit.xml' }]
|
|
179
|
-
],
|
|
107
|
+
reporters: ['default', TestDriver()],
|
|
180
108
|
setupFiles: ['testdriverai/vitest/setup', 'testdriverai/vitest/setup-aws'],
|
|
181
109
|
},
|
|
182
110
|
});
|
|
183
111
|
```
|
|
184
112
|
|
|
185
|
-
|
|
186
|
-
**That's it!** The `setup-aws` hook automatically spawns and terminates instances when `TD_OS=windows` is set. No manual instance management needed.
|
|
187
|
-
</Note>
|
|
113
|
+
The 15-minute `hookTimeout` gives a new instance time to boot, pass its status checks, and register with SSM.
|
|
188
114
|
|
|
189
|
-
## Step 3: Write
|
|
115
|
+
## Step 3: Write your test
|
|
190
116
|
|
|
191
|
-
|
|
117
|
+
Your tests don't change. The setup file passes the instance's IP address and ID to `TestDriver()` for you:
|
|
192
118
|
|
|
193
|
-
```javascript
|
|
194
|
-
import { describe, it } from
|
|
195
|
-
import { TestDriver } from
|
|
119
|
+
```javascript tests/login.test.mjs
|
|
120
|
+
import { describe, it } from 'vitest';
|
|
121
|
+
import { TestDriver } from 'testdriverai/vitest/hooks';
|
|
196
122
|
|
|
197
|
-
describe(
|
|
198
|
-
it(
|
|
199
|
-
const testdriver = TestDriver(context
|
|
200
|
-
|
|
201
|
-
});
|
|
123
|
+
describe('Login', () => {
|
|
124
|
+
it('shows the login page', async (context) => {
|
|
125
|
+
const testdriver = TestDriver(context);
|
|
126
|
+
await testdriver.provision.chrome({ url: 'https://testdriver-sandbox.vercel.app/login' });
|
|
202
127
|
|
|
203
|
-
await testdriver.
|
|
204
|
-
// ... your test steps
|
|
128
|
+
await testdriver.assert('the login form is visible');
|
|
205
129
|
});
|
|
206
130
|
});
|
|
207
131
|
```
|
|
208
132
|
|
|
209
|
-
|
|
210
|
-
**How it works**: When `TD_OS=windows` with AWS credentials, `context.ip` is automatically set by the setup hook. When running without AWS setup (cloud-hosted), both are undefined and TestDriver uses the cloud. When `TD_IP` is provided manually, it takes precedence.
|
|
211
|
-
</Note>
|
|
212
|
-
|
|
213
|
-
## Step 4: Run Tests
|
|
133
|
+
## Step 4: Run the tests
|
|
214
134
|
|
|
215
|
-
|
|
135
|
+
Set `TD_OS=windows` to turn on the setup file, along with your launch template and AMI:
|
|
216
136
|
|
|
217
137
|
```bash
|
|
218
138
|
TD_OS=windows \
|
|
219
139
|
AWS_REGION=us-east-2 \
|
|
220
|
-
AWS_LAUNCH_TEMPLATE_ID=lt-
|
|
221
|
-
AMI_ID=ami-
|
|
222
|
-
vitest run
|
|
140
|
+
AWS_LAUNCH_TEMPLATE_ID=lt-0123456789abcdef0 \
|
|
141
|
+
AMI_ID=ami-0123456789abcdef0 \
|
|
142
|
+
npx vitest run
|
|
223
143
|
```
|
|
224
144
|
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
|
|
145
|
+
Each test gets its own instance. The instance is terminated after the test ends, once its recording has stopped. If the test process exits early, TestDriver terminates any instances still running.
|
|
146
|
+
|
|
147
|
+
When `TD_OS` isn't `windows`, the setup file does nothing and tests run on TestDriver's cloud.
|
|
148
|
+
|
|
149
|
+
### Environment variables
|
|
150
|
+
|
|
151
|
+
| Variable | Default | Description |
|
|
152
|
+
| --- | --- | --- |
|
|
153
|
+
| `TD_OS` | | Set to `windows` to launch instances. Required. |
|
|
154
|
+
| `AWS_LAUNCH_TEMPLATE_ID` | | `LaunchTemplateId` from the stack outputs. Required. |
|
|
155
|
+
| `AMI_ID` | | The TestDriver AMI, or your own AMI built from it. Required. |
|
|
156
|
+
| `AWS_REGION` | `us-east-2` | The region of your stack and AMI. |
|
|
157
|
+
| `AWS_LAUNCH_TEMPLATE_VERSION` | `$Latest` | The launch template version to use. |
|
|
158
|
+
| `RESOLUTION` | `1920x1080` | The instance's screen resolution. |
|
|
159
|
+
| `AWS_TAG_PREFIX` | `td` | Prefix for each instance's `Name` tag, followed by a timestamp. |
|
|
160
|
+
|
|
161
|
+
The AWS CLI reads your credentials from its usual sources, such as `aws configure`, `AWS_PROFILE`, or `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`.
|
|
228
162
|
|
|
229
|
-
###
|
|
163
|
+
### AWS permissions
|
|
230
164
|
|
|
231
|
-
|
|
165
|
+
The credentials that run your tests need these actions:
|
|
232
166
|
|
|
233
|
-
|
|
234
|
-
|
|
167
|
+
- `ec2:RunInstances`, `ec2:CreateTags`, `ec2:DescribeInstances`, `ec2:DescribeInstanceStatus`, and `ec2:TerminateInstances`
|
|
168
|
+
- `iam:PassRole` for the stack's instance role
|
|
169
|
+
- `ssm:DescribeInstanceInformation`, `ssm:SendCommand`, and `ssm:GetCommandInvocation`
|
|
170
|
+
|
|
171
|
+
## Run in GitHub Actions
|
|
172
|
+
|
|
173
|
+
This workflow authenticates to TestDriver with the [TestDriver action](/ci-cd#github-actions) and to AWS with stored access keys:
|
|
174
|
+
|
|
175
|
+
```yaml .github/workflows/testdriver-windows.yml
|
|
176
|
+
name: Windows tests
|
|
235
177
|
|
|
236
178
|
on:
|
|
237
179
|
push:
|
|
@@ -241,208 +183,148 @@ on:
|
|
|
241
183
|
jobs:
|
|
242
184
|
test:
|
|
243
185
|
runs-on: ubuntu-latest
|
|
244
|
-
|
|
186
|
+
permissions:
|
|
187
|
+
id-token: write
|
|
188
|
+
contents: read
|
|
189
|
+
|
|
245
190
|
steps:
|
|
246
|
-
-
|
|
247
|
-
uses: actions/checkout@v4
|
|
191
|
+
- uses: actions/checkout@v4
|
|
248
192
|
|
|
249
|
-
-
|
|
250
|
-
uses: actions/setup-node@v4
|
|
193
|
+
- uses: actions/setup-node@v4
|
|
251
194
|
with:
|
|
252
195
|
node-version: '20'
|
|
253
196
|
cache: 'npm'
|
|
254
197
|
|
|
255
|
-
-
|
|
256
|
-
|
|
198
|
+
- run: npm ci
|
|
199
|
+
|
|
200
|
+
- uses: testdriverai/action@stable
|
|
257
201
|
|
|
258
|
-
-
|
|
259
|
-
|
|
202
|
+
- uses: aws-actions/configure-aws-credentials@v4
|
|
203
|
+
with:
|
|
204
|
+
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
|
205
|
+
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
|
206
|
+
aws-region: us-east-2
|
|
207
|
+
|
|
208
|
+
- name: Run tests on Windows instances
|
|
209
|
+
run: npx vitest run
|
|
260
210
|
env:
|
|
261
|
-
TD_API_KEY: ${{ secrets.TD_API_KEY }}
|
|
262
211
|
TD_OS: windows
|
|
263
|
-
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
|
264
|
-
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
|
265
|
-
AWS_REGION: us-east-2
|
|
266
212
|
AWS_LAUNCH_TEMPLATE_ID: ${{ secrets.AWS_LAUNCH_TEMPLATE_ID }}
|
|
267
213
|
AMI_ID: ${{ secrets.AMI_ID }}
|
|
268
|
-
|
|
269
|
-
- name: Upload test results
|
|
270
|
-
if: always()
|
|
271
|
-
uses: actions/upload-artifact@v4
|
|
272
|
-
with:
|
|
273
|
-
name: test-results
|
|
274
|
-
path: test-report.junit.xml
|
|
275
214
|
```
|
|
276
215
|
|
|
277
|
-
|
|
278
|
-
**Automatic Instance Management**: Setting `TD_OS=windows` with AWS credentials enables automatic instance spawning. Each test gets its own fresh instance that's terminated after the test completes.
|
|
279
|
-
</Note>
|
|
280
|
-
|
|
281
|
-
### Required GitHub Secrets
|
|
282
|
-
|
|
283
|
-
| Secret | Description | Example |
|
|
284
|
-
|--------|-------------|---------|
|
|
285
|
-
| `AWS_ACCESS_KEY_ID` | AWS access key | `AKIAIOSFODNN7EXAMPLE` |
|
|
286
|
-
| `AWS_SECRET_ACCESS_KEY` | AWS secret key | `wJalrXUtnFEMI/K7MDENG...` |
|
|
287
|
-
| `AWS_REGION` | AWS region | `us-east-2` |
|
|
288
|
-
| `AWS_LAUNCH_TEMPLATE_ID` | From CloudFormation output | `lt-07c53ce8349b958d1` |
|
|
289
|
-
| `AMI_ID` | TestDriver AMI ID | `ami-0504bf50fad62f312` |
|
|
290
|
-
| `TD_API_KEY` | Your TestDriver API key | From [console.testdriver.ai](https://console.testdriver.ai) |
|
|
216
|
+
GitHub's `ubuntu-latest` runners already include the AWS CLI and `jq`.
|
|
291
217
|
|
|
292
218
|
<Tip>
|
|
293
|
-
|
|
219
|
+
To avoid storing AWS keys, give `configure-aws-credentials` a `role-to-assume` instead. The job's `id-token: write` permission covers both TestDriver and AWS. See [GitHub's OIDC guide](https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/about-security-hardening-with-openid-connect).
|
|
294
220
|
</Tip>
|
|
295
221
|
|
|
296
|
-
|
|
297
|
-
|
|
298
|
-
For complete production examples, see:
|
|
299
|
-
- **[Scheduled Tests](https://github.com/testdriverai/testdriverai/blob/main/.github/workflows/acceptance-windows-scheduled.yaml)** — Daily automated test runs
|
|
300
|
-
- **[Reusable Workflow](https://github.com/testdriverai/testdriverai/blob/main/.github/workflows/windows-self-hosted.yaml)** — Shared configuration for PR and scheduled tests
|
|
301
|
-
|
|
302
|
-
## Advanced Usage
|
|
303
|
-
|
|
304
|
-
### Using an Existing Instance
|
|
305
|
-
|
|
306
|
-
If you already have a running instance, you can skip automatic spawning by providing `TD_IP`:
|
|
307
|
-
|
|
308
|
-
```bash
|
|
309
|
-
TD_OS=windows TD_IP=1.2.3.4 vitest run
|
|
310
|
-
```
|
|
311
|
-
|
|
312
|
-
The `setup-aws` hook will detect `TD_IP` is already set and skip spawning a new instance.
|
|
313
|
-
|
|
314
|
-
### Manual Instance Management
|
|
315
|
-
|
|
316
|
-
For advanced use cases, you can manually spawn instances using the [`spawn-runner.sh`](https://github.com/testdriverai/testdriverai/blob/main/setup/aws/spawn-runner.sh) script:
|
|
222
|
+
## Connect to an instance
|
|
317
223
|
|
|
318
|
-
|
|
319
|
-
AWS_REGION=us-east-2 \
|
|
320
|
-
AMI_ID=ami-0504bf50fad62f312 \
|
|
321
|
-
AWS_LAUNCH_TEMPLATE_ID=lt-xxx \
|
|
322
|
-
bash setup/aws/spawn-runner.sh
|
|
323
|
-
```
|
|
224
|
+
To watch or debug a test, connect to its instance over RDP on port 3389 while it runs. The setup file prints the instance's IP address when it launches it.
|
|
324
225
|
|
|
325
|
-
|
|
326
|
-
```
|
|
327
|
-
PUBLIC_IP=1.2.3.4
|
|
328
|
-
INSTANCE_ID=i-1234567890abcdef0
|
|
329
|
-
AWS_REGION=us-east-2
|
|
330
|
-
```
|
|
331
|
-
|
|
332
|
-
Then manually terminate when done:
|
|
333
|
-
|
|
334
|
-
```bash
|
|
335
|
-
aws ec2 terminate-instances \
|
|
336
|
-
--instance-ids i-1234567890abcdef0 \
|
|
337
|
-
--region us-east-2
|
|
338
|
-
```
|
|
226
|
+
The TestDriver AMI signs in as the user `testdriver`. Ask TestDriver for the default password, and change it before you build your own image.
|
|
339
227
|
|
|
340
|
-
|
|
341
|
-
- **[Scheduled Tests](https://github.com/testdriverai/testdriverai/blob/main/.github/workflows/acceptance-windows-scheduled.yaml)** — Daily automated test runs
|
|
342
|
-
- **[Reusable Workflow](https://github.com/testdriverai/testdriverai/blob/main/.github/workflows/windows-self-hosted.yaml)** — Shared configuration for PR and scheduled tests
|
|
228
|
+
## Create a custom AMI
|
|
343
229
|
|
|
344
|
-
|
|
230
|
+
Preinstall your apps and test data so tests don't spend time on setup.
|
|
345
231
|
|
|
346
|
-
|
|
347
|
-
|
|
348
|
-
- **VNC** — Access via web browser at `http://<public-ip>:5900`
|
|
349
|
-
- **AWS Console** — Use EC2 Instance Connect or Session Manager
|
|
350
|
-
|
|
351
|
-
<Note>
|
|
352
|
-
Stopped instances retain their EBS volumes and can be restarted later. Terminated instances are permanently deleted. Always terminate instances when done to avoid storage costs.
|
|
353
|
-
</Note>
|
|
232
|
+
<Steps>
|
|
233
|
+
<Step title="Launch an instance and connect">
|
|
354
234
|
|
|
355
|
-
|
|
235
|
+
Launch an instance from the TestDriver AMI with the stack's launch template:
|
|
356
236
|
|
|
357
|
-
|
|
237
|
+
```bash
|
|
238
|
+
aws ec2 run-instances \
|
|
239
|
+
--region us-east-2 \
|
|
240
|
+
--image-id ami-0123456789abcdef0 \
|
|
241
|
+
--launch-template 'LaunchTemplateId=lt-0123456789abcdef0,Version=$Latest'
|
|
242
|
+
```
|
|
358
243
|
|
|
359
|
-
|
|
360
|
-
- VNC + web server for remote access
|
|
361
|
-
- Python, Node.js, Git
|
|
362
|
-
- TestDriver agent and dependencies
|
|
244
|
+
When it's running, connect to its public IP address over RDP.
|
|
363
245
|
|
|
364
|
-
|
|
246
|
+
</Step>
|
|
247
|
+
<Step title="Rotate the default password">
|
|
365
248
|
|
|
366
|
-
|
|
249
|
+
Run the password rotation script and save the new password:
|
|
367
250
|
|
|
368
|
-
<Steps>
|
|
369
|
-
<Step title="Connect via RDP">
|
|
370
|
-
Use the default credentials:
|
|
371
|
-
- **Username**: `testdriver`
|
|
372
|
-
- **Password**: `wwv9uJ0sqlulbN3`
|
|
373
|
-
</Step>
|
|
374
|
-
|
|
375
|
-
<Step title="Change the Password">
|
|
376
|
-
**Critical**: Run the password rotation script immediately:
|
|
377
251
|
```powershell
|
|
378
252
|
C:\testdriver\RotateLocalPasswords.ps1
|
|
379
253
|
```
|
|
380
|
-
|
|
381
|
-
</Step>
|
|
382
|
-
|
|
383
|
-
<Step title="Install Your Software">
|
|
384
|
-
Install any additional dependencies, configure settings, or modify the environment as needed.
|
|
254
|
+
|
|
385
255
|
</Step>
|
|
386
|
-
|
|
387
|
-
|
|
388
|
-
|
|
256
|
+
<Step title="Install your software">
|
|
257
|
+
|
|
258
|
+
Install your apps, dependencies, certificates, and data.
|
|
259
|
+
|
|
389
260
|
</Step>
|
|
390
|
-
|
|
261
|
+
<Step title="Create the AMI">
|
|
391
262
|
|
|
392
|
-
|
|
393
|
-
**Security**: Never use the default password in production. Always rotate passwords before creating custom AMIs.
|
|
394
|
-
</Warning>
|
|
263
|
+
Create an image from the instance:
|
|
395
264
|
|
|
396
|
-
|
|
265
|
+
```bash
|
|
266
|
+
aws ec2 create-image \
|
|
267
|
+
--region us-east-2 \
|
|
268
|
+
--instance-id i-0123456789abcdef0 \
|
|
269
|
+
--name my-testdriver-image
|
|
270
|
+
```
|
|
397
271
|
|
|
398
|
-
|
|
272
|
+
Set `AMI_ID` to the new image's ID when you run tests.
|
|
399
273
|
|
|
400
|
-
|
|
401
|
-
|
|
402
|
-
- **Enable VPC Flow Logs** — For network monitoring
|
|
274
|
+
</Step>
|
|
275
|
+
</Steps>
|
|
403
276
|
|
|
404
|
-
|
|
277
|
+
## Troubleshooting
|
|
405
278
|
|
|
406
|
-
|
|
279
|
+
<AccordionGroup>
|
|
280
|
+
<Accordion title="TD_OS=windows requires AWS_LAUNCH_TEMPLATE_ID and AMI_ID">
|
|
407
281
|
|
|
408
|
-
|
|
409
|
-
permissions:
|
|
410
|
-
id-token: write
|
|
411
|
-
contents: read
|
|
282
|
+
Set both variables in the environment that runs `npx vitest run`. In CI, check that the secrets are passed to the test step.
|
|
412
283
|
|
|
413
|
-
|
|
414
|
-
|
|
415
|
-
uses: aws-actions/configure-aws-credentials@v4
|
|
416
|
-
with:
|
|
417
|
-
role-to-assume: arn:aws:iam::123456789012:role/GitHubActionsRole
|
|
418
|
-
aws-region: us-east-2
|
|
419
|
-
```
|
|
284
|
+
</Accordion>
|
|
285
|
+
<Accordion title="AWS CLI is not installed">
|
|
420
286
|
|
|
421
|
-
|
|
287
|
+
Install the [AWS CLI](https://aws.amazon.com/cli/) on the machine that runs the tests, and check that `aws` is on the `PATH`.
|
|
422
288
|
|
|
423
|
-
|
|
289
|
+
</Accordion>
|
|
290
|
+
<Accordion title="SSM registration timeout">
|
|
424
291
|
|
|
425
|
-
|
|
426
|
-
- **Monitor costs** with AWS billing alerts
|
|
427
|
-
- **Use least-privilege IAM roles**
|
|
428
|
-
- **Enable CloudTrail** for audit logging
|
|
292
|
+
The instance didn't register with Systems Manager. Check that:
|
|
429
293
|
|
|
430
|
-
|
|
294
|
+
- The launch template from the stack is used, so the instance gets the stack's IAM role.
|
|
295
|
+
- The instance is in the stack's public subnet, so it can reach SSM.
|
|
296
|
+
- The AMI is the TestDriver AMI, or an image built from it.
|
|
431
297
|
|
|
432
|
-
|
|
298
|
+
</Accordion>
|
|
299
|
+
<Accordion title="A test fails before it starts with a hook timeout">
|
|
433
300
|
|
|
434
|
-
|
|
435
|
-
- **Verify status checks** — Ensure instance has passed AWS status checks
|
|
436
|
-
- **Check AMI compatibility** — Some instance types don't support certain AMIs
|
|
301
|
+
Launching and preparing an instance takes several minutes. Raise `hookTimeout` in `vitest.config.mjs`.
|
|
437
302
|
|
|
438
|
-
|
|
303
|
+
</Accordion>
|
|
304
|
+
<Accordion title="Instances are left running">
|
|
439
305
|
|
|
440
|
-
|
|
441
|
-
- Check VPC routing and internet gateway configuration
|
|
442
|
-
- Confirm instance is in the correct subnet
|
|
306
|
+
TestDriver terminates instances after each test and when the process exits. A killed CI job can still leave one behind. Find instances by their `Name` tag, which starts with `td-`, and terminate them:
|
|
443
307
|
|
|
444
|
-
|
|
308
|
+
```bash
|
|
309
|
+
aws ec2 terminate-instances --region us-east-2 --instance-ids i-0123456789abcdef0
|
|
310
|
+
```
|
|
445
311
|
|
|
446
|
-
|
|
447
|
-
|
|
448
|
-
|
|
312
|
+
</Accordion>
|
|
313
|
+
</AccordionGroup>
|
|
314
|
+
|
|
315
|
+
## Next steps
|
|
316
|
+
|
|
317
|
+
<CardGroup cols={2}>
|
|
318
|
+
<Card title="Run on your own machine" icon="display" href="/bring-your-own-machine">
|
|
319
|
+
Run the runner on a laptop, lab machine, or on-prem VM.
|
|
320
|
+
</Card>
|
|
321
|
+
<Card title="Run tests in CI/CD" icon="code-branch" href="/ci-cd">
|
|
322
|
+
Authenticate and run tests in any CI provider.
|
|
323
|
+
</Card>
|
|
324
|
+
<Card title="Self-hosted" icon="server" href="/self-hosted">
|
|
325
|
+
What the Enterprise plan includes.
|
|
326
|
+
</Card>
|
|
327
|
+
<Card title="Launch apps" icon="rocket" href="/provision">
|
|
328
|
+
Start Chrome, desktop apps, and installers in the sandbox.
|
|
329
|
+
</Card>
|
|
330
|
+
</CardGroup>
|