testdriverai 6.1.7 → 6.1.8-canary.68bcfa2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -7,6 +7,8 @@ on:
7
7
  # So that we don't do expensive tests until approved
8
8
  push:
9
9
  branches: [main]
10
+ paths-ignore:
11
+ - "docs/**"
10
12
  # So that we can manually trigger tests when there's flake
11
13
  workflow_dispatch:
12
14
 
@@ -5,6 +5,8 @@ on:
5
5
  push:
6
6
  branches:
7
7
  - main
8
+ paths-ignore:
9
+ - "docs/**"
8
10
  pull_request:
9
11
  branches:
10
12
  - main
@@ -1,7 +1,10 @@
1
1
  # Ensure affected code follows standards and is formatted correctly. Otherwise, automatic formatting in future changes will cause larger diffs.
2
2
  name: Lint + Prettier
3
3
 
4
- on: push
4
+ on:
5
+ push:
6
+ paths-ignore:
7
+ - "docs/**"
5
8
 
6
9
  jobs:
7
10
  lint:
@@ -10,6 +10,8 @@ on:
10
10
  # So that we publish for every push to `main`, despite tests
11
11
  push:
12
12
  branches: [main]
13
+ paths-ignore:
14
+ - "docs/**"
13
15
  workflow_dispatch:
14
16
 
15
17
  jobs:
@@ -3,6 +3,8 @@ name: AWS
3
3
  on:
4
4
  workflow_dispatch:
5
5
  push:
6
+ paths-ignore:
7
+ - "docs/**"
6
8
 
7
9
  jobs:
8
10
  gather:
@@ -24,15 +24,15 @@ We specialize in testing scenarios that other tools can't handle - desktop appli
24
24
 
25
25
  ## Enterprise Plans
26
26
 
27
- TestDriver Enterprise plans start at **$995/month** and include:
27
+ TestDriver Enterprise plans start at **$2,000/month** and include:
28
28
 
29
+ - A pilot program with our expert team to create 4 custom tests within your first 4 weeks (4x4 Guarantee).
30
+ - 4 parallel tests
29
31
  - **12,500 runner minutes per month**: Sufficient capacity for continuous testing of your custom test suite.
30
- - Enterprise-grade test dashboards with advanced analytics.
31
32
  - Full CI/CD pipeline integration with custom configurations.
32
33
  - Dedicated infrastructure and ongoing support for complex testing scenarios.
33
- - Expert test creation and maintenance services.
34
34
 
35
- For detailed pricing and contract information, see [Contract Details](#contract-details). For other plans, visit our [Pricing](/account/pricing) page.
35
+ For detailed pricing and contract information our [Pricing](/account/pricing) page. Want unlimited minutes or enhanced security? We also support self-hosted options with 16 parallel tests starting at $2,000/month. See our [Self Hosting](/getting-started/self-hosting) docs for more info.
36
36
 
37
37
  <CardGroup cols={3}>
38
38
  <Card title="Custom Desktop & Extension Testing">
@@ -66,17 +66,14 @@ TestDriver Enterprise provides comprehensive support for fast-moving teams with
66
66
 
67
67
  Testing complex applications requires more than standard automation tools. Desktop applications, browser extensions, and multi-platform workflows demand specialized infrastructure, custom integrations, and deep technical expertise. TestDriver Enterprise provides the complete solution - from initial setup through ongoing maintenance and support.
68
68
 
69
- For more details, see [Contract Details](#contract-details).
70
-
71
69
  ---
72
70
 
73
71
  ## Implementation Process
74
72
 
75
73
  1. **Initial Consultation**: Discuss your specific testing challenges, application architecture, and infrastructure requirements.
76
- 2. **Custom Infrastructure Design**: Configure specialized testing environments tailored to your technology stack and workflow requirements.
77
- 3. **Expert Test Development**: Our team develops 4 custom tests designed specifically for your application's critical user flows and business logic.
78
- 4. **Integration & Deployment**: Implement tests within your CI/CD pipeline with custom monitoring and reporting configurations.
79
- 5. **Team Training & Ongoing Support**: Comprehensive training for your team plus ongoing technical support and consultation.
74
+ 2. **Expert Test Development**: Our team develops 4 custom tests designed specifically for your application's critical user flows and business logic.
75
+ 3. **Integration & Deployment**: Implement tests within your CI/CD pipeline with custom monitoring and reporting configurations.
76
+ 4. **Team Training & Ongoing Support**: Comprehensive training for your team plus ongoing technical support and consultation.
80
77
 
81
78
  Complex applications - particularly desktop software, browser extensions, and multi-platform workflows - present unique testing challenges that require specialized infrastructure and deep technical expertise. TestDriver Enterprise addresses these challenges with custom solutions designed specifically for your application and development process.
82
79
 
@@ -86,7 +83,6 @@ Complex applications - particularly desktop software, browser extensions, and mu
86
83
 
87
84
  | Service | Timeline | Description |
88
85
  | --------------------------------- | ------------- | ------------------------------------------------------------------------------------------------ |
89
- | **Infrastructure Design** | First 7 Days | Analysis and configuration of specialized testing environments for your application stack. |
90
86
  | **Requirements Analysis** | First 7 Days | Comprehensive review of testing requirements and technical specifications. |
91
87
  | **Custom Test Development** | First 4 Weeks | Expert creation of 4 fully customized tests (4x4 Guarantee) tailored to your critical workflows. |
92
88
  | **Training & Knowledge Transfer** | First 30 Days | Technical training for your team and establishment of ongoing support processes. |
@@ -103,7 +99,7 @@ Complex applications - particularly desktop software, browser extensions, and mu
103
99
  - **Service Level**: Dedicated support team and technical consultation included.
104
100
  - **Usage Tracking**: Monthly runner minute allocation with standard overage rates.
105
101
  - **Custom Infrastructure**: Specialized testing environments included for complex applications.
106
- - **Enterprise Options**: On-premises and BYOC (Bring Your Own Cloud) configurations available.
102
+ - **Enterprise Options**: [Self hosting](/getting-started/self-hosting) configurations available.
107
103
 
108
104
  ---
109
105
 
@@ -19,12 +19,12 @@ TestDriver offers a range of pricing plans to suit different needs, from individ
19
19
  </Card>
20
20
  <Card title="Enterprise" icon="shield">
21
21
  Need advanced features? Contact us for tailored solutions. Starting at
22
- $995/month.
22
+ $2,000/month.
23
23
  </Card>
24
24
  </CardGroup>
25
25
 
26
26
  <Tip>
27
- Every plan starts with $100 in TestDriver credits to get you off the starting
27
+ Every plan comes with access to the Playwright SDK to get you off the starting
28
28
  line!
29
29
  </Tip>
30
30
 
@@ -23,6 +23,11 @@ From the Project view, you can see all the replays (Dashes) stored for that proj
23
23
 
24
24
  When you create a new Project, you can also enable the <Icon icon="jira" /> Jira integration to create issues automatically each time a replay (Dash) is created.
25
25
 
26
+ <Info>
27
+ The project ID can be used in conjunction with your `lifecycle/postrun.yaml`
28
+ script to automatically assign a replay to a project. For more info see the
29
+ (Dashcam section)[/guide/dashcam].
30
+ </Info>
26
31
  <Frame caption="Click a Project to view its replays">
27
32
  <img src="/images/content/account/newprojectsettings.png" />
28
33
  </Frame>
@@ -14,10 +14,7 @@ icon: "https://tauri.app/favicon.svg"
14
14
 
15
15
  In this guide, we'll leverage [Playwright](https://playwright.dev/) and the [TestDriver Playwright SDK](/getting-started/playwright) to convert the [Tauri Quick Start](https://tauri.app/start/create-project/) to TestDriver's selectorless, Vision AI.
16
16
 
17
- <Info>
18
- View Source:
19
- https://github.com/testdriverai/demo-tauri-app
20
- </Info>
17
+ <Info>View Source: https://github.com/testdriverai/demo-tauri-app</Info>
21
18
 
22
19
  ### Requirements
23
20
 
@@ -30,7 +27,7 @@ To start testing your Tauri app with TestDriver, you need the following:
30
27
  You will need a [Free TestDriver Account](https://app.testdriver.ai/team) to get an API key.
31
28
 
32
29
  <Card title="Sign Up for TestDriver" icon="user-plus" horizontal href="https://app.testdriver.ai/team">
33
-
30
+
34
31
  </Card>
35
32
  </Step>
36
33
  <Step title="Set up your environment">
@@ -54,6 +51,7 @@ To start testing your Tauri app with TestDriver, you need the following:
54
51
  </Tabs>
55
52
  </Step>
56
53
  </Steps>
54
+
57
55
  </Accordion>
58
56
  <Accordion title="Create a Tauri project">
59
57
  <Note>
@@ -98,6 +96,7 @@ To start testing your Tauri app with TestDriver, you need the following:
98
96
  ✔ Install Playwright browsers (can be done manually via 'npx playwright install')? (Y/n)
99
97
  > Y
100
98
  ```
99
+
101
100
  </Accordion>
102
101
  <Accordion title="Install the TestDriver Playwright SDK">
103
102
  `@testdriver.ai/playwright` is an AI-powered extension of `@playwright/test`.
@@ -119,6 +118,7 @@ To start testing your Tauri app with TestDriver, you need the following:
119
118
  ```
120
119
  </Tab>
121
120
  </Tabs>
121
+
122
122
  </Accordion>
123
123
  </AccordionGroup>
124
124
 
@@ -146,6 +146,7 @@ First, we need to modify the default Playwright configuration and our Tauri proj
146
146
  },
147
147
  });
148
148
  ```
149
+
149
150
  </Step>
150
151
  <Step title="Mock Tauri APIs">
151
152
  Since we're testing the Tauri frontend, we need to [mock IPC Requests](https://tauri.app/develop/tests/mocking/)
@@ -167,6 +168,7 @@ First, we need to modify the default Playwright configuration and our Tauri proj
167
168
  ```
168
169
 
169
170
  We only need to do this once, as we'll be accessing `window.mockIPC` in our tests.
171
+
170
172
  </Step>
171
173
  <Step title="Create a new test file">
172
174
  Create a new file (e.g. `tests/testdriver.spec.ts`) with:
@@ -174,15 +176,16 @@ First, we need to modify the default Playwright configuration and our Tauri proj
174
176
  ```typescript tests/testdriver.spec.ts
175
177
  import type { mockIPC } from "@tauri-apps/api/mocks";
176
178
  import { expect, test } from "@playwright/test";
177
-
179
+
178
180
  test.beforeEach(async ({ page }) => {
179
181
  await page.goto("http://localhost:1420");
180
182
  });
181
-
183
+
182
184
  test("should have title", async ({ page }) => {
183
185
  await expect(page).toHaveTitle("Tauri + React + TypeScript");
184
186
  });
185
187
  ```
188
+
186
189
  </Step>
187
190
  <Step title="Run Playwright in UI Mode">
188
191
  Now we're ready to run Playwright and start working on our tests:
@@ -211,6 +214,7 @@ First, we need to modify the default Playwright configuration and our Tauri proj
211
214
  <Tip>
212
215
  Click the <Icon icon="eye" /> button to automatically re-run tests on save.
213
216
  </Tip>
217
+
214
218
  </Step>
215
219
  </Steps>
216
220
 
@@ -300,6 +304,7 @@ With TestDriver, we can skip the test implementation **entirely** and let AI per
300
304
  });
301
305
  });
302
306
  ```
307
+
303
308
  </Step>
304
309
  <Step title="Add an Agentic Test">
305
310
  Next, wrap a _prompt_ in `test.agent` to perform the test:
@@ -313,6 +318,7 @@ With TestDriver, we can skip the test implementation **entirely** and let AI per
313
318
  `);
314
319
  });
315
320
  ```
321
+
316
322
  </Step>
317
323
  </Steps>
318
324
 
@@ -327,27 +333,13 @@ We can use TestDriver and natural language to test our Tauri desktop app:
327
333
  <Steps>
328
334
  <Step title="Run the Desktop App">
329
335
  <Tabs>
330
- <Tab title="npm">
331
- ```bash
332
- npm run tauri dev
333
- ```
334
- </Tab>
335
- <Tab title="yarn">
336
- ```bash
337
- yarn tauri dev
338
- ```
339
- </Tab>
340
- <Tab title="pnpm">
341
- ```bash
342
- pnpm tauri dev
343
- ```
344
- </Tab>
336
+ <Tab title="npm">```bash npm run tauri dev ```</Tab>
337
+ <Tab title="yarn">```bash yarn tauri dev ```</Tab>
338
+ <Tab title="pnpm">```bash pnpm tauri dev ```</Tab>
345
339
  </Tabs>
346
340
  </Step>
347
341
  <Step title="Continued Reading">
348
- <Note>
349
- See [Desktop Apps](/apps/desktop-apps) for more information.
350
- </Note>
342
+ <Note>See [Desktop Apps](/apps/desktop-apps) for more information.</Note>
351
343
  </Step>
352
344
  </Steps>
353
345
 
@@ -358,26 +350,12 @@ We can use TestDriver and natural language to test our Tauri iOS app:
358
350
  <Steps>
359
351
  <Step title="Run the Mobile App">
360
352
  <Tabs>
361
- <Tab title="npm">
362
- ```bash
363
- npm run tauri ios dev
364
- ```
365
- </Tab>
366
- <Tab title="yarn">
367
- ```bash
368
- yarn tauri ios dev
369
- ```
370
- </Tab>
371
- <Tab title="pnpm">
372
- ```bash
373
- pnpm tauri ios dev
374
- ```
375
- </Tab>
353
+ <Tab title="npm">```bash npm run tauri ios dev ```</Tab>
354
+ <Tab title="yarn">```bash yarn tauri ios dev ```</Tab>
355
+ <Tab title="pnpm">```bash pnpm tauri ios dev ```</Tab>
376
356
  </Tabs>
377
357
  </Step>
378
358
  <Step title="Continued Reading">
379
- <Note>
380
- See [Mobile Apps](/apps/mobile-apps) for more information.
381
- </Note>
359
+ <Note>See [Mobile Apps](/apps/mobile-apps) for more information.</Note>
382
360
  </Step>
383
- </Steps>
361
+ </Steps>
@@ -15,11 +15,11 @@ npx testdriverai@latest <command> [options]
15
15
 
16
16
  ## Available commands
17
17
 
18
- | Command | Description |
19
- | :--------------------: | :----------------------------------------------------------- |
20
- | [`run`](/commands/run) | Executes a TestDriver test. |
21
- | [`edit`](/commands/edit) | Launch interactive mode. |
22
- | [`help`](/commands/help) | Displays help information for the CLI or a specific command. |
18
+ | Command | Description |
19
+ | :----------------------: | :----------------------------------------------------------- |
20
+ | [`run`](/commands/run) | Executes a TestDriver test. |
21
+ | [`edit`](/commands/edit) | Launch interactive mode. |
22
+ | [`help`](/commands/help) | Displays help information for the CLI or a specific command. |
23
23
 
24
24
  ## Available Flags
25
25
 
@@ -27,10 +27,9 @@ npx testdriverai@latest <command> [options]
27
27
  | :------------------ | :----------------------------------------------------------------------------------------- |
28
28
  | `--heal` | Launch exploratory mode and attemp to recover if an error or failing state is encountered. |
29
29
  | `--write` | Ovewrite test file with new commands resulting from agentic testing |
30
- | `--headless` | Run test without opening a browser window (useful for CI/CD environments) |
31
30
  | `--new` | Create a new sandbox environment for the test run. |
32
31
  | `--summary=<value>` | Output file where AI summary should be saved. |
33
- | `--junit=<value>` | Output file where junit report should be saved. |
32
+ | `--junit=<value>` | Output file where junit report should be saved. |
34
33
 
35
34
  ## Example usage
36
35
 
package/docs/docs.json CHANGED
@@ -103,6 +103,7 @@
103
103
  "/guide/authentication",
104
104
  "/guide/variables",
105
105
  "/guide/lifecycle",
106
+ "/guide/dashcam",
106
107
  "/guide/environment-variables",
107
108
  "/action/ami"
108
109
  ]
@@ -50,8 +50,9 @@ test.describe("get started link", () => {
50
50
  You will need a [Free TestDriver Account](https://app.testdriver.ai/team) to get an API key.
51
51
 
52
52
  <Card title="Sign Up for TestDriver" icon="user-plus" horizontal href="https://app.testdriver.ai/team">
53
-
53
+
54
54
  </Card>
55
+
55
56
  </Step>
56
57
  <Step title="Set up your environment">
57
58
  Copy your API key from [the TestDriver dashboard](https://app.testdriver.ai/team), and set it as an environment variable.
@@ -68,6 +69,7 @@ test.describe("get started link", () => {
68
69
  ```
69
70
  </Tab>
70
71
  </Tabs>
72
+
71
73
  </Step>
72
74
  </Steps>
73
75
 
@@ -131,6 +133,7 @@ test.describe("get started link", () => {
131
133
  ```
132
134
  </Tab>
133
135
  </Tabs>
136
+
134
137
  </Step>
135
138
  </Steps>
136
139
 
@@ -157,6 +160,7 @@ test.describe("get started link", () => {
157
160
  ```
158
161
  </Tab>
159
162
  </Tabs>
163
+
160
164
  </Step>
161
165
  <Step title="Run Playwright">
162
166
  Before we start using TestDriver in our tests, run Playwright in [UI Mode](https://playwright.dev/docs/test-ui-mode):
@@ -182,6 +186,7 @@ test.describe("get started link", () => {
182
186
 
183
187
  Clicking the ▶️ button should successfully run the tests in the UI,
184
188
  just as they did before with `playwright test` in the CLI.
189
+
185
190
  </Step>
186
191
  <Step title="Import TestDriver">
187
192
  For the sake of simplicity, we'll be working with one test file for now.
@@ -199,6 +204,7 @@ test.describe("get started link", () => {
199
204
  <Tip>
200
205
  Click the <Icon icon="eye" /> button to automatically re-run tests on save.
201
206
  </Tip>
207
+
202
208
  </Step>
203
209
  </Steps>
204
210
 
@@ -266,14 +272,15 @@ Now, our test uses natural language to both describe & locate the element.
266
272
  <Tip>
267
273
  In the example above, you can still use Playwright to assert that the element is indeed a link for accessibility:
268
274
 
269
- ```typescript tests/example.spec.ts icon=square-js
270
- const link = await testdriver(page).locate("Get started link");
271
- // [!code ++]
272
- expect(link).toHaveRole("link");
273
- await link.click();
274
- ```
275
+ ```typescript tests/example.spec.ts icon=square-js
276
+ const link = await testdriver(page).locate("Get started link");
277
+ // [!code ++]
278
+ expect(link).toHaveRole("link");
279
+ await link.click();
280
+ ```
281
+
282
+ This way you can write user-centric tests _and_ validate the implementation.
275
283
 
276
- This way you can write user-centric tests _and_ validate the implementation.
277
284
  </Tip>
278
285
 
279
286
  ### Performing actions with `testdriver.act`
@@ -332,4 +339,4 @@ but replaced the `test` itself with `test.agent`.
332
339
 
333
340
  ## Conclusion
334
341
 
335
- With `@testdriver.ai/playwright`, you can use as much or as little of Playwright's _or_ TestDriver's API as you need to validate correctness. It's up to you!
342
+ With `@testdriver.ai/playwright`, you can use as much or as little of Playwright's _or_ TestDriver's API as you need to validate correctness. It's up to you!
@@ -15,28 +15,37 @@ Self-hosting TestDriver allows you to run tests on your own infrastructure, givi
15
15
 
16
16
  ## Why self host?
17
17
 
18
+ Self-hosting TestDriver gives you complete control over your test execution environment:
19
+
18
20
  - **Enhanced security**: Get complete control over ingress and egress rules.
19
- - **Complete customization**: Modify the TestDriver Golden Image to include custom dependencies, software, and configurations at launch time.
21
+ - **Complete customization**: Modify the TestDriver Golden Image (our pre-configured AMI) to include custom dependencies, software, and configurations at launch time.
20
22
  - **Powerful Infrastructure**: Run tests on bare metal infrastructure that support emulators and simulators.
21
23
 
24
+ You'll use the [TestDriver CLI repository](https://github.com/testdriverai/cli) which contains all the infrastructure templates and setup scripts needed for self-hosting.
25
+
22
26
  ## Overview
23
27
 
24
- Self-hosting TestDriver gives you complete control over your test execution environment. You'll provision EC2 instances on AWS using our pre-configured AMI and infrastructure templates:
28
+ By the end of this guide, you'll have a complete self-hosted testing infrastructure that can:
29
+
30
+ - Spawn TestDriver instances on-demand in your AWS account
31
+ - Run tests on your own AWS infrastructure with custom configurations
32
+ - Integrate seamlessly with GitHub Actions CI/CD workflows
33
+ - Automatically clean up resources after test completion
25
34
 
26
- 1. **GitHub Workflow**: Use `.github/workflows/self-hosted.yml` as your template to test in CI.
27
- 2. **CloudFormation**: Deploy our `setup/aws/cloudformation.yaml` to provision infrastructure.
28
- 3. **On-Demand Runners**: Use `setup/aws/spawn-runner.sh` with your Launch Template ID from CloudFormation to spawn runners on-demand.
29
- 4. **GitHub Secrets**: Store AWS Credentials to your GitHub repository's secrets.
35
+ The setup process involves three main steps:
36
+
37
+ 1. **CloudFormation Infrastructure**: Deploy our `setup/aws/cloudformation.yaml` template to create the foundational AWS resources (VPC, security groups, IAM roles, and a launch template for instance creation).
38
+ 2. **On-Demand Instance Spawning**: Use `setup/aws/spawn-runner.sh` with your Launch Template ID to programmatically spawn TestDriver instances whenever you need to run tests.
39
+ 3. **GitHub Actions Integration**: Use `.github/workflows/self-hosted.yml` as a template for running tests in CI. This workflow demonstrates the complete lifecycle: spawning an instance, running tests, and shutting down the instance to minimize costs.
30
40
 
31
41
  ## Prerequisites
32
42
 
33
43
  - AWS account with permissions to run CloudFormation.
34
44
  - [AWS CLI](https://aws.amazon.com/cli/) installed locally.
35
45
 
36
- <Tip>
37
- Be sure to run `aws configure` with your credentials
38
- </Tip>
39
- - Access to the TestDriver AMI\
46
+ <Tip>Be sure to run `aws configure` with your credentials</Tip>
47
+
48
+ - Access to the TestDriver AMI (Golden Image)\
40
49
  [Contact us with your preferred AWS Region for access](https://form.typeform.com/to/UECf9rDx?typeform-source=testdriver.ai).
41
50
  - A GitHub repository for committing your tests & workflow.
42
51
 
@@ -57,7 +66,7 @@ This is a one-time setup used to generate a template ID for launching instances.
57
66
  # Deploy the CloudFormation stack
58
67
  aws cloudformation deploy \
59
68
  --template-file setup/aws/cloudformation.yaml \
60
- --stack-name testdriver-infrastructure-11 \
69
+ --stack-name my-testdriver-infrastructure \
61
70
  --parameter-overrides \
62
71
  ProjectTag=testdriver \
63
72
  AllowedIngressCidr=0.0.0.0/0 \
@@ -67,7 +76,8 @@ aws cloudformation deploy \
67
76
  ```
68
77
 
69
78
  <Danger>
70
- **Security**: Replace `AllowedIngressCidr=0.0.0.0/0` with your specific IP ranges to lock down access to your VPC.
79
+ **Security**: Replace `AllowedIngressCidr=0.0.0.0/0` with your specific IP
80
+ ranges to lock down access to your VPC.
71
81
  </Danger>
72
82
 
73
83
  ### Get Launch Template ID
@@ -76,32 +86,35 @@ After CloudFormation completes, find the launch template ID in the stack outputs
76
86
 
77
87
  ```bash
78
88
  aws cloudformation describe-stacks \
79
- --stack-name testdriver-infrastructure-11 \
89
+ --stack-name my-testdriver-infrastructure \
80
90
  --query 'Stacks[0].Outputs[?OutputKey==`LaunchTemplateId`].OutputValue' \
81
91
  --output text
82
92
  ```
83
93
 
84
- <Tip>
85
- **Save this ID** – you'll need it for the next step.
86
- </Tip>
94
+ <Tip>**Save this ID** – you'll need it for the next step.</Tip>
87
95
 
88
96
  ## Step 2: Spawn a New TestDriver Runner
89
97
 
90
- ### Using aws-setup.sh
98
+ This step is performed **every time you want to run tests**. The `spawn-runner.sh` script launches a new EC2 instance on-demand for test execution.
91
99
 
92
- Our [`setup/aws/spawn-runner.sh`](https://github.com/testdriverai/cli/tree/main/setup/aws/spawn-runner.sh) spawns and initializes instances:
100
+ ### Using spawn-runner.sh
93
101
 
94
- - Launches instances using your launch template
95
- - Completes TestDriver handshake
96
- - Returns instance details for CLI usage
102
+ Our [`setup/aws/spawn-runner.sh`](https://github.com/testdriverai/cli/tree/main/setup/aws/spawn-runner.sh) script:
103
+
104
+ - Launches a new EC2 instance using your launch template from Step 1
105
+ - Waits for the instance to become ready
106
+ - Completes the TestDriver handshake
107
+ - Returns instance details (IP, instance ID) for CLI usage
108
+
109
+ The script accepts parameters as either environment variables or CLI arguments:
97
110
 
98
111
  ```bash
99
- # Launch an instance
112
+ # Launch an instance using environment variables
100
113
  export AWS_REGION=us-east-2
101
114
  export AMI_ID=ami-•••••••••• # Your TestDriver AMI (contact us to get one)
102
115
  export AWS_LAUNCH_TEMPLATE_ID=lt-•••••••••• # From CloudFormation output from step 1
103
116
 
104
- /bin/bash ./setup/aws/aws-setup.sh
117
+ /bin/bash ./setup/aws/spawn-runner.sh
105
118
  ```
106
119
 
107
120
  The script outputs:
@@ -112,6 +125,13 @@ INSTANCE_ID=i-1234567890abcdef0
112
125
  AWS_REGION=us-east-2
113
126
  ```
114
127
 
128
+ <Note>
129
+ **Instance Lifecycle**: Instances spawned by this script will continue running
130
+ until you manually terminate them. They are automatically tagged with
131
+ `Name=TestDriverRunner` and `Project=[your ProjectTag value]` for easy
132
+ identification in the AWS console.
133
+ </Note>
134
+
115
135
  ### CLI Usage
116
136
 
117
137
  Once you have an instance IP, run tests directly:
@@ -128,17 +148,26 @@ npx testdriverai@latest run testdriver/your-test.yaml \
128
148
  --ip="$PUBLIC_IP" \
129
149
  ```
130
150
 
131
- Note that the instance will remain running until you terminate it. You can do this manually via the AWS console, or programmatically in your CI workflow:
151
+ ### Terminating Instances
152
+
153
+ After your tests complete, terminate the instance to avoid unnecessary costs:
132
154
 
133
155
  ```bash
134
- aws ec2 terminate-instances --instance-ids $INSTANCE_ID
156
+ # Terminate the instance
157
+ aws ec2 terminate-instances --instance-ids $INSTANCE_ID --region $AWS_REGION
135
158
  ```
136
159
 
160
+ You can also terminate instances manually through the AWS console by searching for instances tagged with `Name=TestDriverRunner`.
161
+
137
162
  ## Step 3: GitHub Actions Integration
138
163
 
164
+ This step shows you how to automate the entire test lifecycle in CI/CD.
165
+
139
166
  ### Example Workflow
140
167
 
141
- Our [`.github/workflows/self-hosted.yml`](https://github.com/testdriverai/cli/tree/main/.github/workflows/self-hosted.yml) demonstrates the complete workflow:
168
+ Our [`.github/workflows/self-hosted.yml`](https://github.com/testdriverai/cli/tree/main/.github/workflows/self-hosted.yml) demonstrates the complete workflow: spawning an EC2 instance, running your tests, and shutting down the instance automatically to minimize costs.
169
+
170
+ The workflow uses the GitHub secrets you configure (see below) to authenticate with AWS and spawn instances on-demand:
142
171
 
143
172
  ```yaml
144
173
  name: TestDriver Self-Hosted
@@ -205,11 +234,24 @@ Configure these secrets in your GitHub repository:
205
234
 
206
235
  ### Using the Base AMI
207
236
 
208
- Our AMI comes pre-configured with:
237
+ Our TestDriver Golden Image (AMI) comes pre-configured with everything you need to run tests:
238
+
239
+ **Operating System & Environment:**
209
240
 
210
241
  - Windows Server with desktop environment
211
- - Required TestDriver dependencies
242
+ - VNC + web server for remote desktop access through the browser
243
+
244
+ **Development Tools:**
245
+
246
+ - Python (with pip)
247
+ - Node.js (with npm)
248
+ - Git
249
+
250
+ **Test Infrastructure:**
251
+
252
+ - TestDriver agent and dependencies
212
253
  - Optimized settings for test execution
254
+ - Pre-configured networking for TestDriver CLI communication
213
255
 
214
256
  ### Modifying the AMI
215
257
 
@@ -281,11 +323,13 @@ steps:
281
323
 
282
324
  ### Common Issues
283
325
 
284
- **Instance not responding:**
326
+ **Instance not responding in TestDriver CLI:**
327
+
328
+ When the CLI displays connection errors or timeouts, check:
285
329
 
286
- - Check security group rules allow necessary ports
287
- - Verify instance has passed all status checks
288
- - Ensure AMI is compatible with selected instance type
330
+ - **Security group rules**: The CloudFormation template configures all necessary ports (RDP 3389, VNC 5900, and TestDriver communication ports). Verify your security group hasn't been modified.
331
+ - **Instance status checks**: Ensure the instance has passed both system and instance status checks in the AWS console.
332
+ - **AMI compatibility**: Verify the AMI is compatible with your selected instance type (some instance types don't support certain AMIs).
289
333
 
290
334
  **Connection timeouts:**
291
335
 
@@ -305,4 +349,4 @@ For enterprise customers:
305
349
 
306
350
  - Contact your account manager for AMI access issues
307
351
  - Use support channels for infrastructure questions
308
- - Check the TestDriver documentation for CLI usage
352
+ - Check the TestDriver documentation for CLI usage
@@ -0,0 +1,118 @@
1
+ ---
2
+ title: "Dashcam Replays"
3
+ sidebarTitle: "Dashcam"
4
+ description: "Learn how to use Dashcam to record and replay test sessions in TestDriver."
5
+ icon: "video"
6
+ ---
7
+
8
+ [Dashcam](https://www.dashcam.io), from the makers of TestDriver, is a powerful feature in TestDriver that allows you to record and replay your test sessions. This is particularly useful for debugging, sharing test runs with team members, or reviewing the steps taken during a test. For the full docs see the [Dashcam docs](https://docs.dashcam.io/dashcam/).
9
+
10
+ ## Recording a Test Session
11
+
12
+ To record a test session, you can use the `dashcam` command in your lifecycle scripts. There are two main lifecycle scripts where you can integrate Dashcam: `lifecycle/prerun.yaml` and `lifecycle/postrun.yaml`.
13
+
14
+ ## Ways to use Dashcam
15
+
16
+ Dashcam comes as a standalone app and a Chrome extension. You can use either or both to capture your test sessions.
17
+
18
+ <Info>
19
+ To capture web logs, make sure to install the Dashcam Chrome extension on the
20
+ browser you are testing with. We recommend installing it via CLI to Chrome for
21
+ Testing. You can also find the extension [in the Chrome
22
+ Webstore](https://chromewebstore.google.com/detail/dashcam/dkcoeknmlfnfimigfagbcjgpokhdcbbp)
23
+ </Info>
24
+
25
+ ### Installing the Dashcam Chrome extension via command line in prerun.yaml
26
+
27
+ In this lifecycle script, we install Chrome for Testing with a user profile that has the password manager disabled and sets up TestDriver Dashcam for replays and logs.
28
+
29
+ ```yaml lifecycle/prerun.yaml [expandable]
30
+ - prompt: launch chrome for testing and setup dashcam
31
+ commands:
32
+ # this script installs chrome for testing with a userprofile that has password manager disabled and sets up TestDriver Dashcam for replays and logs
33
+ - command: exec
34
+ lang: pwsh
35
+ code: |
36
+ cd $env:TEMP
37
+ Write-Host "Changed directory to TEMP: $env:TEMP"
38
+
39
+ Write-Host "Running 'npm init -y'..."
40
+ npm init -y
41
+
42
+ Write-Host "Installing dependencies: @puppeteer/browsers and dashcam-chrome..."
43
+ npm install @puppeteer/browsers dashcam-chrome
44
+
45
+ Write-Host "Installing Chromium via '@puppeteer/browsers'..."
46
+ npx @puppeteer/browsers install chrome
47
+
48
+ # Define paths
49
+ $extensionPath = Join-Path (Get-Location) "node_modules/dashcam-chrome/build"
50
+ $profilePath = Join-Path $env:TEMP "chrome-profile-$(Get-Random)"
51
+
52
+ Write-Host "Extension path: $extensionPath"
53
+ Write-Host "Chrome user data dir: $profilePath"
54
+
55
+ # Validate extension path
56
+ if (-not (Test-Path $extensionPath)) {
57
+ Write-Host "Extension not found at $extensionPath"
58
+ }
59
+
60
+ $chromeArgs = @(
61
+ "--start-maximized",
62
+ "--load-extension=$extensionPath",
63
+ "--user-data-dir=$profilePath",
64
+ "--no-first-run",
65
+ "--no-default-browser-check",
66
+ "--disable-infobars"
67
+ "${TD_WEBSITE}"
68
+ ) -join ' '
69
+
70
+ Start-Process "cmd.exe" -ArgumentList "/c", "npx @puppeteer/browsers launch chrome -- $chromeArgs"
71
+
72
+ Write-Host "Script complete."
73
+ exit 0
74
+ ```
75
+
76
+ ### Using the Chrome extension and capturing web logs
77
+
78
+ Now in the same `lifecycle/prerun.yaml` script, we set up Dashcam to track web logs and application logs. You can customize the patterns to match your needs. Testing Desktop? You can skip the web logs and just track application logs.
79
+
80
+ ```yaml lifecycle/prerun.yaml
81
+ ...
82
+ - command: exec
83
+ lang: pwsh
84
+ code: |
85
+ dashcam track --name="Web Logs" --type="web" --pattern="*"
86
+ dashcam track --name=TestDriver --type=application --pattern="C:\Users\testdriver\Documents\testdriver.log"
87
+ ```
88
+
89
+ ### Starting Dashcam
90
+
91
+ The final step in our `lifecycle/prerun.yaml` script is to start Dashcam recording.
92
+
93
+ ```yaml lifecycle/prerun.yaml
94
+ ...
95
+ - command: exec
96
+ lang: pwsh
97
+ code: dashcam start
98
+ ```
99
+
100
+ ### Publishing replays to a project in your account
101
+
102
+ Lastly, in the `lifecycle/postrun.yaml` script, we publish the recorded Dashcam session to a project in your Dashcam account. Make sure to replace `<YOUR_PROJECT_ID>` with the actual ID of your project.
103
+
104
+ ```yaml lifecycle/postrun.yaml
105
+ - prompt: send dashcam recording to server
106
+ # this script tells TestDriver Dashcam to send the recording to the server
107
+ commands:
108
+ - command: exec
109
+ lang: pwsh
110
+ code: dashcam -t '${TD_THIS_FILE}' -p -k <YOUR_PROJECT_ID> # optional add `-k MYFOLDERID` for the id of a folder in your Projects page at app.testdriver.ai
111
+ ```
112
+
113
+ <Info>
114
+ `${TD_THIS_FILE}` is an environment variable set by TestDriver that contains
115
+ the name of the current test file being executed. This will be used as the
116
+ title of the Dashcam recording. For more info see [parallel testing
117
+ docs](/features/parallel-testing).
118
+ </Info>
@@ -11,13 +11,13 @@ import GitignoreWarning from "/snippets/gitignore-warning.mdx";
11
11
  The supported environment variables in TestDriver are:
12
12
 
13
13
  <div className="env-vars-table">
14
- | Variable | Type | Description |
14
+ | Variable | Type | Description |
15
15
  |:---------------:|:---------:|---------------------------------------------------------------------------------|
16
- | TD_ANALYTICS | boolean | Send analytics to TestDriver servers. This helps provide feedback to inform our roadmap. |
17
- | TD_API_KEY | string | Set this to spawn VMs with TestDriver Pro. |
16
+ | TD_ANALYTICS | boolean | Send analytics to TestDriver servers. This helps
17
+ provide feedback to inform our roadmap. | | TD_API_KEY | string | Set this to
18
+ spawn VMs with TestDriver Pro. |
18
19
  </div>
19
-
20
- <GitignoreWarning/>
20
+ <GitignoreWarning />
21
21
  ## Example
22
22
 
23
23
  ```bash .env
@@ -10,13 +10,10 @@ icon: boxing-glove
10
10
  TestDriver operates a full desktop environment, so it can run any application.
11
11
 
12
12
  <div className="comparison-table">
13
- | Application | TestDriver | Playwright | Selenium |
14
- |:-----------------:|:---------:|:-----------:|:--------:|
15
- | Web Apps | ✅ | ✅ | ✅ |
16
- | Mobile Apps | | | ✅ |
17
- | VS Code | ✅ | ✅ | ✅ |
18
- | Desktop Apps | ✅ | | |
19
- | Chrome Extensions | ✅ | | |
13
+ | Application | TestDriver | Playwright | Selenium |
14
+ |:-----------------:|:---------:|:-----------:|:--------:| | Web Apps | ✅ |
15
+ | | | Mobile Apps | ✅ | ✅ | ✅ | | VS Code | ✅ | ✅ | ✅ | | Desktop
16
+ Apps | | | | | Chrome Extensions | | | |
20
17
  </div>
21
18
 
22
19
  ## Testing features
@@ -24,16 +21,11 @@ TestDriver operates a full desktop environment, so it can run any application.
24
21
  TestDriver is AI first.
25
22
 
26
23
  <div className="comparison-table">
27
- | Feature | TestDriver | Playwright | Selenium |
28
- |:--------------------:|:---------:|:----------:|:--------:|
29
- | Test Generation | ✅ | | |
30
- | Adaptive Testing | ✅ | | |
31
- | Visual Assertions | ✅ | | |
32
- | Self Healing | ✅ | | |
33
- | Application Switching | ✅ | | |
34
- | GitHub Actions | ✅ | ✅ | |
35
- | Team Dashboard | ✅ | | |
36
- | Team Collaboration | ✅ | | |
24
+ | Feature | TestDriver | Playwright | Selenium |
25
+ |:--------------------:|:---------:|:----------:|:--------:| | Test Generation
26
+ | | | | | Adaptive Testing | | | | | Visual Assertions | ✅ | | | | Self
27
+ Healing | | | | | Application Switching | | | | | GitHub Actions | ✅ |
28
+ | | | Team Dashboard | | | | | Team Collaboration | ✅ | | |
37
29
  </div>
38
30
 
39
31
  ## Test coverage
@@ -61,15 +53,11 @@ TestDriver has more coverage than selector-based frameworks.
61
53
  Debugging features are powered by [Dashcam.io](https://dashcam.io).
62
54
 
63
55
  <div className="comparison-table">
64
- | Feature | TestDriver | Playwright | Selenium |
65
- |:------------------:|:----------:|:----------:|:--------:|
66
- | AI Summary | ✅ | | |
67
- | Video Replay | ✅ | ✅ | |
68
- | Browser Logs | | | |
69
- | Desktop Logs | ✅ | | |
70
- | Network Requests | ✅ | ✅ | |
71
- | Team Dashboard | ✅ | | |
72
- | Team Collaboration | ✅ | | |
56
+ | Feature | TestDriver | Playwright | Selenium |
57
+ |:------------------:|:----------:|:----------:|:--------:| | AI Summary | ✅
58
+ | | | | Video Replay | || | | Browser Logs | ✅ | ✅ | | | Desktop Logs
59
+ | | | | | Network Requests | | ✅ | | | Team Dashboard | ✅ | | | | Team
60
+ Collaboration | | | |
73
61
  </div>
74
62
 
75
63
  ## Web browser support
@@ -77,15 +65,10 @@ Debugging features are powered by [Dashcam.io](https://dashcam.io).
77
65
  TestDriver is browser agnostic and supports any version of any browser.
78
66
 
79
67
  <div className="comparison-table">
80
- | Feature | TestDriver | Playwright | Selenium |
81
- |:--------:|:----------:|:----------:|:--------:|
82
- | Chrome | ✅ | ✅ | ✅ |
83
- | Firefox | ✅ | ✅ | ✅ |
84
- | Webkit | ✅ | ✅ | ✅ |
85
- | IE | ✅ | | ✅ |
86
- | Edge | ✅ | ✅ | ✅ |
87
- | Opera | ✅ | | ✅ |
88
- | Safari | ✅ | | ✅ |
68
+ | Feature | TestDriver | Playwright | Selenium |
69
+ |:--------:|:----------:|:----------:|:--------:| | Chrome | ✅ | ✅ | ✅ | |
70
+ Firefox | | ✅ | ✅ | | Webkit | | ✅ | ✅ | | IE | ✅ | | ✅ | | Edge |
71
+ | | ✅ | | Opera | | | | | Safari | ✅ | | ✅ |
89
72
  </div>
90
73
 
91
74
  ## Operating system support
@@ -93,9 +76,7 @@ TestDriver is browser agnostic and supports any version of any browser.
93
76
  TestDriver currently supports Mac and Windows!
94
77
 
95
78
  <div className="comparison-table">
96
- | Feature | TestDriver | Playwright | Selenium |
97
- |:--------:|:----------:|:----------:|:--------:|
98
- | Windows | ✅ | ✅ | ✅ |
99
- | Mac | ✅ | ✅ | ✅ |
100
- | Linux | | ✅ | ✅ |
79
+ | Feature | TestDriver | Playwright | Selenium |
80
+ |:--------:|:----------:|:----------:|:--------:| | Windows | ✅ | ✅ | ✅ | |
81
+ Mac | | ✅ | ✅ | | Linux | | | ✅ |
101
82
  </div>
@@ -6,7 +6,6 @@ icon: "gauge-high"
6
6
  mode: "wide"
7
7
  ---
8
8
 
9
-
10
9
  <Steps>
11
10
  <Step title="Create a TestDriver Account">
12
11
 
@@ -78,7 +77,7 @@ mode: "wide"
78
77
  </Step>
79
78
  <Step title="Run the generated regression test">
80
79
 
81
- After TestDriver has run the exploratory test, you'll see that the `prompt.yaml` file has been updated with commands generated by the agent to make the test faster and more reliable.
80
+ After TestDriver has run the exploratory test, you'll see that the `prompt.yaml` file has been updated with commands generated by the agent to make the test faster and more reliable.
82
81
 
83
82
  ```yaml
84
83
  version: 6.0.0
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "testdriverai",
3
- "version": "6.1.7",
3
+ "version": "6.1.8-canary.68bcfa2.0",
4
4
  "description": "Next generation autonomous AI agent for end-to-end testing of web & desktop",
5
5
  "main": "index.js",
6
6
  "bin": {