testdriverai 6.0.15 → 6.0.16-canary.e337e67.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (29) hide show
  1. package/agent/index.js +2 -1
  2. package/docs/account/enterprise.mdx +64 -51
  3. package/docs/action/performance.mdx +7 -7
  4. package/docs/commands/hover-text.mdx +7 -0
  5. package/docs/docs.json +2 -2
  6. package/docs/getting-started/vscode.mdx +1 -1
  7. package/docs/guide/lifecycle.mdx +0 -8
  8. package/docs/guide/protips.mdx +42 -0
  9. package/package.json +1 -1
  10. package/testdriver/acceptance/remember.yaml +25 -5
  11. package/testdriver/examples/web/lifecycle/prerun.yaml +22 -8
  12. package/docs/commands/issues.mdx +0 -20
  13. package/docs/docs-future.json.txt +0 -114
  14. package/docs/support/known-bugs.mdx +0 -85
  15. package/testdriver/examples/doom/.env.example +0 -1
  16. package/testdriver/examples/doom/desktop.yaml +0 -0
  17. package/testdriver/examples/doom/lifecycle/provision.yaml +0 -46
  18. package/testdriver/examples/doom/readme.md +0 -1
  19. package/testdriver/examples/mobile/readme.md +0 -3
  20. package/testdriver/examples/npm/install.yaml +0 -0
  21. package/testdriver/examples/npm/lifecycle/prerun.yaml +0 -19
  22. package/testdriver/examples/performance/lifecycle/prerun.yaml +0 -40
  23. package/testdriver/examples/performance/monitor.yaml +0 -40
  24. package/testdriver/examples/performance/readme.md +0 -1
  25. package/testdriver/examples/playwright-test-recording/hello-world.yaml +0 -2
  26. package/testdriver/examples/playwright-test-recording/lifecycle/prerun.yaml +0 -28
  27. package/testdriver/examples/playwright-test-recording/readme.md +0 -1
  28. /package/testdriver/examples/{arc-browser → desktop}/desktop.yaml +0 -0
  29. /package/testdriver/examples/{arc-browser → desktop}/lifecycle/prerun.yaml +0 -0
package/agent/index.js CHANGED
@@ -447,13 +447,14 @@ class TestDriverAgent extends EventEmitter2 {
447
447
  sourcePosition: sourcePosition,
448
448
  });
449
449
 
450
- return await this.haveAIResolveError(
450
+ await this.haveAIResolveError(
451
451
  error,
452
452
  yaml.dump({ commands: [yml] }),
453
453
  depth,
454
454
  true,
455
455
  shouldSave,
456
456
  );
457
+ throw error;
457
458
  }
458
459
  }
459
460
 
@@ -7,95 +7,108 @@ icon: gem
7
7
 
8
8
  import { calendar } from "/snippets/calendar-link.mdx";
9
9
 
10
- We understand that QA coverage is critical and often needed immediately. That's why we're offering our **4x4 Guarantee**: try TestDriver Enterprise and we guarantee you will see 4 tests in your first 4 weeks!
10
+ TestDriver Enterprise is designed for fast-moving teams that need comprehensive QA coverage for complex applications. Our **4x4 Guarantee** ensures you get 4 custom tests created specifically for your application within your first 4 weeks.
11
11
 
12
- Unlike other QA services that take months and cost hundreds of thousands of dollars, TestDriver gets you up and running in just 30 days.
12
+ We specialize in testing scenarios that other tools can't handle - desktop applications, browser extensions, multi-platform workflows, and complex integrations that require custom infrastructure and expert support.
13
13
 
14
14
  <Card
15
- title="Book a Call"
15
+ title="Schedule a Consultation"
16
16
  icon="calendar"
17
17
  href="https://calendly.com/d/cq23-qyn-3v6/testdriver-ai-demo"
18
18
  link
19
19
  >
20
- Don't miss out on the cutting-edge of AI and QA testing!
20
+ Discuss your testing requirements and custom infrastructure needs.
21
21
  </Card>
22
22
 
23
23
  ---
24
24
 
25
- ## Pricing
25
+ ## Enterprise Plans
26
26
 
27
- We're confident you'll love TestDriver, which is why we're offering this risk-free trial. After the trial, plans start at just **$995/month**, which includes:
27
+ TestDriver Enterprise plans start at **$995/month** and include:
28
28
 
29
- - **12,500 runner minutes per month**: Enough to run 30 tests **2-5 times per day**.
30
- - Full access to our enterprise test dashboards.
31
- - Seamless integration with any CI/CD pipeline.
29
+ - **12,500 runner minutes per month**: Sufficient capacity for continuous testing of your custom test suite.
30
+ - Enterprise-grade test dashboards with advanced analytics.
31
+ - Full CI/CD pipeline integration with custom configurations.
32
+ - Dedicated infrastructure and ongoing support for complex testing scenarios.
33
+ - Expert test creation and maintenance services.
32
34
 
33
- For more details or to get a demo [Contract Details](#contract-details). To see other plans and pricing, check out our [Pricing](/account/pricing).
35
+ For detailed pricing and contract information, see [Contract Details](#contract-details). For other plans, visit our [Pricing](/account/pricing) page.
34
36
 
35
37
  <CardGroup cols={3}>
36
- <Card title="Quickly Deploy AI QA Tests">
37
- Safeguard your most important user flows with generated tests.
38
+ <Card title="Custom Desktop & Extension Testing">
39
+ Test complex desktop applications and browser extensions that other tools
40
+ can't handle.
38
41
  </Card>
39
- <Card title="Increase Your Total Coverage">
40
- Test flows never possible before with our powerful computer-use agent.
42
+ <Card title="Multi-App Workflow Testing">
43
+ Test sophisticated workflows that span multiple applications and platforms.
41
44
  </Card>
42
- <Card title="Spend Less Time on Maintenance">
43
- TestDriver tests automatically repair themselves.
45
+ <Card title="Enterprise Infrastructure & Support">
46
+ Get dedicated infrastructure and expert support for your unique testing
47
+ challenges.
44
48
  </Card>
45
49
  </CardGroup>
46
50
 
47
- ## Promotion details
51
+ ## Enterprise Features
48
52
 
49
- Here's what's included in all Enterprise plans:
53
+ TestDriver Enterprise provides comprehensive support for fast-moving teams with complex testing requirements:
50
54
 
51
- - **Test Coverage**: Test any publicly available desktop app, Chrome extension, mobile app, or website.
52
- - **AI-Generated Tests**: Choose from **250 AI-generated tests** within the first 7 days.
53
- - **Parallel Testing**: No limits on parallel testing.
54
- - **Daily Test Runs**: Run tests **3-5 times every day**.
55
- - **AI Quality Reports**: Receive detailed reports delivered directly to your email.
56
- - **Self-Healing Tests**: AI automatically fixes and maintains your tests.
57
- - **CI/CD Integration**: Seamlessly integrate with your CI/CD pipeline.
58
- - **Enterprise Dashboards**: Gain full access to our enterprise-grade test dashboards.
55
+ - **Complex Application Support**: Full testing capabilities for desktop applications, Chrome extensions, mobile apps, and web applications that require specialized infrastructure.
56
+ - **4x4 Custom Test Creation**: Our expert team creates 4 fully customized tests tailored to your specific application and workflows within the first 4 weeks.
57
+ - **Multi-Platform Workflows**: Test complex scenarios spanning multiple applications, browsers, and operating systems.
58
+ - **Unlimited Parallel Testing**: Scale your testing across custom infrastructure without limitations.
59
+ - **Continuous Test Execution**: Automated test runs integrated with your development and deployment cycles.
60
+ - **Intelligent Reporting**: Detailed analytics and quality reports delivered to your team.
61
+ - **Self-Healing Test Maintenance**: AI-powered test maintenance that adapts to application changes.
62
+ - **Custom CI/CD Integration**: Tailored integration with your existing development pipeline.
63
+ - **Enterprise Dashboards**: Advanced analytics and monitoring with custom metrics and insights.
64
+ - **Dedicated Infrastructure**: Specialized testing environments configured for your application stack.
65
+ - **Expert Support**: Ongoing technical support and consultation from our testing specialists.
66
+
67
+ Testing complex applications requires more than standard automation tools. Desktop applications, browser extensions, and multi-platform workflows demand specialized infrastructure, custom integrations, and deep technical expertise. TestDriver Enterprise provides the complete solution - from initial setup through ongoing maintenance and support.
59
68
 
60
69
  For more details, see [Contract Details](#contract-details).
61
70
 
62
71
  ---
63
72
 
64
- ## How does it work?
73
+ ## Implementation Process
74
+
75
+ 1. **Initial Consultation**: Discuss your specific testing challenges, application architecture, and infrastructure requirements.
76
+ 2. **Custom Infrastructure Design**: Configure specialized testing environments tailored to your technology stack and workflow requirements.
77
+ 3. **Expert Test Development**: Our team develops 4 custom tests designed specifically for your application's critical user flows and business logic.
78
+ 4. **Integration & Deployment**: Implement tests within your CI/CD pipeline with custom monitoring and reporting configurations.
79
+ 5. **Team Training & Ongoing Support**: Comprehensive training for your team plus ongoing technical support and consultation.
65
80
 
66
- 1. **Book an Onboarding Call**: Share the specific flows you want to test.
67
- 2. **Generate a Test Suite**: We'll explore your app and generate hundreds of tests using our AI.
68
- 3. **Deploy Tests**: Your new tests will be deployed to your CI/CD pipeline.
69
- 4. **Custom Test Creation**: Our support team will work with you to create tests for any features TestDriver might have missed.
70
- 5. **Engineer Training**: We'll train your engineers on best practices for creating and maintaining TestDriver tests.
81
+ Complex applications - particularly desktop software, browser extensions, and multi-platform workflows - present unique testing challenges that require specialized infrastructure and deep technical expertise. TestDriver Enterprise addresses these challenges with custom solutions designed specifically for your application and development process.
71
82
 
72
83
  ---
73
84
 
74
- ## Onboarding process
85
+ ## Implementation Timeline
75
86
 
76
- | Service | Timeline | Description |
77
- | ------------------------ | ------------- | --------------------------------------------------------------------------------------------- |
78
- | **Custom Onboarding** | First 7 Days | Get set up quickly with custom workflows and Prerun Scripts developed by the TestDriver team. |
79
- | **AI Test Generation** | First 7 Days | Instant coverage: We'll generate hundreds of tests for you to choose from. |
80
- | **Custom Test Creation** | First 30 Days | Our team will create 4 tests for any features TestDriver might have missed. |
81
- | **Training** | First 30 Days | Our support team will train your engineers on best practices. |
82
- | **Test Execution** | Recurring | Seamless deployment: Tests are executed on a schedule or via CI/CD. |
87
+ | Service | Timeline | Description |
88
+ | --------------------------------- | ------------- | ------------------------------------------------------------------------------------------------ |
89
+ | **Infrastructure Design** | First 7 Days | Analysis and configuration of specialized testing environments for your application stack. |
90
+ | **Requirements Analysis** | First 7 Days | Comprehensive review of testing requirements and technical specifications. |
91
+ | **Custom Test Development** | First 4 Weeks | Expert creation of 4 fully customized tests (4x4 Guarantee) tailored to your critical workflows. |
92
+ | **Training & Knowledge Transfer** | First 30 Days | Technical training for your team and establishment of ongoing support processes. |
93
+ | **Continuous Execution** | Ongoing | Automated test execution with specialized monitoring, reporting, and maintenance. |
83
94
 
84
95
  ---
85
96
 
86
- ## Contract details
97
+ ## Contract Information
87
98
 
88
- - **Free Trial**: Get 30 custom tests free during a 30-day trial when subscribing to the $995/month plan.
89
- - **Ownership**: The tests are yours! You can modify, duplicate, or distribute them however you wish.
90
- - **Payment**: A payment method is required to begin the trial.
91
- - **Renewal**: The contract renews annually. You can pay monthly or annually.
92
- - **Cancellation**: Cancel anytime during your trial, for any reason.
93
- - **Credits**: Unused credits don't roll over and expire at the end of each month.
94
- - **Additional Usage**: Additional usage is billed at standard rates.
95
- - **On Prem/BYOC**: Option available as an add-on to your contract.
99
+ - **4x4 Guarantee**: 4 custom tests delivered within your first 4 weeks of service.
100
+ - **Test Ownership**: Complete ownership of all tests - modify, duplicate, or distribute as needed.
101
+ - **Payment Terms**: Monthly or annual billing options available.
102
+ - **Contract Term**: Annual contracts with flexible payment schedules.
103
+ - **Service Level**: Dedicated support team and technical consultation included.
104
+ - **Usage Tracking**: Monthly runner minute allocation with standard overage rates.
105
+ - **Custom Infrastructure**: Specialized testing environments included for complex applications.
106
+ - **Enterprise Options**: On-premises and BYOC (Bring Your Own Cloud) configurations available.
96
107
 
97
108
  ---
98
109
 
99
- ## Ready to get started?
110
+ ## Get Started
100
111
 
101
- Don't wait! <a href={calendar}>**Schedule your onboarding call now**</a> and let us help you achieve QA coverage in just 30 days.
112
+ <a href={calendar}>**Schedule a consultation**</a> to discuss your testing
113
+ requirements and learn how TestDriver Enterprise can support your team's
114
+ development process.
@@ -23,9 +23,9 @@ Parallel testing allows you to split your test actions into multiple files and r
23
23
  - Use a test matrix strategy to execute these files in parallel.
24
24
 
25
25
  <Warning>
26
- The `run` command is used in your test files to run other test files. This is
27
- useful for breaking down large tests into smaller, more manageable pieces. To
28
- run multiple separate tests, use a test matrix strategy.
26
+ The [`run`](/commands/run) command is used in your test files to run other
27
+ test files. This is useful for breaking down large tests into smaller, more
28
+ manageable pieces. To run multiple separate tests, use a test matrix strategy.
29
29
  </Warning>
30
30
 
31
31
  Example:
@@ -43,12 +43,12 @@ strategy:
43
43
 
44
44
  ### 2. Use optimized matching methods
45
45
 
46
- For actions like `hover-text`, `wait-for-text`, and `scroll-until-text`, use the `turbo` matching method instead of `ai`. The `turbo` method uses text similarity to quickly compute the most relevant match, making it about 40% faster than the `ai` method.
46
+ For actions like [`hover-text`](/commands/hover-text), [`wait-for-text`](/commands/wait-for-text), and [`scroll-until-text`](/commands/scroll-until-text), use the `turbo` matching method instead of `ai`. The `turbo` method uses text similarity to quickly compute the most relevant match, making it about 40% faster than the `ai` method.
47
47
 
48
48
  #### Example
49
49
 
50
50
  ```yaml
51
- command: hover-text
51
+ command: [`hover-text`](/commands/hover-text)
52
52
  text: Sign In
53
53
  description: login button
54
54
  action: click
@@ -59,12 +59,12 @@ method: turbo
59
59
 
60
60
  ### 3. Use `async` asserts
61
61
 
62
- The `assert` command supports the `async: true` property, allowing you to create non-blocking assertions. This means your tests can continue running while the assertion is being validated, saving valuable time.
62
+ The [`assert`](/commands/assert) command supports the `async: true` property, allowing you to create non-blocking assertions. This means your tests can continue running while the assertion is being validated, saving valuable time.
63
63
 
64
64
  #### Example
65
65
 
66
66
  ```yaml
67
- command: assert
67
+ command: [`assert`](/commands/assert)
68
68
  expect: The user is logged in
69
69
  async: true
70
70
  ```
@@ -1,3 +1,10 @@
1
+ ---
2
+ title: "hover-text"
3
+ sidebarTitle: "hover-text"
4
+ description: "Hover or click on text elements based on a description."
5
+ icon: "text"
6
+ ---
7
+
1
8
  <Replay />
2
9
 
3
10
  <Example />
package/docs/docs.json CHANGED
@@ -62,7 +62,6 @@
62
62
  {
63
63
  "group": "Getting Started",
64
64
  "pages": [
65
- "/getting-started/setup",
66
65
  "/getting-started/vscode",
67
66
  "/getting-started/running",
68
67
  "/getting-started/ci",
@@ -79,7 +78,8 @@
79
78
  "/guide/authentication",
80
79
  "/guide/lifecycle",
81
80
  "/guide/code",
82
- "/guide/environment-variables"
81
+ "/guide/environment-variables",
82
+ "/guide/protips"
83
83
  ]
84
84
  },
85
85
  {
@@ -10,7 +10,7 @@ import YmlWarning from "/snippets/yml-warning.mdx";
10
10
  <Card
11
11
  title="Get the VS Code Extension"
12
12
  icon="link"
13
- href="/getting-started/vscode"
13
+ href="https://marketplace.visualstudio.com/items?itemName=testdriver.testdriver"
14
14
  >
15
15
  Turn your IDE into a powerful computer-use automation tool with our VS Code
16
16
  extension.
@@ -5,14 +5,6 @@ description: "Learn how to customize TestDriver execution with lifecycle files f
5
5
  icon: "arrows-rotate"
6
6
  ---
7
7
 
8
- # Lifecycle Files
9
-
10
- TestDriver supports lifecycle files to customize test execution at different phases. These files allow you to define custom setup, provisioning, and cleanup steps that run automatically during test execution.
11
-
12
- ---
13
-
14
- ## Overview
15
-
16
8
  Lifecycle files are YAML files placed in your repository's `lifecycle/` directory. TestDriver automatically executes these files during the appropriate phases:
17
9
 
18
10
  - **`lifecycle/provision.yaml`**: Executed when a new sandbox is created
@@ -0,0 +1,42 @@
1
+ ---
2
+ title: "Pro Tips"
3
+ sidebarTitle: "Pro Tips"
4
+ description: "Navigate TestDriver like a pro with these tips and tricks."
5
+ icon: "star"
6
+ ---
7
+
8
+ TestDriver is a vision-based testing tool and is not aware of the underlying HTML structure of the page. Pay attention to the following tips to improve your testing experience and reduce test flake.
9
+
10
+ ### Use unique descriptions for `hover-image` and `hover-text`
11
+
12
+ When executing the `hover-image` and `hover-text` command, TestDriver may use the description to identify the image, especially if multiple images are visible. If multiple images on the screen match the description, TestDriver may not be able to determine which one to interact with. To improve accuracy, use specific and unique descriptions for images.
13
+
14
+ TestDriver may not accurately locate matches in certain cases. This can lead to unexpected behavior when using the `hover-image` or `hover-text` command. For example, if you specify a description that's too generic or similar to other images on the screen, TestDriver may not be able to identify the correct image to interact with. To improve accuracy, use specific and unique descriptions for images and text.
15
+ When executing the `hover-image` and `hover-text` commands, TestDriver uses the provided description to identify the target image or text. If the description is too generic or matches multiple elements on the screen, TestDriver may not be able to determine which one to interact with, leading to unexpected behavior. To improve accuracy, always use specific and unique descriptions for both images and text.
16
+ ### Avoid using `hover-text` for single characters (ex: `1`, `>`, `|`)
17
+
18
+ To improve accuracy of our model, we do not include single characters or symbols in the matching data. This means that `hover-text` may not work as expected for these cases. If you need to interact with single characters or symbols, consider using the `match-image` command with a screenshot of the target element instead.
19
+
20
+ ### Use `<label>`s to match empty text inputs.
21
+
22
+ To focus empty text inputs, prefer to `hover-text` on the label associated with the input. This [should](https://www.w3.org/WAI/tutorials/forms/labels/) focus the input field.
23
+
24
+ This is more reliable than trying to match the empty input itself, which may not have any visible text. You can always use `match-image` to match the input field if needed.
25
+
26
+ ### Provide strict criteria for `assert`
27
+
28
+ Assertions use the current state of the page to determine if the step passes or fails. `assert` is not aware of the past, so avoid writing "assert" steps that rely on previous state. Instead, use `assert` to check the current state of the page against strict criteria.
29
+
30
+ ## Reporting issues
31
+
32
+ If you encounter any issues while using TestDriver, please report them to us. You can do this by creating a new issue in the [Feedback Forum](https://forums.testdriver.ai/c/feedback/10). We appreciate your feedback and will work to resolve any problems as quickly as possible.
33
+
34
+ <Card
35
+ title="Report an Issue"
36
+ icon="arrow-turn-down-right"
37
+ iconType="duotone"
38
+ horizontal
39
+ href="https://forums.testdriver.ai/c/feedback/10"
40
+ >
41
+ We'd love to hear from you!
42
+ </Card>
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "testdriverai",
3
- "version": "6.0.15",
3
+ "version": "6.0.16-canary.e337e67.0",
4
4
  "description": "Next generation autonomous AI agent for end-to-end testing of web & desktop",
5
5
  "main": "index.js",
6
6
  "bin": {
@@ -1,7 +1,27 @@
1
- version: 5.7.7
2
- session: 682f66241f5b5f003ee139cb
1
+ version: 6.0.11
2
+ session: 689a163a8d474e20a6142e70
3
3
  steps:
4
- - prompt: remember the username of the user who cannot login
4
+ - prompt: focus chrome, remember the password, enter the username and the remembered password and login
5
5
  commands:
6
- - command: focus-application
7
- name: Google Chrome
6
+ - command: remember
7
+ description: the password
8
+ output: my_password
9
+ - command: hover-text
10
+ text: Username
11
+ description: username input field
12
+ action: click
13
+ - command: type
14
+ text: standard_user
15
+ - command: press-keys
16
+ keys:
17
+ - tab
18
+ - command: type
19
+ text: ${OUTPUT.my_password}
20
+ - command: press-keys
21
+ keys:
22
+ - tab
23
+ - command: press-keys
24
+ keys:
25
+ - enter
26
+ - command: assert
27
+ expect: the homepage is visible
@@ -22,16 +22,33 @@ steps:
22
22
 
23
23
  # Define paths
24
24
  $extensionPath = Join-Path (Get-Location) "node_modules/dashcam-chrome/build"
25
- $profilePath = Join-Path $env:TEMP "chrome-profile-$(Get-Random)"
25
+ $profilePath = Join-Path $env:TEMP "chrome-profile-$(Get-Random)"
26
+ $defaultDir = Join-Path $profilePath "Default"
27
+ $prefsPath = Join-Path $defaultDir "Preferences"
26
28
 
27
29
  Write-Host "Extension path: $extensionPath"
28
30
  Write-Host "Chrome user data dir: $profilePath"
29
31
 
30
- # Validate extension path
31
- if (-not (Test-Path $extensionPath)) {
32
- Write-Host "Extension not found at $extensionPath"
33
- }
32
+ # Create a clean profile + Preferences to disable password manager & autofill
33
+ Remove-Item $profilePath -Recurse -Force -ErrorAction SilentlyContinue
34
+ New-Item $defaultDir -ItemType Directory -Force | Out-Null
34
35
 
36
+ $prefs = @{
37
+ "credentials_enable_service" = $false
38
+ "profile" = @{
39
+ "password_manager_enabled" = $false
40
+ }
41
+ "autofill" = @{
42
+ "profile_enabled" = $false
43
+ "address_enabled" = $false
44
+ "credit_card_enabled" = $false
45
+ }
46
+ } | ConvertTo-Json -Depth 6
47
+
48
+ # Write Preferences before Chrome starts (ASCII/UTF8 is fine)
49
+ $prefs | Set-Content -Path $prefsPath -Encoding ASCII
50
+
51
+ # Build args
35
52
  $chromeArgs = @(
36
53
  "--start-maximized",
37
54
  "--load-extension=$extensionPath",
@@ -43,9 +60,6 @@ steps:
43
60
  ) -join ' '
44
61
 
45
62
  Start-Process "cmd.exe" -ArgumentList "/c", "npx @puppeteer/browsers launch chrome -- $chromeArgs"
46
-
47
- Write-Host "Script complete."
48
- exit
49
63
  - command: wait-for-text
50
64
  text: ${TD_WEBSITE}
51
65
  timeout: 60000
@@ -1,20 +0,0 @@
1
- ---
2
- title: "Known Issues"
3
- sidebarTitle: "Known Issues"
4
- description: "Known issues and limitations of TestDriver."
5
- icon: "bug"
6
- ---
7
-
8
- ## Reporting issues
9
-
10
- If you encounter any issues while using TestDriver, please report them to us. You can do this by creating a new issue in our [GitHub repository](https://github.com/testdriverai/testdriverai/issues/new). We appreciate your feedback and will work to resolve any problems as quickly as possible.
11
-
12
- ## Known issues
13
-
14
- ### Image description matches
15
-
16
- TestDriver may not accurately match image descriptions in certain cases. This can lead to unexpected behavior when using the `hover-image` command. For example, if you specify a description that's too generic or similar to other images on the screen, TestDriver may not be able to identify the correct image to interact with. To improve accuracy, use specific and unique descriptions for images.
17
-
18
- ### Text substrings
19
-
20
- TestDriver may not accurately match text substrings in certain cases. This can lead to unexpected behavior when using the `hover-text` command.
@@ -1,114 +0,0 @@
1
-
2
- {
3
- "group": "Features",
4
- "pages": [
5
- "/features/selectorless",
6
- "/features/generation",
7
- "/features/visual-assertions",
8
- "/features/auto-healing",
9
- "/features/cross-platform",
10
- "/features/reusable-snippets",
11
- "/features/parallel-testing",
12
- "/features/github"
13
- ]
14
- },
15
- {
16
- "tab": "Examples",
17
- "groups": [
18
- {
19
- "group": "Scenarios",
20
- "pages": [
21
- "cookie-banner",
22
- "log-in",
23
- "spell-check",
24
- "form-filling",
25
- "file-upload",
26
- "pdf-generation",
27
- "ai-chatbot"
28
- ]
29
- },
30
- {
31
- "group": "Apps",
32
- "pages": [
33
- "static-websites",
34
- "chrome-extensions",
35
- "mobile-apps",
36
- "desktop-apps"
37
- ]
38
- }]
39
- },
40
- {
41
- "tab": "Integrations",
42
- "groups": [
43
- {
44
- "group": "Buidling",
45
- "pages": [
46
- "/integrations/vercel",
47
- "/integrations/netlify",
48
- "/integrations/electron"
49
- ]
50
- },
51
- {
52
- "group": "Importing",
53
- "pages": [
54
- "/importing/testrail",
55
- "/importing/jira",
56
- "/importing/gherkin",
57
- "/importing/csv"
58
- ]
59
- },
60
- {
61
- "group": "Exporting",
62
- "pages": [
63
- "/exploting/playwright"
64
- ]
65
- },
66
- {
67
- "group": "Bug Reporting",
68
- "pages": [
69
- "/bugs/jira"
70
- ]
71
- }]
72
- },
73
-
74
-
75
-
76
- {
77
- "tab": "Guide",
78
- "groups": [
79
- {
80
- "group": "Getting Started",
81
- "pages": [
82
- "/getting-started/vscode",
83
- "/getting-started/setup",
84
- "/getting-started/writing",
85
- "/getting-started/generating",
86
- "/getting-started/running",
87
- "/getting-started/ci"
88
- ]
89
- },
90
- {
91
- "group": "Writing Tests",
92
- "pages": [
93
- "/guide/locating",
94
- "/guide/waiting",
95
- "/guide/assertions",
96
- "/guide/variables",
97
- "/guide/authentication",
98
- "/guide/setup-teardown",
99
- "/guide/code"
100
- ]
101
- },
102
- {
103
- "group": "Running in CI",
104
- "pages": [
105
- "/action/setup",
106
- "/action/browser",
107
- "/action/os",
108
- "/action/output",
109
- "/action/performance",
110
- "/action/prerun",
111
- "/action/secrets"
112
- ]
113
- }]
114
- },
@@ -1,85 +0,0 @@
1
- ---
2
- title: macOS Automation Limitations
3
- description: Understanding the limitations of synthetic input automation on macOS when using tools like robot.js in TestDriver.
4
- ---
5
-
6
- # macOS Automation Limitations
7
-
8
- macOS includes several system-level security features that affect how synthetic input—like mouse clicks and keyboard presses generated by automation tools—is interpreted. If you're using tools like `robot.js` for test automation, it's important to understand which applications and interactions may be blocked or ignored by the system.
9
-
10
- ## 🔐 System-Level Protected Apps
11
-
12
- Some macOS applications are **heavily protected** by System Integrity Protection (SIP) and will **not respond to synthetic input**, even if Accessibility permissions are granted.
13
-
14
- Examples include:
15
-
16
- - **Finder**
17
- - **System Settings / System Preferences**
18
- - **Dock**
19
- - **Menu Bar**
20
- - **Login Window**
21
- - **Launchpad**
22
- - **Installer**
23
- - **File Open / Save Dialogs**
24
-
25
- These apps typically ignore input generated by tools like `robot.js` or even AppleScript unless scripted via system-sanctioned methods.
26
-
27
- <Alert type="warning" title="Heads up">
28
- Synthetic double-clicks often fail in Finder, even when the correct timing and coordinates are used.
29
- </Alert>
30
-
31
- ## 🛡️ TCC-Protected Applications
32
-
33
- Apps protected by Transparency, Consent, and Control (TCC) **can be automated** if the right permissions are granted.
34
-
35
- To enable automation for these apps:
36
-
37
- 1. Go to **System Settings > Privacy & Security > Accessibility**
38
- 2. Add and check the app or terminal running your automation (e.g., Terminal, iTerm, VS Code).
39
- 3. You may also need to grant:
40
- - **Input Monitoring**
41
- - **Screen Recording**
42
- - **Full Disk Access**
43
-
44
- These permissions allow tools like `robot.js` to simulate clicks, keystrokes, and screen interactions in:
45
-
46
- - **Google Chrome**
47
- - **Safari**
48
- - **Slack**
49
- - **Mail**
50
- - **Calendar**
51
- - **Notes**
52
- - **VS Code**
53
- - **Xcode**
54
-
55
- ## ⚙️ Best Targets for UI Automation
56
-
57
- For reliable automation, use `robot.js` or TestDriver with:
58
-
59
- - Chromium-based browsers (Chrome, Brave, Edge)
60
- - Electron apps (Slack, Discord)
61
- - Most third-party GUI apps
62
- - Terminal emulators
63
-
64
- These apps typically respect synthetic input when appropriate permissions are granted.
65
-
66
- ## 🚫 Actions That Can't Be Automated
67
-
68
- macOS enforces strict security policies on certain sensitive UI elements. Automation is **not possible** for:
69
-
70
- - Login screen interactions
71
- - Security permission prompts (e.g., “Allow Screen Recording”)
72
- - Password input in secure dialogs
73
- - Force Quit window
74
- - Crash reporter popups
75
-
76
- ## ✅ Recommendations
77
-
78
- - Always **grant Accessibility permissions** to the app or terminal running `robot.js`.
79
- - Use **AppleScript** for system apps like Finder or System Settings.
80
- - For GUI testing, prefer apps that are not protected by SIP.
81
- - Consider using native tools like **XCUITest** or **Appium** for native macOS apps.
82
-
83
- <Alert type="info" title="Need Help?">
84
- If you're running into automation issues in specific apps, reach out on Discord or contact the TestDriver team.
85
- </Alert>
@@ -1 +0,0 @@
1
- TD_API_KEY=
File without changes
@@ -1,46 +0,0 @@
1
- version: 6.0.0
2
- session: 67f00511acbd9ccac373edf7
3
- steps:
4
- - prompt: download arc browser
5
- commands:
6
- - command: exec
7
- lang: pwsh
8
- code: |
9
- # === Setup ===
10
- $chocoUrl = "https://www.chocolate-doom.org/downloads/3.0.1/chocolate-doom-3.0.1-win32.zip"
11
- $wadUrl = "https://archive.org/download/DOOMIWADFILE/DOOM.WAD"
12
- $basePath = "$env:USERPROFILE\choco-doom"
13
- $extractDir = "$basePath\choco-extracted"
14
- $wadFile = "$basePath\DOOM.WAD"
15
- $chocoExe = "$extractDir\chocolate-doom.exe"
16
- $zipPath = "$basePath\choco.zip"
17
-
18
- # === Ensure clean folders ===
19
- New-Item -ItemType Directory -Force -Path $basePath | Out-Null
20
- New-Item -ItemType Directory -Force -Path $extractDir | Out-Null
21
-
22
- # === Download Chocolate DOOM if missing ===
23
- if (-not (Test-Path $zipPath)) {
24
- Write-Host "Downloading Chocolate DOOM..."
25
- Invoke-WebRequest -Uri $chocoUrl -OutFile $zipPath
26
- }
27
-
28
- # === Extract if not already extracted ===
29
- if (-not (Test-Path $chocoExe)) {
30
- Write-Host "Extracting Chocolate DOOM..."
31
- Expand-Archive -LiteralPath $zipPath -DestinationPath $extractDir -Force
32
- }
33
-
34
- # === Download DOOM.WAD if missing ===
35
- if (-not (Test-Path $wadFile)) {
36
- Write-Host "Downloading DOOM.WAD..."
37
- Invoke-WebRequest -Uri $wadUrl -OutFile $wadFile
38
- }
39
-
40
- # === Set SDL variables for software rendering ===
41
- $env:SDL_VIDEODRIVER = "windows"
42
- $env:SDL_RENDER_DRIVER = "software"
43
-
44
- # === Launch DOOM ===
45
- Write-Host "Launching Chocolate DOOM with software rendering..."
46
- Start-Process -FilePath $chocoExe -ArgumentList "-iwad `"$wadFile`""
@@ -1 +0,0 @@
1
- Can launch doom, but can not interact with it (probably due to emulation).
@@ -1,3 +0,0 @@
1
- # TBD
2
-
3
- - Alex is working on this
File without changes
@@ -1,19 +0,0 @@
1
- version: 6.0.0
2
- session: 67f00511acbd9ccac373edf7
3
- steps:
4
- - prompt: download and run npm
5
- commands:
6
- - command: exec
7
- lang: pwsh
8
- code: |
9
- # Download and install Chocolatey:
10
- powershell -c "irm https://community.chocolatey.org/install.ps1|iex"
11
-
12
- # Download and install Node.js:
13
- choco install nodejs --version="22.17.1"
14
-
15
- # Verify the Node.js version:
16
- node -v # Should print "v22.17.1".
17
-
18
- # Verify npm version:
19
- npm -v # Should print "10.9.2".
@@ -1,40 +0,0 @@
1
- version: 6.0.0
2
- session: 67f00511acbd9ccac373edf7
3
- steps:
4
- - prompt: download and run npm
5
- commands:
6
- - command: exec
7
- lang: pwsh
8
- code: |
9
- # Set the output log file path
10
- $logFile = "$PSScriptRoot\SystemUsageLog.csv"
11
-
12
- # Write header if the log file does not exist
13
- if (!(Test-Path $logFile)) {
14
- "Timestamp,CPU_Usage_Percent,Memory_Used_MB,Memory_Total_MB,Memory_Usage_Percent" | Out-File $logFile
15
- }
16
-
17
- # Monitor loop
18
- while ($true) {
19
- $timestamp = Get-Date -Format "yyyy-MM-dd HH:mm:ss"
20
-
21
- # Get CPU usage
22
- $cpuUsage = Get-Counter '\Processor(_Total)\% Processor Time'
23
- $cpuValue = [math]::Round($cpuUsage.CounterSamples.CookedValue, 2)
24
-
25
- # Get memory usage
26
- $comp = Get-CimInstance Win32_OperatingSystem
27
- $totalMemory = [math]::Round($comp.TotalVisibleMemorySize / 1024, 2)
28
- $freeMemory = [math]::Round($comp.FreePhysicalMemory / 1024, 2)
29
- $usedMemory = [math]::Round($totalMemory - $freeMemory, 2)
30
- $memPercent = [math]::Round(($usedMemory / $totalMemory) * 100, 2)
31
-
32
- # Append log entry
33
- "$timestamp,$cpuValue,$usedMemory,$totalMemory,$memPercent" | Out-File -Append -Encoding utf8 $logFile
34
-
35
- # Output to screen (optional)
36
- Write-Host "$timestamp | CPU: $cpuValue% | RAM: $usedMemory / $totalMemory MB ($memPercent%)"
37
-
38
- # Wait before next poll
39
- Start-Sleep -Seconds 5
40
- }
@@ -1,40 +0,0 @@
1
- version: 6.0.0
2
- session: 67f00511acbd9ccac373edf7
3
- steps:
4
- - prompt: download and run npm
5
- commands:
6
- - command: exec
7
- lang: pwsh
8
- code: |
9
- # Set the output log file path
10
- $logFile = "$PSScriptRoot\SystemUsageLog.csv"
11
-
12
- # Write header if the log file does not exist
13
- if (!(Test-Path $logFile)) {
14
- "Timestamp,CPU_Usage_Percent,Memory_Used_MB,Memory_Total_MB,Memory_Usage_Percent" | Out-File $logFile
15
- }
16
-
17
- # Monitor loop
18
- while ($true) {
19
- $timestamp = Get-Date -Format "yyyy-MM-dd HH:mm:ss"
20
-
21
- # Get CPU usage
22
- $cpuUsage = Get-Counter '\Processor(_Total)\% Processor Time'
23
- $cpuValue = [math]::Round($cpuUsage.CounterSamples.CookedValue, 2)
24
-
25
- # Get memory usage
26
- $comp = Get-CimInstance Win32_OperatingSystem
27
- $totalMemory = [math]::Round($comp.TotalVisibleMemorySize / 1024, 2)
28
- $freeMemory = [math]::Round($comp.FreePhysicalMemory / 1024, 2)
29
- $usedMemory = [math]::Round($totalMemory - $freeMemory, 2)
30
- $memPercent = [math]::Round(($usedMemory / $totalMemory) * 100, 2)
31
-
32
- # Append log entry
33
- "$timestamp,$cpuValue,$usedMemory,$totalMemory,$memPercent" | Out-File -Append -Encoding utf8 $logFile
34
-
35
- # Output to screen (optional)
36
- Write-Host "$timestamp | CPU: $cpuValue% | RAM: $usedMemory / $totalMemory MB ($memPercent%)"
37
-
38
- # Wait before next poll
39
- Start-Sleep -Seconds 5
40
- }
@@ -1 +0,0 @@
1
- Needs `dashcam track` call
@@ -1,2 +0,0 @@
1
- steps:
2
- - prompt: interact with the webpage
@@ -1,28 +0,0 @@
1
- version: 6.0.0
2
- session: 67f00511acbd9ccac373edf7
3
- steps:
4
- - prompt: launch playwright test recorder
5
- commands:
6
- - command: exec
7
- lang: pwsh
8
- code: |
9
- powershell -c "irm https://community.chocolatey.org/install.ps1|iex"
10
-
11
- # Download and install Node.js:
12
- choco install nodejs --version="22.17.1"
13
-
14
- # Verify the Node.js version:
15
- node -v # Should print "v22.17.1".
16
-
17
- # Verify npm version:
18
- npm -v # Should print "10.9.2".
19
-
20
- echo "Step 3: Installing Playwright dependencies"
21
- npx playwright install --with-deps chromium
22
-
23
- echo "Step 4: Generating Playwright script"
24
- $process = Start-Process -FilePath "npx" -ArgumentList "playwright codegen --target playwright-test https://airbnb.com" -PassThru
25
- Write-Output "Started Playwright with PID: $($process.Id)"
26
-
27
- echo "Step 5: Completed"
28
- exit
@@ -1 +0,0 @@
1
- Worked in v5, `npx` may be failing