@testdriverai/agent 7.8.0-test.38
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/settings.local.json +7 -0
- package/.env.example +4 -0
- package/.prettierignore +4 -0
- package/.prettierrc +1 -0
- package/CHANGELOG.md +953 -0
- package/README.md +81 -0
- package/agent/events.js +135 -0
- package/agent/index.js +2450 -0
- package/agent/interface.js +35 -0
- package/agent/lib/analytics.js +22 -0
- package/agent/lib/censorship.js +75 -0
- package/agent/lib/commander.js +246 -0
- package/agent/lib/commands.js +1684 -0
- package/agent/lib/config.js +60 -0
- package/agent/lib/generator.js +91 -0
- package/agent/lib/http.js +144 -0
- package/agent/lib/logger.js +56 -0
- package/agent/lib/outputs.js +29 -0
- package/agent/lib/parser.js +209 -0
- package/agent/lib/redraw.js +386 -0
- package/agent/lib/resources/cursor-2.png +0 -0
- package/agent/lib/sandbox.js +1104 -0
- package/agent/lib/sdk.js +633 -0
- package/agent/lib/session.js +25 -0
- package/agent/lib/source-mapper.js +342 -0
- package/agent/lib/subimage/index.js +77 -0
- package/agent/lib/subimage/opencv.js +69 -0
- package/agent/lib/system.js +204 -0
- package/agent/lib/theme.js +14 -0
- package/agent/lib/valid-version.js +21 -0
- package/agent/lib/validation.js +169 -0
- package/ai/.claude-plugin/plugin.json +9 -0
- package/ai/agents/testdriver.md +638 -0
- package/ai/skills/testdriver-ai/SKILL.md +204 -0
- package/ai/skills/testdriver-assert/SKILL.md +315 -0
- package/ai/skills/testdriver-aws-setup/SKILL.md +448 -0
- package/ai/skills/testdriver-cache/SKILL.md +221 -0
- package/ai/skills/testdriver-caching/SKILL.md +124 -0
- package/ai/skills/testdriver-captcha/SKILL.md +158 -0
- package/ai/skills/testdriver-ci-cd/SKILL.md +602 -0
- package/ai/skills/testdriver-click/SKILL.md +286 -0
- package/ai/skills/testdriver-client/SKILL.md +477 -0
- package/ai/skills/testdriver-cloud/SKILL.md +119 -0
- package/ai/skills/testdriver-customizing-devices/SKILL.md +319 -0
- package/ai/skills/testdriver-dashcam/SKILL.md +418 -0
- package/ai/skills/testdriver-debugging-with-screenshots/SKILL.md +401 -0
- package/ai/skills/testdriver-device-config/SKILL.md +317 -0
- package/ai/skills/testdriver-double-click/SKILL.md +102 -0
- package/ai/skills/testdriver-elements/SKILL.md +605 -0
- package/ai/skills/testdriver-enterprise/SKILL.md +114 -0
- package/ai/skills/testdriver-errors/SKILL.md +246 -0
- package/ai/skills/testdriver-events/SKILL.md +356 -0
- package/ai/skills/testdriver-examples/SKILL.md +7 -0
- package/ai/skills/testdriver-exec/SKILL.md +317 -0
- package/ai/skills/testdriver-find/SKILL.md +829 -0
- package/ai/skills/testdriver-focus-application/SKILL.md +293 -0
- package/ai/skills/testdriver-generating-tests/SKILL.md +36 -0
- package/ai/skills/testdriver-hover/SKILL.md +278 -0
- package/ai/skills/testdriver-locating-elements/SKILL.md +71 -0
- package/ai/skills/testdriver-making-assertions/SKILL.md +32 -0
- package/ai/skills/testdriver-mcp/SKILL.md +7 -0
- package/ai/skills/testdriver-mcp-workflow/SKILL.md +410 -0
- package/ai/skills/testdriver-mouse-down/SKILL.md +161 -0
- package/ai/skills/testdriver-mouse-up/SKILL.md +164 -0
- package/ai/skills/testdriver-parse/SKILL.md +236 -0
- package/ai/skills/testdriver-performing-actions/SKILL.md +54 -0
- package/ai/skills/testdriver-press-keys/SKILL.md +348 -0
- package/ai/skills/testdriver-provision/SKILL.md +331 -0
- package/ai/skills/testdriver-quickstart/SKILL.md +144 -0
- package/ai/skills/testdriver-redraw/SKILL.md +214 -0
- package/ai/skills/testdriver-reusable-code/SKILL.md +249 -0
- package/ai/skills/testdriver-right-click/SKILL.md +123 -0
- package/ai/skills/testdriver-running-tests/SKILL.md +185 -0
- package/ai/skills/testdriver-screenshot/SKILL.md +248 -0
- package/ai/skills/testdriver-screenshots/SKILL.md +184 -0
- package/ai/skills/testdriver-scroll/SKILL.md +335 -0
- package/ai/skills/testdriver-secrets/SKILL.md +115 -0
- package/ai/skills/testdriver-self-hosted/SKILL.md +65 -0
- package/ai/skills/testdriver-test-writer/SKILL.md +448 -0
- package/ai/skills/testdriver-testdriver/SKILL.md +628 -0
- package/ai/skills/testdriver-testdriver-mechanic/SKILL.md +165 -0
- package/ai/skills/testdriver-type/SKILL.md +357 -0
- package/ai/skills/testdriver-variables/SKILL.md +111 -0
- package/ai/skills/testdriver-wait/SKILL.md +50 -0
- package/ai/skills/testdriver-waiting-for-elements/SKILL.md +90 -0
- package/ai/skills/testdriver-what-is-testdriver/SKILL.md +54 -0
- package/bin/testdriverai.js +22 -0
- package/debugger/bg.png +0 -0
- package/debugger/icon.png +0 -0
- package/debugger/index.html +469 -0
- package/debugger/td.png +0 -0
- package/debugger/tray-buffered.png +0 -0
- package/debugger/tray.png +0 -0
- package/docs/GITHUB_COMMENTS.md +330 -0
- package/docs/GITHUB_COMMENTS_ANNOUNCEMENT.md +167 -0
- package/docs/QUICK-START-GITHUB-COMMENTS.md +84 -0
- package/docs/TEST-GITHUB-COMMENTS.md +129 -0
- package/docs/_data/examples-manifest.json +177 -0
- package/docs/_data/examples-manifest.schema.json +41 -0
- package/docs/_scripts/extract-example-urls.js +165 -0
- package/docs/_scripts/generate-examples.js +560 -0
- package/docs/_scripts/generate-skills.js +154 -0
- package/docs/_scripts/link-replacer.js +164 -0
- package/docs/_scripts/upload-docs-to-openai.js +284 -0
- package/docs/changelog.mdx +161 -0
- package/docs/claude-mcp-plugin.mdx +160 -0
- package/docs/docs.json +442 -0
- package/docs/github-integration-setup.md +266 -0
- package/docs/guide/best-practices-polling.mdx +174 -0
- package/docs/images/content/account/newprojectsettings.png +0 -0
- package/docs/images/content/account/projectpage.png +0 -0
- package/docs/images/content/account/projectreplays.png +0 -0
- package/docs/images/content/account/team-manage.png +0 -0
- package/docs/images/content/account/teampage.png +0 -0
- package/docs/images/content/extension/cursor.svg +1 -0
- package/docs/images/content/extension/vscode.svg +57 -0
- package/docs/images/content/extension/windsurf.svg +3 -0
- package/docs/images/content/parse/output.png +0 -0
- package/docs/images/content/self-hosted/launchtemplateid.png +0 -0
- package/docs/images/content/side-by-side.png +0 -0
- package/docs/images/content/vscode/ide-full.png +0 -0
- package/docs/images/content/vscode/running.png +0 -0
- package/docs/images/content/vscode/v7-chat.png +0 -0
- package/docs/images/content/vscode/v7-choose-agent.png +0 -0
- package/docs/images/content/vscode/v7-full.png +0 -0
- package/docs/images/content/vscode/v7-onboarding.png +0 -0
- package/docs/images/content/vscode/vscode-2-assert.png +0 -0
- package/docs/images/content/vscode/vscode-agent-preview.png +0 -0
- package/docs/images/content/vscode/vscode-copilot-ask.png +0 -0
- package/docs/images/content/vscode/vscode-file-creation.png +0 -0
- package/docs/images/content/vscode/vscode-install.png +0 -0
- package/docs/images/content/vscode/vscode-overview.png +0 -0
- package/docs/images/content/vscode/vscode-setup-walkthrough.png +0 -0
- package/docs/images/content/vscode/vscode-stopchat.png +0 -0
- package/docs/images/content/vscode/vscode-stoptest.png +0 -0
- package/docs/images/content/vscode/vscode-tdservice.png +0 -0
- package/docs/images/content/vscode/vscode-test-output.png +0 -0
- package/docs/images/content/vscode/vscode-testhistory.png +0 -0
- package/docs/images/content/vscode/vscode-testpane-runtests.png +0 -0
- package/docs/images/content/vscode/vscode-testpane.png +0 -0
- package/docs/images/template/dark.png +0 -0
- package/docs/images/template/icon.png +0 -0
- package/docs/images/template/light.png +0 -0
- package/docs/snippets/calendar-link.mdx +4 -0
- package/docs/snippets/gitignore-warning.mdx +7 -0
- package/docs/snippets/lifecycle-warning.mdx +6 -0
- package/docs/snippets/test-prereqs.mdx +12 -0
- package/docs/snippets/tests/assert-replay.mdx +7 -0
- package/docs/snippets/tests/assert-yaml.mdx +8 -0
- package/docs/snippets/tests/exec-js-replay.mdx +7 -0
- package/docs/snippets/tests/exec-js-yaml.mdx +32 -0
- package/docs/snippets/tests/exec-shell-replay.mdx +7 -0
- package/docs/snippets/tests/exec-shell-yaml.mdx +15 -0
- package/docs/snippets/tests/hover-image-replay.mdx +7 -0
- package/docs/snippets/tests/hover-image-yaml.mdx +17 -0
- package/docs/snippets/tests/hover-text-replay.mdx +7 -0
- package/docs/snippets/tests/hover-text-with-description-replay.mdx +7 -0
- package/docs/snippets/tests/hover-text-with-description-yaml.mdx +24 -0
- package/docs/snippets/tests/hover-text-yaml.mdx +14 -0
- package/docs/snippets/tests/match-image-replay.mdx +7 -0
- package/docs/snippets/tests/match-image-yaml.mdx +17 -0
- package/docs/snippets/tests/press-keys-replay.mdx +7 -0
- package/docs/snippets/tests/press-keys-yaml.mdx +36 -0
- package/docs/snippets/tests/remember-replay.mdx +7 -0
- package/docs/snippets/tests/remember-yaml.mdx +28 -0
- package/docs/snippets/tests/scroll-replay.mdx +7 -0
- package/docs/snippets/tests/scroll-until-image-replay.mdx +7 -0
- package/docs/snippets/tests/scroll-until-image-yaml.mdx +14 -0
- package/docs/snippets/tests/scroll-until-text-replay.mdx +7 -0
- package/docs/snippets/tests/scroll-until-text-yaml.mdx +17 -0
- package/docs/snippets/tests/scroll-yaml.mdx +30 -0
- package/docs/snippets/tests/type-repeated-replay.mdx +7 -0
- package/docs/snippets/tests/type-repeated-yaml.mdx +22 -0
- package/docs/snippets/tests/type-replay.mdx +7 -0
- package/docs/snippets/tests/type-yaml.mdx +28 -0
- package/docs/snippets/tests/wait-for-image-replay.mdx +7 -0
- package/docs/snippets/tests/wait-for-image-yaml.mdx +18 -0
- package/docs/snippets/tests/wait-for-text-replay.mdx +7 -0
- package/docs/snippets/tests/wait-for-text-yaml.mdx +18 -0
- package/docs/snippets/tests/wait-replay.mdx +7 -0
- package/docs/snippets/tests/wait-yaml.mdx +13 -0
- package/docs/styles.css +65 -0
- package/docs/v6/account/dashboard.mdx +16 -0
- package/docs/v6/account/enterprise.mdx +110 -0
- package/docs/v6/account/pricing.mdx +33 -0
- package/docs/v6/account/projects.mdx +33 -0
- package/docs/v6/account/team.mdx +35 -0
- package/docs/v6/action/ami.mdx +109 -0
- package/docs/v6/action/performance.mdx +105 -0
- package/docs/v6/action/secrets.mdx +93 -0
- package/docs/v6/apps/chrome-extensions.mdx +48 -0
- package/docs/v6/apps/desktop-apps.mdx +93 -0
- package/docs/v6/apps/mobile-apps.mdx +26 -0
- package/docs/v6/apps/static-websites.mdx +54 -0
- package/docs/v6/apps/tauri-apps.mdx +361 -0
- package/docs/v6/bugs/jira.mdx +232 -0
- package/docs/v6/cli/overview.mdx +66 -0
- package/docs/v6/commands/assert.mdx +45 -0
- package/docs/v6/commands/exec.mdx +276 -0
- package/docs/v6/commands/focus-application.mdx +44 -0
- package/docs/v6/commands/hover-image.mdx +69 -0
- package/docs/v6/commands/hover-text.mdx +47 -0
- package/docs/v6/commands/if.mdx +53 -0
- package/docs/v6/commands/match-image.mdx +67 -0
- package/docs/v6/commands/press-keys.mdx +87 -0
- package/docs/v6/commands/remember.mdx +49 -0
- package/docs/v6/commands/run.mdx +44 -0
- package/docs/v6/commands/scroll-until-image.mdx +66 -0
- package/docs/v6/commands/scroll-until-text.mdx +60 -0
- package/docs/v6/commands/scroll.mdx +69 -0
- package/docs/v6/commands/type.mdx +45 -0
- package/docs/v6/commands/wait-for-image.mdx +54 -0
- package/docs/v6/commands/wait-for-text.mdx +48 -0
- package/docs/v6/commands/wait.mdx +45 -0
- package/docs/v6/exporting/junit.mdx +218 -0
- package/docs/v6/exporting/playwright.mdx +197 -0
- package/docs/v6/features/auto-healing.mdx +144 -0
- package/docs/v6/features/generation.mdx +116 -0
- package/docs/v6/features/parallel-testing.mdx +151 -0
- package/docs/v6/features/reusable-snippets.mdx +131 -0
- package/docs/v6/features/selectorless.mdx +80 -0
- package/docs/v6/features/visual-assertions.mdx +139 -0
- package/docs/v6/getting-started/ci.mdx +146 -0
- package/docs/v6/getting-started/cli.mdx +91 -0
- package/docs/v6/getting-started/editing.mdx +100 -0
- package/docs/v6/getting-started/playwright.mdx +342 -0
- package/docs/v6/getting-started/running.mdx +48 -0
- package/docs/v6/getting-started/self-hosting.mdx +408 -0
- package/docs/v6/getting-started/vscode.mdx +88 -0
- package/docs/v6/guide/assertions.mdx +189 -0
- package/docs/v6/guide/authentication.mdx +136 -0
- package/docs/v6/guide/code.mdx +65 -0
- package/docs/v6/guide/dashcam.mdx +118 -0
- package/docs/v6/guide/environment-variables.mdx +26 -0
- package/docs/v6/guide/lifecycle.mdx +242 -0
- package/docs/v6/guide/locating.mdx +141 -0
- package/docs/v6/guide/protips.mdx +43 -0
- package/docs/v6/guide/variables.mdx +143 -0
- package/docs/v6/guide/waiting.mdx +130 -0
- package/docs/v6/importing/csv.mdx +196 -0
- package/docs/v6/importing/gherkin.mdx +143 -0
- package/docs/v6/importing/jira.mdx +164 -0
- package/docs/v6/importing/testrail.mdx +162 -0
- package/docs/v6/integrations/electron.mdx +146 -0
- package/docs/v6/integrations/netlify.mdx +100 -0
- package/docs/v6/integrations/vercel.mdx +125 -0
- package/docs/v6/interactive/explore.mdx +99 -0
- package/docs/v6/interactive/run.mdx +52 -0
- package/docs/v6/interactive/save.mdx +63 -0
- package/docs/v6/overview/comparison.mdx +101 -0
- package/docs/v6/overview/faq.mdx +162 -0
- package/docs/v6/overview/performance.mdx +52 -0
- package/docs/v6/overview/quickstart.mdx +137 -0
- package/docs/v6/overview/what-is-testdriver.mdx +85 -0
- package/docs/v6/scenarios/ai-chatbot.mdx +28 -0
- package/docs/v6/scenarios/cookie-banner.mdx +32 -0
- package/docs/v6/scenarios/file-upload.mdx +33 -0
- package/docs/v6/scenarios/form-filling.mdx +32 -0
- package/docs/v6/scenarios/log-in.mdx +75 -0
- package/docs/v6/scenarios/pdf-generation.mdx +25 -0
- package/docs/v6/scenarios/spell-check.mdx +22 -0
- package/docs/v6/security/action.mdx +84 -0
- package/docs/v6/security/agent.mdx +73 -0
- package/docs/v6/security/platform.mdx +77 -0
- package/docs/v6/tutorials/advanced-test.mdx +81 -0
- package/docs/v6/tutorials/basic-test.mdx +45 -0
- package/docs/v7/_drafts/agents.mdx +843 -0
- package/docs/v7/_drafts/architecture.mdx +399 -0
- package/docs/v7/_drafts/auto-cache-key.mdx +167 -0
- package/docs/v7/_drafts/awesome-logs-quick-ref.mdx +100 -0
- package/docs/v7/_drafts/best-practices.mdx +486 -0
- package/docs/v7/_drafts/caching-ai.mdx +215 -0
- package/docs/v7/_drafts/caching-selectors.mdx +424 -0
- package/docs/v7/_drafts/caching.mdx +366 -0
- package/docs/v7/_drafts/cli-to-sdk-migration.mdx +425 -0
- package/docs/v7/_drafts/commands/assert.mdx +45 -0
- package/docs/v7/_drafts/commands/exec.mdx +276 -0
- package/docs/v7/_drafts/commands/focus-application.mdx +44 -0
- package/docs/v7/_drafts/commands/hover-image.mdx +69 -0
- package/docs/v7/_drafts/commands/hover-text.mdx +47 -0
- package/docs/v7/_drafts/commands/if.mdx +53 -0
- package/docs/v7/_drafts/commands/match-image.mdx +67 -0
- package/docs/v7/_drafts/commands/press-keys.mdx +87 -0
- package/docs/v7/_drafts/commands/remember.mdx +49 -0
- package/docs/v7/_drafts/commands/run.mdx +44 -0
- package/docs/v7/_drafts/commands/scroll-until-image.mdx +66 -0
- package/docs/v7/_drafts/commands/scroll-until-text.mdx +60 -0
- package/docs/v7/_drafts/commands/scroll.mdx +69 -0
- package/docs/v7/_drafts/commands/type.mdx +45 -0
- package/docs/v7/_drafts/commands/wait-for-image.mdx +54 -0
- package/docs/v7/_drafts/commands/wait-for-text.mdx +48 -0
- package/docs/v7/_drafts/commands/wait.mdx +45 -0
- package/docs/v7/_drafts/configuration.mdx +378 -0
- package/docs/v7/_drafts/contributing.mdx +174 -0
- package/docs/v7/_drafts/core.mdx +458 -0
- package/docs/v7/_drafts/dashcam-title-feature.mdx +89 -0
- package/docs/v7/_drafts/debugging.mdx +349 -0
- package/docs/v7/_drafts/error-handling.mdx +501 -0
- package/docs/v7/_drafts/faq.mdx +393 -0
- package/docs/v7/_drafts/hooks.mdx +360 -0
- package/docs/v7/_drafts/init-command.mdx +95 -0
- package/docs/v7/_drafts/installation.mdx +420 -0
- package/docs/v7/_drafts/migration.mdx +562 -0
- package/docs/v7/_drafts/observable.mdx +604 -0
- package/docs/v7/_drafts/playwright.mdx +342 -0
- package/docs/v7/_drafts/plugin-migration.mdx +220 -0
- package/docs/v7/_drafts/powerful.mdx +419 -0
- package/docs/v7/_drafts/presets.mdx +210 -0
- package/docs/v7/_drafts/progressive-disclosure.mdx +230 -0
- package/docs/v7/_drafts/prompt-cache.mdx +200 -0
- package/docs/v7/_drafts/provision.mdx +390 -0
- package/docs/v7/_drafts/quick-start-test-recording.mdx +214 -0
- package/docs/v7/_drafts/readme.mdx +135 -0
- package/docs/v7/_drafts/reports.mdx +414 -0
- package/docs/v7/_drafts/scalable.mdx +763 -0
- package/docs/v7/_drafts/screenshot.mdx +155 -0
- package/docs/v7/_drafts/sdk-awesome-logs.mdx +468 -0
- package/docs/v7/_drafts/sdk-browser-rendering.mdx +167 -0
- package/docs/v7/_drafts/sdk-migration.mdx +474 -0
- package/docs/v7/_drafts/sdk-v7-complete.mdx +345 -0
- package/docs/v7/_drafts/self-hosting.mdx +369 -0
- package/docs/v7/_drafts/test-recording.mdx +382 -0
- package/docs/v7/_drafts/troubleshooting.mdx +526 -0
- package/docs/v7/_drafts/vitest-plugin.mdx +477 -0
- package/docs/v7/_drafts/vitest.mdx +535 -0
- package/docs/v7/_drafts/writing-tests.mdx +25 -0
- package/docs/v7/ai.mdx +205 -0
- package/docs/v7/assert.mdx +316 -0
- package/docs/v7/aws-setup.mdx +449 -0
- package/docs/v7/cache.mdx +223 -0
- package/docs/v7/caching.mdx +128 -0
- package/docs/v7/captcha.mdx +159 -0
- package/docs/v7/ci-cd.mdx +603 -0
- package/docs/v7/click.mdx +287 -0
- package/docs/v7/client.mdx +478 -0
- package/docs/v7/copilot/auto-healing.mdx +265 -0
- package/docs/v7/copilot/creating-tests.mdx +156 -0
- package/docs/v7/copilot/github.mdx +143 -0
- package/docs/v7/copilot/running-tests.mdx +149 -0
- package/docs/v7/copilot/setup.mdx +143 -0
- package/docs/v7/customizing-devices.mdx +319 -0
- package/docs/v7/dashcam.mdx +419 -0
- package/docs/v7/debugging-with-screenshots.mdx +402 -0
- package/docs/v7/device-config.mdx +317 -0
- package/docs/v7/double-click.mdx +102 -0
- package/docs/v7/elements.mdx +606 -0
- package/docs/v7/enterprise.mdx +9 -0
- package/docs/v7/errors.mdx +248 -0
- package/docs/v7/events.mdx +358 -0
- package/docs/v7/examples/ai.mdx +72 -0
- package/docs/v7/examples/assert.mdx +72 -0
- package/docs/v7/examples/captcha-api.mdx +92 -0
- package/docs/v7/examples/chrome-extension.mdx +132 -0
- package/docs/v7/examples/drag-and-drop.mdx +100 -0
- package/docs/v7/examples/element-not-found.mdx +67 -0
- package/docs/v7/examples/exec-output.mdx +85 -0
- package/docs/v7/examples/exec-pwsh.mdx +83 -0
- package/docs/v7/examples/focus-window.mdx +62 -0
- package/docs/v7/examples/hover-image.mdx +94 -0
- package/docs/v7/examples/hover-text.mdx +69 -0
- package/docs/v7/examples/installer.mdx +91 -0
- package/docs/v7/examples/launch-vscode-linux.mdx +101 -0
- package/docs/v7/examples/match-image.mdx +96 -0
- package/docs/v7/examples/press-keys.mdx +92 -0
- package/docs/v7/examples/scroll-keyboard.mdx +79 -0
- package/docs/v7/examples/scroll-until-image.mdx +81 -0
- package/docs/v7/examples/scroll-until-text.mdx +109 -0
- package/docs/v7/examples/scroll.mdx +81 -0
- package/docs/v7/examples/type.mdx +92 -0
- package/docs/v7/examples/windows-installer.mdx +89 -0
- package/docs/v7/exec.mdx +318 -0
- package/docs/v7/find.mdx +830 -0
- package/docs/v7/focus-application.mdx +294 -0
- package/docs/v7/generating-tests.mdx +36 -0
- package/docs/v7/hosted.mdx +158 -0
- package/docs/v7/hover.mdx +279 -0
- package/docs/v7/locating-elements.mdx +71 -0
- package/docs/v7/making-assertions.mdx +32 -0
- package/docs/v7/mcp.mdx +9 -0
- package/docs/v7/mouse-down.mdx +161 -0
- package/docs/v7/mouse-up.mdx +164 -0
- package/docs/v7/parse.mdx +237 -0
- package/docs/v7/performing-actions.mdx +54 -0
- package/docs/v7/press-keys.mdx +349 -0
- package/docs/v7/provision.mdx +333 -0
- package/docs/v7/quickstart.mdx +173 -0
- package/docs/v7/redraw.mdx +216 -0
- package/docs/v7/reusable-code.mdx +249 -0
- package/docs/v7/right-click.mdx +123 -0
- package/docs/v7/running-tests.mdx +185 -0
- package/docs/v7/screenshot.mdx +249 -0
- package/docs/v7/screenshots.mdx +186 -0
- package/docs/v7/scroll.mdx +336 -0
- package/docs/v7/secrets.mdx +115 -0
- package/docs/v7/self-hosted.mdx +149 -0
- package/docs/v7/type.mdx +358 -0
- package/docs/v7/variables.mdx +111 -0
- package/docs/v7/wait.mdx +52 -0
- package/docs/v7/waiting-for-elements.mdx +90 -0
- package/docs/v7/what-is-testdriver.mdx +54 -0
- package/eslint.config.js +67 -0
- package/examples/ai.test.mjs +31 -0
- package/examples/assert.test.mjs +47 -0
- package/examples/chrome-extension.test.mjs +97 -0
- package/examples/config.mjs +5 -0
- package/examples/element-not-found.test.mjs +27 -0
- package/examples/exec-output.test.mjs +60 -0
- package/examples/exec-pwsh.test.mjs +58 -0
- package/examples/findall-coffee-icons.test.mjs +42 -0
- package/examples/focus-window.test.mjs +37 -0
- package/examples/formatted-logging.test.mjs +27 -0
- package/examples/hover-image.test.mjs +53 -0
- package/examples/hover-text-with-description.test.mjs +57 -0
- package/examples/hover-text.test.mjs +28 -0
- package/examples/installer.test.mjs +50 -0
- package/examples/launch-vscode-linux.test.mjs +55 -0
- package/examples/match-image.test.mjs +55 -0
- package/examples/parse.test.mjs +19 -0
- package/examples/press-keys.test.mjs +44 -0
- package/examples/prompt.test.mjs +34 -0
- package/examples/scroll-keyboard.test.mjs +38 -0
- package/examples/scroll-until-image.test.mjs +40 -0
- package/examples/scroll.test.mjs +42 -0
- package/examples/type.test.mjs +46 -0
- package/examples/windows-installer.test.mjs +54 -0
- package/index.js +2 -0
- package/interfaces/cli/commands/init.js +438 -0
- package/interfaces/cli/commands/setup.js +382 -0
- package/interfaces/cli/lib/base.js +285 -0
- package/interfaces/cli.js +20 -0
- package/interfaces/junit-reporter.js +290 -0
- package/interfaces/logger.js +388 -0
- package/interfaces/readline.js +234 -0
- package/interfaces/shared-test-state.mjs +64 -0
- package/interfaces/vitest-plugin.d.ts +115 -0
- package/interfaces/vitest-plugin.mjs +1698 -0
- package/lib/captcha/solver.js +358 -0
- package/lib/core/Dashcam.js +533 -0
- package/lib/core/index.d.ts +172 -0
- package/lib/core/index.js +12 -0
- package/lib/environments.json +18 -0
- package/lib/github-comment-formatter.js +263 -0
- package/lib/github-comment.mjs +452 -0
- package/lib/init-project.js +575 -0
- package/lib/presets/index.mjs +331 -0
- package/lib/resolve-channel.js +46 -0
- package/lib/sentry.js +417 -0
- package/lib/vitest/hooks.d.ts +57 -0
- package/lib/vitest/hooks.mjs +674 -0
- package/lib/vitest/setup-aws.mjs +247 -0
- package/lib/vitest/setup-self-hosted.mjs +151 -0
- package/lib/vitest/setup.mjs +46 -0
- package/manual/captcha-api.test.mjs +51 -0
- package/manual/drag-and-drop.test.mjs +59 -0
- package/manual/flake-diffthreshold-001.test.mjs +9 -0
- package/manual/flake-diffthreshold-01.test.mjs +9 -0
- package/manual/flake-diffthreshold-05.test.mjs +9 -0
- package/manual/flake-noredraw-cache.test.mjs +9 -0
- package/manual/flake-noredraw-nocache.test.mjs +9 -0
- package/manual/flake-redraw-cache.test.mjs +9 -0
- package/manual/flake-redraw-nocache.test.mjs +9 -0
- package/manual/flake-rocket-match.test.mjs +30 -0
- package/manual/flake-shared.mjs +51 -0
- package/manual/no-provision.test.mjs +31 -0
- package/manual/packer-hover-image.test.mjs +176 -0
- package/manual/scroll-until-text.test.mjs +68 -0
- package/manual/test-init-command.js +223 -0
- package/mcp-server/README.md +322 -0
- package/mcp-server/dist/codegen.d.ts +9 -0
- package/mcp-server/dist/codegen.js +165 -0
- package/mcp-server/dist/mcp-app.html +114 -0
- package/mcp-server/dist/package.json +1 -0
- package/mcp-server/dist/provision-types.d.ts +290 -0
- package/mcp-server/dist/provision-types.js +174 -0
- package/mcp-server/dist/server.d.ts +6 -0
- package/mcp-server/dist/server.mjs +1925 -0
- package/mcp-server/dist/session.d.ts +85 -0
- package/mcp-server/dist/session.js +152 -0
- package/mcp-server/mcp-app.html +28 -0
- package/mcp-server/mcp-config.example.json +19 -0
- package/mcp-server/package-lock.json +4027 -0
- package/mcp-server/package.json +31 -0
- package/mcp-server/src/codegen.ts +189 -0
- package/mcp-server/src/mcp-app.css +360 -0
- package/mcp-server/src/mcp-app.ts +547 -0
- package/mcp-server/src/provision-types.ts +209 -0
- package/mcp-server/src/server.ts +2391 -0
- package/mcp-server/src/session.ts +194 -0
- package/mcp-server/tsconfig.json +16 -0
- package/mcp-server/vite.config.ts +23 -0
- package/package.json +158 -0
- package/schema.json +1046 -0
- package/scripts/generate-skills.js +94 -0
- package/sdk-log-formatter.js +1157 -0
- package/sdk.d.ts +1486 -0
- package/sdk.js +4336 -0
- package/setup/aws/cloudformation.yaml +463 -0
- package/setup/aws/disable-defender.sh +42 -0
- package/setup/aws/install-dev-runner.sh +79 -0
- package/setup/aws/spawn-runner.sh +289 -0
- package/test/captcha-solver.test.mjs +152 -0
- package/test/chrome-remote-debugging.test.mjs +66 -0
- package/test/duckduckgo/experiment.test.mjs +28 -0
- package/test/duckduckgo/setup.test.mjs +29 -0
- package/test/manual/debug-locate-response.js +82 -0
- package/test/manual/reconnect-provision.test.mjs +49 -0
- package/test/manual/test-console-logs.test.mjs +42 -0
- package/test/manual/test-find-api.js +73 -0
- package/test/manual/test-init.sh +54 -0
- package/test/manual/test-prompt-cache.js +97 -0
- package/test/manual/test-provision-auth.mjs +22 -0
- package/test/manual/test-sandbox-render.js +29 -0
- package/test/manual/test-sdk-methods.js +15 -0
- package/test/manual/test-sdk-refactor.js +53 -0
- package/test/manual/test-stack-trace.mjs +57 -0
- package/test/manual/verify-element-api.js +89 -0
- package/test/manual/verify-types.js +0 -0
- package/test/manual-unawaited-promise.test.mjs +31 -0
- package/vitest.config.mjs +58 -0
- package/vitest.runner.config.mjs +33 -0
- package/vscode-extension/.vscodeignore +12 -0
- package/vscode-extension/README.md +94 -0
- package/vscode-extension/media/icon.png +0 -0
- package/vscode-extension/package-lock.json +4126 -0
- package/vscode-extension/package.json +86 -0
- package/vscode-extension/src/extension.ts +829 -0
- package/vscode-extension/testdriverai-0.1.0.vsix +0 -0
- package/vscode-extension/tsconfig.json +16 -0
|
@@ -0,0 +1,162 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "FAQ"
|
|
3
|
+
sidebarTitle: "FAQ"
|
|
4
|
+
description: "Frequently Asked Questions about TestDriver"
|
|
5
|
+
icon: "block-question"
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
## What's TestDriver?
|
|
9
|
+
|
|
10
|
+
TestDriver is an AI-powered testing platform that simulates user interactions to automate end-to-end testing for web, desktop, and mobile applications.
|
|
11
|
+
|
|
12
|
+
## How does TestDriver work?
|
|
13
|
+
|
|
14
|
+
It interprets high-level prompts, interacts with interfaces like a user would, and verifies expected outcomes using visual assertions.
|
|
15
|
+
|
|
16
|
+
## What platforms does TestDriver support?
|
|
17
|
+
|
|
18
|
+
TestDriver supports Windows, Mac, Linux, desktop apps, chrome extensions, web browsers, and mobile interfaces (via emulator or device farm).
|
|
19
|
+
|
|
20
|
+
## Can it be used for exploratory testing?
|
|
21
|
+
|
|
22
|
+
Yes. TestDriver can autonomously navigate the application to generate new test cases.
|
|
23
|
+
|
|
24
|
+
## Can it test desktop applications?
|
|
25
|
+
|
|
26
|
+
Yes. It supports testing native desktop applications by simulating mouse and keyboard input and identifying UI elements.
|
|
27
|
+
|
|
28
|
+
## Can it test mobile apps?
|
|
29
|
+
|
|
30
|
+
Yes, via mobile emulators or integration with device farms.
|
|
31
|
+
|
|
32
|
+
## Can TestDriver generate tests automatically?
|
|
33
|
+
|
|
34
|
+
Yes, it explores the app and creates test cases based on UI flows and user interactions.
|
|
35
|
+
|
|
36
|
+
## Can I create tests from natural language prompts?
|
|
37
|
+
|
|
38
|
+
Yes. You can write high-level instructions in plain language, and TestDriver will interpret and build tests from them.
|
|
39
|
+
|
|
40
|
+
## Can it generate tests from user stories or documentation?
|
|
41
|
+
|
|
42
|
+
Yes. It can use minimal descriptions to produce complete test cases.
|
|
43
|
+
|
|
44
|
+
## Can it turn recorded user sessions into tests?
|
|
45
|
+
|
|
46
|
+
No, an important part of AI-native testing is the intent behind the actions. TestDriver focuses on understanding user goals rather than just recording actions.
|
|
47
|
+
|
|
48
|
+
## What happens when the UI changes?
|
|
49
|
+
|
|
50
|
+
TestDriver adapts using AI—if a button or label changes, it can often infer the correct action without breaking.
|
|
51
|
+
|
|
52
|
+
## Do I need to rewrite tests often?
|
|
53
|
+
|
|
54
|
+
No. TestDriver reduces maintenance by handling common UI changes automatically.
|
|
55
|
+
|
|
56
|
+
## How does it handle flaky tests?
|
|
57
|
+
|
|
58
|
+
TestDriver Enterprise Dashboards provide insights into test stability, helping you identify flaky tests and their causes.
|
|
59
|
+
|
|
60
|
+
## How are tests updated over time?
|
|
61
|
+
|
|
62
|
+
TestDriver will open pull requests in your repository with updated tests when it detects changes in the UI or application behavior. You can also regenerate tests using the original prompts manually.
|
|
63
|
+
|
|
64
|
+
## How does TestDriver report test failures?
|
|
65
|
+
|
|
66
|
+
It provides detailed logs, screenshots, console output, and visual diffs.
|
|
67
|
+
|
|
68
|
+
## What happens when a test fails?
|
|
69
|
+
|
|
70
|
+
It stops execution, flags the failing step, and provides context for debugging.
|
|
71
|
+
|
|
72
|
+
## Can I view why a test failed?
|
|
73
|
+
|
|
74
|
+
Yes. You can view step-by-step logs, network traffic, DOM state, and video playback of the test run.
|
|
75
|
+
|
|
76
|
+
## Can it automatically retry failed actions?
|
|
77
|
+
|
|
78
|
+
Yes. You can configure retry behavior for individual steps or full tests.
|
|
79
|
+
|
|
80
|
+
## Can I run tests in parallel?
|
|
81
|
+
|
|
82
|
+
Yes. TestDriver supports parallel execution using multiple VMs or containers.
|
|
83
|
+
|
|
84
|
+
## Can I track performance metrics during testing?
|
|
85
|
+
|
|
86
|
+
Yes. It can log CPU, memory, load times, and frame rates to help catch performance regressions.
|
|
87
|
+
|
|
88
|
+
## Can it validate non-deterministic output?
|
|
89
|
+
|
|
90
|
+
Yes. It uses AI assertions to verify outcomes even when outputs vary (for example, generated text or dynamic UIs).
|
|
91
|
+
|
|
92
|
+
## Can it test workflows with variable inputs?
|
|
93
|
+
|
|
94
|
+
Yes. It supports data-driven tests using parameterized inputs.
|
|
95
|
+
|
|
96
|
+
## Can it test file uploads and downloads?
|
|
97
|
+
|
|
98
|
+
Yes. TestDriver can interact with file pickers and validate uploaded/downloaded content.
|
|
99
|
+
|
|
100
|
+
## Can it generate tests for PDFs or document output?
|
|
101
|
+
|
|
102
|
+
Yes. It can open and verify generated files for expected text or formatting.
|
|
103
|
+
|
|
104
|
+
## Can I trigger tests based on pull requests or merges?
|
|
105
|
+
|
|
106
|
+
Yes. You can integrate TestDriver with your CI to trigger runs via GitHub Actions or other CI/CD tools.
|
|
107
|
+
|
|
108
|
+
## Does it integrate with CI/CD tools?
|
|
109
|
+
|
|
110
|
+
Yes. TestDriver integrates with pipelines like GitHub Actions, GitLab CI, and CircleCI.
|
|
111
|
+
|
|
112
|
+
## Can I integrate TestDriver with Jira, Slack, etc.?
|
|
113
|
+
|
|
114
|
+
Yes. You can receive alerts and sync test results with third-party tools via API/webhooks.
|
|
115
|
+
|
|
116
|
+
## Does it support cloud and local environments?
|
|
117
|
+
|
|
118
|
+
Yes. You can run tests locally or in the cloud using ephemeral VMs for clean state testing.
|
|
119
|
+
|
|
120
|
+
## Does it work with existing test frameworks?
|
|
121
|
+
|
|
122
|
+
It can complement or convert some existing test cases into its format, though full conversion depends on compatibility.
|
|
123
|
+
|
|
124
|
+
## How does TestDriver measure test coverage?
|
|
125
|
+
|
|
126
|
+
It tracks UI paths, element interaction frequency, and application state changes to infer coverage.
|
|
127
|
+
|
|
128
|
+
## Can it suggest missing test scenarios?
|
|
129
|
+
|
|
130
|
+
Yes. Based on interaction patterns and user behavior, it can propose additional test cases.
|
|
131
|
+
|
|
132
|
+
## Can it analyze test stability over time?
|
|
133
|
+
|
|
134
|
+
Yes. You can view trends in pass/fail rates and test execution consistency.
|
|
135
|
+
|
|
136
|
+
## Is it safe to test sensitive data?
|
|
137
|
+
|
|
138
|
+
Yes. TestDriver supports variable obfuscation, secure containers, and test data isolation.
|
|
139
|
+
|
|
140
|
+
## Can I avoid using production data in tests?
|
|
141
|
+
|
|
142
|
+
Yes. You can configure mock data, sanitize logs, and use test-specific environments.
|
|
143
|
+
|
|
144
|
+
## How does the AI understand what to test?
|
|
145
|
+
|
|
146
|
+
It uses language models to interpret your goals, element names, and interface cues to perform tasks.
|
|
147
|
+
|
|
148
|
+
## Can I adjust how the AI interprets my prompt?
|
|
149
|
+
|
|
150
|
+
Yes. You can rewrite prompts, add constraints, or review and tweak auto-generated steps.
|
|
151
|
+
|
|
152
|
+
## Can I see what the AI is doing behind the scenes?
|
|
153
|
+
|
|
154
|
+
Yes. You can inspect the resolved steps, see element matches, and adjust test flows before execution.
|
|
155
|
+
|
|
156
|
+
## Can I use TestDriver to test new features?
|
|
157
|
+
|
|
158
|
+
Yes. It's great for validating changes, ensuring no regressions, and verifying rollout configurations.
|
|
159
|
+
|
|
160
|
+
## Can it test seasonal or time-based behaviors?
|
|
161
|
+
|
|
162
|
+
Yes. You can schedule tests or run them under specific date/time settings to verify time-sensitive logic.
|
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "Performance"
|
|
3
|
+
sidebarTitle: "Performance"
|
|
4
|
+
description: "Expected Command Performance"
|
|
5
|
+
icon: "rabbit-running"
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
This guide outlines the typical performance characteristics of each TestDriver command based on observed average execution times. These measurements can help guide optimization, identify bottlenecks, and set realistic expectations when designing automated tests.
|
|
9
|
+
|
|
10
|
+
## Fastest Commands
|
|
11
|
+
|
|
12
|
+
These commands execute quickly and can be relied upon for high-frequency usage in complex test sequences.
|
|
13
|
+
|
|
14
|
+
| Command | Avg Duration | Notes |
|
|
15
|
+
| -------------------------------------------- | ------------ | ---------------------------------------------------------------------------------------------- |
|
|
16
|
+
| [`exec`](/commands/exec) | **0.28s** | Fastest command — used for running system-level operations or internal scripting logic. |
|
|
17
|
+
| [`wait-for-image`](/commands/wait-for-image) | 2.21s | Relatively fast if the image is readily present; can be slower if the UI takes time to render. |
|
|
18
|
+
| [`remember`](/commands/remember) | 2.80s | Internal memory operation — used for tracking previous outputs or locations. |
|
|
19
|
+
| [`hover-text`](/commands/hover-text) | 3.11s | Efficient for UI elements with immediate accessibility. |
|
|
20
|
+
| [`scroll`](/commands/scroll) | 3.34s | Smooth and fast in most scrollable containers. |
|
|
21
|
+
| [`assert`](/commands/assert) | 3.47s | Used for validation — usually lightweight unless image or text detection is delayed. |
|
|
22
|
+
|
|
23
|
+
## Medium Performance Commands
|
|
24
|
+
|
|
25
|
+
These are reliable but may involve minor delays due to image processing, UI rendering, or input simulation.
|
|
26
|
+
|
|
27
|
+
| Command | Avg Duration | Notes |
|
|
28
|
+
| -------------------------------------------------- | ------------ | ----------------------------------------------------------------------------- |
|
|
29
|
+
| [`focus-application`](/commands/focus-application) | 4.83s | Includes system-level context switching — may vary by OS and app state. |
|
|
30
|
+
| [`scroll-until-text`](/commands/scroll-until-text) | 5.94s | Slightly slower due to iterative scroll and search logic. |
|
|
31
|
+
| `click` | 6.15s | Includes visual target matching and precise cursor control. |
|
|
32
|
+
| [`press-keys`](/commands/press-keys) | 6.18s | Slightly slower if key sequences involve modifier keys or application delays. |
|
|
33
|
+
| [`type`](/commands/type) | 7.32s | Simulates real typing — intentionally throttled for realism and stability. |
|
|
34
|
+
| [`wait`](/commands/wait) | 7.50s | Direct sleep used for explicit pauses. Use sparingly for faster tests. |
|
|
35
|
+
|
|
36
|
+
## Slower Commands
|
|
37
|
+
|
|
38
|
+
These commands tend to be slower due to intensive image comparison, polling loops, or delays in dynamic content rendering.
|
|
39
|
+
|
|
40
|
+
| Command | Avg Duration | Notes |
|
|
41
|
+
| ------------------------------------------ | ------------ | -------------------------------------------------------------------------------------------------------------------------------------- |
|
|
42
|
+
| [`hover-image`](/commands/hover-image) | 11.95s | Requires locating a target image — performance depends on image quality and rendering time. |
|
|
43
|
+
| [`wait-for-text`](/commands/wait-for-text) | 12.08s | Polls repeatedly for expected text — delay depends on application speed and visibility. |
|
|
44
|
+
| [`match-image`](/commands/match-image) | **16.55s** | Most time-consuming operation — relies on pixel-level image detection which may be affected by resolution, anti-aliasing, and scaling. |
|
|
45
|
+
|
|
46
|
+
---
|
|
47
|
+
|
|
48
|
+
- **Avoid overusing [`match-image`](/commands/match-image)** unless strictly necessary. Prefer [`wait-for-text`](/commands/wait-for-text) or [`hover-text`](/commands/hover-text) when working with text-based UIs.
|
|
49
|
+
- **Use [`remember`](/commands/remember) and [`assert`](/commands/assert) early** in the test to catch failures before expensive commands.
|
|
50
|
+
- **Favor [`exec`](/commands/exec) for background operations** like launching processes or setting up test conditions.
|
|
51
|
+
- **Use [`wait`](/commands/wait) intentionally** and sparingly — prefer dynamic waits ([`wait-for-image`](/commands/wait-for-image), [`wait-for-text`](/commands/wait-for-text)) when possible.
|
|
52
|
+
- **Monitor cumulative test time**. Replacing slower commands can dramatically improve test suite duration.
|
|
@@ -0,0 +1,137 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "Quickstart"
|
|
3
|
+
sidebarTitle: "Quickstart"
|
|
4
|
+
description: "Get started with TestDriver in minutes."
|
|
5
|
+
icon: "gauge-high"
|
|
6
|
+
mode: "wide"
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
<Steps>
|
|
10
|
+
<Step title="Create a TestDriver Account">
|
|
11
|
+
|
|
12
|
+
You will need a [TestDriver Pro](https://app.testdriver.ai/team) account ($20/month) to get an API key.
|
|
13
|
+
|
|
14
|
+
<Card
|
|
15
|
+
title="Sign Up for TestDriver"
|
|
16
|
+
icon="user-plus"
|
|
17
|
+
href="https://app.testdriver.ai/team"
|
|
18
|
+
arrow
|
|
19
|
+
horizontal
|
|
20
|
+
></Card>
|
|
21
|
+
|
|
22
|
+
</Step>
|
|
23
|
+
<Step title="Set up your environment">
|
|
24
|
+
|
|
25
|
+
Copy your API key from [the TestDriver dashboard](https://app.testdriver.ai/team), and set it as an environment variable.
|
|
26
|
+
|
|
27
|
+
<Tabs>
|
|
28
|
+
<Tab title="macOS / Linux">
|
|
29
|
+
```bash Export an environment variable on macOS or Linux systems
|
|
30
|
+
export TD_API_KEY="your_api_key_here"
|
|
31
|
+
```
|
|
32
|
+
</Tab>
|
|
33
|
+
<Tab title="Windows">
|
|
34
|
+
```powershell Export an environment variable in PowerShell
|
|
35
|
+
setx TD_API_KEY "your_api_key_here"
|
|
36
|
+
```
|
|
37
|
+
</Tab>
|
|
38
|
+
</Tabs>
|
|
39
|
+
|
|
40
|
+
<Tip>Using VS Code, Cursor, or Windsurf? [Try our VS Code Extension (beta)](/getting-started/vscode).</Tip>
|
|
41
|
+
|
|
42
|
+
</Step>
|
|
43
|
+
<Step title="Check out an example test">
|
|
44
|
+
|
|
45
|
+
Download the TestDriver GitHub repository and run the example test.
|
|
46
|
+
|
|
47
|
+
```bash
|
|
48
|
+
git clone --depth 1 https://github.com/testdriverai/cli testdriverai
|
|
49
|
+
cd testdriverai/testdriver/acceptance
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
TestDriver tests are written in YAML, a human-readable data format. The `prompt.yaml` file contains an example series of steps for the agent to execute.
|
|
53
|
+
|
|
54
|
+
```yaml testdriver/acceptance/prompt.yaml
|
|
55
|
+
steps:
|
|
56
|
+
- prompt: log in
|
|
57
|
+
- prompt: add an item to the cart
|
|
58
|
+
- prompt: click on the cart icon
|
|
59
|
+
- prompt: complete checkout
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
Each step has a `prompt` that describes what the agent should do. The agent will use the prompt to generate [commands](/commands/assert) that make the tests faster and more reliable the next time you run the test.
|
|
63
|
+
|
|
64
|
+
</Step>
|
|
65
|
+
<Step title="Generate regression test from prompts">
|
|
66
|
+
|
|
67
|
+
Run the following command to run the test file. TestDriver will spawn a virtual machine, launch the sandbox test page, and execute the steps defined in the `prompt.yaml` file.
|
|
68
|
+
|
|
69
|
+
```bash
|
|
70
|
+
npx testdriverai@latest run prompt.yaml --write --heal
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
The `--write` flag tells TestDriver to save any generated commands to the test file, and the `--heal` flag allows TestDriver to recover from unexpected issues during the test run.
|
|
74
|
+
|
|
75
|
+
<Tip>You can use an interactive CLI to generate test steps with the [explore command](/interactive/explore)</Tip>
|
|
76
|
+
|
|
77
|
+
</Step>
|
|
78
|
+
<Step title="Run the generated regression test">
|
|
79
|
+
|
|
80
|
+
After TestDriver has run the exploratory test, you'll see that the `prompt.yaml` file has been updated with commands generated by the agent to make the test faster and more reliable.
|
|
81
|
+
|
|
82
|
+
```yaml
|
|
83
|
+
version: 6.0.0
|
|
84
|
+
steps:
|
|
85
|
+
- prompt: focus chrome
|
|
86
|
+
commands:
|
|
87
|
+
- command: focus-application
|
|
88
|
+
name: Google Chrome
|
|
89
|
+
- prompt: enter a username
|
|
90
|
+
commands:
|
|
91
|
+
- command: hover-text
|
|
92
|
+
text: Username
|
|
93
|
+
description: username input field
|
|
94
|
+
action: click
|
|
95
|
+
- command: type
|
|
96
|
+
text: standard_user
|
|
97
|
+
- prompt: enter a password
|
|
98
|
+
commands:
|
|
99
|
+
- command: hover-text
|
|
100
|
+
text: Password
|
|
101
|
+
description: password input field
|
|
102
|
+
action: click
|
|
103
|
+
- command: type
|
|
104
|
+
text: secret_password
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
The `--write` command tells the agent to save the generated commands to the test file, and the `--heal` command gives the agent permission to recover if something goes wrong.
|
|
108
|
+
|
|
109
|
+
</Step>
|
|
110
|
+
|
|
111
|
+
<Step title="Managing your test suite">
|
|
112
|
+
|
|
113
|
+
There's so much more you can do with TestDriver! Here are some popular features to explore:
|
|
114
|
+
|
|
115
|
+
<Columns cols={2}>
|
|
116
|
+
<Card title="Importing Tests" icon="table" href="/importing/csv">
|
|
117
|
+
Generate tests from other formats like CSV, Gherkin, etc.
|
|
118
|
+
</Card>
|
|
119
|
+
<Card title="Self Healing" icon="bandage" href="/features/auto-healing">
|
|
120
|
+
Automatically update tests when the application changes.
|
|
121
|
+
</Card>
|
|
122
|
+
<Card title="Variable Test Data" icon="square-root-variable" href="/guide/variables">
|
|
123
|
+
Use variables run the same test with different data sets.
|
|
124
|
+
</Card>
|
|
125
|
+
<Card title="Reusable Snippets" icon="repeat" href="/features/reusable-snippets">
|
|
126
|
+
Embed reusable snippets to simplify your test scripts.
|
|
127
|
+
</Card>
|
|
128
|
+
<Card title="Running Custom Code" icon="code" href="/guide/code">
|
|
129
|
+
Use custom powershell or NodeJS code to extend the capabilities of your tests.
|
|
130
|
+
</Card>
|
|
131
|
+
<Card title="Setup & Teardown" icon="wrench" href="/guide/lifecycle">
|
|
132
|
+
Systemically prepare the sandbox before running tests and clean up afterward.
|
|
133
|
+
</Card>
|
|
134
|
+
</Columns>
|
|
135
|
+
|
|
136
|
+
</Step>
|
|
137
|
+
</Steps>
|
|
@@ -0,0 +1,85 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "What Is TestDriver?"
|
|
3
|
+
sidebarTitle: "What Is TestDriver?"
|
|
4
|
+
description: "TestDriver is a computer-use agent for QA testing of user interfaces."
|
|
5
|
+
icon: "circle-info"
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
TestDriver uses AI vision and keyboard and mouse control to automate end-to-end testing. TestDriver is `selectorless`, meaning it doesn't rely on CSS selectors or static analysis to find elements.
|
|
9
|
+
|
|
10
|
+
<CardGroup cols={3}>
|
|
11
|
+
<Card title="Cross-Platform" icon="bolt">
|
|
12
|
+
Test web, mobile, desktop and more with a single tool.
|
|
13
|
+
</Card>
|
|
14
|
+
<Card title="Easy Setup" icon="wrench">
|
|
15
|
+
No need to craft or maintain complex selectors.
|
|
16
|
+
</Card>
|
|
17
|
+
<Card title="Less Maintenance" icon="bandage">
|
|
18
|
+
Tests won't break when code changes.
|
|
19
|
+
</Card>
|
|
20
|
+
</CardGroup>
|
|
21
|
+
|
|
22
|
+
TestDriver is different from other computer-use agents as it produces `YAML` based test script which increases the speed and repeatability of testing.
|
|
23
|
+
|
|
24
|
+
### Selectorless testing
|
|
25
|
+
|
|
26
|
+
Unlike traditional frameworks (for example, Selenium, Playwright), TestDriver doesn't rely on CSS selectors or static analysis. Instead, tests are described in plain English, such as:
|
|
27
|
+
|
|
28
|
+
```
|
|
29
|
+
> Open Google Chrome and search for "testdriver"
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
This means that you can write tests without worrying about the underlying code structure:
|
|
33
|
+
|
|
34
|
+
- Test any user flow on any website in any browser
|
|
35
|
+
- Clone, build, and test any desktop app
|
|
36
|
+
- Render multiple browser windows and popups like 3rd party auth
|
|
37
|
+
- Test `<canvas>`, `<iframe>`, and `<video>` tags with ease
|
|
38
|
+
- Use file selectors to upload files to the browser
|
|
39
|
+
- Resize the browser
|
|
40
|
+
- Test chrome extensions
|
|
41
|
+
- Test integrations between applications
|
|
42
|
+
|
|
43
|
+
### The problem with current approach to end-to-end testing
|
|
44
|
+
|
|
45
|
+
End-to-end is commonly described as the most expensive and time-consuming test method. Right now we write end-to-end tests using complex selectors that are tightly coupled with the code.
|
|
46
|
+
|
|
47
|
+
```js
|
|
48
|
+
const e = await page.$(
|
|
49
|
+
'div[class="product-card"] >> text="Add to Cart" >> nth=2',
|
|
50
|
+
);
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
This tight coupling means developers need to spend time to understand the codebase and maintain the tests every time the code changes. And code is always changing!
|
|
54
|
+
|
|
55
|
+
### End-to-end is about users, not code
|
|
56
|
+
|
|
57
|
+
In end-to-end testing the business priority is usability. All that really matters is that the user can accomplish the goal.
|
|
58
|
+
|
|
59
|
+
TestDriver uses human language to define test requirements. Then our simulated software tester figures out how to accomplish those goals.
|
|
60
|
+
|
|
61
|
+
| Old and Busted (Selectors) | New Hotness (TestDriver) |
|
|
62
|
+
| :--------------------------------------------------------- | :----------------------- |
|
|
63
|
+
| `div[class="product-card"] >> text="Add to Cart" >> nth=2` | buy the 2nd product |
|
|
64
|
+
|
|
65
|
+
These high level instructions are easier to create and maintain because they're loosely coupled from the codebase. We're describing a high level goal, not a low level interaction.
|
|
66
|
+
|
|
67
|
+
The tests will still continue to work even when the junior developer changes .product-card to .product.card or the designers change Add to Cart to Buy Now . The concepts remain the same so the AI will adapt.
|
|
68
|
+
|
|
69
|
+
## How exactly does this work?
|
|
70
|
+
|
|
71
|
+
TestDriver uses a combination of reinforcement learning and computer vision. The context from successful text executions inform future executions. Here's an example of the context our model considers when locating a text match:
|
|
72
|
+
|
|
73
|
+
| Context | What's it? | Touchpoint |
|
|
74
|
+
| :----------------- | :---------------------------------------------------------------------------------- | :------------- |
|
|
75
|
+
| Prompt | Desired outcome | User Input |
|
|
76
|
+
| Screenshot | Image of computer desktop | Runtime |
|
|
77
|
+
| OCR | All possible text found on screen Runtime |
|
|
78
|
+
| Text Similarity | Closely matching text | Runtime |
|
|
79
|
+
| Redraw | Visual difference between previous and current desktop screenshots | Runtime |
|
|
80
|
+
| Network | The current network activity compared to a baseline | Runtime |
|
|
81
|
+
| Execution History | Previous test steps | Runtime |
|
|
82
|
+
| System Information | Platform, Display Size, etc | Runtime |
|
|
83
|
+
| Mouse Position | X, Y coordinates of mouse | Runtime |
|
|
84
|
+
| Description | An elaborate description of the target element including it's position and function | Past Execution |
|
|
85
|
+
| Text | The exact text value clicked | Past Execution |
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "AI Chatbots"
|
|
3
|
+
sidebarTitle: "AI Chatbots"
|
|
4
|
+
description: "Integrate AI chatbots into your testing workflow."
|
|
5
|
+
icon: "message-bot"
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
import TestPrereqs from "/snippets/test-prereqs.mdx";
|
|
9
|
+
|
|
10
|
+
# Testing AI Chatbots with TestDriver
|
|
11
|
+
|
|
12
|
+
TestDriver can be used to test AI chatbots, or any two way assertions like:
|
|
13
|
+
|
|
14
|
+
- Approval workflows
|
|
15
|
+
- Chatbots
|
|
16
|
+
- Email or SMS notifications
|
|
17
|
+
- And more!
|
|
18
|
+
|
|
19
|
+
This scenario is an example of how to set up and run tests for an AI chatbot using TestDriver.
|
|
20
|
+
|
|
21
|
+
<TestPrereqs />
|
|
22
|
+
|
|
23
|
+
## Scenario overview
|
|
24
|
+
|
|
25
|
+
1. Load a website with an AI chatbot.
|
|
26
|
+
2. Type a chat message or ask a question.
|
|
27
|
+
3. Wait for the chatbot to respond.
|
|
28
|
+
4. Verify that the chatbot's response is correct and meets the expected criteria.
|
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "Cookie Banner"
|
|
3
|
+
sidebarTitle: "Cookie Banner"
|
|
4
|
+
description: "Test cookie banners with TestDriver"
|
|
5
|
+
icon: "cookie"
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
import TestPrereqs from "/snippets/test-prereqs.mdx";
|
|
9
|
+
|
|
10
|
+
<iframe
|
|
11
|
+
width="600"
|
|
12
|
+
height="400"
|
|
13
|
+
src="https://www.youtube.com/embed/VUmjDQ4rM9M"
|
|
14
|
+
title="Testing Cookie Banners with TestDriver"
|
|
15
|
+
frameborder="0"
|
|
16
|
+
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
|
|
17
|
+
allowfullscreen
|
|
18
|
+
></iframe>
|
|
19
|
+
|
|
20
|
+
---
|
|
21
|
+
|
|
22
|
+
# Testing Cookie Banners with TestDriver
|
|
23
|
+
|
|
24
|
+
With TestDriver, you can automate the testing of cookie banners on your web application. This scenario demonstrates how to check if a cookie banner appears when a user visits the site and how to interact with it.
|
|
25
|
+
|
|
26
|
+
<TestPrereqs />
|
|
27
|
+
|
|
28
|
+
## Scenario overview
|
|
29
|
+
|
|
30
|
+
1. **Visit the Site**: The test will navigate to the target URL.
|
|
31
|
+
2. **Check for Cookie Banner**: It will verify if the cookie banner is displayed.
|
|
32
|
+
3. **Decline Cookies**: The test will simulate a user clicking the "Decline All" button on the cookie banner.
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "File Upload"
|
|
3
|
+
sidebarTitle: "File Upload"
|
|
4
|
+
description: "Test file upload functionality with TestDriver"
|
|
5
|
+
icon: "upload"
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
import TestPrereqs from "/snippets/test-prereqs.mdx";
|
|
9
|
+
|
|
10
|
+
<iframe
|
|
11
|
+
width="600"
|
|
12
|
+
height="400"
|
|
13
|
+
src="https://www.youtube.com/embed/9fMAJ0AC6Qw"
|
|
14
|
+
title="File Upload Testing with TestDriver"
|
|
15
|
+
frameborder="0"
|
|
16
|
+
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
|
|
17
|
+
allowfullscreen
|
|
18
|
+
></iframe>
|
|
19
|
+
|
|
20
|
+
---
|
|
21
|
+
|
|
22
|
+
## Testing file upload functionality with TestDriver
|
|
23
|
+
|
|
24
|
+
This scenario demonstrates how to automate testing the file upload functionality of a web application using TestDriver's capabilities. The test will check if the file upload process works correctly and if the uploaded file is processed as expected.
|
|
25
|
+
|
|
26
|
+
<TestPrereqs />
|
|
27
|
+
|
|
28
|
+
## Scenario overview
|
|
29
|
+
|
|
30
|
+
1. Load a website with a file upload feature.
|
|
31
|
+
2. Generate a file that will be used for the upload.
|
|
32
|
+
3. Use TestDriver to perform the file upload process.
|
|
33
|
+
4. Check if the uploaded file is processed correctly by the application.
|
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "Form Filling"
|
|
3
|
+
sidebarTitle: "Form Filling"
|
|
4
|
+
description: "Test form filling functionality with TestDriver"
|
|
5
|
+
icon: "list"
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
import TestPrereqs from "/snippets/test-prereqs.mdx";
|
|
9
|
+
|
|
10
|
+
<iframe
|
|
11
|
+
width="600"
|
|
12
|
+
height="400"
|
|
13
|
+
src="https://www.youtube.com/embed/R0xHyuaBy3A"
|
|
14
|
+
title="Form Filling With TestDriver"
|
|
15
|
+
frameborder="0"
|
|
16
|
+
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
|
|
17
|
+
allowfullscreen
|
|
18
|
+
></iframe>
|
|
19
|
+
|
|
20
|
+
---
|
|
21
|
+
|
|
22
|
+
# Testing Form Filling with TestDriver
|
|
23
|
+
|
|
24
|
+
TestDriver provides a powerful solution for testing form filling functionality on web applications. This scenario demonstrates how to automate the process of filling out forms, such as login or registration forms, using TestDriver's capabilities.
|
|
25
|
+
|
|
26
|
+
<TestPrereqs />
|
|
27
|
+
|
|
28
|
+
## Scenario overview
|
|
29
|
+
|
|
30
|
+
1. Open a webpage containing a form (Log in, Registration, etc.).
|
|
31
|
+
2. Use TestDriver to fill in the form fields with test data.
|
|
32
|
+
3. Submit the form and verify the expected outcome (for example, successful login, registration confirmation, redirect etc.).
|
|
@@ -0,0 +1,75 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "Login"
|
|
3
|
+
sidebarTitle: "Login"
|
|
4
|
+
description: "Test login functionality with TestDriver"
|
|
5
|
+
icon: "lock"
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
import TestPrereqs from "/snippets/test-prereqs.mdx";
|
|
9
|
+
|
|
10
|
+
# Testing Login Functionality with TestDriver
|
|
11
|
+
|
|
12
|
+
Test login functionality with TestDriver. This scenario demonstrates how to automate testing the login process for a web application using TestDriver.
|
|
13
|
+
|
|
14
|
+
<TestPrereqs />
|
|
15
|
+
|
|
16
|
+
## Scenario overview
|
|
17
|
+
|
|
18
|
+
1. Visit the login page of the web application.
|
|
19
|
+
2. Enter the username and password into the respective fields (see [Reusable Snippets](/features/reusable-snippets#How-to Create-and-Use-Reusable-Snippets)).
|
|
20
|
+
3. Click the "Login" button.
|
|
21
|
+
4. Verify that the user is redirected to the dashboard or home page after a successful login.
|
|
22
|
+
5. Optionally, check if the user is logged in by verifying the presence of a logout button or user profile information.
|
|
23
|
+
|
|
24
|
+
### Setup your test environment with predefined credentials.
|
|
25
|
+
|
|
26
|
+
This example uses GitHub secret management to store credentials.
|
|
27
|
+
|
|
28
|
+
<Tip>
|
|
29
|
+
You can also use the [`exec`](commands/exec) command to fetch a test user
|
|
30
|
+
dynamically from a database or API.
|
|
31
|
+
</Tip>
|
|
32
|
+
|
|
33
|
+
To use GitHub secrets, create or modify an existing `.env` file in the root of your project and add the following lines:
|
|
34
|
+
|
|
35
|
+
```bash
|
|
36
|
+
TD_TEST_USERNAME=your_test_username
|
|
37
|
+
TD_TEST_PASSWORD=your_test_password
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
2. Create a test file and use the credentials like this:
|
|
41
|
+
|
|
42
|
+
```yaml login.yaml
|
|
43
|
+
version: 6.0.0
|
|
44
|
+
steps:
|
|
45
|
+
- prompt: Log in to the application
|
|
46
|
+
commands:
|
|
47
|
+
- command: hover-text
|
|
48
|
+
text: Email address
|
|
49
|
+
description: email input field label
|
|
50
|
+
action: click
|
|
51
|
+
- command: type
|
|
52
|
+
text: ${TD_USERNAME} # Use environment variable for username
|
|
53
|
+
- command: hover-text
|
|
54
|
+
text: Password
|
|
55
|
+
description: password input field label
|
|
56
|
+
action: click
|
|
57
|
+
- command: type
|
|
58
|
+
text: ${TD_PASSWORD} # Use environment variable for password
|
|
59
|
+
- command: hover-text
|
|
60
|
+
text: Log In
|
|
61
|
+
description: log in button
|
|
62
|
+
action: click
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
3. Run the test using the command line:
|
|
66
|
+
|
|
67
|
+
```bash
|
|
68
|
+
npx testdriverai@latest run login.yaml
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
4. Watch replays in [Your account](https://app.testdriver.ai)
|
|
72
|
+
|
|
73
|
+
### Conclusion
|
|
74
|
+
|
|
75
|
+
In this scenario, we demonstrated how to automate the login process for a web application using TestDriver. By leveraging reusable snippets and environment variables, you can create efficient and maintainable tests for your applications. This approach not only saves time but also ensures that your tests are easily adaptable to changes in the application or test data.
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "PDF Generation"
|
|
3
|
+
sidebarTitle: "PDF Generation"
|
|
4
|
+
description: "Test PDF generation functionality with TestDriver"
|
|
5
|
+
icon: "file-pdf"
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
import TestPrereqs from "/snippets/test-prereqs.mdx";
|
|
9
|
+
|
|
10
|
+
# Testing PDF Generation with TestDriver
|
|
11
|
+
|
|
12
|
+
Test PDF generation functionality with TestDriver. This scenario demonstrates how to automate testing the PDF generation process for a web application using TestDriver's capabilities.
|
|
13
|
+
|
|
14
|
+
<TestPrereqs />
|
|
15
|
+
|
|
16
|
+
## Scenario overview
|
|
17
|
+
|
|
18
|
+
1. Open a webpage or desktop app that will allow you to generate a PDF.
|
|
19
|
+
2. Trigger the PDF generation process (for example, by clicking a button).
|
|
20
|
+
3. Verify that the PDF is generated successfully and contains the expected content.
|
|
21
|
+
4. Optionally, check the file size and format of the generated PDF.
|
|
22
|
+
|
|
23
|
+
## What's next
|
|
24
|
+
|
|
25
|
+
From here you can automate and add coverage to your PDF generation process, removing manual testing from the equation.
|