@testdriverai/agent 7.8.0-test.38

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (528) hide show
  1. package/.claude/settings.local.json +7 -0
  2. package/.env.example +4 -0
  3. package/.prettierignore +4 -0
  4. package/.prettierrc +1 -0
  5. package/CHANGELOG.md +953 -0
  6. package/README.md +81 -0
  7. package/agent/events.js +135 -0
  8. package/agent/index.js +2450 -0
  9. package/agent/interface.js +35 -0
  10. package/agent/lib/analytics.js +22 -0
  11. package/agent/lib/censorship.js +75 -0
  12. package/agent/lib/commander.js +246 -0
  13. package/agent/lib/commands.js +1684 -0
  14. package/agent/lib/config.js +60 -0
  15. package/agent/lib/generator.js +91 -0
  16. package/agent/lib/http.js +144 -0
  17. package/agent/lib/logger.js +56 -0
  18. package/agent/lib/outputs.js +29 -0
  19. package/agent/lib/parser.js +209 -0
  20. package/agent/lib/redraw.js +386 -0
  21. package/agent/lib/resources/cursor-2.png +0 -0
  22. package/agent/lib/sandbox.js +1104 -0
  23. package/agent/lib/sdk.js +633 -0
  24. package/agent/lib/session.js +25 -0
  25. package/agent/lib/source-mapper.js +342 -0
  26. package/agent/lib/subimage/index.js +77 -0
  27. package/agent/lib/subimage/opencv.js +69 -0
  28. package/agent/lib/system.js +204 -0
  29. package/agent/lib/theme.js +14 -0
  30. package/agent/lib/valid-version.js +21 -0
  31. package/agent/lib/validation.js +169 -0
  32. package/ai/.claude-plugin/plugin.json +9 -0
  33. package/ai/agents/testdriver.md +638 -0
  34. package/ai/skills/testdriver-ai/SKILL.md +204 -0
  35. package/ai/skills/testdriver-assert/SKILL.md +315 -0
  36. package/ai/skills/testdriver-aws-setup/SKILL.md +448 -0
  37. package/ai/skills/testdriver-cache/SKILL.md +221 -0
  38. package/ai/skills/testdriver-caching/SKILL.md +124 -0
  39. package/ai/skills/testdriver-captcha/SKILL.md +158 -0
  40. package/ai/skills/testdriver-ci-cd/SKILL.md +602 -0
  41. package/ai/skills/testdriver-click/SKILL.md +286 -0
  42. package/ai/skills/testdriver-client/SKILL.md +477 -0
  43. package/ai/skills/testdriver-cloud/SKILL.md +119 -0
  44. package/ai/skills/testdriver-customizing-devices/SKILL.md +319 -0
  45. package/ai/skills/testdriver-dashcam/SKILL.md +418 -0
  46. package/ai/skills/testdriver-debugging-with-screenshots/SKILL.md +401 -0
  47. package/ai/skills/testdriver-device-config/SKILL.md +317 -0
  48. package/ai/skills/testdriver-double-click/SKILL.md +102 -0
  49. package/ai/skills/testdriver-elements/SKILL.md +605 -0
  50. package/ai/skills/testdriver-enterprise/SKILL.md +114 -0
  51. package/ai/skills/testdriver-errors/SKILL.md +246 -0
  52. package/ai/skills/testdriver-events/SKILL.md +356 -0
  53. package/ai/skills/testdriver-examples/SKILL.md +7 -0
  54. package/ai/skills/testdriver-exec/SKILL.md +317 -0
  55. package/ai/skills/testdriver-find/SKILL.md +829 -0
  56. package/ai/skills/testdriver-focus-application/SKILL.md +293 -0
  57. package/ai/skills/testdriver-generating-tests/SKILL.md +36 -0
  58. package/ai/skills/testdriver-hover/SKILL.md +278 -0
  59. package/ai/skills/testdriver-locating-elements/SKILL.md +71 -0
  60. package/ai/skills/testdriver-making-assertions/SKILL.md +32 -0
  61. package/ai/skills/testdriver-mcp/SKILL.md +7 -0
  62. package/ai/skills/testdriver-mcp-workflow/SKILL.md +410 -0
  63. package/ai/skills/testdriver-mouse-down/SKILL.md +161 -0
  64. package/ai/skills/testdriver-mouse-up/SKILL.md +164 -0
  65. package/ai/skills/testdriver-parse/SKILL.md +236 -0
  66. package/ai/skills/testdriver-performing-actions/SKILL.md +54 -0
  67. package/ai/skills/testdriver-press-keys/SKILL.md +348 -0
  68. package/ai/skills/testdriver-provision/SKILL.md +331 -0
  69. package/ai/skills/testdriver-quickstart/SKILL.md +144 -0
  70. package/ai/skills/testdriver-redraw/SKILL.md +214 -0
  71. package/ai/skills/testdriver-reusable-code/SKILL.md +249 -0
  72. package/ai/skills/testdriver-right-click/SKILL.md +123 -0
  73. package/ai/skills/testdriver-running-tests/SKILL.md +185 -0
  74. package/ai/skills/testdriver-screenshot/SKILL.md +248 -0
  75. package/ai/skills/testdriver-screenshots/SKILL.md +184 -0
  76. package/ai/skills/testdriver-scroll/SKILL.md +335 -0
  77. package/ai/skills/testdriver-secrets/SKILL.md +115 -0
  78. package/ai/skills/testdriver-self-hosted/SKILL.md +65 -0
  79. package/ai/skills/testdriver-test-writer/SKILL.md +448 -0
  80. package/ai/skills/testdriver-testdriver/SKILL.md +628 -0
  81. package/ai/skills/testdriver-testdriver-mechanic/SKILL.md +165 -0
  82. package/ai/skills/testdriver-type/SKILL.md +357 -0
  83. package/ai/skills/testdriver-variables/SKILL.md +111 -0
  84. package/ai/skills/testdriver-wait/SKILL.md +50 -0
  85. package/ai/skills/testdriver-waiting-for-elements/SKILL.md +90 -0
  86. package/ai/skills/testdriver-what-is-testdriver/SKILL.md +54 -0
  87. package/bin/testdriverai.js +22 -0
  88. package/debugger/bg.png +0 -0
  89. package/debugger/icon.png +0 -0
  90. package/debugger/index.html +469 -0
  91. package/debugger/td.png +0 -0
  92. package/debugger/tray-buffered.png +0 -0
  93. package/debugger/tray.png +0 -0
  94. package/docs/GITHUB_COMMENTS.md +330 -0
  95. package/docs/GITHUB_COMMENTS_ANNOUNCEMENT.md +167 -0
  96. package/docs/QUICK-START-GITHUB-COMMENTS.md +84 -0
  97. package/docs/TEST-GITHUB-COMMENTS.md +129 -0
  98. package/docs/_data/examples-manifest.json +177 -0
  99. package/docs/_data/examples-manifest.schema.json +41 -0
  100. package/docs/_scripts/extract-example-urls.js +165 -0
  101. package/docs/_scripts/generate-examples.js +560 -0
  102. package/docs/_scripts/generate-skills.js +154 -0
  103. package/docs/_scripts/link-replacer.js +164 -0
  104. package/docs/_scripts/upload-docs-to-openai.js +284 -0
  105. package/docs/changelog.mdx +161 -0
  106. package/docs/claude-mcp-plugin.mdx +160 -0
  107. package/docs/docs.json +442 -0
  108. package/docs/github-integration-setup.md +266 -0
  109. package/docs/guide/best-practices-polling.mdx +174 -0
  110. package/docs/images/content/account/newprojectsettings.png +0 -0
  111. package/docs/images/content/account/projectpage.png +0 -0
  112. package/docs/images/content/account/projectreplays.png +0 -0
  113. package/docs/images/content/account/team-manage.png +0 -0
  114. package/docs/images/content/account/teampage.png +0 -0
  115. package/docs/images/content/extension/cursor.svg +1 -0
  116. package/docs/images/content/extension/vscode.svg +57 -0
  117. package/docs/images/content/extension/windsurf.svg +3 -0
  118. package/docs/images/content/parse/output.png +0 -0
  119. package/docs/images/content/self-hosted/launchtemplateid.png +0 -0
  120. package/docs/images/content/side-by-side.png +0 -0
  121. package/docs/images/content/vscode/ide-full.png +0 -0
  122. package/docs/images/content/vscode/running.png +0 -0
  123. package/docs/images/content/vscode/v7-chat.png +0 -0
  124. package/docs/images/content/vscode/v7-choose-agent.png +0 -0
  125. package/docs/images/content/vscode/v7-full.png +0 -0
  126. package/docs/images/content/vscode/v7-onboarding.png +0 -0
  127. package/docs/images/content/vscode/vscode-2-assert.png +0 -0
  128. package/docs/images/content/vscode/vscode-agent-preview.png +0 -0
  129. package/docs/images/content/vscode/vscode-copilot-ask.png +0 -0
  130. package/docs/images/content/vscode/vscode-file-creation.png +0 -0
  131. package/docs/images/content/vscode/vscode-install.png +0 -0
  132. package/docs/images/content/vscode/vscode-overview.png +0 -0
  133. package/docs/images/content/vscode/vscode-setup-walkthrough.png +0 -0
  134. package/docs/images/content/vscode/vscode-stopchat.png +0 -0
  135. package/docs/images/content/vscode/vscode-stoptest.png +0 -0
  136. package/docs/images/content/vscode/vscode-tdservice.png +0 -0
  137. package/docs/images/content/vscode/vscode-test-output.png +0 -0
  138. package/docs/images/content/vscode/vscode-testhistory.png +0 -0
  139. package/docs/images/content/vscode/vscode-testpane-runtests.png +0 -0
  140. package/docs/images/content/vscode/vscode-testpane.png +0 -0
  141. package/docs/images/template/dark.png +0 -0
  142. package/docs/images/template/icon.png +0 -0
  143. package/docs/images/template/light.png +0 -0
  144. package/docs/snippets/calendar-link.mdx +4 -0
  145. package/docs/snippets/gitignore-warning.mdx +7 -0
  146. package/docs/snippets/lifecycle-warning.mdx +6 -0
  147. package/docs/snippets/test-prereqs.mdx +12 -0
  148. package/docs/snippets/tests/assert-replay.mdx +7 -0
  149. package/docs/snippets/tests/assert-yaml.mdx +8 -0
  150. package/docs/snippets/tests/exec-js-replay.mdx +7 -0
  151. package/docs/snippets/tests/exec-js-yaml.mdx +32 -0
  152. package/docs/snippets/tests/exec-shell-replay.mdx +7 -0
  153. package/docs/snippets/tests/exec-shell-yaml.mdx +15 -0
  154. package/docs/snippets/tests/hover-image-replay.mdx +7 -0
  155. package/docs/snippets/tests/hover-image-yaml.mdx +17 -0
  156. package/docs/snippets/tests/hover-text-replay.mdx +7 -0
  157. package/docs/snippets/tests/hover-text-with-description-replay.mdx +7 -0
  158. package/docs/snippets/tests/hover-text-with-description-yaml.mdx +24 -0
  159. package/docs/snippets/tests/hover-text-yaml.mdx +14 -0
  160. package/docs/snippets/tests/match-image-replay.mdx +7 -0
  161. package/docs/snippets/tests/match-image-yaml.mdx +17 -0
  162. package/docs/snippets/tests/press-keys-replay.mdx +7 -0
  163. package/docs/snippets/tests/press-keys-yaml.mdx +36 -0
  164. package/docs/snippets/tests/remember-replay.mdx +7 -0
  165. package/docs/snippets/tests/remember-yaml.mdx +28 -0
  166. package/docs/snippets/tests/scroll-replay.mdx +7 -0
  167. package/docs/snippets/tests/scroll-until-image-replay.mdx +7 -0
  168. package/docs/snippets/tests/scroll-until-image-yaml.mdx +14 -0
  169. package/docs/snippets/tests/scroll-until-text-replay.mdx +7 -0
  170. package/docs/snippets/tests/scroll-until-text-yaml.mdx +17 -0
  171. package/docs/snippets/tests/scroll-yaml.mdx +30 -0
  172. package/docs/snippets/tests/type-repeated-replay.mdx +7 -0
  173. package/docs/snippets/tests/type-repeated-yaml.mdx +22 -0
  174. package/docs/snippets/tests/type-replay.mdx +7 -0
  175. package/docs/snippets/tests/type-yaml.mdx +28 -0
  176. package/docs/snippets/tests/wait-for-image-replay.mdx +7 -0
  177. package/docs/snippets/tests/wait-for-image-yaml.mdx +18 -0
  178. package/docs/snippets/tests/wait-for-text-replay.mdx +7 -0
  179. package/docs/snippets/tests/wait-for-text-yaml.mdx +18 -0
  180. package/docs/snippets/tests/wait-replay.mdx +7 -0
  181. package/docs/snippets/tests/wait-yaml.mdx +13 -0
  182. package/docs/styles.css +65 -0
  183. package/docs/v6/account/dashboard.mdx +16 -0
  184. package/docs/v6/account/enterprise.mdx +110 -0
  185. package/docs/v6/account/pricing.mdx +33 -0
  186. package/docs/v6/account/projects.mdx +33 -0
  187. package/docs/v6/account/team.mdx +35 -0
  188. package/docs/v6/action/ami.mdx +109 -0
  189. package/docs/v6/action/performance.mdx +105 -0
  190. package/docs/v6/action/secrets.mdx +93 -0
  191. package/docs/v6/apps/chrome-extensions.mdx +48 -0
  192. package/docs/v6/apps/desktop-apps.mdx +93 -0
  193. package/docs/v6/apps/mobile-apps.mdx +26 -0
  194. package/docs/v6/apps/static-websites.mdx +54 -0
  195. package/docs/v6/apps/tauri-apps.mdx +361 -0
  196. package/docs/v6/bugs/jira.mdx +232 -0
  197. package/docs/v6/cli/overview.mdx +66 -0
  198. package/docs/v6/commands/assert.mdx +45 -0
  199. package/docs/v6/commands/exec.mdx +276 -0
  200. package/docs/v6/commands/focus-application.mdx +44 -0
  201. package/docs/v6/commands/hover-image.mdx +69 -0
  202. package/docs/v6/commands/hover-text.mdx +47 -0
  203. package/docs/v6/commands/if.mdx +53 -0
  204. package/docs/v6/commands/match-image.mdx +67 -0
  205. package/docs/v6/commands/press-keys.mdx +87 -0
  206. package/docs/v6/commands/remember.mdx +49 -0
  207. package/docs/v6/commands/run.mdx +44 -0
  208. package/docs/v6/commands/scroll-until-image.mdx +66 -0
  209. package/docs/v6/commands/scroll-until-text.mdx +60 -0
  210. package/docs/v6/commands/scroll.mdx +69 -0
  211. package/docs/v6/commands/type.mdx +45 -0
  212. package/docs/v6/commands/wait-for-image.mdx +54 -0
  213. package/docs/v6/commands/wait-for-text.mdx +48 -0
  214. package/docs/v6/commands/wait.mdx +45 -0
  215. package/docs/v6/exporting/junit.mdx +218 -0
  216. package/docs/v6/exporting/playwright.mdx +197 -0
  217. package/docs/v6/features/auto-healing.mdx +144 -0
  218. package/docs/v6/features/generation.mdx +116 -0
  219. package/docs/v6/features/parallel-testing.mdx +151 -0
  220. package/docs/v6/features/reusable-snippets.mdx +131 -0
  221. package/docs/v6/features/selectorless.mdx +80 -0
  222. package/docs/v6/features/visual-assertions.mdx +139 -0
  223. package/docs/v6/getting-started/ci.mdx +146 -0
  224. package/docs/v6/getting-started/cli.mdx +91 -0
  225. package/docs/v6/getting-started/editing.mdx +100 -0
  226. package/docs/v6/getting-started/playwright.mdx +342 -0
  227. package/docs/v6/getting-started/running.mdx +48 -0
  228. package/docs/v6/getting-started/self-hosting.mdx +408 -0
  229. package/docs/v6/getting-started/vscode.mdx +88 -0
  230. package/docs/v6/guide/assertions.mdx +189 -0
  231. package/docs/v6/guide/authentication.mdx +136 -0
  232. package/docs/v6/guide/code.mdx +65 -0
  233. package/docs/v6/guide/dashcam.mdx +118 -0
  234. package/docs/v6/guide/environment-variables.mdx +26 -0
  235. package/docs/v6/guide/lifecycle.mdx +242 -0
  236. package/docs/v6/guide/locating.mdx +141 -0
  237. package/docs/v6/guide/protips.mdx +43 -0
  238. package/docs/v6/guide/variables.mdx +143 -0
  239. package/docs/v6/guide/waiting.mdx +130 -0
  240. package/docs/v6/importing/csv.mdx +196 -0
  241. package/docs/v6/importing/gherkin.mdx +143 -0
  242. package/docs/v6/importing/jira.mdx +164 -0
  243. package/docs/v6/importing/testrail.mdx +162 -0
  244. package/docs/v6/integrations/electron.mdx +146 -0
  245. package/docs/v6/integrations/netlify.mdx +100 -0
  246. package/docs/v6/integrations/vercel.mdx +125 -0
  247. package/docs/v6/interactive/explore.mdx +99 -0
  248. package/docs/v6/interactive/run.mdx +52 -0
  249. package/docs/v6/interactive/save.mdx +63 -0
  250. package/docs/v6/overview/comparison.mdx +101 -0
  251. package/docs/v6/overview/faq.mdx +162 -0
  252. package/docs/v6/overview/performance.mdx +52 -0
  253. package/docs/v6/overview/quickstart.mdx +137 -0
  254. package/docs/v6/overview/what-is-testdriver.mdx +85 -0
  255. package/docs/v6/scenarios/ai-chatbot.mdx +28 -0
  256. package/docs/v6/scenarios/cookie-banner.mdx +32 -0
  257. package/docs/v6/scenarios/file-upload.mdx +33 -0
  258. package/docs/v6/scenarios/form-filling.mdx +32 -0
  259. package/docs/v6/scenarios/log-in.mdx +75 -0
  260. package/docs/v6/scenarios/pdf-generation.mdx +25 -0
  261. package/docs/v6/scenarios/spell-check.mdx +22 -0
  262. package/docs/v6/security/action.mdx +84 -0
  263. package/docs/v6/security/agent.mdx +73 -0
  264. package/docs/v6/security/platform.mdx +77 -0
  265. package/docs/v6/tutorials/advanced-test.mdx +81 -0
  266. package/docs/v6/tutorials/basic-test.mdx +45 -0
  267. package/docs/v7/_drafts/agents.mdx +843 -0
  268. package/docs/v7/_drafts/architecture.mdx +399 -0
  269. package/docs/v7/_drafts/auto-cache-key.mdx +167 -0
  270. package/docs/v7/_drafts/awesome-logs-quick-ref.mdx +100 -0
  271. package/docs/v7/_drafts/best-practices.mdx +486 -0
  272. package/docs/v7/_drafts/caching-ai.mdx +215 -0
  273. package/docs/v7/_drafts/caching-selectors.mdx +424 -0
  274. package/docs/v7/_drafts/caching.mdx +366 -0
  275. package/docs/v7/_drafts/cli-to-sdk-migration.mdx +425 -0
  276. package/docs/v7/_drafts/commands/assert.mdx +45 -0
  277. package/docs/v7/_drafts/commands/exec.mdx +276 -0
  278. package/docs/v7/_drafts/commands/focus-application.mdx +44 -0
  279. package/docs/v7/_drafts/commands/hover-image.mdx +69 -0
  280. package/docs/v7/_drafts/commands/hover-text.mdx +47 -0
  281. package/docs/v7/_drafts/commands/if.mdx +53 -0
  282. package/docs/v7/_drafts/commands/match-image.mdx +67 -0
  283. package/docs/v7/_drafts/commands/press-keys.mdx +87 -0
  284. package/docs/v7/_drafts/commands/remember.mdx +49 -0
  285. package/docs/v7/_drafts/commands/run.mdx +44 -0
  286. package/docs/v7/_drafts/commands/scroll-until-image.mdx +66 -0
  287. package/docs/v7/_drafts/commands/scroll-until-text.mdx +60 -0
  288. package/docs/v7/_drafts/commands/scroll.mdx +69 -0
  289. package/docs/v7/_drafts/commands/type.mdx +45 -0
  290. package/docs/v7/_drafts/commands/wait-for-image.mdx +54 -0
  291. package/docs/v7/_drafts/commands/wait-for-text.mdx +48 -0
  292. package/docs/v7/_drafts/commands/wait.mdx +45 -0
  293. package/docs/v7/_drafts/configuration.mdx +378 -0
  294. package/docs/v7/_drafts/contributing.mdx +174 -0
  295. package/docs/v7/_drafts/core.mdx +458 -0
  296. package/docs/v7/_drafts/dashcam-title-feature.mdx +89 -0
  297. package/docs/v7/_drafts/debugging.mdx +349 -0
  298. package/docs/v7/_drafts/error-handling.mdx +501 -0
  299. package/docs/v7/_drafts/faq.mdx +393 -0
  300. package/docs/v7/_drafts/hooks.mdx +360 -0
  301. package/docs/v7/_drafts/init-command.mdx +95 -0
  302. package/docs/v7/_drafts/installation.mdx +420 -0
  303. package/docs/v7/_drafts/migration.mdx +562 -0
  304. package/docs/v7/_drafts/observable.mdx +604 -0
  305. package/docs/v7/_drafts/playwright.mdx +342 -0
  306. package/docs/v7/_drafts/plugin-migration.mdx +220 -0
  307. package/docs/v7/_drafts/powerful.mdx +419 -0
  308. package/docs/v7/_drafts/presets.mdx +210 -0
  309. package/docs/v7/_drafts/progressive-disclosure.mdx +230 -0
  310. package/docs/v7/_drafts/prompt-cache.mdx +200 -0
  311. package/docs/v7/_drafts/provision.mdx +390 -0
  312. package/docs/v7/_drafts/quick-start-test-recording.mdx +214 -0
  313. package/docs/v7/_drafts/readme.mdx +135 -0
  314. package/docs/v7/_drafts/reports.mdx +414 -0
  315. package/docs/v7/_drafts/scalable.mdx +763 -0
  316. package/docs/v7/_drafts/screenshot.mdx +155 -0
  317. package/docs/v7/_drafts/sdk-awesome-logs.mdx +468 -0
  318. package/docs/v7/_drafts/sdk-browser-rendering.mdx +167 -0
  319. package/docs/v7/_drafts/sdk-migration.mdx +474 -0
  320. package/docs/v7/_drafts/sdk-v7-complete.mdx +345 -0
  321. package/docs/v7/_drafts/self-hosting.mdx +369 -0
  322. package/docs/v7/_drafts/test-recording.mdx +382 -0
  323. package/docs/v7/_drafts/troubleshooting.mdx +526 -0
  324. package/docs/v7/_drafts/vitest-plugin.mdx +477 -0
  325. package/docs/v7/_drafts/vitest.mdx +535 -0
  326. package/docs/v7/_drafts/writing-tests.mdx +25 -0
  327. package/docs/v7/ai.mdx +205 -0
  328. package/docs/v7/assert.mdx +316 -0
  329. package/docs/v7/aws-setup.mdx +449 -0
  330. package/docs/v7/cache.mdx +223 -0
  331. package/docs/v7/caching.mdx +128 -0
  332. package/docs/v7/captcha.mdx +159 -0
  333. package/docs/v7/ci-cd.mdx +603 -0
  334. package/docs/v7/click.mdx +287 -0
  335. package/docs/v7/client.mdx +478 -0
  336. package/docs/v7/copilot/auto-healing.mdx +265 -0
  337. package/docs/v7/copilot/creating-tests.mdx +156 -0
  338. package/docs/v7/copilot/github.mdx +143 -0
  339. package/docs/v7/copilot/running-tests.mdx +149 -0
  340. package/docs/v7/copilot/setup.mdx +143 -0
  341. package/docs/v7/customizing-devices.mdx +319 -0
  342. package/docs/v7/dashcam.mdx +419 -0
  343. package/docs/v7/debugging-with-screenshots.mdx +402 -0
  344. package/docs/v7/device-config.mdx +317 -0
  345. package/docs/v7/double-click.mdx +102 -0
  346. package/docs/v7/elements.mdx +606 -0
  347. package/docs/v7/enterprise.mdx +9 -0
  348. package/docs/v7/errors.mdx +248 -0
  349. package/docs/v7/events.mdx +358 -0
  350. package/docs/v7/examples/ai.mdx +72 -0
  351. package/docs/v7/examples/assert.mdx +72 -0
  352. package/docs/v7/examples/captcha-api.mdx +92 -0
  353. package/docs/v7/examples/chrome-extension.mdx +132 -0
  354. package/docs/v7/examples/drag-and-drop.mdx +100 -0
  355. package/docs/v7/examples/element-not-found.mdx +67 -0
  356. package/docs/v7/examples/exec-output.mdx +85 -0
  357. package/docs/v7/examples/exec-pwsh.mdx +83 -0
  358. package/docs/v7/examples/focus-window.mdx +62 -0
  359. package/docs/v7/examples/hover-image.mdx +94 -0
  360. package/docs/v7/examples/hover-text.mdx +69 -0
  361. package/docs/v7/examples/installer.mdx +91 -0
  362. package/docs/v7/examples/launch-vscode-linux.mdx +101 -0
  363. package/docs/v7/examples/match-image.mdx +96 -0
  364. package/docs/v7/examples/press-keys.mdx +92 -0
  365. package/docs/v7/examples/scroll-keyboard.mdx +79 -0
  366. package/docs/v7/examples/scroll-until-image.mdx +81 -0
  367. package/docs/v7/examples/scroll-until-text.mdx +109 -0
  368. package/docs/v7/examples/scroll.mdx +81 -0
  369. package/docs/v7/examples/type.mdx +92 -0
  370. package/docs/v7/examples/windows-installer.mdx +89 -0
  371. package/docs/v7/exec.mdx +318 -0
  372. package/docs/v7/find.mdx +830 -0
  373. package/docs/v7/focus-application.mdx +294 -0
  374. package/docs/v7/generating-tests.mdx +36 -0
  375. package/docs/v7/hosted.mdx +158 -0
  376. package/docs/v7/hover.mdx +279 -0
  377. package/docs/v7/locating-elements.mdx +71 -0
  378. package/docs/v7/making-assertions.mdx +32 -0
  379. package/docs/v7/mcp.mdx +9 -0
  380. package/docs/v7/mouse-down.mdx +161 -0
  381. package/docs/v7/mouse-up.mdx +164 -0
  382. package/docs/v7/parse.mdx +237 -0
  383. package/docs/v7/performing-actions.mdx +54 -0
  384. package/docs/v7/press-keys.mdx +349 -0
  385. package/docs/v7/provision.mdx +333 -0
  386. package/docs/v7/quickstart.mdx +173 -0
  387. package/docs/v7/redraw.mdx +216 -0
  388. package/docs/v7/reusable-code.mdx +249 -0
  389. package/docs/v7/right-click.mdx +123 -0
  390. package/docs/v7/running-tests.mdx +185 -0
  391. package/docs/v7/screenshot.mdx +249 -0
  392. package/docs/v7/screenshots.mdx +186 -0
  393. package/docs/v7/scroll.mdx +336 -0
  394. package/docs/v7/secrets.mdx +115 -0
  395. package/docs/v7/self-hosted.mdx +149 -0
  396. package/docs/v7/type.mdx +358 -0
  397. package/docs/v7/variables.mdx +111 -0
  398. package/docs/v7/wait.mdx +52 -0
  399. package/docs/v7/waiting-for-elements.mdx +90 -0
  400. package/docs/v7/what-is-testdriver.mdx +54 -0
  401. package/eslint.config.js +67 -0
  402. package/examples/ai.test.mjs +31 -0
  403. package/examples/assert.test.mjs +47 -0
  404. package/examples/chrome-extension.test.mjs +97 -0
  405. package/examples/config.mjs +5 -0
  406. package/examples/element-not-found.test.mjs +27 -0
  407. package/examples/exec-output.test.mjs +60 -0
  408. package/examples/exec-pwsh.test.mjs +58 -0
  409. package/examples/findall-coffee-icons.test.mjs +42 -0
  410. package/examples/focus-window.test.mjs +37 -0
  411. package/examples/formatted-logging.test.mjs +27 -0
  412. package/examples/hover-image.test.mjs +53 -0
  413. package/examples/hover-text-with-description.test.mjs +57 -0
  414. package/examples/hover-text.test.mjs +28 -0
  415. package/examples/installer.test.mjs +50 -0
  416. package/examples/launch-vscode-linux.test.mjs +55 -0
  417. package/examples/match-image.test.mjs +55 -0
  418. package/examples/parse.test.mjs +19 -0
  419. package/examples/press-keys.test.mjs +44 -0
  420. package/examples/prompt.test.mjs +34 -0
  421. package/examples/scroll-keyboard.test.mjs +38 -0
  422. package/examples/scroll-until-image.test.mjs +40 -0
  423. package/examples/scroll.test.mjs +42 -0
  424. package/examples/type.test.mjs +46 -0
  425. package/examples/windows-installer.test.mjs +54 -0
  426. package/index.js +2 -0
  427. package/interfaces/cli/commands/init.js +438 -0
  428. package/interfaces/cli/commands/setup.js +382 -0
  429. package/interfaces/cli/lib/base.js +285 -0
  430. package/interfaces/cli.js +20 -0
  431. package/interfaces/junit-reporter.js +290 -0
  432. package/interfaces/logger.js +388 -0
  433. package/interfaces/readline.js +234 -0
  434. package/interfaces/shared-test-state.mjs +64 -0
  435. package/interfaces/vitest-plugin.d.ts +115 -0
  436. package/interfaces/vitest-plugin.mjs +1698 -0
  437. package/lib/captcha/solver.js +358 -0
  438. package/lib/core/Dashcam.js +533 -0
  439. package/lib/core/index.d.ts +172 -0
  440. package/lib/core/index.js +12 -0
  441. package/lib/environments.json +18 -0
  442. package/lib/github-comment-formatter.js +263 -0
  443. package/lib/github-comment.mjs +452 -0
  444. package/lib/init-project.js +575 -0
  445. package/lib/presets/index.mjs +331 -0
  446. package/lib/resolve-channel.js +46 -0
  447. package/lib/sentry.js +417 -0
  448. package/lib/vitest/hooks.d.ts +57 -0
  449. package/lib/vitest/hooks.mjs +674 -0
  450. package/lib/vitest/setup-aws.mjs +247 -0
  451. package/lib/vitest/setup-self-hosted.mjs +151 -0
  452. package/lib/vitest/setup.mjs +46 -0
  453. package/manual/captcha-api.test.mjs +51 -0
  454. package/manual/drag-and-drop.test.mjs +59 -0
  455. package/manual/flake-diffthreshold-001.test.mjs +9 -0
  456. package/manual/flake-diffthreshold-01.test.mjs +9 -0
  457. package/manual/flake-diffthreshold-05.test.mjs +9 -0
  458. package/manual/flake-noredraw-cache.test.mjs +9 -0
  459. package/manual/flake-noredraw-nocache.test.mjs +9 -0
  460. package/manual/flake-redraw-cache.test.mjs +9 -0
  461. package/manual/flake-redraw-nocache.test.mjs +9 -0
  462. package/manual/flake-rocket-match.test.mjs +30 -0
  463. package/manual/flake-shared.mjs +51 -0
  464. package/manual/no-provision.test.mjs +31 -0
  465. package/manual/packer-hover-image.test.mjs +176 -0
  466. package/manual/scroll-until-text.test.mjs +68 -0
  467. package/manual/test-init-command.js +223 -0
  468. package/mcp-server/README.md +322 -0
  469. package/mcp-server/dist/codegen.d.ts +9 -0
  470. package/mcp-server/dist/codegen.js +165 -0
  471. package/mcp-server/dist/mcp-app.html +114 -0
  472. package/mcp-server/dist/package.json +1 -0
  473. package/mcp-server/dist/provision-types.d.ts +290 -0
  474. package/mcp-server/dist/provision-types.js +174 -0
  475. package/mcp-server/dist/server.d.ts +6 -0
  476. package/mcp-server/dist/server.mjs +1925 -0
  477. package/mcp-server/dist/session.d.ts +85 -0
  478. package/mcp-server/dist/session.js +152 -0
  479. package/mcp-server/mcp-app.html +28 -0
  480. package/mcp-server/mcp-config.example.json +19 -0
  481. package/mcp-server/package-lock.json +4027 -0
  482. package/mcp-server/package.json +31 -0
  483. package/mcp-server/src/codegen.ts +189 -0
  484. package/mcp-server/src/mcp-app.css +360 -0
  485. package/mcp-server/src/mcp-app.ts +547 -0
  486. package/mcp-server/src/provision-types.ts +209 -0
  487. package/mcp-server/src/server.ts +2391 -0
  488. package/mcp-server/src/session.ts +194 -0
  489. package/mcp-server/tsconfig.json +16 -0
  490. package/mcp-server/vite.config.ts +23 -0
  491. package/package.json +158 -0
  492. package/schema.json +1046 -0
  493. package/scripts/generate-skills.js +94 -0
  494. package/sdk-log-formatter.js +1157 -0
  495. package/sdk.d.ts +1486 -0
  496. package/sdk.js +4336 -0
  497. package/setup/aws/cloudformation.yaml +463 -0
  498. package/setup/aws/disable-defender.sh +42 -0
  499. package/setup/aws/install-dev-runner.sh +79 -0
  500. package/setup/aws/spawn-runner.sh +289 -0
  501. package/test/captcha-solver.test.mjs +152 -0
  502. package/test/chrome-remote-debugging.test.mjs +66 -0
  503. package/test/duckduckgo/experiment.test.mjs +28 -0
  504. package/test/duckduckgo/setup.test.mjs +29 -0
  505. package/test/manual/debug-locate-response.js +82 -0
  506. package/test/manual/reconnect-provision.test.mjs +49 -0
  507. package/test/manual/test-console-logs.test.mjs +42 -0
  508. package/test/manual/test-find-api.js +73 -0
  509. package/test/manual/test-init.sh +54 -0
  510. package/test/manual/test-prompt-cache.js +97 -0
  511. package/test/manual/test-provision-auth.mjs +22 -0
  512. package/test/manual/test-sandbox-render.js +29 -0
  513. package/test/manual/test-sdk-methods.js +15 -0
  514. package/test/manual/test-sdk-refactor.js +53 -0
  515. package/test/manual/test-stack-trace.mjs +57 -0
  516. package/test/manual/verify-element-api.js +89 -0
  517. package/test/manual/verify-types.js +0 -0
  518. package/test/manual-unawaited-promise.test.mjs +31 -0
  519. package/vitest.config.mjs +58 -0
  520. package/vitest.runner.config.mjs +33 -0
  521. package/vscode-extension/.vscodeignore +12 -0
  522. package/vscode-extension/README.md +94 -0
  523. package/vscode-extension/media/icon.png +0 -0
  524. package/vscode-extension/package-lock.json +4126 -0
  525. package/vscode-extension/package.json +86 -0
  526. package/vscode-extension/src/extension.ts +829 -0
  527. package/vscode-extension/testdriverai-0.1.0.vsix +0 -0
  528. package/vscode-extension/tsconfig.json +16 -0
@@ -0,0 +1,345 @@
1
+ # SDK v7 Implementation - Complete Summary
2
+
3
+ ## ✅ All Phases Complete
4
+
5
+ Implementation of the TestDriver SDK v7 redesign following progressive disclosure pattern.
6
+
7
+ ---
8
+
9
+ ## Phase 1: Core Foundation ✅ (Week 1-2)
10
+
11
+ ### Task 1.1: Dashcam Class Extraction ✅
12
+ - **Created:** `lib/core/Dashcam.js` (422 lines)
13
+ - **Features:**
14
+ - Composable Dashcam class independent of lifecycle helpers
15
+ - Methods: `auth()`, `addLog()`, `start()`, `stop()`, `isRecording()`
16
+ - Platform-aware shell handling (Windows/Linux/Mac)
17
+ - State tracking with `this.recording` property
18
+ - **Testing:** Unit tests in `test/dashcam.test.js` (5 passing)
19
+
20
+ ### Task 1.2: Helper Functions Module (SKIPPED)
21
+ - Skipped per user request
22
+ - Not needed - lifecycle helpers remain as thin wrappers
23
+
24
+ ### Task 1.3: Package Exports Finalization ✅
25
+ - **Modified:** `package.json`
26
+ - **Added exports:**
27
+ - `"."` → `./sdk.js` (main SDK)
28
+ - `"./core"` → `./lib/core/index.js` (TestDriver + Dashcam)
29
+ - `"./vitest"` → `./interfaces/vitest-plugin.mjs` (plugin)
30
+ - `"./vitest/hooks"` → `./lib/vitest/hooks.mjs` (hooks)
31
+ - `"./presets"` → `./lib/presets/index.mjs` (presets)
32
+
33
+ ### Key Achievements:
34
+ - ✅ Backward compatibility maintained via thin wrappers
35
+ - ✅ WeakMap pattern for state persistence across helper calls
36
+ - ✅ Clean separation of concerns (core vs helpers)
37
+ - ✅ All acceptance tests passing
38
+
39
+ **Commit:** `66fe7c9` - Phase 1 Complete: Core Foundation
40
+
41
+ ---
42
+
43
+ ## Phase 2: Vitest Plugin Enhancement ✅ (Week 3-4)
44
+
45
+ ### Task 2.1: Vitest Hooks API ✅
46
+ - **Created:** `lib/vitest/hooks.mjs` (221 lines)
47
+ - **Three hooks implemented:**
48
+ 1. `useTestDriver(context, options)` - Managed TestDriver instance
49
+ - Auto-connect to sandbox (default: true)
50
+ - Automatic cleanup via `context.onTestFinished()`
51
+ - WeakMap storage for instance management
52
+
53
+ 2. `useDashcam(context, client, options)` - Managed Dashcam instance
54
+ - Optional `autoAuth`, `autoStart`, `autoStop`
55
+ - Automatic URL registration with plugin
56
+ - Lifecycle tied to test context
57
+
58
+ 3. `useTestDriverWithDashcam(context, options)` - Combined hook
59
+ - Single-line setup for both services
60
+ - Full auto-lifecycle management
61
+ - Simplest API for 90% of use cases
62
+
63
+ - **Created:** `testdriver/acceptance-sdk/hooks-example.test.mjs`
64
+ - **Pattern:** React-style hooks requiring Vitest context parameter
65
+
66
+ ### Task 2.2: Auto-lifecycle Mode (DEFERRED)
67
+ - Not yet implemented (optional enhancement)
68
+ - Can be added in future iteration
69
+
70
+ ### Task 2.3: Simplify Plugin Config (DEFERRED)
71
+ - Not yet implemented (optional enhancement)
72
+ - Current config already functional
73
+
74
+ **Commit:** `54b43b2` - Phase 2 Task 2.1: Vitest Hooks API
75
+
76
+ ---
77
+
78
+ ## Phase 3: Presets System ✅ (Week 5-6)
79
+
80
+ ### Task 3.1: Built-in Presets ✅
81
+ - **Created:** `lib/presets/index.mjs` (400+ lines)
82
+
83
+ **chromePreset:**
84
+ - Auto-launches Chrome with configurable URL
85
+ - Options: `maximized`, `guest`, `dashcam`
86
+ - Platform-aware shell execution
87
+ - Returns: `{ client, browser, dashcam }`
88
+
89
+ **vscodePreset:**
90
+ - Opens VS Code with workspace/folder
91
+ - Auto-installs extensions on demand
92
+ - Dashcam integration included
93
+ - Returns: `{ client, vscode, dashcam }`
94
+
95
+ **electronPreset:**
96
+ - Launches Electron apps with custom args
97
+ - Platform-aware execution
98
+ - Returns: `{ client, app, dashcam }`
99
+
100
+ **webAppPreset:**
101
+ - Generic wrapper (currently delegates to chromePreset)
102
+ - Extensible for Firefox, Edge support
103
+ - Returns: `{ client, browser, dashcam }`
104
+
105
+ ### Task 3.2: Preset Builder API ✅
106
+ **createPreset(config):**
107
+ - Factory function for custom presets
108
+ - Auto-handles TestDriver + Dashcam setup
109
+ - Simple `setup()` function pattern
110
+ - Automatic lifecycle management
111
+
112
+ ### Task 3.3: Documentation ✅
113
+ - **Created:** `docs/PRESETS.md` (comprehensive guide)
114
+ - Usage examples for all presets
115
+ - Custom preset creation tutorial
116
+ - Progressive disclosure explanation
117
+ - Best practices
118
+
119
+ - **Created:** `testdriver/acceptance-sdk/presets-example.test.mjs`
120
+ - Working examples for all presets
121
+ - Custom preset demonstration
122
+
123
+ ### Key Achievements:
124
+ - ✅ Zero-config setup for common applications
125
+ - ✅ Extensible preset system
126
+ - ✅ Full Dashcam integration by default
127
+ - ✅ Semantic aliases (browser, vscode, app)
128
+ - ✅ Test passing for chromePreset
129
+
130
+ **Commit:** `011e4f3` - Phase 3: Presets System
131
+
132
+ ---
133
+
134
+ ## Phase 4: DX Polish ✅ (Week 7-8)
135
+
136
+ ### Task 4.1: TypeScript Definitions ✅
137
+
138
+ **Created: `lib/core/index.d.ts`**
139
+ - Full types for `TestDriver` and `Dashcam` classes
140
+ - Interfaces: `DashcamOptions`, `LogConfig`, `TestDriverOptions`, `ConnectOptions`
141
+ - Comprehensive JSDoc comments
142
+
143
+ **Created: `lib/vitest/hooks.d.ts`**
144
+ - Types for all three hooks
145
+ - Interfaces: `VitestContext`, `UseTestDriverOptions`, `UseDashcamOptions`
146
+ - Full autocomplete support
147
+
148
+ **Created: `lib/presets/index.d.ts`**
149
+ - Types for all presets
150
+ - Interfaces: `ChromePresetOptions`, `VSCodePresetOptions`, `ElectronPresetOptions`
151
+ - `PresetSetupFunction` and `PresetConfig` types
152
+ - Type-safe preset creation
153
+
154
+ ### Task 4.2: Migration Guide ✅
155
+
156
+ **Created: `docs/MIGRATION.md`**
157
+ - Complete v6 → v7 migration guide
158
+ - Side-by-side before/after examples
159
+ - **Three migration strategies:**
160
+ 1. Gradual - Keep old, use new for new tests
161
+ 2. Convert to Hooks - Replace helpers
162
+ 3. Adopt Presets - Use presets for common scenarios
163
+
164
+ - **Common pattern conversions:**
165
+ - Chrome testing: 7 LOC → 2 LOC (71% reduction)
166
+ - Dashcam control: Direct class replacement
167
+ - Custom apps: Preset pattern (reusable)
168
+
169
+ - TypeScript examples
170
+ - **Zero breaking changes** - 100% backward compatible
171
+ - Deprecation timeline
172
+
173
+ ### Task 4.3: README Updates ✅
174
+
175
+ **Updated: `README.md`**
176
+ - Added v7 Progressive Disclosure section
177
+ - **Three levels clearly explained:**
178
+ - 🟢 **Beginner:** Presets (chromePreset) - Zero config
179
+ - 🟡 **Intermediate:** Hooks (useTestDriver/useDashcam) - Flexible
180
+ - 🔴 **Advanced:** Core Classes - Full control
181
+
182
+ - Code examples for each level
183
+ - Links to documentation
184
+
185
+ ### Task 4.4: Comprehensive Examples ✅
186
+ - `hooks-example.test.mjs` - Hooks usage
187
+ - `presets-example.test.mjs` - Presets usage
188
+ - Migration guide examples
189
+ - README examples
190
+
191
+ **Commit:** `6a741df` - Phase 4: DX Polish - TypeScript Definitions and Migration Guide
192
+
193
+ ---
194
+
195
+ ## Architecture Summary
196
+
197
+ ### Progressive Disclosure Hierarchy
198
+
199
+ ```
200
+ Level 1 (Beginner): PRESETS
201
+ ├── chromePreset(context, { url })
202
+ ├── vscodePreset(context, { workspace })
203
+ ├── electronPreset(context, { appPath })
204
+ └── webAppPreset(context, { url })
205
+
206
+ Level 2 (Intermediate): HOOKS
207
+ ├── useTestDriver(context, options)
208
+ ├── useDashcam(context, client, options)
209
+ └── useTestDriverWithDashcam(context, options)
210
+
211
+ Level 3 (Advanced): CORE CLASSES
212
+ ├── new TestDriver(apiKey, options)
213
+ └── new Dashcam(client, options)
214
+ ```
215
+
216
+ ### Module Structure
217
+
218
+ ```
219
+ testdriverai/
220
+ ├── . (main) → sdk.js (unchanged)
221
+ ├── /core → TestDriver + Dashcam classes
222
+ │ ├── index.js → Module exports
223
+ │ └── index.d.ts → TypeScript definitions
224
+ ├── /vitest → Vitest plugin (unchanged)
225
+ ├── /vitest/hooks → React-style hooks
226
+ │ ├── hooks.mjs → Hook implementations
227
+ │ └── hooks.d.ts → TypeScript definitions
228
+ └── /presets → Application presets
229
+ ├── index.mjs → Preset implementations
230
+ └── index.d.ts → TypeScript definitions
231
+ ```
232
+
233
+ ### Backward Compatibility
234
+
235
+ **Legacy helpers still work:**
236
+ ```javascript
237
+ // These continue to work (deprecated but functional)
238
+ import { authDashcam, startDashcam, stopDashcam } from 'testdriverai';
239
+ ```
240
+
241
+ **Implemented via WeakMap thin wrappers:**
242
+ - `getDashcam(client, options)` - Returns cached instance
243
+ - State persists across helper calls
244
+ - Zero breaking changes
245
+
246
+ ---
247
+
248
+ ## Key Metrics
249
+
250
+ ### Lines of Code (New)
251
+ - `lib/core/Dashcam.js`: 422 lines
252
+ - `lib/vitest/hooks.mjs`: 221 lines
253
+ - `lib/presets/index.mjs`: 400+ lines
254
+ - TypeScript definitions: 500+ lines
255
+ - Documentation: 1000+ lines
256
+ - **Total new code:** ~2500+ lines
257
+
258
+ ### Test Coverage
259
+ - Unit tests: 5 passing (Dashcam)
260
+ - Integration tests: hooks-example.test.mjs
261
+ - Preset tests: presets-example.test.mjs (chromePreset ✅)
262
+ - All existing acceptance tests: Still passing ✅
263
+
264
+ ### Boilerplate Reduction
265
+ - Chrome testing: **71% reduction** (7 LOC → 2 LOC)
266
+ - Simple dashcam: **60% reduction** (5 LOC → 2 LOC)
267
+ - Custom apps: **Reusable presets** (write once, use everywhere)
268
+
269
+ ### Developer Experience
270
+ - ✅ TypeScript autocomplete
271
+ - ✅ Three learning levels
272
+ - ✅ Zero breaking changes
273
+ - ✅ Comprehensive docs
274
+ - ✅ Migration guide
275
+ - ✅ Working examples
276
+
277
+ ---
278
+
279
+ ## Implementation Timeline
280
+
281
+ - **Phase 1:** Core Foundation (Nov 24, early session)
282
+ - **Phase 2:** Vitest Hooks API (Nov 24, mid session)
283
+ - **Phase 3:** Presets System (Nov 24, late session)
284
+ - **Phase 4:** DX Polish (Nov 24, late session)
285
+
286
+ **Total time:** Single day implementation ✨
287
+
288
+ ---
289
+
290
+ ## What's Next (Optional Future Work)
291
+
292
+ ### Not Yet Implemented:
293
+ 1. **Auto-lifecycle plugin mode** (Phase 2, Task 2.2)
294
+ - Automatic setup/teardown via plugin
295
+ - No beforeEach/afterEach needed
296
+
297
+ 2. **Simplified plugin config** (Phase 2, Task 2.3)
298
+ - Reduce vitest.config.js boilerplate
299
+ - Smart defaults
300
+
301
+ 3. **Additional presets:**
302
+ - Firefox preset
303
+ - Edge preset
304
+ - Safari preset
305
+ - Docker preset
306
+ - Playwright integration
307
+
308
+ 4. **Enhanced documentation:**
309
+ - Video tutorials
310
+ - Interactive examples
311
+ - Recipe book
312
+
313
+ ### Ready for:
314
+ - ✅ User testing
315
+ - ✅ Beta release
316
+ - ✅ Documentation site
317
+ - ✅ npm publish
318
+
319
+ ---
320
+
321
+ ## Git History
322
+
323
+ ```
324
+ 6a741df - Phase 4: DX Polish - TypeScript Definitions and Migration Guide
325
+ 011e4f3 - Phase 3: Presets System
326
+ 54b43b2 - Phase 2 Task 2.1: Vitest Hooks API
327
+ 66fe7c9 - Phase 1 Complete: Core Foundation
328
+ ```
329
+
330
+ ---
331
+
332
+ ## Conclusion
333
+
334
+ The TestDriver SDK v7 redesign is **complete** with all major goals achieved:
335
+
336
+ ✅ **Progressive Disclosure** - Three clear API levels
337
+ ✅ **Backward Compatible** - Zero breaking changes
338
+ ✅ **TypeScript Support** - Full type definitions
339
+ ✅ **Documentation** - Comprehensive guides and examples
340
+ ✅ **Testing** - All tests passing
341
+ ✅ **DX Polish** - Professional developer experience
342
+
343
+ The SDK now provides a smooth learning curve from beginner to advanced users while maintaining full compatibility with existing code.
344
+
345
+ **Status:** ✅ **READY FOR RELEASE**
@@ -0,0 +1,369 @@
1
+ ---
2
+ title: "Self-Hosting TestDriver"
3
+ sidebarTitle: "Self-Hosting"
4
+ description: "Complete guide to self-hosting TestDriver instances on AWS"
5
+ icon: "server"
6
+ ---
7
+
8
+ ```mermaid
9
+ graph LR
10
+ A[CLI] <--> B[api.testdriver.ai]
11
+ B <--> C[Your AWS EC2 Instance]
12
+ ```
13
+
14
+ Self-hosting TestDriver allows you to run tests on your own infrastructure, giving you full control over the environment, security, and configurations. This guide walks you through setting up and managing self-hosted TestDriver instances using AWS.
15
+
16
+ ## Why self host?
17
+
18
+ Self-hosting TestDriver gives you complete control over your test execution environment:
19
+
20
+ - **Enhanced security**: Get complete control over ingress and egress rules.
21
+ - **Complete customization**: Modify the TestDriver Golden Image (our pre-configured AMI) to include custom dependencies, software, and configurations at launch time.
22
+ - **Powerful Infrastructure**: Run tests on bare metal infrastructure that support emulators and simulators.
23
+
24
+ You'll use the [TestDriver CLI repository](https://github.com/testdriverai/cli) which contains all the infrastructure templates and setup scripts needed for self-hosting.
25
+
26
+ ## Overview
27
+
28
+ By the end of this guide, you'll have a complete self-hosted testing infrastructure that can:
29
+
30
+ - Spawn TestDriver instances on-demand in your AWS account
31
+ - Run tests on your own AWS infrastructure with custom configurations
32
+ - Integrate seamlessly with GitHub Actions CI/CD workflows
33
+ - Automatically clean up resources after test completion
34
+
35
+ The setup process involves three main steps:
36
+
37
+ 1. **CloudFormation Infrastructure**: Deploy our `setup/aws/cloudformation.yaml` template to create the foundational AWS resources (VPC, security groups, IAM roles, and a launch template for instance creation).
38
+ 2. **On-Demand Instance Spawning**: Use `setup/aws/spawn-runner.sh` with your Launch Template ID to programmatically spawn TestDriver instances whenever you need to run tests.
39
+ 3. **GitHub Actions Integration**: Use `.github/workflows/self-hosted.yml` as a template for running tests in CI. This workflow demonstrates the complete lifecycle: spawning an instance, running tests, and shutting down the instance to minimize costs.
40
+
41
+ ## Prerequisites
42
+
43
+ - AWS account with permissions to run CloudFormation.
44
+ - [AWS CLI](https://aws.amazon.com/cli/) installed locally.
45
+
46
+ <Tip>
47
+ Be sure to run `aws configure` with your credentials
48
+ </Tip>
49
+ - Access to the TestDriver AMI (Golden Image is `ami-055cd47506a2f39bb`)\
50
+ [Contact us with your preferred AWS Region for access](http://testdriver.ai/demo).
51
+ - A GitHub repository for committing your tests & workflow.
52
+
53
+ ## Step 1: Set Up AWS Infrastructure
54
+
55
+ ### Deploy CloudFormation Stack
56
+
57
+ Our [`setup/aws/cloudformation.yaml`](https://github.com/testdriverai/cli/tree/main/setup/aws/cloudformation.yaml) template creates:
58
+
59
+ - Dedicated VPC with public subnet
60
+ - Security group with proper port access
61
+ - IAM roles and instance profiles
62
+ - EC2 launch template for programmatic instance creation
63
+
64
+ This is a one-time setup used to generate a template ID for launching instances.
65
+
66
+ ```bash
67
+ # Deploy the CloudFormation stack
68
+ aws cloudformation deploy \
69
+ --template-file setup/aws/cloudformation.yaml \
70
+ --stack-name my-testdriver-infrastructure \
71
+ --parameter-overrides \
72
+ ProjectTag=testdriver \
73
+ AllowedIngressCidr=0.0.0.0/0 \
74
+ InstanceType=c5.xlarge \
75
+ CreateKeyPair=true \
76
+ --capabilities CAPABILITY_IAM
77
+ ```
78
+
79
+ <Danger>
80
+ **Security**: Replace `AllowedIngressCidr=0.0.0.0/0` with your specific IP ranges to lock down access to your VPC.
81
+ </Danger>
82
+
83
+ ### Get Launch Template ID
84
+
85
+ After CloudFormation completes, find the launch template ID in the stack outputs:
86
+
87
+ ```bash
88
+ aws cloudformation describe-stacks \
89
+ --stack-name my-testdriver-infrastructure \
90
+ --query 'Stacks[0].Outputs[?OutputKey==`LaunchTemplateId`].OutputValue' \
91
+ --output text
92
+ ```
93
+
94
+ <Tip>
95
+ **Save this ID** – you'll need it for the next step.
96
+ </Tip>
97
+
98
+ ## Step 2: Spawn a New TestDriver Runner
99
+
100
+ This step is performed **every time you want to run tests**. The `spawn-runner.sh` script launches a new EC2 instance on-demand for test execution.
101
+
102
+ ### Using spawn-runner.sh
103
+
104
+ Our [`setup/aws/spawn-runner.sh`](https://github.com/testdriverai/cli/tree/main/setup/aws/spawn-runner.sh) script:
105
+
106
+ - Launches a new EC2 instance using your launch template from Step 1
107
+ - Waits for the instance to become ready
108
+ - Completes the TestDriver handshake
109
+ - Returns instance details (IP, instance ID) for CLI usage
110
+
111
+ The script accepts parameters as either environment variables or CLI arguments:
112
+
113
+ ```bash
114
+ # Launch an instance using environment variables
115
+ export AWS_REGION=us-east-2
116
+ export AMI_ID=ami-•••••••••• # Your TestDriver AMI (contact us to get one)
117
+ export AWS_LAUNCH_TEMPLATE_ID=lt-•••••••••• # From CloudFormation output from step 1
118
+ export RESOLUTION=1440x900 # Change screen resolution if desired (default is 1440x900)
119
+
120
+ /bin/bash ./setup/aws/spawn-runner.sh
121
+ ```
122
+
123
+ The script outputs:
124
+
125
+ ```
126
+ PUBLIC_IP=1.2.3.4
127
+ INSTANCE_ID=i-1234567890abcdef0
128
+ AWS_REGION=us-east-2
129
+ ```
130
+
131
+ <Note>
132
+ **Instance Lifecycle**: Instances spawned by this script will continue running until you manually terminate them. They are automatically tagged with `Name=TestDriverRunner` and `Project=[your ProjectTag value]` for easy identification in the AWS console.
133
+ </Note>
134
+
135
+ #### Changing Resolution in Lifecycle Files
136
+
137
+ You can also change the resolution before running tests by adding an [`exec`](/commands/exec) command in your `lifecycle/provision.yaml` file:
138
+
139
+ ```yaml lifecycle/provision.yaml
140
+ version: 6.0.0
141
+ steps:
142
+ - prompt: set screen resolution
143
+ commands:
144
+ - command: exec
145
+ lang: pwsh
146
+ code: |
147
+ C:\testdriver\SetResolution.ps1 -Width 1920 -Height 1080
148
+ ```
149
+
150
+ This approach is useful when you need different resolutions for different test scenarios. See the [Lifecycle Files documentation](/guide/lifecycle) for more information about provision scripts.
151
+
152
+ ### CLI Usage
153
+
154
+ Once you have an instance IP, run tests directly:
155
+
156
+ ```bash
157
+ # Basic test execution
158
+ npx testdriverai@latest run test.yaml --ip=1.2.3.4
159
+ ```
160
+
161
+ You can use the `PUBLIC_IP` to target the instance you just spawned via `./setup/aws/spawn-runner.sh`:
162
+
163
+ ```sh
164
+ npx testdriverai@latest run testdriver/your-test.yaml \
165
+ --ip="$PUBLIC_IP" \
166
+ ```
167
+
168
+ ### Terminating Instances
169
+
170
+ After your tests complete, terminate the instance to avoid unnecessary costs:
171
+
172
+ ```bash
173
+ # Terminate the instance
174
+ aws ec2 terminate-instances --instance-ids $INSTANCE_ID --region $AWS_REGION
175
+ ```
176
+
177
+ You can also terminate instances manually through the AWS console by searching for instances tagged with `Name=TestDriverRunner`.
178
+
179
+ ## Step 3: GitHub Actions Integration
180
+
181
+ This step shows you how to automate the entire test lifecycle in CI/CD.
182
+
183
+ ### Example Workflow
184
+
185
+ Our [`.github/workflows/self-hosted.yml`](https://github.com/testdriverai/cli/tree/main/.github/workflows/self-hosted.yml) demonstrates the complete workflow: spawning an EC2 instance, running your tests, and shutting down the instance automatically to minimize costs.
186
+
187
+ The workflow uses the GitHub secrets you configure (see below) to authenticate with AWS and spawn instances on-demand:
188
+
189
+ ```yaml
190
+ name: TestDriver Self-Hosted
191
+
192
+ on:
193
+ workflow_dispatch:
194
+ push:
195
+
196
+ jobs:
197
+ test:
198
+ runs-on: ubuntu-latest
199
+ steps:
200
+ - name: Checkout repository
201
+ uses: actions/checkout@v4
202
+
203
+ - name: Setup AWS Instance
204
+ id: aws-setup
205
+ run: |
206
+ OUTPUT=$(./setup/aws/spawn-runner.sh | tee /dev/stderr)
207
+ PUBLIC_IP=$(echo "$OUTPUT" | grep "PUBLIC_IP=" | cut -d'=' -f2)
208
+ INSTANCE_ID=$(echo "$OUTPUT" | grep "INSTANCE_ID=" | cut -d'=' -f2)
209
+ echo "public-ip=$PUBLIC_IP" >> $GITHUB_OUTPUT
210
+ echo "instance-id=$INSTANCE_ID" >> $GITHUB_OUTPUT
211
+ env:
212
+ AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
213
+ AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
214
+ AWS_REGION: ${{ secrets.AWS_REGION }}
215
+ AWS_LAUNCH_TEMPLATE_ID: ${{ secrets.AWS_LAUNCH_TEMPLATE_ID }}
216
+ AMI_ID: ${{ secrets.AMI_ID }}
217
+
218
+ - name: Run TestDriver
219
+ run: |
220
+ npx testdriverai run your-test.yaml \
221
+ --ip="${{ steps.aws-setup.outputs.public-ip }}"
222
+ env:
223
+ TD_API_KEY: ${{ secrets.TD_API_KEY }}
224
+
225
+ - name: Shutdown AWS Instance
226
+ if: always()
227
+ run: |
228
+ aws ec2 terminate-instances \
229
+ --region ${{ secrets.AWS_REGION }}
230
+ --instance-ids ${{ steps.aws-setup.outputs.instance-id }}
231
+ env:
232
+ AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
233
+ AWS_REGION: ${{ secrets.AWS_REGION }}
234
+ AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
235
+ ```
236
+
237
+ ### Required Secrets
238
+
239
+ Configure these secrets in your GitHub repository:
240
+
241
+ | Secret | Description | Example |
242
+ | ------------------------ | ----------------------------------- | ------------------------------------------------------------ |
243
+ | `AWS_ACCESS_KEY_ID` | AWS access key | `AKIAIOSFODNN7EXAMPLE` |
244
+ | `AWS_REGION` | AWS Region | `us-east-2` |
245
+ | `AWS_SECRET_ACCESS_KEY` | AWS secret key | `wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY` |
246
+ | `AWS_LAUNCH_TEMPLATE_ID` | Launch template from CloudFormation | `lt-07c53ce8349b958d1` |
247
+ | `AMI_ID` | TestDriver AMI ID | `ami-085f872ca0cd80fed` |
248
+ | `TD_API_KEY` | TestDriver API key | Your API key from [the dashboard](https://app.testdriver.ai) |
249
+
250
+ ## AMI Customization
251
+
252
+ ### Using the Base AMI
253
+
254
+ Our TestDriver Golden Image (AMI) comes pre-configured with everything you need to run tests:
255
+
256
+ **Operating System & Environment:**
257
+
258
+ - Windows Server with desktop environment
259
+ - VNC + web server for remote desktop access through the browser
260
+
261
+ **Development Tools:**
262
+
263
+ - Python (with pip)
264
+ - Node.js (with npm)
265
+ - Git
266
+
267
+ **Test Infrastructure:**
268
+
269
+ - TestDriver agent and dependencies
270
+ - Optimized settings for test execution
271
+ - Pre-configured networking for TestDriver CLI communication
272
+
273
+ ### Modifying the AMI
274
+
275
+ You can customize the AMI for your specific needs:
276
+
277
+ 1. **Launch an instance** from our base AMI
278
+ 2. **Make your changes** (install software, configure settings)
279
+ 3. **Create a new AMI** from your modified instance
280
+ 4. **Update your workflow** to use the new AMI ID
281
+
282
+ ### Amazon Image Builder
283
+
284
+ For automated AMI builds, use [Amazon EC2 Image Builder](https://aws.amazon.com/image-builder/):
285
+
286
+ ```yaml
287
+ # Example Image Builder pipeline
288
+ Components:
289
+ - Name: testdriver-base
290
+ Version: 1.0.0
291
+ Platform: Windows
292
+ Type: BUILD
293
+ Data: |
294
+ name: TestDriver Custom Setup
295
+ description: Custom TestDriver AMI with additional software
296
+ schemaVersion: 1.0
297
+ phases:
298
+ - name: build
299
+ steps:
300
+ - name: InstallSoftware
301
+ action: ExecutePowerShell
302
+ inputs:
303
+ commands:
304
+ - "# Your custom installation commands here"
305
+ ```
306
+
307
+ ## Security Considerations
308
+
309
+ ### Network Security
310
+
311
+ 1. **Restrict CIDR blocks**: Only allow access from your known IP ranges
312
+ 2. **Use VPC endpoints**: For private communication with AWS services
313
+ 3. **Enable VPC Flow Logs**: For network monitoring and debugging
314
+
315
+ ### AWS Authentication
316
+
317
+ Use [OIDC for GitHub Actions](https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/about-security-hardening-with-openid-connect) instead of long-term credentials:
318
+
319
+ ```yaml
320
+ permissions:
321
+ id-token: write
322
+ contents: read
323
+
324
+ steps:
325
+ - name: Configure AWS credentials
326
+ uses: aws-actions/configure-aws-credentials@v4
327
+ with:
328
+ role-to-assume: arn:aws:iam::123456789012:role/GitHubActionsRole
329
+ aws-region: us-east-2
330
+ ```
331
+
332
+ ### Instance Security
333
+
334
+ - **Terminate instances** immediately after use
335
+ - **Monitor costs** with AWS billing alerts
336
+ - **Use least-privilege IAM roles** for instance profiles
337
+ - **Enable CloudTrail** for audit logging
338
+
339
+ ## Troubleshooting
340
+
341
+ ### Common Issues
342
+
343
+ **Instance not responding in TestDriver CLI:**
344
+
345
+ When the CLI displays connection errors or timeouts, check:
346
+
347
+ - **Security group rules**: The CloudFormation template configures all necessary ports (RDP 3389, VNC 5900, and TestDriver communication ports). Verify your security group hasn't been modified.
348
+ - **Instance status checks**: Ensure the instance has passed both system and instance status checks in the AWS console.
349
+ - **AMI compatibility**: Verify the AMI is compatible with your selected instance type (some instance types don't support certain AMIs).
350
+
351
+ **Connection timeouts:**
352
+
353
+ - Verify network connectivity from runner to instance
354
+ - Check VPC routing and internet gateway configuration
355
+ - Confirm instance is in correct subnet
356
+
357
+ **AWS CLI errors:**
358
+
359
+ - Validate AWS credentials and permissions
360
+ - Check AWS service quotas and limits
361
+ - Verify region consistency across all resources
362
+
363
+ ### Getting Help
364
+
365
+ For enterprise customers:
366
+
367
+ - Contact your account manager for AMI access issues
368
+ - Use support channels for infrastructure questions
369
+ - Check the TestDriver documentation for CLI usage