testeiya 0.2.1 → 0.3.10

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (402) hide show
  1. package/LICENSE +21 -0
  2. package/README.md +284 -171
  3. package/dist/prompt/context.js +79 -0
  4. package/dist/prompt/context.js.map +1 -0
  5. package/dist/prompt/index.js +60 -0
  6. package/dist/prompt/index.js.map +1 -0
  7. package/dist/prompt/print.js +30 -0
  8. package/dist/prompt/print.js.map +1 -0
  9. package/dist/prompt/project-info.js +2 -0
  10. package/dist/prompt/project-info.js.map +1 -0
  11. package/{src/prompt/system-prompt.ts → dist/prompt/system-prompt.js} +38 -25
  12. package/dist/prompt/system-prompt.js.map +1 -0
  13. package/dist/prompt/testomatio.js +264 -0
  14. package/dist/prompt/testomatio.js.map +1 -0
  15. package/dist/prompt/tools.js +57 -0
  16. package/dist/prompt/tools.js.map +1 -0
  17. package/dist/prompt/vocab.js +8 -0
  18. package/dist/prompt/vocab.js.map +1 -0
  19. package/dist/src/args.js +172 -0
  20. package/dist/src/args.js.map +1 -0
  21. package/dist/src/cli.js +97 -0
  22. package/dist/src/cli.js.map +1 -0
  23. package/dist/src/doctor.js +186 -0
  24. package/dist/src/doctor.js.map +1 -0
  25. package/dist/src/env.js +93 -0
  26. package/dist/src/env.js.map +1 -0
  27. package/dist/src/mcp-extension.js +9 -0
  28. package/dist/src/mcp-extension.js.map +1 -0
  29. package/dist/src/mcp.js +139 -0
  30. package/dist/src/mcp.js.map +1 -0
  31. package/dist/src/model.js +71 -0
  32. package/dist/src/model.js.map +1 -0
  33. package/dist/src/models.js +38 -0
  34. package/dist/src/models.js.map +1 -0
  35. package/dist/src/output.js +185 -0
  36. package/dist/src/output.js.map +1 -0
  37. package/dist/src/result.js +41 -0
  38. package/dist/src/result.js.map +1 -0
  39. package/dist/src/run.js +291 -0
  40. package/dist/src/run.js.map +1 -0
  41. package/dist/src/session.js +92 -0
  42. package/dist/src/session.js.map +1 -0
  43. package/dist/src/sessions.js +61 -0
  44. package/dist/src/sessions.js.map +1 -0
  45. package/dist/vendor/mcp.js +14801 -0
  46. package/mcp-tools.json +2967 -0
  47. package/package.json +34 -43
  48. package/prompt/context.ts +101 -0
  49. package/prompt/index.ts +102 -0
  50. package/prompt/print.ts +32 -0
  51. package/prompt/project-info.ts +30 -0
  52. package/prompt/system-prompt.ts +187 -0
  53. package/{src/prompt → prompt}/testomatio.ts +120 -9
  54. package/prompt/tools.ts +57 -0
  55. package/prompt/vocab.ts +7 -0
  56. package/skills/codeceptjs/ci-fix-tests/SKILL.md +7 -8
  57. package/skills/codeceptjs/codeceptjs-auth/SKILL.md +47 -64
  58. package/skills/codeceptjs/codeceptjs-exploration/SKILL.md +56 -47
  59. package/skills/codeceptjs/codeceptjs-fundamentals/SKILL.md +144 -106
  60. package/skills/codeceptjs/codeceptjs-run-analysis/SKILL.md +50 -57
  61. package/skills/codeceptjs/debugging-codeceptjs-tests/SKILL.md +88 -196
  62. package/skills/codeceptjs/migrate-codeceptjs-4/SKILL.md +4 -5
  63. package/skills/codeceptjs/migrate-cypress-to-codeceptjs/SKILL.md +25 -36
  64. package/skills/codeceptjs/migrate-protractor-to-codeceptjs/SKILL.md +23 -36
  65. package/skills/codeceptjs/migrate-selenium-java-to-codeceptjs/SKILL.md +23 -36
  66. package/skills/codeceptjs/migrate-testcafe-to-codeceptjs/SKILL.md +23 -35
  67. package/skills/codeceptjs/refactoring-codeceptjs-tests/SKILL.md +50 -53
  68. package/skills/codeceptjs/writing-codeceptjs-tests/SKILL.md +87 -191
  69. package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/SKILL.md +1 -1
  70. package/{skills.lock.json → skills/skills.lock.json} +43 -36
  71. package/skills/skills.yaml +34 -0
  72. package/skills/testomatio/explorbot/explorbot-fundamentals/SKILL.md +49 -61
  73. package/skills/testomatio/explorbot/explorbot-fundamentals/references/no-install.md +36 -0
  74. package/skills/testomatio/explorbot/explorbot-plan/SKILL.md +20 -7
  75. package/skills/testomatio/explorbot/explorbot-setup/SKILL.md +41 -289
  76. package/skills/testomatio/qa-process/qa-explain-behavior/SKILL.md +111 -0
  77. package/skills/testomatio/qa-process/qa-explain-behavior/references/answer-shapes.md +116 -0
  78. package/skills/testomatio/qa-process/qa-explain-behavior/references/examples.md +157 -0
  79. package/skills/testomatio/qa-process/qa-lead-strategy-advisor/SKILL.md +91 -0
  80. package/skills/testomatio/qa-process/qa-lead-strategy-advisor/references/output-format.md +71 -0
  81. package/skills/testomatio/qa-process/testing-workflow/SKILL.md +227 -0
  82. package/skills/testomatio/test-automation/qa-data-seeder/SKILL.md +83 -0
  83. package/skills/testomatio/test-automation/run-tests-with-testomatio-reporter/SKILL.md +5 -3
  84. package/skills/testomatio/test-automation/setup-change-aware-pr-testing/SKILL.md +190 -0
  85. package/skills/testomatio/test-automation/setup-ci-automation/SKILL.md +76 -0
  86. package/skills/testomatio/test-management/qa-split-testing-levels-pyramid/SKILL.md +55 -0
  87. package/skills/testomatio/test-management/qa-sprint-report-by-testomatio/SKILL.md +220 -0
  88. package/skills/testomatio/test-management/qa-sprint-report-by-testomatio/references/qa-sprint-report.md +126 -0
  89. package/skills/testomatio/test-management/qa-test-code-coverage/SKILL.md +1 -1
  90. package/skills/testomatio/test-management/qa-thinking/SKILL.md +25 -25
  91. package/skills/testomatio/test-management/testomatio-mcp/SKILL.md +14 -1
  92. package/skills/testomatio/test-management/testomatio-mcp/references/MCP_SETUP.md +8 -0
  93. package/assets/testeiya-logo-dark.svg +0 -16
  94. package/assets/testeiya-logo.svg +0 -15
  95. package/models.catalog.json +0 -6963
  96. package/skills/playwright-best-practices-skill/LICENSE.md +0 -7
  97. package/skills/playwright-best-practices-skill/README.md +0 -147
  98. package/skills/testeiya/README.md +0 -29
  99. package/skills/testeiya/answer-formatting/SKILL.md +0 -30
  100. package/skills/testeiya/check-cucumber/SKILL.md +0 -200
  101. package/skills/testeiya/check-tests/SKILL.md +0 -260
  102. package/skills/testeiya/manual-run-assistant/SKILL.md +0 -61
  103. package/skills/testeiya/testomatio-docs/INDEX.md +0 -175
  104. package/skills/testeiya/testomatio-docs/SKILL.md +0 -102
  105. package/skills/testeiya/testomatio-docs/docs/advanced/ai-powered-features/ai-agents.md +0 -144
  106. package/skills/testeiya/testomatio-docs/docs/advanced/ai-powered-features/ai-powered-features.md +0 -531
  107. package/skills/testeiya/testomatio-docs/docs/advanced/ai-powered-features/ai-requirements.md +0 -437
  108. package/skills/testeiya/testomatio-docs/docs/advanced/api-access/index.md +0 -191
  109. package/skills/testeiya/testomatio-docs/docs/advanced/branches-folder/index.md +0 -277
  110. package/skills/testeiya/testomatio-docs/docs/advanced/bulk-edit-folder/bulk-edit-demos.mdx +0 -39
  111. package/skills/testeiya/testomatio-docs/docs/advanced/bulk-edit-folder/bulk-edit-on-suite-and-test-level.md +0 -122
  112. package/skills/testeiya/testomatio-docs/docs/advanced/bulk-edit-folder/what-is-bulk-edit.md +0 -170
  113. package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/branches.md +0 -53
  114. package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/index.mdx +0 -98
  115. package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/jira-issue.md +0 -151
  116. package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/runs-and-reports.md +0 -377
  117. package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/work-in-jira.md +0 -267
  118. package/skills/testeiya/testomatio-docs/docs/advanced/living-doc/index.md +0 -138
  119. package/skills/testeiya/testomatio-docs/docs/advanced/milestones/index.md +0 -220
  120. package/skills/testeiya/testomatio-docs/docs/advanced/shortcuts/index.md +0 -48
  121. package/skills/testeiya/testomatio-docs/docs/advanced/tags-labels/labels-and-custom-fields.md +0 -201
  122. package/skills/testeiya/testomatio-docs/docs/advanced/tags-labels/tags-or-labels.md +0 -94
  123. package/skills/testeiya/testomatio-docs/docs/advanced/tags-labels/tags.md +0 -243
  124. package/skills/testeiya/testomatio-docs/docs/advanced/test-artifacts/index.md +0 -135
  125. package/skills/testeiya/testomatio-docs/docs/advanced/tql/index.md +0 -394
  126. package/skills/testeiya/testomatio-docs/docs/getting-started/index.md +0 -290
  127. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/azure.md +0 -96
  128. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/bamboo.md +0 -94
  129. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/bitbucket.md +0 -106
  130. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/circle.md +0 -115
  131. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/github.md +0 -120
  132. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/gitlab.md +0 -88
  133. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/index.mdx +0 -151
  134. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/jenkins.md +0 -121
  135. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/teamcity.md +0 -96
  136. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/azure.md +0 -31
  137. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/clickup.md +0 -54
  138. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/confluence.md +0 -34
  139. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/github.md +0 -31
  140. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/gitlab.md +0 -93
  141. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/index.md +0 -131
  142. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/jira.md +0 -164
  143. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/linear.md +0 -29
  144. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/shortcut.md +0 -67
  145. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/youtrack.md +0 -94
  146. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/azure-devops.md +0 -53
  147. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/email.md +0 -71
  148. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/jira.md +0 -52
  149. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/ms-teams.md +0 -60
  150. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/rules.md +0 -267
  151. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/slack.md +0 -82
  152. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/telegram.md +0 -73
  153. package/skills/testeiya/testomatio-docs/docs/integrations/scim-provisioning/index.mdx +0 -207
  154. package/skills/testeiya/testomatio-docs/docs/integrations/scim-provisioning/okta.md +0 -204
  155. package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/azure.md +0 -95
  156. package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/google.md +0 -75
  157. package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/index.mdx +0 -33
  158. package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/okta.md +0 -108
  159. package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/saml.md +0 -66
  160. package/skills/testeiya/testomatio-docs/docs/legal/compliance/gdpr.md +0 -95
  161. package/skills/testeiya/testomatio-docs/docs/legal/compliance/index.mdx +0 -94
  162. package/skills/testeiya/testomatio-docs/docs/legal/compliance/privacy.md +0 -26
  163. package/skills/testeiya/testomatio-docs/docs/legal/compliance/terms.md +0 -32
  164. package/skills/testeiya/testomatio-docs/docs/legal/security/index.mdx +0 -242
  165. package/skills/testeiya/testomatio-docs/docs/legal/security/jira.md +0 -140
  166. package/skills/testeiya/testomatio-docs/docs/legal/security/subprocessors.md +0 -76
  167. package/skills/testeiya/testomatio-docs/docs/management/company/administration.md +0 -363
  168. package/skills/testeiya/testomatio-docs/docs/management/company/audit-log.md +0 -41
  169. package/skills/testeiya/testomatio-docs/docs/management/company/plan-features-comparison.md +0 -240
  170. package/skills/testeiya/testomatio-docs/docs/management/company/subscriptions.md +0 -326
  171. package/skills/testeiya/testomatio-docs/docs/management/company/trials.md +0 -75
  172. package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/accountant.md +0 -49
  173. package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/index.mdx +0 -285
  174. package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/read-only-user.md +0 -67
  175. package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/teams.md +0 -138
  176. package/skills/testeiya/testomatio-docs/docs/management/project/settings.md +0 -87
  177. package/skills/testeiya/testomatio-docs/docs/management/project/templates.md +0 -546
  178. package/skills/testeiya/testomatio-docs/docs/project/analytics/index.md +0 -326
  179. package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/classical-tests-markdown-format.md +0 -386
  180. package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/download-manual-tests-as-files.md +0 -378
  181. package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/export-to-obsidian.md +0 -112
  182. package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/export-to-spreadsheet.md +0 -127
  183. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/auto-import.md +0 -63
  184. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-Allure-TestOps.md +0 -49
  185. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-QTest.md +0 -41
  186. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-Qase.md +0 -39
  187. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-Zephyr.md +0 -37
  188. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-csv-xlsx.md +0 -117
  189. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-cucumber.md +0 -93
  190. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-qmetry.md +0 -44
  191. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-source-code.md +0 -117
  192. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-testcaselabs.md +0 -44
  193. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-testmo.md +0 -47
  194. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-testrail.md +0 -82
  195. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-xray.md +0 -55
  196. package/skills/testeiya/testomatio-docs/docs/project/plans/index.md +0 -334
  197. package/skills/testeiya/testomatio-docs/docs/project/pulse/index.md +0 -104
  198. package/skills/testeiya/testomatio-docs/docs/project/runs/archive-runs-and-groups.md +0 -176
  199. package/skills/testeiya/testomatio-docs/docs/project/runs/custom-statuses.md +0 -195
  200. package/skills/testeiya/testomatio-docs/docs/project/runs/environments.md +0 -203
  201. package/skills/testeiya/testomatio-docs/docs/project/runs/managing-runs.md +0 -455
  202. package/skills/testeiya/testomatio-docs/docs/project/runs/merge-strategies.md +0 -137
  203. package/skills/testeiya/testomatio-docs/docs/project/runs/reports.md +0 -329
  204. package/skills/testeiya/testomatio-docs/docs/project/runs/rungroups.md +0 -342
  205. package/skills/testeiya/testomatio-docs/docs/project/runs/running-automated-tests.md +0 -256
  206. package/skills/testeiya/testomatio-docs/docs/project/runs/running-manual-and-automated-tests.md +0 -331
  207. package/skills/testeiya/testomatio-docs/docs/project/runs/running-tests-manually.md +0 -633
  208. package/skills/testeiya/testomatio-docs/docs/project/runs/temporary-tests-notes.md +0 -223
  209. package/skills/testeiya/testomatio-docs/docs/project/steps-snippets/snippets.md +0 -68
  210. package/skills/testeiya/testomatio-docs/docs/project/steps-snippets/steps.md +0 -181
  211. package/skills/testeiya/testomatio-docs/docs/project/tests/bdd-test-case-editor.md +0 -98
  212. package/skills/testeiya/testomatio-docs/docs/project/tests/classical-test-case-editor.md +0 -354
  213. package/skills/testeiya/testomatio-docs/docs/project/tests/classical-vs-bdd.md +0 -169
  214. package/skills/testeiya/testomatio-docs/docs/project/tests/copy-and-move-your-tests.md +0 -156
  215. package/skills/testeiya/testomatio-docs/docs/project/tests/other-features-for-test-case-design.md +0 -412
  216. package/skills/testeiya/testomatio-docs/docs/project/tests/test-case-creation-and-editing.md +0 -527
  217. package/skills/testeiya/testomatio-docs/docs/support/index.md +0 -75
  218. package/skills/testeiya/testomatio-docs/docs/test-reporting/artifacts.md +0 -388
  219. package/skills/testeiya/testomatio-docs/docs/test-reporting/cli.md +0 -215
  220. package/skills/testeiya/testomatio-docs/docs/test-reporting/configuration.md +0 -247
  221. package/skills/testeiya/testomatio-docs/docs/test-reporting/frameworks.md +0 -490
  222. package/skills/testeiya/testomatio-docs/docs/test-reporting/functions.md +0 -144
  223. package/skills/testeiya/testomatio-docs/docs/test-reporting/index.mdx +0 -48
  224. package/skills/testeiya/testomatio-docs/docs/test-reporting/junit.md +0 -248
  225. package/skills/testeiya/testomatio-docs/docs/test-reporting/logger.md +0 -138
  226. package/skills/testeiya/testomatio-docs/docs/test-reporting/php.md +0 -77
  227. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/bitbucket.md +0 -72
  228. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/csv.md +0 -23
  229. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/debug.md +0 -18
  230. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/github.md +0 -63
  231. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/gitlab.md +0 -29
  232. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/html.md +0 -80
  233. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/index.mdx +0 -57
  234. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/testomatio.md +0 -331
  235. package/skills/testeiya/testomatio-docs/docs/test-reporting/python.md +0 -229
  236. package/skills/testeiya/testomatio-docs/docs/test-reporting/workflows.md +0 -325
  237. package/skills/testeiya/testomatio-docs/docs/tutorials/Interacting-via-API.md +0 -159
  238. package/skills/testeiya/testomatio-docs/docs/tutorials/Manual-Testing-Classic.md +0 -164
  239. package/skills/testeiya/testomatio-docs/docs/tutorials/Set-Up-Read-Only-Access.md +0 -85
  240. package/skills/testeiya/testomatio-docs/docs/tutorials/Using-AI-in-Testing.md +0 -163
  241. package/skills/testeiya/testomatio-docs/docs/tutorials/Working-with-Jira.md +0 -125
  242. package/skills/testeiya/testomatio-docs/docs/tutorials/java.md +0 -478
  243. package/skills/testeiya/testomatio-docs/docs/tutorials/playwright.md +0 -207
  244. package/skills/testeiya/testomatio-docs/docs/tutorials/webdriver.md +0 -218
  245. package/skills/testeiya/testomatio-docs/docs.lock.json +0 -6
  246. package/skills/testeiya/testomatio-reporter/SKILL.md +0 -307
  247. package/skills/testomatio/test-automation/setup-pr-testing/SKILL.md +0 -199
  248. package/skills.yaml +0 -21
  249. package/src/ai-debug.ts +0 -188
  250. package/src/api/agent-get.ts +0 -22
  251. package/src/api/agent-start.ts +0 -384
  252. package/src/api/client-log.ts +0 -29
  253. package/src/api/context.ts +0 -332
  254. package/src/api/debug-layout.ts +0 -47
  255. package/src/api/debug-report.ts +0 -29
  256. package/src/api/debug-snapshot.ts +0 -30
  257. package/src/api/debug-stream.ts +0 -52
  258. package/src/api/files-delete.ts +0 -184
  259. package/src/api/files-read.ts +0 -41
  260. package/src/api/files-rename.ts +0 -99
  261. package/src/api/files-tree.ts +0 -292
  262. package/src/api/files-write.ts +0 -41
  263. package/src/api/mcp.ts +0 -603
  264. package/src/api/memory.ts +0 -69
  265. package/src/api/open-external.ts +0 -39
  266. package/src/api/playwright-cli.ts +0 -606
  267. package/src/api/providers.ts +0 -457
  268. package/src/api/read-env-token.ts +0 -33
  269. package/src/api/sessions.ts +0 -79
  270. package/src/api/settings.ts +0 -120
  271. package/src/api/skills.ts +0 -58
  272. package/src/api/testomatio-attachment.ts +0 -89
  273. package/src/api/testomatio-auth.ts +0 -284
  274. package/src/api/testomatio-proxy.ts +0 -197
  275. package/src/api/testomatio-run-stats.ts +0 -81
  276. package/src/api/testomatio-target.ts +0 -225
  277. package/src/api/testomatio-transcription.ts +0 -82
  278. package/src/api/workspace-search.ts +0 -96
  279. package/src/api/workspace-sync.ts +0 -151
  280. package/src/api/workspace.ts +0 -165
  281. package/src/app-server.ts +0 -556
  282. package/src/bridge.ts +0 -251
  283. package/src/check-tests.ts +0 -156
  284. package/src/cli.ts +0 -17
  285. package/src/commands.ts +0 -244
  286. package/src/config.ts +0 -183
  287. package/src/connection.ts +0 -658
  288. package/src/context-store.ts +0 -265
  289. package/src/debug-bus.ts +0 -405
  290. package/src/extensions/tool-gate.ts +0 -220
  291. package/src/extensions/webui/ask-channel.ts +0 -52
  292. package/src/extensions/webui/index.ts +0 -184
  293. package/src/extensions/webui/tools/ask-question.ts +0 -88
  294. package/src/extensions/webui/tools/query-result.ts +0 -79
  295. package/src/extensions/webui/tools/render-chart.ts +0 -53
  296. package/src/extensions/webui/tools/render-item.ts +0 -61
  297. package/src/extensions/webui/tools/render-list.ts +0 -170
  298. package/src/extensions/webui/tools/render-result.ts +0 -85
  299. package/src/extensions/webui/tools/render-tree.ts +0 -71
  300. package/src/extensions/webui/tools/ui-widget.ts +0 -54
  301. package/src/extensions/webui/tools/widget-result.ts +0 -13
  302. package/src/extensions/webui/widget-channel.ts +0 -50
  303. package/src/file-log.ts +0 -196
  304. package/src/git-tracked.ts +0 -68
  305. package/src/gitignore.ts +0 -53
  306. package/src/json-store.ts +0 -43
  307. package/src/load-env.ts +0 -70
  308. package/src/loader-style.ts +0 -96
  309. package/src/main.ts +0 -140
  310. package/src/mcp-catalog.ts +0 -96
  311. package/src/models-catalog.ts +0 -72
  312. package/src/permissions.ts +0 -120
  313. package/src/project-dir.ts +0 -189
  314. package/src/project-info.ts +0 -108
  315. package/src/prompt/app-ui.ts +0 -59
  316. package/src/prompt/browser.ts +0 -27
  317. package/src/prompt/index.ts +0 -62
  318. package/src/prompt/tools.ts +0 -27
  319. package/src/server-info.ts +0 -62
  320. package/src/session-factory.ts +0 -442
  321. package/src/session-store.ts +0 -115
  322. package/src/shell-env.ts +0 -45
  323. package/src/skills.ts +0 -163
  324. package/src/sync-snapshot.ts +0 -140
  325. package/src/telemetry.ts +0 -239
  326. package/src/testomatio-auth.ts +0 -299
  327. package/src/testomatio.ts +0 -98
  328. package/src/theme.ts +0 -30
  329. package/src/user-env.ts +0 -94
  330. package/src/welcome.ts +0 -194
  331. package/src/workspace/safe-path.ts +0 -85
  332. package/src/workspace/test-md.ts +0 -70
  333. package/src/workspace-model.ts +0 -440
  334. package/testeiya.config.json +0 -14
  335. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/authentication-flows.md +0 -0
  336. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/authentication.md +0 -0
  337. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/clock-mocking.md +0 -0
  338. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/mobile-testing.md +0 -0
  339. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/multi-context.md +0 -0
  340. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/multi-user.md +0 -0
  341. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/network-advanced.md +0 -0
  342. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/third-party.md +0 -0
  343. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/architecture/pom-vs-fixtures.md +0 -0
  344. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/architecture/test-architecture.md +0 -0
  345. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/architecture/when-to-mock.md +0 -0
  346. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/browser-apis/browser-apis.md +0 -0
  347. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/browser-apis/iframes.md +0 -0
  348. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/browser-apis/service-workers.md +0 -0
  349. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/browser-apis/websockets.md +0 -0
  350. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/annotations.md +0 -0
  351. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/assertions-waiting.md +0 -0
  352. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/configuration.md +0 -0
  353. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/fixtures-hooks.md +0 -0
  354. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/global-setup.md +0 -0
  355. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/locators.md +0 -0
  356. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/page-object-model.md +0 -0
  357. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/projects-dependencies.md +0 -0
  358. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/test-data.md +0 -0
  359. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/test-suite-structure.md +0 -0
  360. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/test-tags.md +0 -0
  361. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/debugging/console-errors.md +0 -0
  362. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/debugging/debugging.md +0 -0
  363. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/debugging/error-testing.md +0 -0
  364. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/debugging/flaky-tests.md +0 -0
  365. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/frameworks/angular.md +0 -0
  366. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/frameworks/nextjs.md +0 -0
  367. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/frameworks/react.md +0 -0
  368. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/frameworks/vue.md +0 -0
  369. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/ci-cd.md +0 -0
  370. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/docker.md +0 -0
  371. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/github-actions.md +0 -0
  372. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/gitlab.md +0 -0
  373. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/other-providers.md +0 -0
  374. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/parallel-sharding.md +0 -0
  375. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/performance.md +0 -0
  376. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/reporting.md +0 -0
  377. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/test-coverage.md +0 -0
  378. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/accessibility.md +0 -0
  379. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/api-testing.md +0 -0
  380. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/browser-extensions.md +0 -0
  381. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/canvas-webgl.md +0 -0
  382. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/component-testing.md +0 -0
  383. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/drag-drop.md +0 -0
  384. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/electron.md +0 -0
  385. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/file-operations.md +0 -0
  386. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/file-upload-download.md +0 -0
  387. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/forms-validation.md +0 -0
  388. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/graphql-testing.md +0 -0
  389. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/i18n.md +0 -0
  390. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/performance-testing.md +0 -0
  391. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/security-testing.md +0 -0
  392. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/visual-regression.md +0 -0
  393. /package/skills/{playwright-cli → playwright/playwright-cli}/SKILL.md +0 -0
  394. /package/skills/{playwright-cli → playwright/playwright-cli}/references/element-attributes.md +0 -0
  395. /package/skills/{playwright-cli → playwright/playwright-cli}/references/playwright-tests.md +0 -0
  396. /package/skills/{playwright-cli → playwright/playwright-cli}/references/request-mocking.md +0 -0
  397. /package/skills/{playwright-cli → playwright/playwright-cli}/references/running-code.md +0 -0
  398. /package/skills/{playwright-cli → playwright/playwright-cli}/references/session-management.md +0 -0
  399. /package/skills/{playwright-cli → playwright/playwright-cli}/references/storage-state.md +0 -0
  400. /package/skills/{playwright-cli → playwright/playwright-cli}/references/test-generation.md +0 -0
  401. /package/skills/{playwright-cli → playwright/playwright-cli}/references/tracing.md +0 -0
  402. /package/skills/{playwright-cli → playwright/playwright-cli}/references/video-recording.md +0 -0
@@ -1,80 +0,0 @@
1
- ---
2
- title: HTML
3
- description: Discover how to use Testomat.io's HTML Pipe to generate detailed HTML reports from your test runs. Learn how to configure environment variables to save reports, customize output paths, and view essential test data like execution time and status.
4
- type: article
5
- url: https://docs.testomat.io/test-reporting/pipes/html
6
- head:
7
- - tag: meta
8
- attrs:
9
- name: keywords
10
- content: Testomat.io, HTML reports, test reporting, test automation, environment variables, CI/CD, CodeceptJS, test result visualization, report generation
11
- ---
12
- ## Testomat.io HTML Pipe
13
-
14
- The Testomat.io HTML Pipe is a feature that allows you to generate and save HTML reports based on your test results. This functionality is part of the Testomat.io testing ecosystem.
15
-
16
- ### Prerequisites
17
-
18
- **🔌 TESTOMATIO users now have the option to generate an HTML report based on their test execution results. If a user hasn't SET `TESTOMATIO` environment, the report will still be generated**
19
-
20
- ```
21
- TESTOMATIO_HTML_REPORT_SAVE=1 <actual run command>
22
- ```
23
-
24
- ### Enabling HTML Reports
25
-
26
- To enable HTML reports, set the TESTOMATIO_HTML_REPORT_SAVE environment variable to 1. Additionally, you can customize the HTML report folder and filename using the following variables:
27
-
28
- - TESTOMATIO_HTML_REPORT_SAVE=1
29
- - TESTOMATIO_HTML_REPORT_FOLDER: Specify the folder for HTML reports
30
- - TESTOMATIO_HTML_FILENAME: Set the desired filename for the HTML report
31
-
32
- _!!!Please note that the name must include the extension ".html". If the extension is missing, the report will be saved with the default name = testomatio-report.html_
33
-
34
- ### Usage
35
-
36
- The HTML Pipe operates as part of the test execution process. When running your tests with the specified environment variables, the HTML Pipe will generate reports based on the test results.
37
-
38
- ### Report Generation
39
-
40
- Once the test run is complete, the HTML Pipe compiles the test results and generates an HTML report. The generated report is saved to the specified HTML output path.
41
-
42
- The HTML report includes essential information, such as run ID, status, parallel execution indication, run URL, execution time, execution date, and details of individual tests.
43
-
44
- #### Example Command
45
-
46
- 📊 Generate a report without triggering the TESTOMATIO pipe (no data sent to the client)
47
-
48
- _Execute the following command to generate a report without sending data to TESTOMATIO:_
49
-
50
- ```
51
- TESTOMATIO_HTML_REPORT_SAVE=1 npx codeceptjs run
52
-
53
- ```
54
-
55
- 📝 Generate a report based on the default folder/html-file name: This command triggers the test execution process and enables the HTML Pipe to generate reports based on the test results:
56
-
57
- - Output folder: html-report/
58
- - Output file: testomatio-report.html
59
-
60
- _Execute the following command to generate a report with default settings:_
61
-
62
- ```
63
- TESTOMATIO_HTML_REPORT_SAVE=1 TESTOMATIO={API_KEY} npx codeceptjs run
64
-
65
- ```
66
-
67
- 📂 Generate a report based on a custom HTML folder/html-file name.
68
- **!!!You can SET only one of the following parameters if needed: folder or file name**
69
-
70
- _Execute the following command to generate a report with both folder & file names:_
71
-
72
- ```
73
- TESTOMATIO_HTML_REPORT_SAVE=1 TESTOMATIO_HTML_REPORT_FOLDER=user-html-reporter TESTOMATIO_HTML_FILENAME=custom.html TESTOMATIO={API_KEY} npx codeceptjs run
74
-
75
- ```
76
-
77
- 🖥️ An example of a report can be seen in the screenshot below:
78
-
79
- ![HTML example](./images/html-pipe.png)
80
-
@@ -1,57 +0,0 @@
1
- ---
2
- title: Pipes
3
- description: Testomat.io's Pipes feature allows you to forward test data from your test framework to other services like GitHub, GitLab, or CSV reports. Pipes are configured via environment variables and can be customized to send data to analytics, databases, or notifications. You can also create custom pipes for specific needs, ensuring compatibility across different test frameworks.
4
- type: article
5
- url: https://docs.testomat.io/test-reporting/pipes/overview
6
- head:
7
- - tag: meta
8
- attrs:
9
- name: keywords
10
- content: Testomat.io, Pipes, test reporting, GitHub integration, GitLab, CSV reports, CI pipelines, custom pipes, environment variables, test frameworks
11
- ---
12
-
13
- import DirectoryLinks from '/src/components/DirectoryLinks.astro';
14
-
15
- <DirectoryLinks directory="test-reporting/pipes" />
16
-
17
- ## 📯 Pipes
18
-
19
- ### What are Pipes?
20
-
21
- Unlike other reporters, Testomat.io Reporter doesn't force you to report data only to Testomat.io application. Testomat.io Reporter collects data from test frameworks and submits it via pipes to other services.
22
-
23
- ![](./images/reporter-pipes.png)
24
-
25
- For instance, you can enable GitHub Pipe to create a comment on a GitHub Pull Request and CSV Export Pipe to write a report in CSV format. To report data to Testomat.io App, Testomat.io Pipe should be enabled, but not required if you don't plan to use Testomat.io App. You can also write your own pipe to:
26
-
27
- - 📈 send data to analytics storage
28
- - 🗄 store data into database
29
- - 📧 send emails
30
- - 💬 send notifications to chats
31
- - etc....
32
-
33
- Pipes work in the very same manner for all supported test frameworks, as well as for JUnit reports. So writing a pipe instead of a custom reporter for a particular framework, saves your time as you can be sure, this pipe will work with any attached test result.
34
-
35
- Testomat.io Reporter doesn't store test data in any internal format. Test data is processed in memory. While a test is running, all pipes are notified of changes.
36
-
37
- Pipes Concepts:
38
-
39
- - **Pipes are configured via environment variables** - each test framework has its own configuration (or not). To ensure that a pipe receives its configuration it is independent from a running test framework.
40
- - **Pipes are enabled if a corresponding environment variables are set** - for instance, if `TESTOMATIO` variable is present, Testomat.io Pipe is enabled, if `GH_PAT` variable is set, GitHub Pipe is enabled.
41
- - **Custom Pipes** can be created as modules and added to `package.json` file.
42
- - When used locally environment variables are expected to be set in `.env` file. When running on CI server use Environment Configuration a CI server provides to pass environment variables into the pipeline.
43
-
44
- ### Built-in Pipes
45
-
46
- - [Testomat.io](https://docs.testomat.io/test-reporting/pipes/testomatio)
47
- - [GitHub](https://docs.testomat.io/test-reporting/pipes/github)
48
- - [Gitlab](https://docs.testomat.io/test-reporting/pipes/gitlab)
49
- - [CSV](https://docs.testomat.io/test-reporting/pipes/csv)
50
- - [Bitbucket](https://docs.testomat.io/test-reporting/pipes/bitbucket)
51
- - [Debugging](https://docs.testomat.io/test-reporting/pipes/debug)
52
-
53
- ## Custom Pipe
54
-
55
- _to be updated_
56
-
57
- ....
@@ -1,331 +0,0 @@
1
- ---
2
- title: Advanced Options
3
- description: Learn how to use the Testomat.io Pipe for advanced reporting configurations in test automation. The guide covers setting environment variables, adding reports to test runs, managing parallel executions, filtering tests, and integrating with JIRA for issue tracking.
4
- type: article
5
- url: https://docs.testomat.io/test-reporting/pipes/testomatio
6
- head:
7
- - tag: meta
8
- attrs:
9
- name: keywords
10
- content: Test reporting, test automation, Testomat.io Pipe, environment variables, CI integration, parallel execution, test filtering, JIRA integration, build URL, test management
11
- ---
12
- ## Testomat.io Pipe
13
-
14
- ![](./images/testomatio.png)
15
-
16
- Testomat.io Pipe sends data to [Testomat.io Application](https://app.testomat.io). Testomat.io provides free projects forever and unlimited test runs. Even it is a cloud application you can use it for free.
17
-
18
- **🔌 To enable Testomat.io pipe set `TESTOMATIO` environment variable with API key provided by Testomatio.**
19
-
20
- ```
21
- TESTOMATIO={API_KEY} <actual run command>
22
- ```
23
-
24
- Here are some possible use cases where you can use additional configuration on reporter:
25
-
26
- - [Create Unmatched Tests](#create-unmatched-tests)
27
- - [Add Report to Run by ID](#add-report-to-run-by-id)
28
- - [Do Not Finalize Run](#do-not-finalize-run)
29
- - [Setting Report Title](#setting-report-title)
30
- - [Reporting Parallel Execution to To Same Run](#reporting-parallel-execution-to-to-same-run)
31
- - [Adding Report to RunGroup](#adding-report-to-rungroup)
32
- - [Adding Environments to Run](#adding-environments-to-run)
33
- - [Starting an Empty Run](#starting-an-empty-run)
34
- - [Manually Finishing Run](#manually-finishing-run)
35
- - [Setting Build URL](#setting-build-url)
36
- - [Publish Run](#publish-run)
37
- - [Assign Label To Run](#assign-label-to-run)
38
- - [Create Jira Issue for Run](#create-jira-issue-for-run)
39
- - [Filter Tests](#filter-tests)
40
-
41
- ### Create Unmatched Tests
42
-
43
- Testomat.io will not create tests from the report if they have not been previously imported. To create tests during the report `TESTOMATIO_CREATE` option can be used:
44
-
45
- ```bash
46
- TESTOMATIO={API_KEY} TESTOMATIO_CREATE=1 <actual run command>
47
- ```
48
-
49
- ### Add Report to Run by ID
50
-
51
- This feature is widely used when a run is executed on CI.
52
- A run is created before the test is started and it is marked as `scheduled`. Then
53
- a report is assigned to that run using `TESTOMATIO_RUN` environment variable and `{RUN_ID}` of a run:
54
-
55
- ```bash
56
- TESTOMATIO={API_KEY} TESTOMATIO_RUN={RUN_ID} <actual run command>
57
- ```
58
-
59
- ### Do Not Finalize Run
60
-
61
- If multiple reports are added to the same run, each of them should not finalize the run.
62
- In this case use `TESTOMATIO_PROCEED=1` environment variable, so the Run will be shown as `Running`
63
-
64
- ```
65
- TESTOMATIO={API_KEY} TESTOMATIO_PROCEED=1 TESTOMATIO_RUN={RUN_ID} <actual run command>
66
- ```
67
-
68
- After all reports were attached and run can be execute the following command:
69
-
70
- ```
71
- TESTOMATIO={API_KEY} TESTOMATIO_RUN={RUN_ID} npx @testomatio/reporter finish
72
- ```
73
-
74
- ### Setting Report Title
75
-
76
- Give a title to your reports by passing it as environment variable to `TESTOMATIO_TITLE`.
77
-
78
- ```bash
79
- TESTOMATIO={API_KEY} TESTOMATIO_TITLE="title for the report" <actual run command>
80
- ```
81
-
82
- ### Reporting Parallel Execution to To Same Run
83
-
84
- Provide a shared unique title to all runs that will be running in parallel, and add `TESTOMATIO_SHARED_RUN` environment var. So all reports will be sent to this run.
85
-
86
- ```bash
87
- TESTOMATIO={API_KEY} TESTOMATIO_TITLE="report for commit ${GIT_COMMIT}" TESTOMATIO_SHARED_RUN=1 <actual run command>
88
- ```
89
-
90
- We recommend using a commit hash as a title to create a new Run. In this case we ensure that run title is unique and will be the same for all parallel jobs running exactly for this commit.
91
-
92
- By default you can report to the same run if a run was created no more than 20 minutes ago. If you want to increase this time you can use `TESTOMATIO_SHARED_RUN_TIMEOUT` environment variable. `TESTOMATIO_SHARED_RUN_TIMEOUT`accepts value in minutes, so to increase timeout to 2 hours (120 minutes) use the following command:
93
-
94
- ```bash
95
- TESTOMATIO={API_KEY} TESTOMATIO_TITLE="report for commit ${GIT_COMMIT}" TESTOMATIO_SHARED_RUN=1 TESTOMATIO_SHARED_RUN_TIMEOUT=120 <actual run command>
96
- ```
97
-
98
- ### Adding Report to RunGroup
99
-
100
- Create/Add run to group by providing `TESTOMATIO_RUNGROUP_TITLE`:
101
-
102
- ```sh
103
- TESTOMATIO={API_KEY} TESTOMATIO_RUNGROUP_TITLE="Build ${BUILD_ID}" <actual run command>
104
- ```
105
-
106
- ### Adding Environments to Run
107
-
108
- Add environments to run by providing `TESTOMATIO_ENV` as comma seperated values:
109
-
110
- ```bash
111
- TESTOMATIO={API_KEY} TESTOMATIO_ENV="Windows, Chrome" <actual run command>
112
- ```
113
-
114
- ### Starting an Empty Run
115
-
116
- If you want to create a run and obtain its `{RUN_ID}` from [testomat.io](https://testomat.io) you can use `--launch` option:
117
-
118
- ```bash
119
- TESTOMATIO={API_KEY} npx @testomatio/reporter start
120
- ```
121
-
122
- This command will return `{RUN_ID}` which you can pass to other jobs in a workflow.
123
-
124
- > When executed with `--launch` a command provided by `-c` flag is ignored
125
-
126
- ### Manually Finishing Run
127
-
128
- If you want to finish a run started by `--launch` use `--finish` option. `TESTOMATIO_RUN` environment variable is required:
129
-
130
- ```bash
131
- TESTOMATIO={API_KEY} TESTOMATIO_RUN={RUN_ID} npx @testomatio/reporter finish
132
- ```
133
-
134
- ### Setting Build URL
135
-
136
- When running on CI reporter tries to detect automatically the URL of the current build. This URL will be set to Run report. Reporter automatically detects build URL for following CI services:
137
-
138
- - GitHub Actions
139
- - Azure Devops
140
- - Jenkins
141
- - CircleCi
142
- - Gitlab CI
143
-
144
- If you are using a different CI or you want to override build url run tests with `BUILD_URL` environment variable:
145
-
146
- ```bash
147
- TESTOMATIO={API_KEY} BUILD_URL=<build-url> <actual run command>
148
- ```
149
-
150
- ### Publish Run
151
-
152
- To make report publicly available pass `TESTOMATIO_PUBLISH=1` variable:
153
-
154
- ```
155
- TESTOMATIO={API_KEY} TESTOMATIO_PUBLISH=1 <actual run command>
156
- ```
157
-
158
- ### Assign Label To Run
159
-
160
- To assign a label to run, a **label with scope `runs` must exist**.
161
- Then a label can be passed via `TESTOMATIO_LABEL` variable:
162
-
163
- ```
164
- TESTOMATIO={API_KEY} TESTOMATIO_LABEL=release <actual run command>
165
- ```
166
-
167
- There is a way to pass a value to label via `:`:
168
-
169
- ```
170
- TESTOMATIO={API_KEY} TESTOMATIO_LABEL="module:checkout" <actual run command>
171
- ```
172
-
173
- Or you can set multiple labels separating them with comma:
174
-
175
- ```
176
- TESTOMATIO={API_KEY} TESTOMATIO_LABEL="release,module:checkout" <actual run command>
177
- ```
178
-
179
- ### Create Jira Issue for Run
180
-
181
- To automatically assign Run to Jira Issue pass the ID of that issue as `TESTOMATIO_JIRA_ID` variable
182
-
183
- ```
184
- TESTOMATIO={API_KEY} TESTOMATIO_JIRA_ID={Jira_Issue_ID} <actual run command>
185
- ```
186
-
187
- For example, if your Jira issue id is `TST-1`, command should be:
188
-
189
- ```
190
- TESTOMATIO={API_KEY} TESTOMATIO_JIRA_ID=TST-1 <actual run command>
191
- ```
192
-
193
- ### Filter Tests
194
-
195
- Testomatio allows you to enhance your test execution with the option to filter and manage test runs based on various criteria. This can help you efficiently organize and execute your tests. Below, we explain how to use the `--filter` option in conjunction with Testomatio for customized test runs.
196
-
197
- > The filtering functionality for test execution is currently available for the following frameworks: Playwright & CodeceptJS.
198
-
199
- To use the `--filter` option with Testomatio, follow the format:
200
-
201
- ```bash
202
- TESTOMATIO={API_KEY} npx @testomatio/reporter run <actual run command> --filter 'testomatio:{filter-type}={filter-value}'
203
- ```
204
-
205
- #### Available Filter Types
206
-
207
- **Tag Name Filter**
208
-
209
- Filter tests by tag names. Use this filter to select tests associated with specific tags.
210
-
211
- ![Tag Name Filter](./images/by-tag-name.png)
212
-
213
- _Command execution example_
214
-
215
- ```bash
216
- TESTOMATIO={API_KEY} npx @testomatio/reporter run 'actual run command' --filter 'testomatio:tag-name=smoke'
217
- ```
218
-
219
- **Plan ID Filter**
220
-
221
- Filter tests by Testomatio plan IDs. Use this filter to organize and execute tests based on plan IDs.
222
-
223
- ![Plan ID Filter](./images/by-plan-id.png)
224
-
225
- _Command execution example_
226
-
227
- ```bash
228
- TESTOMATIO={API_KEY} npx @testomatio/reporter run 'actual run command' --filter 'testomatio:plan-id=b62f3170'
229
- ```
230
-
231
- **Label Filter**
232
-
233
- Filter tests using custom labels. Labels allow you to categorize tests for better management and execution.
234
-
235
- Label Filter (by label name) Example 1:
236
- ![Label Filter Example 3](./images/by-label-1.png)
237
-
238
- Label Filter (by label id) Example 2:
239
- ![Label Filter Example 1](./images/by-label.png)
240
-
241
- Another Label Filter Example 3:
242
- ![Label Filter Example 2](./images/by-label-3.png)
243
-
244
- _Command execution example_
245
-
246
- ```bash
247
- TESTOMATIO={API_KEY} npx @testomatio/reporter run 'actual run command' --filter 'testomatio:label=new-label-test'
248
- ```
249
-
250
- OR
251
-
252
- ```bash
253
- TESTOMATIO={API_KEY} npx @testomatio/reporter run 'actual run command' --filter 'testomatio:label=NEW label TEST'
254
- ```
255
-
256
- **Label Filter based on the Severity type**
257
-
258
- If you want to use a specific severity type, you should join the "severity" label id + severity type(e.g.: `s1-id2:⛔ Blocker`).
259
-
260
- Here's a list of available types that you can copy and add to the run command:
261
-
262
- - ⚠️ Critical
263
- - ⛔ Blocker
264
- - 🔥 Major
265
- - 👌 Normal
266
- - ❄️ Minor
267
- - 💤 Trivial
268
-
269
- Label Filter Example based on the Severity type:
270
- ![Label Filter Example 3](./images/by-label-severity.png)
271
-
272
- _Command execution example_
273
-
274
- ```bash
275
- TESTOMATIO={API_KEY} npx @testomatio/reporter run 'actual run command' --filter 'testomatio:label=severity-f1b11:🔥 Major'
276
- ```
277
-
278
- OR
279
-
280
- ```bash
281
- TESTOMATIO={API_KEY} npx @testomatio/reporter run 'actual run command' --filter 'testomatio:label=NEW label TEST:💤 Trivial'
282
- ```
283
-
284
- **JIRA Ticket Filter**
285
-
286
- Filter tests linked to JIRA tickets. Use this filter to coordinate testing efforts with your JIRA workflow.
287
-
288
- ![JIRA Ticket Filter](./images/by-jira-id.png)
289
-
290
- _Command execution example_
291
-
292
- ```bash
293
- TESTOMATIO={API_KEY} npx @testomatio/reporter run 'actual run command' --filter 'testomatio:jira-ticket=LMP-51'
294
- ```
295
-
296
- **📚 Example Usage**
297
-
298
- For instance, if you want to run tests with a specific label named "new-label-test," you can use the following command:
299
-
300
- ```bash
301
- TESTOMATIO={API_KEY} npx @testomatio/reporter run 'npx codeceptjs run' --filter 'testomatio:label=new-label'
302
- ```
303
-
304
- OR using Playwright:
305
-
306
- ```bash
307
- TESTOMATIO={API_KEY} npx @testomatio/reporter run 'npx playwright test' --filter 'testomatio:label=new-label'
308
- ```
309
-
310
- > Remember to replace {API_KEY} with your actual Testomatio API key and adjust the `<actual run command>` according to your testing setup
311
-
312
- > It's important to note that the provided filter value must match exactly the corresponding tag name, plan ID, label, or JIRA ticket for the desired tests
313
-
314
- Please note, that this functionality allows you to easily filter and execute tests based on specific criteria, enhancing your testing experience.
315
-
316
- ### Exclude Tests from Report by Glob Pattern
317
-
318
- To exclude tests from the report by [glob pattern](https://www.npmjs.com/package/glob) use `TESTOMATIO_EXCLUDE_FILES_FROM_REPORT_GLOB_PATTERN` environment variable:
319
-
320
- ```bash
321
- TESTOMATIO={API_KEY} TESTOMATIO_EXCLUDE_FILES_FROM_REPORT_GLOB_PATTERN="**/*.setup.ts" <actual run command>
322
- ```
323
-
324
- You can use multiple patterns separated by `;`:
325
-
326
- ```bash
327
- TESTOMATIO={API_KEY} TESTOMATIO_EXCLUDE_FILES_FROM_REPORT_GLOB_PATTERN="**/*.setup.ts;tests/*.auth.js" <actual run command>
328
- ```
329
-
330
- (Any files in `node_modules` will be ignored).
331
-
@@ -1,229 +0,0 @@
1
- ---
2
- title: Python Test Frameworks
3
- description: Learn how to integrate the Testomat.io reporter with Python test frameworks like Pytest. The documentation covers installation, synchronizing tests, configuring test environments, reporting results, and submitting test artifacts. It also includes detailed configuration options such as managing test artifacts through S3 and setting up environment variables for reporting.
4
- type: article
5
- url: https://docs.testomat.io/test-reporting/python
6
- head:
7
- - tag: meta
8
- attrs:
9
- name: keywords
10
- content: Python test frameworks, Pytest, Testomat.io, test synchronization, test reporting, test artifacts, automation, S3 storage, test configuration, API integration, continuous integration, environment setup
11
- ---
12
-
13
- :::note
14
- Taken from [Pytestomatio Reporter Readme](https://github.com/testomatio/pytestomatio)
15
- :::
16
-
17
- [![Support Ukraine Badge](https://bit.ly/support-ukraine-now)](https://github.com/support-ukraine/support-ukraine)
18
-
19
- # testomat.io plugin for pytest
20
-
21
- ## uses testomat.io API:
22
-
23
- - https://testomatio.github.io/check-tests/
24
- - https://testomatio.github.io/reporter/
25
-
26
- ## Installation
27
-
28
- ```bash
29
- pip install pytestomatio
30
- ```
31
-
32
- ## Usage
33
-
34
- Synchronize tests to testomat.io and get back test id.
35
- Tests will not be executed
36
-
37
- ```bash
38
- pytest --testomatio sync
39
- ```
40
-
41
- Remove all test ids from source code. Tests will not be executed
42
-
43
- ```bash
44
- pytest --testomatio remove
45
- ```
46
-
47
- Run pytest and send test results into testomat.io.
48
- Test can be sent to testomat.io without ids in your test code. If testomat.io failed to match tests by title, it will create
49
- new tests for the run
50
-
51
- ```bash
52
- pytest --testomatio report
53
- ```
54
-
55
- Run pytest with debug parameter to get test data collected in metadata.json file
56
-
57
- ```bash
58
- pytest --testomatio debug
59
- ```
60
-
61
- ## Configuration with environment variables
62
- You can use environment variable to control certain features of testomat.io
63
-
64
- | Env variable | What it does | Examples |
65
- |--------|--------|-------|
66
- | TESTOMATIO| Provides token for pytestomatio to access and push data to testomat.io | TESTOMATIO=tstmt_***** pytest --testomatio sync|
67
- |TESTOMATIO_RUN|Name of a test run to create on testomat.io|TESTOMATIO_RUN="Smoke test" pytest --testomatio report|
68
- |TESTOMATIO_RUN_ID| Id of existing test run to use for sending test results to | TESTOMATIO_RUN_ID=98dfas0 pytest --testomatio report |
69
- |TESTOMATIO_RUNGROUP_TITLE| Create a group (folder) for a test run | TESTOMATIO_RUNGROUP_TITLE="Release 2.0" pytest --testomatio report|
70
- |TESTOMATIO_ENV|Assign environment to a test run |TESTOMATIO_ENV="linux,chrome,1920x1080" pytest --testomatio report|
71
- |TESTOMATIO_LABELS|Assign labels to a test run |TESTOMATIO_ENV="smoke,regression" pytest --testomatio report|
72
- |TESTOMATIO_SYNC_LABELS|Assign labels to a test case when you synchronise test from code with testomat.io|TESTOMATIO_SYNC_LABELS="number:1,list:one,standalone" pytest --testomatio report|
73
- |TESTOMATIO_CODE_STYLE|If you are not sure, don't set this variable. Default value is 'default'|TESTOMATIO_CODE_STYLE=pep8 pytest --testomatio sync|
74
- |TESTOMATIO_CI_DOWNSTREAM|If set, pytestomatio will not set or update build url for a test run. This is useful in scenarios where build url is already set in the test run by Testomat.io for test runs that a created directly on Testomat.io.|TESTOMATIO_CI_DOWNSTREAM=true pytest --testomatio report|
75
-
76
-
77
- ### Run groups parameter
78
- There is environment variable `TESTOMATIO_RUNGROUP_TITLE` that can be used to specify run tests with specific group title.
79
-
80
- ### pytest.ini
81
- In case you are using private testomat.io service, create `pytest.ini` file in your project root directory. Specify
82
- testomat.io url in it
83
-
84
- ```ini
85
- [pytest]
86
- testomatio_url = https://app.testomat.io
87
-
88
- ```
89
-
90
- ### Submitting Test Run Environment
91
-
92
- to configure test environment, you can use additional option:
93
-
94
- ```bash
95
- pytest --testomatio report --testRunEnv "windows11,chrome,1920x1080"
96
- ```
97
-
98
- Environment values are comma separated, please use double quotation.
99
-
100
-
101
- ### Submitting Test Artifacts
102
-
103
- testomat.io does not store any screenshots, logs or other artifacts.
104
-
105
- In order to manage them it is advised to use S3 Buckets (GCP Storage).
106
- https://docs.testomat.io/usage/test-artifacts/
107
-
108
- Analyser needs to be aware of the cloud storage credentials.
109
- There are two options:
110
- 1. Enable **Share credentials with testomat.io Reporter** option in testomat.io Settings -> Artifacts.
111
- 2. Use environment variables `ACCESS_KEY_ID, SECRET_ACCESS_KEY, ENDPOINT, BUCKET, BUCKET_PATH`
112
-
113
- You would need to decide when you want to upload your test artifacts to cloud storage
114
-
115
- 1) Upload page screenshot when test fails, using fixtures [reference](https://docs.pytest.org/en/latest/example/simple.html#making-test-result-information-available-in-fixtures)
116
-
117
- ```python
118
- # content of conftest.py
119
- import pytest
120
- import random
121
- import os
122
- from typing import Dict
123
- from pytest import StashKey, CollectReport
124
- from playwright.sync_api import Page
125
-
126
- phase_report_key = StashKey[Dict[str, CollectReport]]()
127
-
128
- @pytest.hookimpl(wrapper=True, tryfirst=True)
129
- def pytest_runtest_makereport(item, call):
130
- rep = yield
131
- item.stash.setdefault(phase_report_key, {})[rep.when] = rep
132
- return rep
133
-
134
-
135
- @pytest.fixture(scope="function")
136
- def handle_artifacts(page: Page, request):
137
- yield
138
- report = request.node.stash[phase_report_key]
139
- if ("call" not in report) or report["setup"].failed or report["call"].failed:
140
- random_string = ''.join(random.choices(string.ascii_letters + string.digits, k=8))
141
-
142
- filename = f"{random_string}.png"
143
- screenshot_path = os.path.join(artifacts_dir, filename)
144
- page.screenshot(path=screenshot_path)
145
- # file_path - required, path to file to be uploaded
146
- # file_bytes - required, bytes of the file to be uploaded
147
- # key - required, file name in the s3 bucket
148
- # bucket_name - optional,name of the bucket to upload file to. Default value is taken from testomat.io
149
- artifact_url = pytest.testomatio.upload_file(screenshot_path, filename)
150
- # or
151
- # artifact_url = pytest.testomatio.upload_file_object(file_bytes, key, bucket_name)
152
- pytest.testomatio.add_artifacts(request.node, [artifact_url])
153
- page.close()
154
- ```
155
-
156
- ⚠️ Please take into account s3_connector available only after **pytest_collection_modifyitems()** hook is executed.
157
-
158
- 2) If you prefer to use pytest hooks - add `pytest_runtest_makereport` hook in your `conftest.py` file.
159
-
160
- ```python
161
- def pytest_runtest_makereport(item, call):
162
- artifact_url = pytest.testomatio.upload_file(screenshot_path, filename)
163
- pytest.testomatio.add_artifacts([artifact_url])
164
- ```
165
-
166
- ### Clarifications
167
-
168
- - tests can be synced even without `@patest.mark.testomatio('@T96c700e6')` decorator.
169
- - test title in testomat.io == test name in pytest
170
- - test suit title in testomat.io == test file name in pytest
171
-
172
- ## Example of test
173
-
174
- To make the experience more consistent, it uses standard pytest markers.
175
- testomat.io test id is a string value that starts with `@T` and has 8 symbols after.
176
-
177
- ```python
178
- from pytest import mark
179
-
180
-
181
- @mark.testomatio('@T96c700e6')
182
- def test_example():
183
- assert 2 + 2 == 4
184
- ```
185
-
186
- ### Compatibility table with [Testomatio check-tests](https://github.com/testomatio/check-tests)
187
-
188
- | Action | Compatibility | Method |
189
- |--------|--------|-------|
190
- | Report tests to testomat.io | complete | `pytest --testomatio report` |
191
- | Filter test by id to run | complete | `pytest --testomatio report --test-id="T00C73028\|T00C73029"` |
192
- | Importing test into testomat.io | complete | `pytest --testomatio sync` |
193
- | Exclude hook code of a test | N/A | N/A |
194
- | Include line number code of a test | N/A | N/A |
195
- | Import Parametrized Tests | complete | default behaviour |
196
- | Disable Detached Tests | complete | `pytest --testomatio sync --no-detach` |
197
- | Synchronous Import | complete | default behaviour |
198
- | Auto-assign Test IDs in Source Code | complete | default behaviour |
199
- | Keep Test IDs Between Projects | complete | `pytest --testomatio sync --create` |
200
- | Clean Test IDs | complete | `pytest --testomatio remove` |
201
- | Import Into a Branch | N/A | N/A |
202
- | Keep Structure of Source Code | complete | `pytest --testomatio sync --keep-structure` |
203
- | Delete Empty Suites | complete | `pytest --testomatio sync --no-empty` |
204
- | Import Into a Specific Folder | complete | `pytest --testomatio sync --directory "Windows\smoke"` |
205
- | Debugging | parity | `pytest --testomatio debug` |
206
-
207
-
208
- ## Test
209
- - import into empty project
210
- - updated test - (resync)
211
- - test run
212
- - test run into a folder
213
- - test run labels, tags
214
-
215
- ## TODO
216
- - retry test run update with less attributes, we get 500 from api
217
- - handler non configured s3 bucket error
218
- - Fix test duration
219
-
220
- ## Contribution
221
- Use python 3.12
222
-
223
- 1. `pip install ".[dev]"`
224
- 1. `python ./smoke.py`
225
- 1. Test things manually
226
- 1. Verify no regression bugs
227
- 1. `cz commit`
228
- 1. `cz bump`
229
- 1. `git push remoteName branchName --tags`