testeiya 0.2.1 → 0.3.10

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (402) hide show
  1. package/LICENSE +21 -0
  2. package/README.md +284 -171
  3. package/dist/prompt/context.js +79 -0
  4. package/dist/prompt/context.js.map +1 -0
  5. package/dist/prompt/index.js +60 -0
  6. package/dist/prompt/index.js.map +1 -0
  7. package/dist/prompt/print.js +30 -0
  8. package/dist/prompt/print.js.map +1 -0
  9. package/dist/prompt/project-info.js +2 -0
  10. package/dist/prompt/project-info.js.map +1 -0
  11. package/{src/prompt/system-prompt.ts → dist/prompt/system-prompt.js} +38 -25
  12. package/dist/prompt/system-prompt.js.map +1 -0
  13. package/dist/prompt/testomatio.js +264 -0
  14. package/dist/prompt/testomatio.js.map +1 -0
  15. package/dist/prompt/tools.js +57 -0
  16. package/dist/prompt/tools.js.map +1 -0
  17. package/dist/prompt/vocab.js +8 -0
  18. package/dist/prompt/vocab.js.map +1 -0
  19. package/dist/src/args.js +172 -0
  20. package/dist/src/args.js.map +1 -0
  21. package/dist/src/cli.js +97 -0
  22. package/dist/src/cli.js.map +1 -0
  23. package/dist/src/doctor.js +186 -0
  24. package/dist/src/doctor.js.map +1 -0
  25. package/dist/src/env.js +93 -0
  26. package/dist/src/env.js.map +1 -0
  27. package/dist/src/mcp-extension.js +9 -0
  28. package/dist/src/mcp-extension.js.map +1 -0
  29. package/dist/src/mcp.js +139 -0
  30. package/dist/src/mcp.js.map +1 -0
  31. package/dist/src/model.js +71 -0
  32. package/dist/src/model.js.map +1 -0
  33. package/dist/src/models.js +38 -0
  34. package/dist/src/models.js.map +1 -0
  35. package/dist/src/output.js +185 -0
  36. package/dist/src/output.js.map +1 -0
  37. package/dist/src/result.js +41 -0
  38. package/dist/src/result.js.map +1 -0
  39. package/dist/src/run.js +291 -0
  40. package/dist/src/run.js.map +1 -0
  41. package/dist/src/session.js +92 -0
  42. package/dist/src/session.js.map +1 -0
  43. package/dist/src/sessions.js +61 -0
  44. package/dist/src/sessions.js.map +1 -0
  45. package/dist/vendor/mcp.js +14801 -0
  46. package/mcp-tools.json +2967 -0
  47. package/package.json +34 -43
  48. package/prompt/context.ts +101 -0
  49. package/prompt/index.ts +102 -0
  50. package/prompt/print.ts +32 -0
  51. package/prompt/project-info.ts +30 -0
  52. package/prompt/system-prompt.ts +187 -0
  53. package/{src/prompt → prompt}/testomatio.ts +120 -9
  54. package/prompt/tools.ts +57 -0
  55. package/prompt/vocab.ts +7 -0
  56. package/skills/codeceptjs/ci-fix-tests/SKILL.md +7 -8
  57. package/skills/codeceptjs/codeceptjs-auth/SKILL.md +47 -64
  58. package/skills/codeceptjs/codeceptjs-exploration/SKILL.md +56 -47
  59. package/skills/codeceptjs/codeceptjs-fundamentals/SKILL.md +144 -106
  60. package/skills/codeceptjs/codeceptjs-run-analysis/SKILL.md +50 -57
  61. package/skills/codeceptjs/debugging-codeceptjs-tests/SKILL.md +88 -196
  62. package/skills/codeceptjs/migrate-codeceptjs-4/SKILL.md +4 -5
  63. package/skills/codeceptjs/migrate-cypress-to-codeceptjs/SKILL.md +25 -36
  64. package/skills/codeceptjs/migrate-protractor-to-codeceptjs/SKILL.md +23 -36
  65. package/skills/codeceptjs/migrate-selenium-java-to-codeceptjs/SKILL.md +23 -36
  66. package/skills/codeceptjs/migrate-testcafe-to-codeceptjs/SKILL.md +23 -35
  67. package/skills/codeceptjs/refactoring-codeceptjs-tests/SKILL.md +50 -53
  68. package/skills/codeceptjs/writing-codeceptjs-tests/SKILL.md +87 -191
  69. package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/SKILL.md +1 -1
  70. package/{skills.lock.json → skills/skills.lock.json} +43 -36
  71. package/skills/skills.yaml +34 -0
  72. package/skills/testomatio/explorbot/explorbot-fundamentals/SKILL.md +49 -61
  73. package/skills/testomatio/explorbot/explorbot-fundamentals/references/no-install.md +36 -0
  74. package/skills/testomatio/explorbot/explorbot-plan/SKILL.md +20 -7
  75. package/skills/testomatio/explorbot/explorbot-setup/SKILL.md +41 -289
  76. package/skills/testomatio/qa-process/qa-explain-behavior/SKILL.md +111 -0
  77. package/skills/testomatio/qa-process/qa-explain-behavior/references/answer-shapes.md +116 -0
  78. package/skills/testomatio/qa-process/qa-explain-behavior/references/examples.md +157 -0
  79. package/skills/testomatio/qa-process/qa-lead-strategy-advisor/SKILL.md +91 -0
  80. package/skills/testomatio/qa-process/qa-lead-strategy-advisor/references/output-format.md +71 -0
  81. package/skills/testomatio/qa-process/testing-workflow/SKILL.md +227 -0
  82. package/skills/testomatio/test-automation/qa-data-seeder/SKILL.md +83 -0
  83. package/skills/testomatio/test-automation/run-tests-with-testomatio-reporter/SKILL.md +5 -3
  84. package/skills/testomatio/test-automation/setup-change-aware-pr-testing/SKILL.md +190 -0
  85. package/skills/testomatio/test-automation/setup-ci-automation/SKILL.md +76 -0
  86. package/skills/testomatio/test-management/qa-split-testing-levels-pyramid/SKILL.md +55 -0
  87. package/skills/testomatio/test-management/qa-sprint-report-by-testomatio/SKILL.md +220 -0
  88. package/skills/testomatio/test-management/qa-sprint-report-by-testomatio/references/qa-sprint-report.md +126 -0
  89. package/skills/testomatio/test-management/qa-test-code-coverage/SKILL.md +1 -1
  90. package/skills/testomatio/test-management/qa-thinking/SKILL.md +25 -25
  91. package/skills/testomatio/test-management/testomatio-mcp/SKILL.md +14 -1
  92. package/skills/testomatio/test-management/testomatio-mcp/references/MCP_SETUP.md +8 -0
  93. package/assets/testeiya-logo-dark.svg +0 -16
  94. package/assets/testeiya-logo.svg +0 -15
  95. package/models.catalog.json +0 -6963
  96. package/skills/playwright-best-practices-skill/LICENSE.md +0 -7
  97. package/skills/playwright-best-practices-skill/README.md +0 -147
  98. package/skills/testeiya/README.md +0 -29
  99. package/skills/testeiya/answer-formatting/SKILL.md +0 -30
  100. package/skills/testeiya/check-cucumber/SKILL.md +0 -200
  101. package/skills/testeiya/check-tests/SKILL.md +0 -260
  102. package/skills/testeiya/manual-run-assistant/SKILL.md +0 -61
  103. package/skills/testeiya/testomatio-docs/INDEX.md +0 -175
  104. package/skills/testeiya/testomatio-docs/SKILL.md +0 -102
  105. package/skills/testeiya/testomatio-docs/docs/advanced/ai-powered-features/ai-agents.md +0 -144
  106. package/skills/testeiya/testomatio-docs/docs/advanced/ai-powered-features/ai-powered-features.md +0 -531
  107. package/skills/testeiya/testomatio-docs/docs/advanced/ai-powered-features/ai-requirements.md +0 -437
  108. package/skills/testeiya/testomatio-docs/docs/advanced/api-access/index.md +0 -191
  109. package/skills/testeiya/testomatio-docs/docs/advanced/branches-folder/index.md +0 -277
  110. package/skills/testeiya/testomatio-docs/docs/advanced/bulk-edit-folder/bulk-edit-demos.mdx +0 -39
  111. package/skills/testeiya/testomatio-docs/docs/advanced/bulk-edit-folder/bulk-edit-on-suite-and-test-level.md +0 -122
  112. package/skills/testeiya/testomatio-docs/docs/advanced/bulk-edit-folder/what-is-bulk-edit.md +0 -170
  113. package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/branches.md +0 -53
  114. package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/index.mdx +0 -98
  115. package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/jira-issue.md +0 -151
  116. package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/runs-and-reports.md +0 -377
  117. package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/work-in-jira.md +0 -267
  118. package/skills/testeiya/testomatio-docs/docs/advanced/living-doc/index.md +0 -138
  119. package/skills/testeiya/testomatio-docs/docs/advanced/milestones/index.md +0 -220
  120. package/skills/testeiya/testomatio-docs/docs/advanced/shortcuts/index.md +0 -48
  121. package/skills/testeiya/testomatio-docs/docs/advanced/tags-labels/labels-and-custom-fields.md +0 -201
  122. package/skills/testeiya/testomatio-docs/docs/advanced/tags-labels/tags-or-labels.md +0 -94
  123. package/skills/testeiya/testomatio-docs/docs/advanced/tags-labels/tags.md +0 -243
  124. package/skills/testeiya/testomatio-docs/docs/advanced/test-artifacts/index.md +0 -135
  125. package/skills/testeiya/testomatio-docs/docs/advanced/tql/index.md +0 -394
  126. package/skills/testeiya/testomatio-docs/docs/getting-started/index.md +0 -290
  127. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/azure.md +0 -96
  128. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/bamboo.md +0 -94
  129. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/bitbucket.md +0 -106
  130. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/circle.md +0 -115
  131. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/github.md +0 -120
  132. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/gitlab.md +0 -88
  133. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/index.mdx +0 -151
  134. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/jenkins.md +0 -121
  135. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/teamcity.md +0 -96
  136. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/azure.md +0 -31
  137. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/clickup.md +0 -54
  138. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/confluence.md +0 -34
  139. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/github.md +0 -31
  140. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/gitlab.md +0 -93
  141. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/index.md +0 -131
  142. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/jira.md +0 -164
  143. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/linear.md +0 -29
  144. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/shortcut.md +0 -67
  145. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/youtrack.md +0 -94
  146. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/azure-devops.md +0 -53
  147. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/email.md +0 -71
  148. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/jira.md +0 -52
  149. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/ms-teams.md +0 -60
  150. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/rules.md +0 -267
  151. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/slack.md +0 -82
  152. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/telegram.md +0 -73
  153. package/skills/testeiya/testomatio-docs/docs/integrations/scim-provisioning/index.mdx +0 -207
  154. package/skills/testeiya/testomatio-docs/docs/integrations/scim-provisioning/okta.md +0 -204
  155. package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/azure.md +0 -95
  156. package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/google.md +0 -75
  157. package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/index.mdx +0 -33
  158. package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/okta.md +0 -108
  159. package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/saml.md +0 -66
  160. package/skills/testeiya/testomatio-docs/docs/legal/compliance/gdpr.md +0 -95
  161. package/skills/testeiya/testomatio-docs/docs/legal/compliance/index.mdx +0 -94
  162. package/skills/testeiya/testomatio-docs/docs/legal/compliance/privacy.md +0 -26
  163. package/skills/testeiya/testomatio-docs/docs/legal/compliance/terms.md +0 -32
  164. package/skills/testeiya/testomatio-docs/docs/legal/security/index.mdx +0 -242
  165. package/skills/testeiya/testomatio-docs/docs/legal/security/jira.md +0 -140
  166. package/skills/testeiya/testomatio-docs/docs/legal/security/subprocessors.md +0 -76
  167. package/skills/testeiya/testomatio-docs/docs/management/company/administration.md +0 -363
  168. package/skills/testeiya/testomatio-docs/docs/management/company/audit-log.md +0 -41
  169. package/skills/testeiya/testomatio-docs/docs/management/company/plan-features-comparison.md +0 -240
  170. package/skills/testeiya/testomatio-docs/docs/management/company/subscriptions.md +0 -326
  171. package/skills/testeiya/testomatio-docs/docs/management/company/trials.md +0 -75
  172. package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/accountant.md +0 -49
  173. package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/index.mdx +0 -285
  174. package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/read-only-user.md +0 -67
  175. package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/teams.md +0 -138
  176. package/skills/testeiya/testomatio-docs/docs/management/project/settings.md +0 -87
  177. package/skills/testeiya/testomatio-docs/docs/management/project/templates.md +0 -546
  178. package/skills/testeiya/testomatio-docs/docs/project/analytics/index.md +0 -326
  179. package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/classical-tests-markdown-format.md +0 -386
  180. package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/download-manual-tests-as-files.md +0 -378
  181. package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/export-to-obsidian.md +0 -112
  182. package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/export-to-spreadsheet.md +0 -127
  183. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/auto-import.md +0 -63
  184. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-Allure-TestOps.md +0 -49
  185. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-QTest.md +0 -41
  186. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-Qase.md +0 -39
  187. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-Zephyr.md +0 -37
  188. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-csv-xlsx.md +0 -117
  189. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-cucumber.md +0 -93
  190. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-qmetry.md +0 -44
  191. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-source-code.md +0 -117
  192. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-testcaselabs.md +0 -44
  193. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-testmo.md +0 -47
  194. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-testrail.md +0 -82
  195. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-xray.md +0 -55
  196. package/skills/testeiya/testomatio-docs/docs/project/plans/index.md +0 -334
  197. package/skills/testeiya/testomatio-docs/docs/project/pulse/index.md +0 -104
  198. package/skills/testeiya/testomatio-docs/docs/project/runs/archive-runs-and-groups.md +0 -176
  199. package/skills/testeiya/testomatio-docs/docs/project/runs/custom-statuses.md +0 -195
  200. package/skills/testeiya/testomatio-docs/docs/project/runs/environments.md +0 -203
  201. package/skills/testeiya/testomatio-docs/docs/project/runs/managing-runs.md +0 -455
  202. package/skills/testeiya/testomatio-docs/docs/project/runs/merge-strategies.md +0 -137
  203. package/skills/testeiya/testomatio-docs/docs/project/runs/reports.md +0 -329
  204. package/skills/testeiya/testomatio-docs/docs/project/runs/rungroups.md +0 -342
  205. package/skills/testeiya/testomatio-docs/docs/project/runs/running-automated-tests.md +0 -256
  206. package/skills/testeiya/testomatio-docs/docs/project/runs/running-manual-and-automated-tests.md +0 -331
  207. package/skills/testeiya/testomatio-docs/docs/project/runs/running-tests-manually.md +0 -633
  208. package/skills/testeiya/testomatio-docs/docs/project/runs/temporary-tests-notes.md +0 -223
  209. package/skills/testeiya/testomatio-docs/docs/project/steps-snippets/snippets.md +0 -68
  210. package/skills/testeiya/testomatio-docs/docs/project/steps-snippets/steps.md +0 -181
  211. package/skills/testeiya/testomatio-docs/docs/project/tests/bdd-test-case-editor.md +0 -98
  212. package/skills/testeiya/testomatio-docs/docs/project/tests/classical-test-case-editor.md +0 -354
  213. package/skills/testeiya/testomatio-docs/docs/project/tests/classical-vs-bdd.md +0 -169
  214. package/skills/testeiya/testomatio-docs/docs/project/tests/copy-and-move-your-tests.md +0 -156
  215. package/skills/testeiya/testomatio-docs/docs/project/tests/other-features-for-test-case-design.md +0 -412
  216. package/skills/testeiya/testomatio-docs/docs/project/tests/test-case-creation-and-editing.md +0 -527
  217. package/skills/testeiya/testomatio-docs/docs/support/index.md +0 -75
  218. package/skills/testeiya/testomatio-docs/docs/test-reporting/artifacts.md +0 -388
  219. package/skills/testeiya/testomatio-docs/docs/test-reporting/cli.md +0 -215
  220. package/skills/testeiya/testomatio-docs/docs/test-reporting/configuration.md +0 -247
  221. package/skills/testeiya/testomatio-docs/docs/test-reporting/frameworks.md +0 -490
  222. package/skills/testeiya/testomatio-docs/docs/test-reporting/functions.md +0 -144
  223. package/skills/testeiya/testomatio-docs/docs/test-reporting/index.mdx +0 -48
  224. package/skills/testeiya/testomatio-docs/docs/test-reporting/junit.md +0 -248
  225. package/skills/testeiya/testomatio-docs/docs/test-reporting/logger.md +0 -138
  226. package/skills/testeiya/testomatio-docs/docs/test-reporting/php.md +0 -77
  227. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/bitbucket.md +0 -72
  228. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/csv.md +0 -23
  229. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/debug.md +0 -18
  230. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/github.md +0 -63
  231. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/gitlab.md +0 -29
  232. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/html.md +0 -80
  233. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/index.mdx +0 -57
  234. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/testomatio.md +0 -331
  235. package/skills/testeiya/testomatio-docs/docs/test-reporting/python.md +0 -229
  236. package/skills/testeiya/testomatio-docs/docs/test-reporting/workflows.md +0 -325
  237. package/skills/testeiya/testomatio-docs/docs/tutorials/Interacting-via-API.md +0 -159
  238. package/skills/testeiya/testomatio-docs/docs/tutorials/Manual-Testing-Classic.md +0 -164
  239. package/skills/testeiya/testomatio-docs/docs/tutorials/Set-Up-Read-Only-Access.md +0 -85
  240. package/skills/testeiya/testomatio-docs/docs/tutorials/Using-AI-in-Testing.md +0 -163
  241. package/skills/testeiya/testomatio-docs/docs/tutorials/Working-with-Jira.md +0 -125
  242. package/skills/testeiya/testomatio-docs/docs/tutorials/java.md +0 -478
  243. package/skills/testeiya/testomatio-docs/docs/tutorials/playwright.md +0 -207
  244. package/skills/testeiya/testomatio-docs/docs/tutorials/webdriver.md +0 -218
  245. package/skills/testeiya/testomatio-docs/docs.lock.json +0 -6
  246. package/skills/testeiya/testomatio-reporter/SKILL.md +0 -307
  247. package/skills/testomatio/test-automation/setup-pr-testing/SKILL.md +0 -199
  248. package/skills.yaml +0 -21
  249. package/src/ai-debug.ts +0 -188
  250. package/src/api/agent-get.ts +0 -22
  251. package/src/api/agent-start.ts +0 -384
  252. package/src/api/client-log.ts +0 -29
  253. package/src/api/context.ts +0 -332
  254. package/src/api/debug-layout.ts +0 -47
  255. package/src/api/debug-report.ts +0 -29
  256. package/src/api/debug-snapshot.ts +0 -30
  257. package/src/api/debug-stream.ts +0 -52
  258. package/src/api/files-delete.ts +0 -184
  259. package/src/api/files-read.ts +0 -41
  260. package/src/api/files-rename.ts +0 -99
  261. package/src/api/files-tree.ts +0 -292
  262. package/src/api/files-write.ts +0 -41
  263. package/src/api/mcp.ts +0 -603
  264. package/src/api/memory.ts +0 -69
  265. package/src/api/open-external.ts +0 -39
  266. package/src/api/playwright-cli.ts +0 -606
  267. package/src/api/providers.ts +0 -457
  268. package/src/api/read-env-token.ts +0 -33
  269. package/src/api/sessions.ts +0 -79
  270. package/src/api/settings.ts +0 -120
  271. package/src/api/skills.ts +0 -58
  272. package/src/api/testomatio-attachment.ts +0 -89
  273. package/src/api/testomatio-auth.ts +0 -284
  274. package/src/api/testomatio-proxy.ts +0 -197
  275. package/src/api/testomatio-run-stats.ts +0 -81
  276. package/src/api/testomatio-target.ts +0 -225
  277. package/src/api/testomatio-transcription.ts +0 -82
  278. package/src/api/workspace-search.ts +0 -96
  279. package/src/api/workspace-sync.ts +0 -151
  280. package/src/api/workspace.ts +0 -165
  281. package/src/app-server.ts +0 -556
  282. package/src/bridge.ts +0 -251
  283. package/src/check-tests.ts +0 -156
  284. package/src/cli.ts +0 -17
  285. package/src/commands.ts +0 -244
  286. package/src/config.ts +0 -183
  287. package/src/connection.ts +0 -658
  288. package/src/context-store.ts +0 -265
  289. package/src/debug-bus.ts +0 -405
  290. package/src/extensions/tool-gate.ts +0 -220
  291. package/src/extensions/webui/ask-channel.ts +0 -52
  292. package/src/extensions/webui/index.ts +0 -184
  293. package/src/extensions/webui/tools/ask-question.ts +0 -88
  294. package/src/extensions/webui/tools/query-result.ts +0 -79
  295. package/src/extensions/webui/tools/render-chart.ts +0 -53
  296. package/src/extensions/webui/tools/render-item.ts +0 -61
  297. package/src/extensions/webui/tools/render-list.ts +0 -170
  298. package/src/extensions/webui/tools/render-result.ts +0 -85
  299. package/src/extensions/webui/tools/render-tree.ts +0 -71
  300. package/src/extensions/webui/tools/ui-widget.ts +0 -54
  301. package/src/extensions/webui/tools/widget-result.ts +0 -13
  302. package/src/extensions/webui/widget-channel.ts +0 -50
  303. package/src/file-log.ts +0 -196
  304. package/src/git-tracked.ts +0 -68
  305. package/src/gitignore.ts +0 -53
  306. package/src/json-store.ts +0 -43
  307. package/src/load-env.ts +0 -70
  308. package/src/loader-style.ts +0 -96
  309. package/src/main.ts +0 -140
  310. package/src/mcp-catalog.ts +0 -96
  311. package/src/models-catalog.ts +0 -72
  312. package/src/permissions.ts +0 -120
  313. package/src/project-dir.ts +0 -189
  314. package/src/project-info.ts +0 -108
  315. package/src/prompt/app-ui.ts +0 -59
  316. package/src/prompt/browser.ts +0 -27
  317. package/src/prompt/index.ts +0 -62
  318. package/src/prompt/tools.ts +0 -27
  319. package/src/server-info.ts +0 -62
  320. package/src/session-factory.ts +0 -442
  321. package/src/session-store.ts +0 -115
  322. package/src/shell-env.ts +0 -45
  323. package/src/skills.ts +0 -163
  324. package/src/sync-snapshot.ts +0 -140
  325. package/src/telemetry.ts +0 -239
  326. package/src/testomatio-auth.ts +0 -299
  327. package/src/testomatio.ts +0 -98
  328. package/src/theme.ts +0 -30
  329. package/src/user-env.ts +0 -94
  330. package/src/welcome.ts +0 -194
  331. package/src/workspace/safe-path.ts +0 -85
  332. package/src/workspace/test-md.ts +0 -70
  333. package/src/workspace-model.ts +0 -440
  334. package/testeiya.config.json +0 -14
  335. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/authentication-flows.md +0 -0
  336. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/authentication.md +0 -0
  337. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/clock-mocking.md +0 -0
  338. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/mobile-testing.md +0 -0
  339. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/multi-context.md +0 -0
  340. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/multi-user.md +0 -0
  341. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/network-advanced.md +0 -0
  342. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/third-party.md +0 -0
  343. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/architecture/pom-vs-fixtures.md +0 -0
  344. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/architecture/test-architecture.md +0 -0
  345. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/architecture/when-to-mock.md +0 -0
  346. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/browser-apis/browser-apis.md +0 -0
  347. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/browser-apis/iframes.md +0 -0
  348. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/browser-apis/service-workers.md +0 -0
  349. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/browser-apis/websockets.md +0 -0
  350. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/annotations.md +0 -0
  351. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/assertions-waiting.md +0 -0
  352. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/configuration.md +0 -0
  353. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/fixtures-hooks.md +0 -0
  354. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/global-setup.md +0 -0
  355. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/locators.md +0 -0
  356. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/page-object-model.md +0 -0
  357. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/projects-dependencies.md +0 -0
  358. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/test-data.md +0 -0
  359. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/test-suite-structure.md +0 -0
  360. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/test-tags.md +0 -0
  361. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/debugging/console-errors.md +0 -0
  362. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/debugging/debugging.md +0 -0
  363. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/debugging/error-testing.md +0 -0
  364. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/debugging/flaky-tests.md +0 -0
  365. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/frameworks/angular.md +0 -0
  366. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/frameworks/nextjs.md +0 -0
  367. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/frameworks/react.md +0 -0
  368. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/frameworks/vue.md +0 -0
  369. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/ci-cd.md +0 -0
  370. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/docker.md +0 -0
  371. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/github-actions.md +0 -0
  372. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/gitlab.md +0 -0
  373. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/other-providers.md +0 -0
  374. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/parallel-sharding.md +0 -0
  375. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/performance.md +0 -0
  376. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/reporting.md +0 -0
  377. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/test-coverage.md +0 -0
  378. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/accessibility.md +0 -0
  379. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/api-testing.md +0 -0
  380. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/browser-extensions.md +0 -0
  381. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/canvas-webgl.md +0 -0
  382. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/component-testing.md +0 -0
  383. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/drag-drop.md +0 -0
  384. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/electron.md +0 -0
  385. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/file-operations.md +0 -0
  386. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/file-upload-download.md +0 -0
  387. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/forms-validation.md +0 -0
  388. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/graphql-testing.md +0 -0
  389. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/i18n.md +0 -0
  390. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/performance-testing.md +0 -0
  391. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/security-testing.md +0 -0
  392. /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/visual-regression.md +0 -0
  393. /package/skills/{playwright-cli → playwright/playwright-cli}/SKILL.md +0 -0
  394. /package/skills/{playwright-cli → playwright/playwright-cli}/references/element-attributes.md +0 -0
  395. /package/skills/{playwright-cli → playwright/playwright-cli}/references/playwright-tests.md +0 -0
  396. /package/skills/{playwright-cli → playwright/playwright-cli}/references/request-mocking.md +0 -0
  397. /package/skills/{playwright-cli → playwright/playwright-cli}/references/running-code.md +0 -0
  398. /package/skills/{playwright-cli → playwright/playwright-cli}/references/session-management.md +0 -0
  399. /package/skills/{playwright-cli → playwright/playwright-cli}/references/storage-state.md +0 -0
  400. /package/skills/{playwright-cli → playwright/playwright-cli}/references/test-generation.md +0 -0
  401. /package/skills/{playwright-cli → playwright/playwright-cli}/references/tracing.md +0 -0
  402. /package/skills/{playwright-cli → playwright/playwright-cli}/references/video-recording.md +0 -0
@@ -1,329 +0,0 @@
1
- ---
2
- title: Run Reports
3
- description: Learn how Testomat.io generates comprehensive Run Reports for both manual and automated tests, offering basic and extended views. This guide covers features like filtering, sorting, customizing report layouts, viewing test case history, and accessing run statistics. It also details options for downloading reports as spreadsheets or PDFs, sharing reports via email or public links, and comparing test runs for in-depth analysis.
4
- type: article
5
- url: https://docs.testomat.io/project/runs/reports
6
- head:
7
- - tag: meta
8
- attrs:
9
- name: keywords
10
- content: managing test runs, run reports, rungroups, test management, Testomat.io,multi-selection, test reporting, sharing reports, test run statistics, customize test cases view, download run report, run report in Spreadsheet, export run report, PDF format, Share Run Report, Share Report by Email, Share Report Publicly, Public Report, Combined Report.
11
- ---
12
-
13
- When you run tests with Testomat.io, the system generates a **Run Report** displaying your test results. You can view these reports in two ways: **basic view** and **extended view**. Let's explore each option.
14
-
15
- ## Run Report Basic View
16
-
17
- Clicking on the Test Run will toggle Run Report screen in basic view.
18
-
19
- ![Testomat.io - RunReport](./images/Run_Report_1.png)
20
-
21
- **In this basic Run Report, you can:**
22
-
23
- 1. View a list of test cases.
24
- 2. Check their statuses (Passed, Failed, Skipped, Pending, Custom statuses).
25
- 3. Review general information on the Run Result.
26
-
27
- ![Testomat.io - RunReport](./images/Run_Report_2.png)
28
-
29
- 4. Sort test cases by suite, test case names or failure status.
30
- 5. Filter test cases by status, type, added messages, custom statuses, or assignees.
31
-
32
- ![Testomat.io - RunReport](./images/Run_Report_7.png)
33
-
34
- 6. Search test cases by title or message.
35
-
36
- ![Testomat.io - RunReport](./images/Run_Report_8.png)
37
-
38
- 7. Customize test cases view - (learn more in the [How to Customize Test Cases View](https://docs.testomat.io/project/runs/reports/#how-to-customize-test-cases-view) section).
39
-
40
- ![Testomat.io - RunReport](./images/Custom_view_8.png)
41
-
42
- Additionally, click on any test case within the run report to open it and view its details.
43
-
44
- ![Testomat.io - RunReport](./images/Run_Report_3.png)
45
-
46
- Moreover, you can view the history of run results for selected test case by filtering them by group name, defects, labels or environment.
47
-
48
- ![Testomat.io - Test result history in RunGroup](./images/Run_Report_4.gif)
49
-
50
- You can also check overall **Statistics** by suites/tags/labels/assignees/priorities/custom statuses for the selected test run.
51
-
52
- ![Testomat.io - Test run Statistics](./images/Run_Report_5.gif)
53
-
54
- ### How to Customize Test Cases View
55
-
56
- Testomat.io allows you to customize the dispalay of test cases within a Run Report.
57
-
58
- **This feature helps you:**
59
-
60
- - **Show or hide columns** such as test status, duration, labels, tags, priority, etc.
61
- - **Focus only on the data** most reelvant to you or your team.
62
- - **Tailor the report layout** for different review needs.
63
-
64
- To cusmozite your Run Report view, follow next steps:
65
-
66
- 1. Select any Run.
67
- 2. Click the **'Custom view'** button to open the configurable view instantly.
68
-
69
- ![Testomat.io - RunReport Customization](./images/Custom_view_9.png)
70
-
71
- 3. Click the **'Settings'** icon to access **'Runs List Settings'**.
72
- 4. Adjust Run view:
73
-
74
- - Select/deselect columns to show only the data you need.
75
- - Set each column width (px), to suit your preferences (or leave empty for automatic sizing).
76
-
77
- 5. Click the **'Save'** button to apply your changes.
78
-
79
- ![Testomat.io - RunReport Customization](./images/Custom_view_10.png)
80
-
81
- You can also adjust each column width manually, in this case changes will save automatically.
82
-
83
- ![Testomat.io - RunReport Customization](./images/Custom_view_11.gif)
84
-
85
- :::note
86
-
87
- Customized Test Cases view within a Run Report automatically applies to all Run Reports.
88
-
89
- :::
90
-
91
- ### Tree View for Runs
92
-
93
- Runs can be displayed in a hierarchical tree structure that mirrors the test organization. To switch to tree view, open a run and click the tree view icon in the top right corner of the test list.
94
-
95
- You can:
96
-
97
- * Expand folders
98
- * Collapse sections
99
- * Navigate large test sets faster
100
- * Focus on specific areas of a project
101
-
102
- :::note
103
-
104
- This is particularly useful for large projects containing hundreds or thousands of tests.
105
-
106
- :::
107
-
108
- ## Run Report Extended View
109
-
110
- If you click the **'Report'** button from Basic Run Report view, you will see the extended view of the Run Report.
111
-
112
- ![Testomat.io - RunReport](./images/Run_Report_6.png)
113
-
114
- **In the extended report view, you can:**
115
-
116
- 1. View a list of tests.
117
- 2. Check their status (Passed, Failed, Skipped, Pending, Custom statuses).
118
- 3. See overview of all executed tests grouped by suites, tags, labels, assignees, or priorities.
119
- 4. Review the Run Report Summary.
120
-
121
- ![Testomat.io - RunReport](./images/Run_Report_9.png)
122
-
123
- 5. Check **'Flaky Tests'** Analytics, if applicable.
124
-
125
- ![Testomat.io - RunReport](./images/Run_Report_12.png)
126
-
127
- 6. Sort suites by Name and Failed status (as well works for Tags, Labels, Assignees, Priorities, Custom Statuses) in the Overview widget.
128
-
129
- ![Testomat.io - RunReport](./images/Run_Report_10.gif)
130
-
131
- 7. Filter test cases by status, type, added messages, custom statuses, or assignees.
132
-
133
- ![Testomat.io - RunReport](./images/Run_Report_11.gif)
134
-
135
- 8. Search test cases by title or message.
136
-
137
- ![Testomat.io - RunReport](./images/Run_Report_13.png)
138
-
139
- :::note
140
-
141
- Use ↓ and ↑ keys to navigate tests and quickly view details for each test.
142
-
143
- ![Report view](./images/gif2_7537.gif)
144
-
145
- :::
146
-
147
- Similar to the basic run report view, click on a test case to see its details or the history of run results.
148
-
149
- ![All tabs view](./images/gif1_7537.gif)
150
-
151
- ## RunGroup Report
152
-
153
- The same as for Runs, you can also view RunGroups Reportes.
154
- To open **RunGroup Report basic view**, simply click on its name - RunGroup Report window will be displayed with general information and runs summary.
155
-
156
- ![Testomat.io - Test Runs in a created RunGroup](./images/RunGroup_6.png)
157
-
158
- For more information go to the [RunGroup Report Basic View](https://docs.testomat.io/project/runs/rungroups#rungroup-report-basic-view) section **'RunGroups'** page.
159
-
160
- From RunGroup Report Basic View you can open **Combined Report for RunGroups**, by clicking on the **'Combined Report'** button. This Report is designed to help you aggregate and analyze the results of multiple test runs within a single view. You can find more information about this report in [Combined Report for RunGroups](https://docs.testomat.io/project/runs/rungroups/#combined-report-for-rungroups) section on **'RunGroups'** page.
161
-
162
- ## How to Download Run Report as Spreadsheet
163
-
164
- Testomat.io allows you to export Run Reports as a spreadsheet (XLSX file) to obtain your test data.
165
-
166
- **Download Multiple Run Reports in One File**:
167
-
168
- 1. Go to Runs.
169
- 2. Enable **'Multi-select'**.
170
- 3. Select the Runs you want to download.
171
- 4. Click the **'Extra menu'** button.
172
- 5. Select **'Download'** option.
173
-
174
- ![Testomat.io - export as Spreadsheet](./images/Export_1.png)
175
-
176
- **Download a Separate Run Report**:
177
-
178
- 1. Go to Runs.
179
- 2. Select a Run Report.
180
- 3. Click the **'Report'** button.
181
-
182
- ![Testomat.io - export as Spreadsheet](./images/Export_2.png)
183
-
184
- 4. Click the **'Extra menu'** button.
185
- 5. Select **'Download as Spreadsheet'** option.
186
-
187
- ![Testomat.io - export as Spreadsheet](./images/Export_3.png)
188
-
189
- ## How to Export Run Report in PDF Format
190
-
191
- Testomat.io also allows you to export your test run results as a PDF report. This feature enables easy sharing and archiving of test outcomes outside of Testomat.io.
192
-
193
- **You can generate a structured PDF report with test execution details in two ways:**
194
-
195
- 1. Go to Runs.
196
- 2. Select a Run Report.
197
- 3. Click the **'Extra menu'** button.
198
- 4. Select **'Export as PDF'** option.
199
-
200
- ![Testomat.io - export as PDF](./images/Export_4.png)
201
-
202
- **OR**
203
-
204
- 1. Go to Runs.
205
- 2. Select a Run Report.
206
- 3. Click the **'Report'** button.
207
-
208
- ![Testomat.io - export as PDF](./images/Export_5.png)
209
-
210
- 4. Click the **'Extra menu'** button.
211
- 5. Select **'Export as PDF'** option.
212
-
213
- ![Testomat.io - export as PDF](./images/Export_6.png)
214
-
215
- **Example of Test Run Report in PDF format**
216
-
217
- ![Testomat.io - export as PDF](./images/Export_7.png)
218
-
219
- **Use cases for PDF Reports:**
220
-
221
- - **Sharing results with stakeholders:** Quickly generate a portable summary of test outcomes for product owners, QA managers, or external partners who may not have access to the platform.
222
-
223
- - **Audit and compliance documentation:** Maintain a formal record of test results in a fixed format for compliance, certification, or audit purposes.
224
-
225
- - **Team retrospectives and reviews:** Use the PDF report to review test coverage and outcomes during retrospectives or sprint reviews without relying on live access to the system.
226
-
227
- ## How to Share Run Report
228
-
229
- You can share your Run Report for external access from the Extended report view using two options: **'Share Report by Email'** or **'Share Report Publicly'**.
230
-
231
- ### Share Report by Email
232
-
233
- 1. Open Extended report view.
234
-
235
- ![Testomat.io - Share Report](./images/Expend_view.png)
236
-
237
- 2. Click the **'Extra menu'** button.
238
- 3. Select **'Share Report by Email'** option.
239
-
240
- ![Testomat.io - Share Report](./images/Share_Report_1.png)
241
-
242
- 4. Enter one or multiple emails, separated with commas, in the displayed modal.
243
- 5. Click the **'Send'** button.
244
-
245
- ![Testomat.io - Share Report](./images/Share_Report_2.png)
246
-
247
- :::note
248
-
249
- This action also creates public report without passcode for external access.
250
-
251
- :::
252
-
253
- **Example of the email with Run Report**
254
-
255
- ![Testomat.io - Share Report](./images/Share_Report_3.png)
256
-
257
- ### Share Report Publicly
258
-
259
- :::note
260
-
261
- To enable the **'Share Report Publicly'** option, first enable this feature at the **Company level** and then enable it on **Project level** (Project Settings -> Project -> ‘Sharing’ section).
262
- Check the instruction in the [Share Options](https://docs.testomat.io/management/company/administration/#share-options) section on **'Administration'** page.
263
-
264
- :::
265
-
266
- 1. Open Extended report view.
267
-
268
- ![Testomat.io - Share Report](./images/Expend_view.png)
269
-
270
- 2. Click the **'Extra menu'** button.
271
- 3. Select **'Share Report Publicly'** option.
272
-
273
- ![Testomat.io - Share Report](./images/Share_Report_4.png)
274
-
275
- 4. Select **'Expiration Date'** (7 days is set by default).
276
- 5. **'Protect by passcode'** is enabled by default for better security (you can disable it if needed).
277
- 6. Click the **'Share'** button in the displayed modal.
278
-
279
- ![Testomat.io - Share Report](./images/Share_Report_5.png)
280
-
281
- 7. Copy the **URL** for sharing the report.
282
- 8. Copy and share the **Passcode** with the link.
283
-
284
- :::note
285
-
286
- Copy and save the **Passcode** in a secure place. For security reasons, you will not be able to see it again after the Run Report Public URL is generated.
287
-
288
- If a shared run report is protected with a **Passcode**, only users with the valid **Passcode** will have access to it.
289
-
290
- :::
291
-
292
- 9. Copy and share the **Expiration date**, if needed.
293
-
294
- ![Testomat.io - Share Report](./images/Share_Report_6.png)
295
-
296
- After the Run Report Public URL is generated, **'Share'** button will be displayed on the Extended run report page.
297
-
298
- ![Testomat.io - Share Report](./images/Share_Report_7.png)
299
-
300
- :::note
301
-
302
- You can stop sharing at any time by clicking the **'Stop Sharing'** button.
303
-
304
- ![Testomat.io - Share Report](./images/Share_Report_8.png)
305
-
306
- :::
307
-
308
- Users with access to **'Public Run Report'** can:
309
-
310
- 1. Filter test cases by statuses.
311
- 2. Search test cases by name.
312
- 3. Copy the URL for sharing the Run report.
313
- 4. Check **'Full Report'** (only users with project access).
314
-
315
- ![Testomat.io - Share Report](./images/Share_Report_9.png)
316
-
317
- ## Compare Test Runs
318
-
319
- Select test runs to see a visual display of their similarities and differences. Compare your run reports and analyze them by parameters such as Flaky, Reviewed, Degraded, and more.
320
-
321
- To compare Runs:
322
-
323
- 1. Enable Multiselection.
324
- 2. Select the Runs you want to compare.
325
- 3. Click the **'Compare'** button.
326
-
327
- ![Testomat.io - Compare Test Runs](./images/compare-runs.gif)
328
-
329
- This feature is also available for RunGroups, allowing you to compare test results from different runs within a single RunGroup. You can find more information about it in the [Combined Report for RunGroups](https://docs.testomat.io/project/runs/rungroups/#combined-report-for-rungroups) section on **'RunGroups'** page.
@@ -1,342 +0,0 @@
1
- ---
2
- title: RunGroups
3
- description: Testomat.io's 'RunGroups' feature allows users to organize multiple test runs based on criteria like sprint or release, providing a consolidated view of testing efforts. This guide explains how to create, customize, move runs into, and copy RunGroups, along with options for managing their visibility, such as pinning and archiving, to streamline test management.
4
- type: article
5
- url: https://docs.testomat.io/project/runs/rungroups
6
- head:
7
- - tag: meta
8
- attrs:
9
- name: keywords
10
- content: managing test runs, run reports, merge strategy, rungroups, relaunch runs, test management, Testomat.io, multi-selection, archiving, purge run groups, purge runs, purge logic, test reporting, runGroup flow, runs dashboard flow, RunGroup chart, copy RunGroup, pin a RunGroup, archive RunGroup, unarchive RunGroup, delete run group, purge flow.
11
- ---
12
-
13
- First, let's clarify what **'RunGroups'** are in Testomat.io.
14
- **RunGroups** allow you to organize and group multiple test runs together based on criteria like sprint, release, functional area, or any other logical grouping. This helps in analyzing aggregated results and providing a consolidated view of your testing efforts.
15
-
16
- ## How to Create a RunGroup
17
-
18
- **To create a new RunGroup:**
19
-
20
- 1. Open **'Runs'** page.
21
- 2. Click the **'Additional options'** arrow button next to the 'Manual run' button.
22
- 3. Select **'New group'** option from the dropdown menu.
23
-
24
- ![Testomat.io - New Group button](./images/RunGroup_1.png)
25
-
26
- On New RunGroup screen you need:
27
-
28
- 1. Choose a **'Group Type'** (optional).
29
- 2. Add a **'Name of a Group'**.
30
- 3. Select **'Merge strategy'** (For more details, refer to the [Merge Strategies](https://docs.testomat.io/project/runs/merge-strategies) page in documentation).
31
- 4. Add **'Description of a group'**, if needed.
32
- 5. Click the **'Save'** button.
33
-
34
- ![Testomat.io - New RunGroup screen](./images/RunGroup_2.png)
35
-
36
- The new RunGroup will appear on Runs page and will open automatically after creation.
37
-
38
- ![Testomat.io - RunGroup will appea](./images/RunGroup_3.png)
39
-
40
- After RunGroup is added, you can create new Manual/Automated/Mixed Runs inside it.
41
-
42
- ## How to Create a Run in RunGroup
43
-
44
- - To create a new **Manual Run**, open the RunGroup and click on **'Manual Run'** button.
45
-
46
- ![Testomat.io - put Manual Run](./images/RunGroup_4.png)
47
-
48
- After this in the displayed window select a test plan from the list or create a new one, and fill in other data that you need for your run (like: assignee, title, environment, description).
49
-
50
- :::note
51
-
52
- When you create a manual run from RunGroup, it will be automatically pre-populated on the **'New Manual Run'** window, but you can change it if needed.
53
-
54
- :::
55
-
56
- ![Testomat.io - put Manual Run](./images/RunGroup_7.png)
57
-
58
- - To add a new **Automated Run** to a RunGroup, execute your tests with Testomat.io, providing TESTOMATIO_RUNGROUP_TITLE="Build ${BUILD_ID}".
59
-
60
- Now, by clicking on expand arrow, you can view Test Runs and their results within created RunGroup.
61
-
62
- ![Testomat.io - Test Runs in a created RunGroup](./images/RunGroup_5.png)
63
-
64
- Clicking on RunGroup name will toggle RunGroup Report screen in basic view. Here you can view more detailed information about the runs inside selected RunGroup and total results.
65
-
66
- ![Testomat.io - Test Runs in a created RunGroup](./images/RunGroup_6.png)
67
-
68
- For more details go to the relevant section - [RunGroup Report in Basic View](https://docs.testomat.io/project/runs/reports/#rungroup-report-in-basic-view) on **'Reports'** page.
69
-
70
- ## How to Move a Run to a RunGroup
71
-
72
- You may need to move a Run to a specific RunGroup (e.g., to associate it with a particular release or build). There are two ways to do this in Testomat.io:
73
-
74
- Method 1: Move a Run to a RunGroup
75
-
76
- 1. Navigate to the **'Runs'** page
77
- 2. Select the Run you want to move
78
- 3. Click the **'Extra menu'** button
79
- 4. Select the **'Move'** from the dropdown menu
80
-
81
- ![Testomat.io - Move Run option in the dropdown menu](./images/Move_Run_1.png)
82
-
83
- 5. In the pop-up window, choose the **destination RunGroup**
84
- 6. Click the **'Move'** button to confirm the action
85
-
86
- ![Testomat.io - Destination RunGroup selection dialog](./images/Move_Run_2.png)
87
-
88
- Method 2: Add Existing Runs to the RunGroup
89
-
90
- 1. Navigate to the **'Runs'** page
91
- 2. Click the **'Extra menu'** next to the RunGroup you want to move Runs into
92
- 3. Select the **'Add Existing Run'** option
93
-
94
- ![Add Existing Run](./images/att1.png)
95
-
96
- 4. In the modal window, select the Run(s) you want to move
97
- 5. Click the **Move Runs to Group** button to confirm
98
-
99
- ![Move Runs to Group](./images/att2.png)
100
-
101
- Moved or added Runs will appear in the selected RunGroup.
102
-
103
- ![List of added runs into a group](./images/att3.png)
104
-
105
- ## RunGroup Report Basic View
106
-
107
- You can view RunGroup Report by clicking on its name.
108
-
109
- ![Testomat.io - Test Runs in a created RunGroup](./images/RunGroup_6.png)
110
-
111
- **In this basic RunGroup Report, you can:**
112
-
113
- 1. View the [RunGroup Chart](https://docs.testomat.io/project/runs/rungroups/#rungroup-chart).
114
- 2. Review **'RunGroup summary section'** with general information based on the selected [Merge Strategy](https://docs.testomat.io/project/runs/merge-strategies/).
115
- 3. View a list of test runs, their statuses with the number of Passed, Failed, Skipped tests, along with the run assignee and executed time.
116
- 4. Customize your RunGroup Report view (Read more in the [How to Customize Runs List View ](https://docs.testomat.io/project/runs/rungroups/#how-to-customize-runs-list-view) section below).
117
- 5. Check **'Combined Report'** (Read more in the [Combined Report for RunGroups](https://docs.testomat.io/project/runs/rungroups/#combined-report-for-rungroups) section below).
118
- 6. Add a new Manual Run to the selected RunGroup.
119
- 7. Generate a [Rungroup Statistic Report](https://docs.testomat.io/advanced/ai-powered-features/ai-powered-features/#rungroup-statistic-report)(AI-feature should be enabled on **'Company Settings'** page).
120
-
121
- ![Testomat.io - RunGroup Report](./images/RunGroup_Report_1.png)
122
-
123
- The **'RunGroup summary section'** displays a counter with the number of included and analyzed runs in the RunGroup. Archived runs are excluded from this counter and do not affect the total RunGroup result. You can see how many runs were archived, and by clicking on archived counter, you will be redirected to the **'Groups Archive'** page, where runs are already filtered by selected RunGroup.
124
-
125
- ![Testomat.io - RunGroup Report](./images/RunGroup_Report_2.gif)
126
-
127
- ## How to Customize Runs List View
128
-
129
- When working with test runs inside RunGroup or Runs Dashboard Flow, you can adjust the table layout to fit your needs. Instead of using the default view, you can customize the runs table layout within **RunGroup** page or directly from the main **Runs Dashboard**.
130
-
131
- **RunGroup Flow**
132
-
133
- 1. Go to **'Runs'** page.
134
- 2. Select RunGroup.
135
- 3. Click the **'Custom view'** button.
136
-
137
- ![Testomat.io - Custom View option](images/Custom_view_1.png)
138
-
139
- You can use the default custom view or personalize the table layout:
140
-
141
- 4. Click the **'Settings'** icon to access **'Runs List Settings'**.
142
-
143
- ![Testomat.io - Custom View](images/Custom_view_2.png)
144
-
145
- 5. Select or deselect columns to show only the data you need.
146
- 6. Set the column width (px), if needed, to improve readability.
147
- 7. Click the **'Save'** button to apply your changes.
148
-
149
- ![Testomat.io - Save changes](images/Custom_view_3.png)
150
-
151
- Here's how your customized table within a RunGroup will appear after customization:
152
-
153
- ![Testomat.io - Updated Custom View](images/Custom_view_4.png)
154
-
155
- **Runs Dashboard Flow**
156
-
157
- Similarly, customize the runs view from the main Runs Dashboard:
158
-
159
- 1. Go to **'Runs'** page.
160
- 2. Click the **'Custom view'** button.
161
-
162
- ![Testomat.io - Custom view button](images/Custom_view_5.png)
163
-
164
- 3. Click the **'Settings'** icon to access **'Runs List Settings'**.
165
- 4. Customize your view in the **'Runs list settings'**.
166
- 5. Click the **'Save'** button to apply your changes.
167
-
168
- ![Testomat.io - Save changes](images/Custom_view_6.png)
169
-
170
- Here's how your customized table will appear.
171
-
172
- ![Testomat.io - Updated view](images/Custom_view_7.png)
173
-
174
- These customization options allow you to tailor the Runs View to display only the most relevant details, improving clarity and efficiency.
175
-
176
- :::note
177
-
178
- A customized runs list view automatically applies to all your RunGroups and main Runs Dashboard.
179
-
180
- :::
181
-
182
- ## RunGroup Chart
183
-
184
- The chart displays up to 50 of the latest test runs belonging to the group. If you have more runs, use the pagination arrows to view the results of the previous runs.
185
-
186
- ![Testomat.io - RunGroup Chart](./images/RunGroup_Chart.png)
187
-
188
- ## Combined Report for RunGroups
189
-
190
- Testomat.io provides a **'Combined Report'** feature for RunGroups. This report is designed to help you aggregate and analyze the results of all test runs within the RunGroup in a single view.
191
-
192
- You can open **'Combined Report'** for a RunGroup by clicking on **'Combined Report'** button from the RunGroup Report Basic View window.
193
-
194
- ![Testomat.io - Combined Report](./images/Combined_report.png)
195
-
196
- The comparison of runs is based on the first (main) launch. The statuses counters on the left side are calculated based on the main run.
197
-
198
- To change the main run, click the **'Compare To'** button. Note that the counters of loaded tests is above them. The final summary on the right side is calculated based on the loaded tests.
199
-
200
- ![Testomat.io - Combined Report](./images/Combined_report_1.png)
201
-
202
- :::note
203
-
204
- The **'Combined Report'** feature for RunGroups does **not** combine test results of all runs inside one RunGroup into a single report. It only allows you **to compare** test results of all runs within this RunGroup.
205
-
206
- :::
207
-
208
- Inside the **'Combined Report'**, you can get a quick overview of the pass/fail rates for all tests within the Group. You can also see the total number of tests present in all compared runs, as well as the total number of flaky (unstable tests that passed or failed in different runs), reviewed (previously failing that passed in next runs), and degraded (previously passed that failed in next runs) tests.
209
-
210
- Additionally, you can use filters and search features to view the data most relevant to your needs.
211
-
212
- ![Testomat.io - Combined Report](./images/Combined_report_2.gif)
213
-
214
- **Key benefits:**
215
-
216
- - **Analyze Trends:** By seeing the results from multiple runs in one place, you can identify trends in test performance and stability over time.
217
-
218
- - **Consolidate Data:** The combined report is especially useful for "mixed runs" (runs that include both manual and automated tests), as it consolidates all results into a single, cohesive report.
219
-
220
- - **Compare Runs:** Testomat.io also has a separate **'Compare Test Runs'** feature that can be used within a RunGroup. This allows you to side-by-side compare the results of different runs to pinpoint changes, regressions, or improvements. (Go to [Compare Test Runs](https://docs.testomat.io/project/runs/reports/#compare-test-runs) section on **'Run reports'** page to read more about this feature.)
221
-
222
- This feature is a powerful tool for project managers, QA leads, and stakeholders who need to make informed decisions based on a holistic view of testing activities, without having to manually compile data from individual reports.
223
-
224
- ## How to Copy RunGroup
225
-
226
- You can easily create a new RunGroup, completely independent of any previous runs, by copying all relevant tests exactly as they are. You can configure what data should be copied, namely:
227
-
228
- - **Assignee**: Define assignee details separately, preventing them from being copied.
229
- - **Issues**: Choose whether to include or exclude linked issues during duplication.
230
- - **Labels**: Decide whether labels should be duplicated or omitted in the new test run.
231
- - **Environments**: Control the duplication of environment settings based on your requirements.
232
- - **Nested Structure**: Preserve or exclude the nested structure of your test groups as you duplicate them.
233
-
234
- **Follow these steps:**
235
-
236
- 1. Open the RunGroup.
237
- 2. Click the **'Extra menu'** button.
238
- 3. Select **'Copy'** option form the dropdown menu.
239
-
240
- ![Testomat.io - Copy group 1](./images/Copy_group_1.png)
241
-
242
- 4. Select configuration options.
243
- 5. Click the **'Copy'** button.
244
-
245
- ![Testomat.io - Copy group 2](./images/Copy_group_2.png)
246
-
247
- 6. Verify the created RunGroup.
248
-
249
- ![Testomat.io - Copy group 3](./images/Copy_group_3.png)
250
-
251
- :::note
252
-
253
- Only manual runs will be copied with the RunGroup. If a copied RunGroup contains a mixed test run, automated test cases will be marked as skipped by default.
254
-
255
- :::
256
-
257
- ## How to Pin a RunGroup
258
-
259
- Testomat.io allows users to pin RunGroups to the top of the Dashboard. This provides quick access to critical or frequently used tests or test runs, helping teams stay focused on the most relevant tasks. Pinning supports faster navigation, improved focus, and customizable workflows—ideal for monitoring regression tests, environment-specific runs, or production hotfixes.
260
-
261
- For more details, refer to the [How to Pin a Run or RunGroup](https://docs.testomat.io/project/runs/managing-runs/#how-to-pin-a-run-or-rungroup) section.
262
-
263
- ## How to Archive RunGroup
264
-
265
- Testomat.io gives you an opportunity to archive a RunGroup, including all its contained Test Runs. This helps maintain better visibility on your main Run Dashboard.
266
-
267
- For more information on archiving, visit the [Archive Runs & RunGroups](https://docs.testomat.io/project/runs/archive-runs-and-groups) page.
268
-
269
- ## How to Unarchive RunGroup
270
-
271
- Archived RunGroups can be unarchived and moved back to the main Run Dashboard. Restoring a RunGroup also restores all its archived runs.
272
-
273
- Read more in the [How to Unarchive Runs & RunGroups](https://docs.testomat.io/project/runs/archive-runs-and-groups/#how-to-unarchive-runs-&-groups) section.
274
-
275
- ## Purge Run Groups
276
-
277
- Starting from **October 2025**, Testomat.io introduces the new **Purge** approach for **Run Groups**. This update extends the same **two-step purge-to-archive logic** that was introduced for individual [Runs](https://docs.testomat.io/project/runs/managing-runs/#purge-runs) in September 2025, ensuring data safety and consistency across all Run types.
278
-
279
- The **Delete** option for Run Groups is now replaced with a **Purge** option.
280
-
281
- ![Purge button in RunGroup](./images/att1_6779.png)
282
-
283
- ### Compress and Move to Archive (Purge)
284
-
285
- When a **Run Group** is purged (either manually or automatically):
286
-
287
- - The **group itself** is **deleted**
288
- - All **nested Runs** are moved to the **[Archive](https://docs.testomat.io/project/runs/archive-runs-and-groups/)** page
289
- - All **nested Runs** are marked with the **purged** badge for easy identification
290
-
291
- ![Purged button confirmation](./images/att2_6779.png)
292
-
293
- During this step:
294
-
295
- - Stack traces are removed to reduce storage size
296
- - Essential data is preserved, including:
297
- - Test results
298
- - Artifacts (attachments, logs, screenshots)
299
- - Custom statuses and metadata
300
- - Purged Runs remain available in the Archive page and can be restored at any time
301
-
302
- :::note
303
-
304
- The purge logic for **ongoing (pending) Runs** within a Run Group follows the **same rules** as described for individual Runs. For detailed information about the purge flow of ongoing Runs, see [**Purge Logic for Ongoing Runs**](https://docs.testomat.io/project/runs/managing-runs/#purge-logic-for-ongoing-runs).
305
-
306
- :::
307
-
308
- ### Automatic Purge Flow
309
-
310
- The same **two-step purge-to-archive logic** applies to automatically purged **Run Groups** via **Project Settings → [Purge Old Runs](https://docs.testomat.io/management/project/settings/#purge-old-runs)**.
311
-
312
- When an automatic purge is triggered:
313
-
314
- - The **group itself** is **deleted**
315
- - All **nested Runs** are moved to the **[Archive](https://docs.testomat.io/project/runs/archive-runs-and-groups/)** page
316
- - All **nested Runs** are marked with the **purged** badge for easy identification
317
-
318
- ### Limitations
319
-
320
- :::note
321
-
322
- Run Groups have a **limit of 20,000 Runs** for the purge process. If a group contains **more than 20,000 Runs**, only the **first 20,000** will be purged and moved to the Archive.
323
-
324
- All remaining Runs beyond this limit will be **permanently deleted** and **cannot be restored**.
325
-
326
- :::
327
-
328
- ### Permanent Deletion from Archive
329
-
330
- :::note
331
-
332
- If you remove a Run (that was part of a purged Run Group) from the Archive page, it is permanently deleted. This step is **irreversible** — once **deleted**, the Run cannot be restored.
333
-
334
- :::
335
-
336
- ![Permanent Deletion from Archive](./images/att3_6779.png)
337
-
338
- ### Summary
339
-
340
- By extending the **Purge** functionality to Run Groups, Testomat.io ensures that all grouped executions follow the same safe, transparent, and recoverable process as individual Runs.
341
-
342
- Teams can efficiently manage large-scale test executions while maintaining complete data traceability and control.