testeiya 0.2.0 → 0.3.9

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (412) hide show
  1. package/LICENSE +21 -0
  2. package/README.md +37 -175
  3. package/dist/prompt/context.js +79 -0
  4. package/dist/prompt/context.js.map +1 -0
  5. package/dist/prompt/index.js +56 -0
  6. package/dist/prompt/index.js.map +1 -0
  7. package/dist/prompt/print.js +22 -0
  8. package/dist/prompt/print.js.map +1 -0
  9. package/dist/prompt/project-info.js +2 -0
  10. package/dist/prompt/project-info.js.map +1 -0
  11. package/{src/prompt/system-prompt.ts → dist/prompt/system-prompt.js} +24 -21
  12. package/dist/prompt/system-prompt.js.map +1 -0
  13. package/dist/prompt/testomatio.js +264 -0
  14. package/dist/prompt/testomatio.js.map +1 -0
  15. package/dist/prompt/tools.js +49 -0
  16. package/dist/prompt/tools.js.map +1 -0
  17. package/dist/prompt/vocab.js +8 -0
  18. package/dist/prompt/vocab.js.map +1 -0
  19. package/dist/src/args.js +72 -0
  20. package/dist/src/args.js.map +1 -0
  21. package/dist/src/cli.js +61 -0
  22. package/dist/src/cli.js.map +1 -0
  23. package/dist/src/env.js +69 -0
  24. package/dist/src/env.js.map +1 -0
  25. package/dist/src/mcp-extension.js +9 -0
  26. package/dist/src/mcp-extension.js.map +1 -0
  27. package/dist/src/mcp.js +139 -0
  28. package/dist/src/mcp.js.map +1 -0
  29. package/dist/src/model.js +66 -0
  30. package/dist/src/model.js.map +1 -0
  31. package/dist/src/result.js +41 -0
  32. package/dist/src/result.js.map +1 -0
  33. package/dist/src/run.js +199 -0
  34. package/dist/src/run.js.map +1 -0
  35. package/dist/src/session.js +77 -0
  36. package/dist/src/session.js.map +1 -0
  37. package/dist/vendor/mcp.js +14801 -0
  38. package/mcp-tools.json +2967 -0
  39. package/package.json +29 -41
  40. package/prompt/context.ts +101 -0
  41. package/prompt/index.ts +93 -0
  42. package/prompt/print.ts +23 -0
  43. package/prompt/project-info.ts +30 -0
  44. package/prompt/system-prompt.ts +173 -0
  45. package/{src/prompt → prompt}/testomatio.ts +119 -8
  46. package/{src/prompt → prompt}/tools.ts +30 -8
  47. package/prompt/vocab.ts +7 -0
  48. package/{skills.lock.json → skills/skills.lock.json} +11 -7
  49. package/skills/skills.yaml +34 -0
  50. package/models.catalog.json +0 -6963
  51. package/skills/codeceptjs/ci-fix-tests/SKILL.md +0 -120
  52. package/skills/codeceptjs/codeceptjs-auth/SKILL.md +0 -158
  53. package/skills/codeceptjs/codeceptjs-exploration/SKILL.md +0 -91
  54. package/skills/codeceptjs/codeceptjs-fundamentals/SKILL.md +0 -158
  55. package/skills/codeceptjs/codeceptjs-run-analysis/SKILL.md +0 -90
  56. package/skills/codeceptjs/debugging-codeceptjs-tests/SKILL.md +0 -234
  57. package/skills/codeceptjs/migrate-codeceptjs-4/SKILL.md +0 -110
  58. package/skills/codeceptjs/migrate-cypress-to-codeceptjs/SKILL.md +0 -306
  59. package/skills/codeceptjs/migrate-protractor-to-codeceptjs/SKILL.md +0 -352
  60. package/skills/codeceptjs/migrate-selenium-java-to-codeceptjs/SKILL.md +0 -472
  61. package/skills/codeceptjs/migrate-testcafe-to-codeceptjs/SKILL.md +0 -350
  62. package/skills/codeceptjs/refactoring-codeceptjs-tests/SKILL.md +0 -81
  63. package/skills/codeceptjs/writing-codeceptjs-tests/SKILL.md +0 -206
  64. package/skills/playwright-best-practices-skill/LICENSE.md +0 -7
  65. package/skills/playwright-best-practices-skill/README.md +0 -147
  66. package/skills/playwright-best-practices-skill/SKILL.md +0 -303
  67. package/skills/playwright-best-practices-skill/advanced/authentication-flows.md +0 -360
  68. package/skills/playwright-best-practices-skill/advanced/authentication.md +0 -871
  69. package/skills/playwright-best-practices-skill/advanced/clock-mocking.md +0 -364
  70. package/skills/playwright-best-practices-skill/advanced/mobile-testing.md +0 -409
  71. package/skills/playwright-best-practices-skill/advanced/multi-context.md +0 -288
  72. package/skills/playwright-best-practices-skill/advanced/multi-user.md +0 -393
  73. package/skills/playwright-best-practices-skill/advanced/network-advanced.md +0 -452
  74. package/skills/playwright-best-practices-skill/advanced/third-party.md +0 -464
  75. package/skills/playwright-best-practices-skill/architecture/pom-vs-fixtures.md +0 -363
  76. package/skills/playwright-best-practices-skill/architecture/test-architecture.md +0 -369
  77. package/skills/playwright-best-practices-skill/architecture/when-to-mock.md +0 -383
  78. package/skills/playwright-best-practices-skill/browser-apis/browser-apis.md +0 -391
  79. package/skills/playwright-best-practices-skill/browser-apis/iframes.md +0 -403
  80. package/skills/playwright-best-practices-skill/browser-apis/service-workers.md +0 -504
  81. package/skills/playwright-best-practices-skill/browser-apis/websockets.md +0 -403
  82. package/skills/playwright-best-practices-skill/core/annotations.md +0 -424
  83. package/skills/playwright-best-practices-skill/core/assertions-waiting.md +0 -361
  84. package/skills/playwright-best-practices-skill/core/configuration.md +0 -452
  85. package/skills/playwright-best-practices-skill/core/fixtures-hooks.md +0 -417
  86. package/skills/playwright-best-practices-skill/core/global-setup.md +0 -434
  87. package/skills/playwright-best-practices-skill/core/locators.md +0 -242
  88. package/skills/playwright-best-practices-skill/core/page-object-model.md +0 -315
  89. package/skills/playwright-best-practices-skill/core/projects-dependencies.md +0 -453
  90. package/skills/playwright-best-practices-skill/core/test-data.md +0 -492
  91. package/skills/playwright-best-practices-skill/core/test-suite-structure.md +0 -361
  92. package/skills/playwright-best-practices-skill/core/test-tags.md +0 -298
  93. package/skills/playwright-best-practices-skill/debugging/console-errors.md +0 -420
  94. package/skills/playwright-best-practices-skill/debugging/debugging.md +0 -504
  95. package/skills/playwright-best-practices-skill/debugging/error-testing.md +0 -360
  96. package/skills/playwright-best-practices-skill/debugging/flaky-tests.md +0 -496
  97. package/skills/playwright-best-practices-skill/frameworks/angular.md +0 -530
  98. package/skills/playwright-best-practices-skill/frameworks/nextjs.md +0 -469
  99. package/skills/playwright-best-practices-skill/frameworks/react.md +0 -531
  100. package/skills/playwright-best-practices-skill/frameworks/vue.md +0 -574
  101. package/skills/playwright-best-practices-skill/infrastructure-ci-cd/ci-cd.md +0 -468
  102. package/skills/playwright-best-practices-skill/infrastructure-ci-cd/docker.md +0 -283
  103. package/skills/playwright-best-practices-skill/infrastructure-ci-cd/github-actions.md +0 -546
  104. package/skills/playwright-best-practices-skill/infrastructure-ci-cd/gitlab.md +0 -397
  105. package/skills/playwright-best-practices-skill/infrastructure-ci-cd/other-providers.md +0 -521
  106. package/skills/playwright-best-practices-skill/infrastructure-ci-cd/parallel-sharding.md +0 -371
  107. package/skills/playwright-best-practices-skill/infrastructure-ci-cd/performance.md +0 -453
  108. package/skills/playwright-best-practices-skill/infrastructure-ci-cd/reporting.md +0 -424
  109. package/skills/playwright-best-practices-skill/infrastructure-ci-cd/test-coverage.md +0 -497
  110. package/skills/playwright-best-practices-skill/testing-patterns/accessibility.md +0 -359
  111. package/skills/playwright-best-practices-skill/testing-patterns/api-testing.md +0 -719
  112. package/skills/playwright-best-practices-skill/testing-patterns/browser-extensions.md +0 -506
  113. package/skills/playwright-best-practices-skill/testing-patterns/canvas-webgl.md +0 -493
  114. package/skills/playwright-best-practices-skill/testing-patterns/component-testing.md +0 -500
  115. package/skills/playwright-best-practices-skill/testing-patterns/drag-drop.md +0 -576
  116. package/skills/playwright-best-practices-skill/testing-patterns/electron.md +0 -509
  117. package/skills/playwright-best-practices-skill/testing-patterns/file-operations.md +0 -377
  118. package/skills/playwright-best-practices-skill/testing-patterns/file-upload-download.md +0 -562
  119. package/skills/playwright-best-practices-skill/testing-patterns/forms-validation.md +0 -561
  120. package/skills/playwright-best-practices-skill/testing-patterns/graphql-testing.md +0 -331
  121. package/skills/playwright-best-practices-skill/testing-patterns/i18n.md +0 -508
  122. package/skills/playwright-best-practices-skill/testing-patterns/performance-testing.md +0 -476
  123. package/skills/playwright-best-practices-skill/testing-patterns/security-testing.md +0 -430
  124. package/skills/playwright-best-practices-skill/testing-patterns/visual-regression.md +0 -634
  125. package/skills/playwright-cli/SKILL.md +0 -420
  126. package/skills/playwright-cli/references/element-attributes.md +0 -23
  127. package/skills/playwright-cli/references/playwright-tests.md +0 -39
  128. package/skills/playwright-cli/references/request-mocking.md +0 -87
  129. package/skills/playwright-cli/references/running-code.md +0 -241
  130. package/skills/playwright-cli/references/session-management.md +0 -225
  131. package/skills/playwright-cli/references/storage-state.md +0 -275
  132. package/skills/playwright-cli/references/test-generation.md +0 -433
  133. package/skills/playwright-cli/references/tracing.md +0 -139
  134. package/skills/playwright-cli/references/video-recording.md +0 -143
  135. package/skills/testeiya/README.md +0 -29
  136. package/skills/testeiya/answer-formatting/SKILL.md +0 -30
  137. package/skills/testeiya/check-cucumber/SKILL.md +0 -200
  138. package/skills/testeiya/check-tests/SKILL.md +0 -260
  139. package/skills/testeiya/manual-run-assistant/SKILL.md +0 -61
  140. package/skills/testeiya/testomatio-docs/INDEX.md +0 -175
  141. package/skills/testeiya/testomatio-docs/SKILL.md +0 -102
  142. package/skills/testeiya/testomatio-docs/docs/advanced/ai-powered-features/ai-agents.md +0 -144
  143. package/skills/testeiya/testomatio-docs/docs/advanced/ai-powered-features/ai-powered-features.md +0 -531
  144. package/skills/testeiya/testomatio-docs/docs/advanced/ai-powered-features/ai-requirements.md +0 -437
  145. package/skills/testeiya/testomatio-docs/docs/advanced/api-access/index.md +0 -191
  146. package/skills/testeiya/testomatio-docs/docs/advanced/branches-folder/index.md +0 -277
  147. package/skills/testeiya/testomatio-docs/docs/advanced/bulk-edit-folder/bulk-edit-demos.mdx +0 -39
  148. package/skills/testeiya/testomatio-docs/docs/advanced/bulk-edit-folder/bulk-edit-on-suite-and-test-level.md +0 -122
  149. package/skills/testeiya/testomatio-docs/docs/advanced/bulk-edit-folder/what-is-bulk-edit.md +0 -170
  150. package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/branches.md +0 -53
  151. package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/index.mdx +0 -98
  152. package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/jira-issue.md +0 -151
  153. package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/runs-and-reports.md +0 -377
  154. package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/work-in-jira.md +0 -267
  155. package/skills/testeiya/testomatio-docs/docs/advanced/living-doc/index.md +0 -138
  156. package/skills/testeiya/testomatio-docs/docs/advanced/milestones/index.md +0 -220
  157. package/skills/testeiya/testomatio-docs/docs/advanced/shortcuts/index.md +0 -48
  158. package/skills/testeiya/testomatio-docs/docs/advanced/tags-labels/labels-and-custom-fields.md +0 -201
  159. package/skills/testeiya/testomatio-docs/docs/advanced/tags-labels/tags-or-labels.md +0 -94
  160. package/skills/testeiya/testomatio-docs/docs/advanced/tags-labels/tags.md +0 -243
  161. package/skills/testeiya/testomatio-docs/docs/advanced/test-artifacts/index.md +0 -135
  162. package/skills/testeiya/testomatio-docs/docs/advanced/tql/index.md +0 -394
  163. package/skills/testeiya/testomatio-docs/docs/getting-started/index.md +0 -290
  164. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/azure.md +0 -96
  165. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/bamboo.md +0 -94
  166. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/bitbucket.md +0 -106
  167. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/circle.md +0 -115
  168. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/github.md +0 -120
  169. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/gitlab.md +0 -88
  170. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/index.mdx +0 -151
  171. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/jenkins.md +0 -121
  172. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/teamcity.md +0 -96
  173. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/azure.md +0 -31
  174. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/clickup.md +0 -54
  175. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/confluence.md +0 -34
  176. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/github.md +0 -31
  177. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/gitlab.md +0 -93
  178. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/index.md +0 -131
  179. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/jira.md +0 -164
  180. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/linear.md +0 -29
  181. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/shortcut.md +0 -67
  182. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/youtrack.md +0 -94
  183. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/azure-devops.md +0 -53
  184. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/email.md +0 -71
  185. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/jira.md +0 -52
  186. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/ms-teams.md +0 -60
  187. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/rules.md +0 -267
  188. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/slack.md +0 -82
  189. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/telegram.md +0 -73
  190. package/skills/testeiya/testomatio-docs/docs/integrations/scim-provisioning/index.mdx +0 -207
  191. package/skills/testeiya/testomatio-docs/docs/integrations/scim-provisioning/okta.md +0 -204
  192. package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/azure.md +0 -95
  193. package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/google.md +0 -75
  194. package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/index.mdx +0 -33
  195. package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/okta.md +0 -108
  196. package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/saml.md +0 -66
  197. package/skills/testeiya/testomatio-docs/docs/legal/compliance/gdpr.md +0 -95
  198. package/skills/testeiya/testomatio-docs/docs/legal/compliance/index.mdx +0 -94
  199. package/skills/testeiya/testomatio-docs/docs/legal/compliance/privacy.md +0 -26
  200. package/skills/testeiya/testomatio-docs/docs/legal/compliance/terms.md +0 -32
  201. package/skills/testeiya/testomatio-docs/docs/legal/security/index.mdx +0 -242
  202. package/skills/testeiya/testomatio-docs/docs/legal/security/jira.md +0 -140
  203. package/skills/testeiya/testomatio-docs/docs/legal/security/subprocessors.md +0 -76
  204. package/skills/testeiya/testomatio-docs/docs/management/company/administration.md +0 -363
  205. package/skills/testeiya/testomatio-docs/docs/management/company/audit-log.md +0 -41
  206. package/skills/testeiya/testomatio-docs/docs/management/company/plan-features-comparison.md +0 -240
  207. package/skills/testeiya/testomatio-docs/docs/management/company/subscriptions.md +0 -326
  208. package/skills/testeiya/testomatio-docs/docs/management/company/trials.md +0 -75
  209. package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/accountant.md +0 -49
  210. package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/index.mdx +0 -285
  211. package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/read-only-user.md +0 -67
  212. package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/teams.md +0 -138
  213. package/skills/testeiya/testomatio-docs/docs/management/project/settings.md +0 -87
  214. package/skills/testeiya/testomatio-docs/docs/management/project/templates.md +0 -546
  215. package/skills/testeiya/testomatio-docs/docs/project/analytics/index.md +0 -326
  216. package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/classical-tests-markdown-format.md +0 -386
  217. package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/download-manual-tests-as-files.md +0 -378
  218. package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/export-to-obsidian.md +0 -112
  219. package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/export-to-spreadsheet.md +0 -127
  220. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/auto-import.md +0 -63
  221. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-Allure-TestOps.md +0 -49
  222. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-QTest.md +0 -41
  223. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-Qase.md +0 -39
  224. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-Zephyr.md +0 -37
  225. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-csv-xlsx.md +0 -117
  226. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-cucumber.md +0 -93
  227. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-qmetry.md +0 -44
  228. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-source-code.md +0 -117
  229. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-testcaselabs.md +0 -44
  230. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-testmo.md +0 -47
  231. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-testrail.md +0 -82
  232. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-xray.md +0 -55
  233. package/skills/testeiya/testomatio-docs/docs/project/plans/index.md +0 -334
  234. package/skills/testeiya/testomatio-docs/docs/project/pulse/index.md +0 -104
  235. package/skills/testeiya/testomatio-docs/docs/project/runs/archive-runs-and-groups.md +0 -176
  236. package/skills/testeiya/testomatio-docs/docs/project/runs/custom-statuses.md +0 -195
  237. package/skills/testeiya/testomatio-docs/docs/project/runs/environments.md +0 -203
  238. package/skills/testeiya/testomatio-docs/docs/project/runs/managing-runs.md +0 -455
  239. package/skills/testeiya/testomatio-docs/docs/project/runs/merge-strategies.md +0 -137
  240. package/skills/testeiya/testomatio-docs/docs/project/runs/reports.md +0 -329
  241. package/skills/testeiya/testomatio-docs/docs/project/runs/rungroups.md +0 -342
  242. package/skills/testeiya/testomatio-docs/docs/project/runs/running-automated-tests.md +0 -256
  243. package/skills/testeiya/testomatio-docs/docs/project/runs/running-manual-and-automated-tests.md +0 -331
  244. package/skills/testeiya/testomatio-docs/docs/project/runs/running-tests-manually.md +0 -633
  245. package/skills/testeiya/testomatio-docs/docs/project/runs/temporary-tests-notes.md +0 -223
  246. package/skills/testeiya/testomatio-docs/docs/project/steps-snippets/snippets.md +0 -68
  247. package/skills/testeiya/testomatio-docs/docs/project/steps-snippets/steps.md +0 -181
  248. package/skills/testeiya/testomatio-docs/docs/project/tests/bdd-test-case-editor.md +0 -98
  249. package/skills/testeiya/testomatio-docs/docs/project/tests/classical-test-case-editor.md +0 -354
  250. package/skills/testeiya/testomatio-docs/docs/project/tests/classical-vs-bdd.md +0 -169
  251. package/skills/testeiya/testomatio-docs/docs/project/tests/copy-and-move-your-tests.md +0 -156
  252. package/skills/testeiya/testomatio-docs/docs/project/tests/other-features-for-test-case-design.md +0 -412
  253. package/skills/testeiya/testomatio-docs/docs/project/tests/test-case-creation-and-editing.md +0 -527
  254. package/skills/testeiya/testomatio-docs/docs/support/index.md +0 -75
  255. package/skills/testeiya/testomatio-docs/docs/test-reporting/artifacts.md +0 -388
  256. package/skills/testeiya/testomatio-docs/docs/test-reporting/cli.md +0 -215
  257. package/skills/testeiya/testomatio-docs/docs/test-reporting/configuration.md +0 -247
  258. package/skills/testeiya/testomatio-docs/docs/test-reporting/frameworks.md +0 -490
  259. package/skills/testeiya/testomatio-docs/docs/test-reporting/functions.md +0 -144
  260. package/skills/testeiya/testomatio-docs/docs/test-reporting/index.mdx +0 -48
  261. package/skills/testeiya/testomatio-docs/docs/test-reporting/junit.md +0 -248
  262. package/skills/testeiya/testomatio-docs/docs/test-reporting/logger.md +0 -138
  263. package/skills/testeiya/testomatio-docs/docs/test-reporting/php.md +0 -77
  264. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/bitbucket.md +0 -72
  265. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/csv.md +0 -23
  266. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/debug.md +0 -18
  267. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/github.md +0 -63
  268. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/gitlab.md +0 -29
  269. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/html.md +0 -80
  270. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/index.mdx +0 -57
  271. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/testomatio.md +0 -331
  272. package/skills/testeiya/testomatio-docs/docs/test-reporting/python.md +0 -229
  273. package/skills/testeiya/testomatio-docs/docs/test-reporting/workflows.md +0 -325
  274. package/skills/testeiya/testomatio-docs/docs/tutorials/Interacting-via-API.md +0 -159
  275. package/skills/testeiya/testomatio-docs/docs/tutorials/Manual-Testing-Classic.md +0 -164
  276. package/skills/testeiya/testomatio-docs/docs/tutorials/Set-Up-Read-Only-Access.md +0 -85
  277. package/skills/testeiya/testomatio-docs/docs/tutorials/Using-AI-in-Testing.md +0 -163
  278. package/skills/testeiya/testomatio-docs/docs/tutorials/Working-with-Jira.md +0 -125
  279. package/skills/testeiya/testomatio-docs/docs/tutorials/java.md +0 -478
  280. package/skills/testeiya/testomatio-docs/docs/tutorials/playwright.md +0 -207
  281. package/skills/testeiya/testomatio-docs/docs/tutorials/webdriver.md +0 -218
  282. package/skills/testeiya/testomatio-docs/docs.lock.json +0 -6
  283. package/skills/testeiya/testomatio-reporter/SKILL.md +0 -307
  284. package/skills/testomatio/explorbot/explorbot-fundamentals/SKILL.md +0 -88
  285. package/skills/testomatio/explorbot/explorbot-plan/SKILL.md +0 -95
  286. package/skills/testomatio/explorbot/explorbot-setup/SKILL.md +0 -334
  287. package/skills/testomatio/test-automation/automate-manual-test-cases/CLAUDE.md +0 -56
  288. package/skills/testomatio/test-automation/automate-manual-test-cases/SKILL.md +0 -242
  289. package/skills/testomatio/test-automation/automate-manual-test-cases/references/CODECEPTJS_BEST_PRACTICES.md +0 -182
  290. package/skills/testomatio/test-automation/automate-manual-test-cases/references/FINAL_SUMMARY_TEMPLATE.md +0 -22
  291. package/skills/testomatio/test-automation/automate-manual-test-cases/references/PLAYWRIGHT_BEST_PRACTICES.md +0 -90
  292. package/skills/testomatio/test-automation/automate-manual-test-cases/references/POM_BEST_PRACTICES.md +0 -53
  293. package/skills/testomatio/test-automation/automate-manual-test-cases/references/TEST_DATA_MANAGEMENT.md +0 -52
  294. package/skills/testomatio/test-automation/debug-fix-failed-flaky-autotests/SKILL.md +0 -107
  295. package/skills/testomatio/test-automation/debug-fix-failed-flaky-autotests/references/DEBUGGING_QUICK_REFERENCE.md +0 -113
  296. package/skills/testomatio/test-automation/qa-automation-test-consolidation/SKILL.md +0 -29
  297. package/skills/testomatio/test-automation/run-tests-with-testomatio-reporter/SKILL.md +0 -151
  298. package/skills/testomatio/test-automation/setup-pr-testing/SKILL.md +0 -199
  299. package/skills/testomatio/test-automation/testomat-allure-adapter/SKILL.md +0 -289
  300. package/skills/testomatio/test-management/detect-duplicate-test-cases/SKILL.md +0 -86
  301. package/skills/testomatio/test-management/detect-duplicate-test-cases/references/DUPLICATE_INSTRUCTIONS.md +0 -54
  302. package/skills/testomatio/test-management/improve-test-cases/SKILL.md +0 -114
  303. package/skills/testomatio/test-management/improve-test-cases/references/TESTOMAT_MARKDOWN_EXAMPLE.md +0 -66
  304. package/skills/testomatio/test-management/pull-request-diff-analyzer/SKILL.md +0 -141
  305. package/skills/testomatio/test-management/qa-e2e-tests-reporting/SKILL.md +0 -354
  306. package/skills/testomatio/test-management/qa-e2e-tests-reporting/references/TESTOMATIO_ARTIFACTS.md +0 -82
  307. package/skills/testomatio/test-management/qa-e2e-tests-reporting/references/TESTOMATIO_HTML_REPORT.md +0 -37
  308. package/skills/testomatio/test-management/qa-e2e-tests-reporting/references/TESTOMATIO_REPORTERS_CONFIG.md +0 -118
  309. package/skills/testomatio/test-management/qa-pr-requirements-analyzer/SKILL.md +0 -149
  310. package/skills/testomatio/test-management/qa-pr-requirements-analyzer/references/summary-example.md +0 -54
  311. package/skills/testomatio/test-management/qa-requirement-reviewer/SKILL.md +0 -126
  312. package/skills/testomatio/test-management/qa-requirement-reviewer/references/requirements_reviewer_examples.md +0 -128
  313. package/skills/testomatio/test-management/qa-test-code-coverage/SKILL.md +0 -174
  314. package/skills/testomatio/test-management/qa-test-code-coverage/references/COVERAGE_FILE_FORMAT.md +0 -138
  315. package/skills/testomatio/test-management/qa-test-code-coverage/references/E2E_FRAMEWORKS.md +0 -65
  316. package/skills/testomatio/test-management/qa-test-code-coverage/scripts/check-coverage.mjs +0 -40
  317. package/skills/testomatio/test-management/qa-thinking/SKILL.md +0 -36
  318. package/skills/testomatio/test-management/qa-write-test-cases/SKILL.md +0 -267
  319. package/skills/testomatio/test-management/qa-write-test-cases/references/test-case-format.md +0 -387
  320. package/skills/testomatio/test-management/qa-write-test-cases/references/testomat-tms-guide.md +0 -72
  321. package/skills/testomatio/test-management/qa-write-test-cases/references/writing-rule.md +0 -159
  322. package/skills/testomatio/test-management/scan-automation-project/SKILL.md +0 -152
  323. package/skills/testomatio/test-management/sync-test-cases-with-tms/SKILL.md +0 -107
  324. package/skills/testomatio/test-management/sync-test-cases-with-tms/references/TESTOMATIO_CLI.md +0 -140
  325. package/skills/testomatio/test-management/testomatio-mcp/SKILL.md +0 -182
  326. package/skills/testomatio/test-management/testomatio-mcp/references/MCP_SETUP.md +0 -169
  327. package/skills.yaml +0 -21
  328. package/src/ai-debug.ts +0 -188
  329. package/src/api/agent-get.ts +0 -22
  330. package/src/api/agent-start.ts +0 -384
  331. package/src/api/client-log.ts +0 -29
  332. package/src/api/context.ts +0 -332
  333. package/src/api/debug-layout.ts +0 -47
  334. package/src/api/debug-report.ts +0 -29
  335. package/src/api/debug-snapshot.ts +0 -30
  336. package/src/api/debug-stream.ts +0 -52
  337. package/src/api/files-delete.ts +0 -184
  338. package/src/api/files-read.ts +0 -41
  339. package/src/api/files-rename.ts +0 -99
  340. package/src/api/files-tree.ts +0 -292
  341. package/src/api/files-write.ts +0 -41
  342. package/src/api/mcp.ts +0 -603
  343. package/src/api/memory.ts +0 -69
  344. package/src/api/open-external.ts +0 -39
  345. package/src/api/playwright-cli.ts +0 -606
  346. package/src/api/providers.ts +0 -457
  347. package/src/api/read-env-token.ts +0 -33
  348. package/src/api/sessions.ts +0 -79
  349. package/src/api/settings.ts +0 -120
  350. package/src/api/skills.ts +0 -58
  351. package/src/api/testomatio-attachment.ts +0 -89
  352. package/src/api/testomatio-auth.ts +0 -284
  353. package/src/api/testomatio-proxy.ts +0 -197
  354. package/src/api/testomatio-run-stats.ts +0 -81
  355. package/src/api/testomatio-target.ts +0 -225
  356. package/src/api/testomatio-transcription.ts +0 -82
  357. package/src/api/workspace-search.ts +0 -96
  358. package/src/api/workspace-sync.ts +0 -151
  359. package/src/api/workspace.ts +0 -165
  360. package/src/app-server.ts +0 -556
  361. package/src/bridge.ts +0 -251
  362. package/src/check-tests.ts +0 -156
  363. package/src/cli.ts +0 -17
  364. package/src/commands.ts +0 -244
  365. package/src/config.ts +0 -183
  366. package/src/connection.ts +0 -658
  367. package/src/context-store.ts +0 -265
  368. package/src/debug-bus.ts +0 -405
  369. package/src/extensions/tool-gate.ts +0 -220
  370. package/src/extensions/webui/ask-channel.ts +0 -52
  371. package/src/extensions/webui/index.ts +0 -184
  372. package/src/extensions/webui/tools/ask-question.ts +0 -88
  373. package/src/extensions/webui/tools/query-result.ts +0 -79
  374. package/src/extensions/webui/tools/render-chart.ts +0 -53
  375. package/src/extensions/webui/tools/render-item.ts +0 -61
  376. package/src/extensions/webui/tools/render-list.ts +0 -170
  377. package/src/extensions/webui/tools/render-result.ts +0 -85
  378. package/src/extensions/webui/tools/render-tree.ts +0 -71
  379. package/src/extensions/webui/tools/ui-widget.ts +0 -54
  380. package/src/extensions/webui/tools/widget-result.ts +0 -13
  381. package/src/extensions/webui/widget-channel.ts +0 -50
  382. package/src/file-log.ts +0 -196
  383. package/src/git-tracked.ts +0 -68
  384. package/src/gitignore.ts +0 -53
  385. package/src/json-store.ts +0 -43
  386. package/src/load-env.ts +0 -70
  387. package/src/loader-style.ts +0 -96
  388. package/src/main.ts +0 -140
  389. package/src/mcp-catalog.ts +0 -96
  390. package/src/models-catalog.ts +0 -72
  391. package/src/permissions.ts +0 -120
  392. package/src/project-dir.ts +0 -189
  393. package/src/project-info.ts +0 -108
  394. package/src/prompt/app-ui.ts +0 -59
  395. package/src/prompt/browser.ts +0 -27
  396. package/src/prompt/index.ts +0 -62
  397. package/src/server-info.ts +0 -62
  398. package/src/session-factory.ts +0 -442
  399. package/src/session-store.ts +0 -115
  400. package/src/shell-env.ts +0 -45
  401. package/src/skills.ts +0 -163
  402. package/src/sync-snapshot.ts +0 -140
  403. package/src/telemetry.ts +0 -239
  404. package/src/testomatio-auth.ts +0 -299
  405. package/src/testomatio.ts +0 -98
  406. package/src/theme.ts +0 -30
  407. package/src/user-env.ts +0 -94
  408. package/src/welcome.ts +0 -127
  409. package/src/workspace/safe-path.ts +0 -85
  410. package/src/workspace/test-md.ts +0 -70
  411. package/src/workspace-model.ts +0 -440
  412. package/testeiya.config.json +0 -14
@@ -1,329 +0,0 @@
1
- ---
2
- title: Run Reports
3
- description: Learn how Testomat.io generates comprehensive Run Reports for both manual and automated tests, offering basic and extended views. This guide covers features like filtering, sorting, customizing report layouts, viewing test case history, and accessing run statistics. It also details options for downloading reports as spreadsheets or PDFs, sharing reports via email or public links, and comparing test runs for in-depth analysis.
4
- type: article
5
- url: https://docs.testomat.io/project/runs/reports
6
- head:
7
- - tag: meta
8
- attrs:
9
- name: keywords
10
- content: managing test runs, run reports, rungroups, test management, Testomat.io,multi-selection, test reporting, sharing reports, test run statistics, customize test cases view, download run report, run report in Spreadsheet, export run report, PDF format, Share Run Report, Share Report by Email, Share Report Publicly, Public Report, Combined Report.
11
- ---
12
-
13
- When you run tests with Testomat.io, the system generates a **Run Report** displaying your test results. You can view these reports in two ways: **basic view** and **extended view**. Let's explore each option.
14
-
15
- ## Run Report Basic View
16
-
17
- Clicking on the Test Run will toggle Run Report screen in basic view.
18
-
19
- ![Testomat.io - RunReport](./images/Run_Report_1.png)
20
-
21
- **In this basic Run Report, you can:**
22
-
23
- 1. View a list of test cases.
24
- 2. Check their statuses (Passed, Failed, Skipped, Pending, Custom statuses).
25
- 3. Review general information on the Run Result.
26
-
27
- ![Testomat.io - RunReport](./images/Run_Report_2.png)
28
-
29
- 4. Sort test cases by suite, test case names or failure status.
30
- 5. Filter test cases by status, type, added messages, custom statuses, or assignees.
31
-
32
- ![Testomat.io - RunReport](./images/Run_Report_7.png)
33
-
34
- 6. Search test cases by title or message.
35
-
36
- ![Testomat.io - RunReport](./images/Run_Report_8.png)
37
-
38
- 7. Customize test cases view - (learn more in the [How to Customize Test Cases View](https://docs.testomat.io/project/runs/reports/#how-to-customize-test-cases-view) section).
39
-
40
- ![Testomat.io - RunReport](./images/Custom_view_8.png)
41
-
42
- Additionally, click on any test case within the run report to open it and view its details.
43
-
44
- ![Testomat.io - RunReport](./images/Run_Report_3.png)
45
-
46
- Moreover, you can view the history of run results for selected test case by filtering them by group name, defects, labels or environment.
47
-
48
- ![Testomat.io - Test result history in RunGroup](./images/Run_Report_4.gif)
49
-
50
- You can also check overall **Statistics** by suites/tags/labels/assignees/priorities/custom statuses for the selected test run.
51
-
52
- ![Testomat.io - Test run Statistics](./images/Run_Report_5.gif)
53
-
54
- ### How to Customize Test Cases View
55
-
56
- Testomat.io allows you to customize the dispalay of test cases within a Run Report.
57
-
58
- **This feature helps you:**
59
-
60
- - **Show or hide columns** such as test status, duration, labels, tags, priority, etc.
61
- - **Focus only on the data** most reelvant to you or your team.
62
- - **Tailor the report layout** for different review needs.
63
-
64
- To cusmozite your Run Report view, follow next steps:
65
-
66
- 1. Select any Run.
67
- 2. Click the **'Custom view'** button to open the configurable view instantly.
68
-
69
- ![Testomat.io - RunReport Customization](./images/Custom_view_9.png)
70
-
71
- 3. Click the **'Settings'** icon to access **'Runs List Settings'**.
72
- 4. Adjust Run view:
73
-
74
- - Select/deselect columns to show only the data you need.
75
- - Set each column width (px), to suit your preferences (or leave empty for automatic sizing).
76
-
77
- 5. Click the **'Save'** button to apply your changes.
78
-
79
- ![Testomat.io - RunReport Customization](./images/Custom_view_10.png)
80
-
81
- You can also adjust each column width manually, in this case changes will save automatically.
82
-
83
- ![Testomat.io - RunReport Customization](./images/Custom_view_11.gif)
84
-
85
- :::note
86
-
87
- Customized Test Cases view within a Run Report automatically applies to all Run Reports.
88
-
89
- :::
90
-
91
- ### Tree View for Runs
92
-
93
- Runs can be displayed in a hierarchical tree structure that mirrors the test organization. To switch to tree view, open a run and click the tree view icon in the top right corner of the test list.
94
-
95
- You can:
96
-
97
- * Expand folders
98
- * Collapse sections
99
- * Navigate large test sets faster
100
- * Focus on specific areas of a project
101
-
102
- :::note
103
-
104
- This is particularly useful for large projects containing hundreds or thousands of tests.
105
-
106
- :::
107
-
108
- ## Run Report Extended View
109
-
110
- If you click the **'Report'** button from Basic Run Report view, you will see the extended view of the Run Report.
111
-
112
- ![Testomat.io - RunReport](./images/Run_Report_6.png)
113
-
114
- **In the extended report view, you can:**
115
-
116
- 1. View a list of tests.
117
- 2. Check their status (Passed, Failed, Skipped, Pending, Custom statuses).
118
- 3. See overview of all executed tests grouped by suites, tags, labels, assignees, or priorities.
119
- 4. Review the Run Report Summary.
120
-
121
- ![Testomat.io - RunReport](./images/Run_Report_9.png)
122
-
123
- 5. Check **'Flaky Tests'** Analytics, if applicable.
124
-
125
- ![Testomat.io - RunReport](./images/Run_Report_12.png)
126
-
127
- 6. Sort suites by Name and Failed status (as well works for Tags, Labels, Assignees, Priorities, Custom Statuses) in the Overview widget.
128
-
129
- ![Testomat.io - RunReport](./images/Run_Report_10.gif)
130
-
131
- 7. Filter test cases by status, type, added messages, custom statuses, or assignees.
132
-
133
- ![Testomat.io - RunReport](./images/Run_Report_11.gif)
134
-
135
- 8. Search test cases by title or message.
136
-
137
- ![Testomat.io - RunReport](./images/Run_Report_13.png)
138
-
139
- :::note
140
-
141
- Use ↓ and ↑ keys to navigate tests and quickly view details for each test.
142
-
143
- ![Report view](./images/gif2_7537.gif)
144
-
145
- :::
146
-
147
- Similar to the basic run report view, click on a test case to see its details or the history of run results.
148
-
149
- ![All tabs view](./images/gif1_7537.gif)
150
-
151
- ## RunGroup Report
152
-
153
- The same as for Runs, you can also view RunGroups Reportes.
154
- To open **RunGroup Report basic view**, simply click on its name - RunGroup Report window will be displayed with general information and runs summary.
155
-
156
- ![Testomat.io - Test Runs in a created RunGroup](./images/RunGroup_6.png)
157
-
158
- For more information go to the [RunGroup Report Basic View](https://docs.testomat.io/project/runs/rungroups#rungroup-report-basic-view) section **'RunGroups'** page.
159
-
160
- From RunGroup Report Basic View you can open **Combined Report for RunGroups**, by clicking on the **'Combined Report'** button. This Report is designed to help you aggregate and analyze the results of multiple test runs within a single view. You can find more information about this report in [Combined Report for RunGroups](https://docs.testomat.io/project/runs/rungroups/#combined-report-for-rungroups) section on **'RunGroups'** page.
161
-
162
- ## How to Download Run Report as Spreadsheet
163
-
164
- Testomat.io allows you to export Run Reports as a spreadsheet (XLSX file) to obtain your test data.
165
-
166
- **Download Multiple Run Reports in One File**:
167
-
168
- 1. Go to Runs.
169
- 2. Enable **'Multi-select'**.
170
- 3. Select the Runs you want to download.
171
- 4. Click the **'Extra menu'** button.
172
- 5. Select **'Download'** option.
173
-
174
- ![Testomat.io - export as Spreadsheet](./images/Export_1.png)
175
-
176
- **Download a Separate Run Report**:
177
-
178
- 1. Go to Runs.
179
- 2. Select a Run Report.
180
- 3. Click the **'Report'** button.
181
-
182
- ![Testomat.io - export as Spreadsheet](./images/Export_2.png)
183
-
184
- 4. Click the **'Extra menu'** button.
185
- 5. Select **'Download as Spreadsheet'** option.
186
-
187
- ![Testomat.io - export as Spreadsheet](./images/Export_3.png)
188
-
189
- ## How to Export Run Report in PDF Format
190
-
191
- Testomat.io also allows you to export your test run results as a PDF report. This feature enables easy sharing and archiving of test outcomes outside of Testomat.io.
192
-
193
- **You can generate a structured PDF report with test execution details in two ways:**
194
-
195
- 1. Go to Runs.
196
- 2. Select a Run Report.
197
- 3. Click the **'Extra menu'** button.
198
- 4. Select **'Export as PDF'** option.
199
-
200
- ![Testomat.io - export as PDF](./images/Export_4.png)
201
-
202
- **OR**
203
-
204
- 1. Go to Runs.
205
- 2. Select a Run Report.
206
- 3. Click the **'Report'** button.
207
-
208
- ![Testomat.io - export as PDF](./images/Export_5.png)
209
-
210
- 4. Click the **'Extra menu'** button.
211
- 5. Select **'Export as PDF'** option.
212
-
213
- ![Testomat.io - export as PDF](./images/Export_6.png)
214
-
215
- **Example of Test Run Report in PDF format**
216
-
217
- ![Testomat.io - export as PDF](./images/Export_7.png)
218
-
219
- **Use cases for PDF Reports:**
220
-
221
- - **Sharing results with stakeholders:** Quickly generate a portable summary of test outcomes for product owners, QA managers, or external partners who may not have access to the platform.
222
-
223
- - **Audit and compliance documentation:** Maintain a formal record of test results in a fixed format for compliance, certification, or audit purposes.
224
-
225
- - **Team retrospectives and reviews:** Use the PDF report to review test coverage and outcomes during retrospectives or sprint reviews without relying on live access to the system.
226
-
227
- ## How to Share Run Report
228
-
229
- You can share your Run Report for external access from the Extended report view using two options: **'Share Report by Email'** or **'Share Report Publicly'**.
230
-
231
- ### Share Report by Email
232
-
233
- 1. Open Extended report view.
234
-
235
- ![Testomat.io - Share Report](./images/Expend_view.png)
236
-
237
- 2. Click the **'Extra menu'** button.
238
- 3. Select **'Share Report by Email'** option.
239
-
240
- ![Testomat.io - Share Report](./images/Share_Report_1.png)
241
-
242
- 4. Enter one or multiple emails, separated with commas, in the displayed modal.
243
- 5. Click the **'Send'** button.
244
-
245
- ![Testomat.io - Share Report](./images/Share_Report_2.png)
246
-
247
- :::note
248
-
249
- This action also creates public report without passcode for external access.
250
-
251
- :::
252
-
253
- **Example of the email with Run Report**
254
-
255
- ![Testomat.io - Share Report](./images/Share_Report_3.png)
256
-
257
- ### Share Report Publicly
258
-
259
- :::note
260
-
261
- To enable the **'Share Report Publicly'** option, first enable this feature at the **Company level** and then enable it on **Project level** (Project Settings -> Project -> ‘Sharing’ section).
262
- Check the instruction in the [Share Options](https://docs.testomat.io/management/company/administration/#share-options) section on **'Administration'** page.
263
-
264
- :::
265
-
266
- 1. Open Extended report view.
267
-
268
- ![Testomat.io - Share Report](./images/Expend_view.png)
269
-
270
- 2. Click the **'Extra menu'** button.
271
- 3. Select **'Share Report Publicly'** option.
272
-
273
- ![Testomat.io - Share Report](./images/Share_Report_4.png)
274
-
275
- 4. Select **'Expiration Date'** (7 days is set by default).
276
- 5. **'Protect by passcode'** is enabled by default for better security (you can disable it if needed).
277
- 6. Click the **'Share'** button in the displayed modal.
278
-
279
- ![Testomat.io - Share Report](./images/Share_Report_5.png)
280
-
281
- 7. Copy the **URL** for sharing the report.
282
- 8. Copy and share the **Passcode** with the link.
283
-
284
- :::note
285
-
286
- Copy and save the **Passcode** in a secure place. For security reasons, you will not be able to see it again after the Run Report Public URL is generated.
287
-
288
- If a shared run report is protected with a **Passcode**, only users with the valid **Passcode** will have access to it.
289
-
290
- :::
291
-
292
- 9. Copy and share the **Expiration date**, if needed.
293
-
294
- ![Testomat.io - Share Report](./images/Share_Report_6.png)
295
-
296
- After the Run Report Public URL is generated, **'Share'** button will be displayed on the Extended run report page.
297
-
298
- ![Testomat.io - Share Report](./images/Share_Report_7.png)
299
-
300
- :::note
301
-
302
- You can stop sharing at any time by clicking the **'Stop Sharing'** button.
303
-
304
- ![Testomat.io - Share Report](./images/Share_Report_8.png)
305
-
306
- :::
307
-
308
- Users with access to **'Public Run Report'** can:
309
-
310
- 1. Filter test cases by statuses.
311
- 2. Search test cases by name.
312
- 3. Copy the URL for sharing the Run report.
313
- 4. Check **'Full Report'** (only users with project access).
314
-
315
- ![Testomat.io - Share Report](./images/Share_Report_9.png)
316
-
317
- ## Compare Test Runs
318
-
319
- Select test runs to see a visual display of their similarities and differences. Compare your run reports and analyze them by parameters such as Flaky, Reviewed, Degraded, and more.
320
-
321
- To compare Runs:
322
-
323
- 1. Enable Multiselection.
324
- 2. Select the Runs you want to compare.
325
- 3. Click the **'Compare'** button.
326
-
327
- ![Testomat.io - Compare Test Runs](./images/compare-runs.gif)
328
-
329
- This feature is also available for RunGroups, allowing you to compare test results from different runs within a single RunGroup. You can find more information about it in the [Combined Report for RunGroups](https://docs.testomat.io/project/runs/rungroups/#combined-report-for-rungroups) section on **'RunGroups'** page.
@@ -1,342 +0,0 @@
1
- ---
2
- title: RunGroups
3
- description: Testomat.io's 'RunGroups' feature allows users to organize multiple test runs based on criteria like sprint or release, providing a consolidated view of testing efforts. This guide explains how to create, customize, move runs into, and copy RunGroups, along with options for managing their visibility, such as pinning and archiving, to streamline test management.
4
- type: article
5
- url: https://docs.testomat.io/project/runs/rungroups
6
- head:
7
- - tag: meta
8
- attrs:
9
- name: keywords
10
- content: managing test runs, run reports, merge strategy, rungroups, relaunch runs, test management, Testomat.io, multi-selection, archiving, purge run groups, purge runs, purge logic, test reporting, runGroup flow, runs dashboard flow, RunGroup chart, copy RunGroup, pin a RunGroup, archive RunGroup, unarchive RunGroup, delete run group, purge flow.
11
- ---
12
-
13
- First, let's clarify what **'RunGroups'** are in Testomat.io.
14
- **RunGroups** allow you to organize and group multiple test runs together based on criteria like sprint, release, functional area, or any other logical grouping. This helps in analyzing aggregated results and providing a consolidated view of your testing efforts.
15
-
16
- ## How to Create a RunGroup
17
-
18
- **To create a new RunGroup:**
19
-
20
- 1. Open **'Runs'** page.
21
- 2. Click the **'Additional options'** arrow button next to the 'Manual run' button.
22
- 3. Select **'New group'** option from the dropdown menu.
23
-
24
- ![Testomat.io - New Group button](./images/RunGroup_1.png)
25
-
26
- On New RunGroup screen you need:
27
-
28
- 1. Choose a **'Group Type'** (optional).
29
- 2. Add a **'Name of a Group'**.
30
- 3. Select **'Merge strategy'** (For more details, refer to the [Merge Strategies](https://docs.testomat.io/project/runs/merge-strategies) page in documentation).
31
- 4. Add **'Description of a group'**, if needed.
32
- 5. Click the **'Save'** button.
33
-
34
- ![Testomat.io - New RunGroup screen](./images/RunGroup_2.png)
35
-
36
- The new RunGroup will appear on Runs page and will open automatically after creation.
37
-
38
- ![Testomat.io - RunGroup will appea](./images/RunGroup_3.png)
39
-
40
- After RunGroup is added, you can create new Manual/Automated/Mixed Runs inside it.
41
-
42
- ## How to Create a Run in RunGroup
43
-
44
- - To create a new **Manual Run**, open the RunGroup and click on **'Manual Run'** button.
45
-
46
- ![Testomat.io - put Manual Run](./images/RunGroup_4.png)
47
-
48
- After this in the displayed window select a test plan from the list or create a new one, and fill in other data that you need for your run (like: assignee, title, environment, description).
49
-
50
- :::note
51
-
52
- When you create a manual run from RunGroup, it will be automatically pre-populated on the **'New Manual Run'** window, but you can change it if needed.
53
-
54
- :::
55
-
56
- ![Testomat.io - put Manual Run](./images/RunGroup_7.png)
57
-
58
- - To add a new **Automated Run** to a RunGroup, execute your tests with Testomat.io, providing TESTOMATIO_RUNGROUP_TITLE="Build ${BUILD_ID}".
59
-
60
- Now, by clicking on expand arrow, you can view Test Runs and their results within created RunGroup.
61
-
62
- ![Testomat.io - Test Runs in a created RunGroup](./images/RunGroup_5.png)
63
-
64
- Clicking on RunGroup name will toggle RunGroup Report screen in basic view. Here you can view more detailed information about the runs inside selected RunGroup and total results.
65
-
66
- ![Testomat.io - Test Runs in a created RunGroup](./images/RunGroup_6.png)
67
-
68
- For more details go to the relevant section - [RunGroup Report in Basic View](https://docs.testomat.io/project/runs/reports/#rungroup-report-in-basic-view) on **'Reports'** page.
69
-
70
- ## How to Move a Run to a RunGroup
71
-
72
- You may need to move a Run to a specific RunGroup (e.g., to associate it with a particular release or build). There are two ways to do this in Testomat.io:
73
-
74
- Method 1: Move a Run to a RunGroup
75
-
76
- 1. Navigate to the **'Runs'** page
77
- 2. Select the Run you want to move
78
- 3. Click the **'Extra menu'** button
79
- 4. Select the **'Move'** from the dropdown menu
80
-
81
- ![Testomat.io - Move Run option in the dropdown menu](./images/Move_Run_1.png)
82
-
83
- 5. In the pop-up window, choose the **destination RunGroup**
84
- 6. Click the **'Move'** button to confirm the action
85
-
86
- ![Testomat.io - Destination RunGroup selection dialog](./images/Move_Run_2.png)
87
-
88
- Method 2: Add Existing Runs to the RunGroup
89
-
90
- 1. Navigate to the **'Runs'** page
91
- 2. Click the **'Extra menu'** next to the RunGroup you want to move Runs into
92
- 3. Select the **'Add Existing Run'** option
93
-
94
- ![Add Existing Run](./images/att1.png)
95
-
96
- 4. In the modal window, select the Run(s) you want to move
97
- 5. Click the **Move Runs to Group** button to confirm
98
-
99
- ![Move Runs to Group](./images/att2.png)
100
-
101
- Moved or added Runs will appear in the selected RunGroup.
102
-
103
- ![List of added runs into a group](./images/att3.png)
104
-
105
- ## RunGroup Report Basic View
106
-
107
- You can view RunGroup Report by clicking on its name.
108
-
109
- ![Testomat.io - Test Runs in a created RunGroup](./images/RunGroup_6.png)
110
-
111
- **In this basic RunGroup Report, you can:**
112
-
113
- 1. View the [RunGroup Chart](https://docs.testomat.io/project/runs/rungroups/#rungroup-chart).
114
- 2. Review **'RunGroup summary section'** with general information based on the selected [Merge Strategy](https://docs.testomat.io/project/runs/merge-strategies/).
115
- 3. View a list of test runs, their statuses with the number of Passed, Failed, Skipped tests, along with the run assignee and executed time.
116
- 4. Customize your RunGroup Report view (Read more in the [How to Customize Runs List View ](https://docs.testomat.io/project/runs/rungroups/#how-to-customize-runs-list-view) section below).
117
- 5. Check **'Combined Report'** (Read more in the [Combined Report for RunGroups](https://docs.testomat.io/project/runs/rungroups/#combined-report-for-rungroups) section below).
118
- 6. Add a new Manual Run to the selected RunGroup.
119
- 7. Generate a [Rungroup Statistic Report](https://docs.testomat.io/advanced/ai-powered-features/ai-powered-features/#rungroup-statistic-report)(AI-feature should be enabled on **'Company Settings'** page).
120
-
121
- ![Testomat.io - RunGroup Report](./images/RunGroup_Report_1.png)
122
-
123
- The **'RunGroup summary section'** displays a counter with the number of included and analyzed runs in the RunGroup. Archived runs are excluded from this counter and do not affect the total RunGroup result. You can see how many runs were archived, and by clicking on archived counter, you will be redirected to the **'Groups Archive'** page, where runs are already filtered by selected RunGroup.
124
-
125
- ![Testomat.io - RunGroup Report](./images/RunGroup_Report_2.gif)
126
-
127
- ## How to Customize Runs List View
128
-
129
- When working with test runs inside RunGroup or Runs Dashboard Flow, you can adjust the table layout to fit your needs. Instead of using the default view, you can customize the runs table layout within **RunGroup** page or directly from the main **Runs Dashboard**.
130
-
131
- **RunGroup Flow**
132
-
133
- 1. Go to **'Runs'** page.
134
- 2. Select RunGroup.
135
- 3. Click the **'Custom view'** button.
136
-
137
- ![Testomat.io - Custom View option](images/Custom_view_1.png)
138
-
139
- You can use the default custom view or personalize the table layout:
140
-
141
- 4. Click the **'Settings'** icon to access **'Runs List Settings'**.
142
-
143
- ![Testomat.io - Custom View](images/Custom_view_2.png)
144
-
145
- 5. Select or deselect columns to show only the data you need.
146
- 6. Set the column width (px), if needed, to improve readability.
147
- 7. Click the **'Save'** button to apply your changes.
148
-
149
- ![Testomat.io - Save changes](images/Custom_view_3.png)
150
-
151
- Here's how your customized table within a RunGroup will appear after customization:
152
-
153
- ![Testomat.io - Updated Custom View](images/Custom_view_4.png)
154
-
155
- **Runs Dashboard Flow**
156
-
157
- Similarly, customize the runs view from the main Runs Dashboard:
158
-
159
- 1. Go to **'Runs'** page.
160
- 2. Click the **'Custom view'** button.
161
-
162
- ![Testomat.io - Custom view button](images/Custom_view_5.png)
163
-
164
- 3. Click the **'Settings'** icon to access **'Runs List Settings'**.
165
- 4. Customize your view in the **'Runs list settings'**.
166
- 5. Click the **'Save'** button to apply your changes.
167
-
168
- ![Testomat.io - Save changes](images/Custom_view_6.png)
169
-
170
- Here's how your customized table will appear.
171
-
172
- ![Testomat.io - Updated view](images/Custom_view_7.png)
173
-
174
- These customization options allow you to tailor the Runs View to display only the most relevant details, improving clarity and efficiency.
175
-
176
- :::note
177
-
178
- A customized runs list view automatically applies to all your RunGroups and main Runs Dashboard.
179
-
180
- :::
181
-
182
- ## RunGroup Chart
183
-
184
- The chart displays up to 50 of the latest test runs belonging to the group. If you have more runs, use the pagination arrows to view the results of the previous runs.
185
-
186
- ![Testomat.io - RunGroup Chart](./images/RunGroup_Chart.png)
187
-
188
- ## Combined Report for RunGroups
189
-
190
- Testomat.io provides a **'Combined Report'** feature for RunGroups. This report is designed to help you aggregate and analyze the results of all test runs within the RunGroup in a single view.
191
-
192
- You can open **'Combined Report'** for a RunGroup by clicking on **'Combined Report'** button from the RunGroup Report Basic View window.
193
-
194
- ![Testomat.io - Combined Report](./images/Combined_report.png)
195
-
196
- The comparison of runs is based on the first (main) launch. The statuses counters on the left side are calculated based on the main run.
197
-
198
- To change the main run, click the **'Compare To'** button. Note that the counters of loaded tests is above them. The final summary on the right side is calculated based on the loaded tests.
199
-
200
- ![Testomat.io - Combined Report](./images/Combined_report_1.png)
201
-
202
- :::note
203
-
204
- The **'Combined Report'** feature for RunGroups does **not** combine test results of all runs inside one RunGroup into a single report. It only allows you **to compare** test results of all runs within this RunGroup.
205
-
206
- :::
207
-
208
- Inside the **'Combined Report'**, you can get a quick overview of the pass/fail rates for all tests within the Group. You can also see the total number of tests present in all compared runs, as well as the total number of flaky (unstable tests that passed or failed in different runs), reviewed (previously failing that passed in next runs), and degraded (previously passed that failed in next runs) tests.
209
-
210
- Additionally, you can use filters and search features to view the data most relevant to your needs.
211
-
212
- ![Testomat.io - Combined Report](./images/Combined_report_2.gif)
213
-
214
- **Key benefits:**
215
-
216
- - **Analyze Trends:** By seeing the results from multiple runs in one place, you can identify trends in test performance and stability over time.
217
-
218
- - **Consolidate Data:** The combined report is especially useful for "mixed runs" (runs that include both manual and automated tests), as it consolidates all results into a single, cohesive report.
219
-
220
- - **Compare Runs:** Testomat.io also has a separate **'Compare Test Runs'** feature that can be used within a RunGroup. This allows you to side-by-side compare the results of different runs to pinpoint changes, regressions, or improvements. (Go to [Compare Test Runs](https://docs.testomat.io/project/runs/reports/#compare-test-runs) section on **'Run reports'** page to read more about this feature.)
221
-
222
- This feature is a powerful tool for project managers, QA leads, and stakeholders who need to make informed decisions based on a holistic view of testing activities, without having to manually compile data from individual reports.
223
-
224
- ## How to Copy RunGroup
225
-
226
- You can easily create a new RunGroup, completely independent of any previous runs, by copying all relevant tests exactly as they are. You can configure what data should be copied, namely:
227
-
228
- - **Assignee**: Define assignee details separately, preventing them from being copied.
229
- - **Issues**: Choose whether to include or exclude linked issues during duplication.
230
- - **Labels**: Decide whether labels should be duplicated or omitted in the new test run.
231
- - **Environments**: Control the duplication of environment settings based on your requirements.
232
- - **Nested Structure**: Preserve or exclude the nested structure of your test groups as you duplicate them.
233
-
234
- **Follow these steps:**
235
-
236
- 1. Open the RunGroup.
237
- 2. Click the **'Extra menu'** button.
238
- 3. Select **'Copy'** option form the dropdown menu.
239
-
240
- ![Testomat.io - Copy group 1](./images/Copy_group_1.png)
241
-
242
- 4. Select configuration options.
243
- 5. Click the **'Copy'** button.
244
-
245
- ![Testomat.io - Copy group 2](./images/Copy_group_2.png)
246
-
247
- 6. Verify the created RunGroup.
248
-
249
- ![Testomat.io - Copy group 3](./images/Copy_group_3.png)
250
-
251
- :::note
252
-
253
- Only manual runs will be copied with the RunGroup. If a copied RunGroup contains a mixed test run, automated test cases will be marked as skipped by default.
254
-
255
- :::
256
-
257
- ## How to Pin a RunGroup
258
-
259
- Testomat.io allows users to pin RunGroups to the top of the Dashboard. This provides quick access to critical or frequently used tests or test runs, helping teams stay focused on the most relevant tasks. Pinning supports faster navigation, improved focus, and customizable workflows—ideal for monitoring regression tests, environment-specific runs, or production hotfixes.
260
-
261
- For more details, refer to the [How to Pin a Run or RunGroup](https://docs.testomat.io/project/runs/managing-runs/#how-to-pin-a-run-or-rungroup) section.
262
-
263
- ## How to Archive RunGroup
264
-
265
- Testomat.io gives you an opportunity to archive a RunGroup, including all its contained Test Runs. This helps maintain better visibility on your main Run Dashboard.
266
-
267
- For more information on archiving, visit the [Archive Runs & RunGroups](https://docs.testomat.io/project/runs/archive-runs-and-groups) page.
268
-
269
- ## How to Unarchive RunGroup
270
-
271
- Archived RunGroups can be unarchived and moved back to the main Run Dashboard. Restoring a RunGroup also restores all its archived runs.
272
-
273
- Read more in the [How to Unarchive Runs & RunGroups](https://docs.testomat.io/project/runs/archive-runs-and-groups/#how-to-unarchive-runs-&-groups) section.
274
-
275
- ## Purge Run Groups
276
-
277
- Starting from **October 2025**, Testomat.io introduces the new **Purge** approach for **Run Groups**. This update extends the same **two-step purge-to-archive logic** that was introduced for individual [Runs](https://docs.testomat.io/project/runs/managing-runs/#purge-runs) in September 2025, ensuring data safety and consistency across all Run types.
278
-
279
- The **Delete** option for Run Groups is now replaced with a **Purge** option.
280
-
281
- ![Purge button in RunGroup](./images/att1_6779.png)
282
-
283
- ### Compress and Move to Archive (Purge)
284
-
285
- When a **Run Group** is purged (either manually or automatically):
286
-
287
- - The **group itself** is **deleted**
288
- - All **nested Runs** are moved to the **[Archive](https://docs.testomat.io/project/runs/archive-runs-and-groups/)** page
289
- - All **nested Runs** are marked with the **purged** badge for easy identification
290
-
291
- ![Purged button confirmation](./images/att2_6779.png)
292
-
293
- During this step:
294
-
295
- - Stack traces are removed to reduce storage size
296
- - Essential data is preserved, including:
297
- - Test results
298
- - Artifacts (attachments, logs, screenshots)
299
- - Custom statuses and metadata
300
- - Purged Runs remain available in the Archive page and can be restored at any time
301
-
302
- :::note
303
-
304
- The purge logic for **ongoing (pending) Runs** within a Run Group follows the **same rules** as described for individual Runs. For detailed information about the purge flow of ongoing Runs, see [**Purge Logic for Ongoing Runs**](https://docs.testomat.io/project/runs/managing-runs/#purge-logic-for-ongoing-runs).
305
-
306
- :::
307
-
308
- ### Automatic Purge Flow
309
-
310
- The same **two-step purge-to-archive logic** applies to automatically purged **Run Groups** via **Project Settings → [Purge Old Runs](https://docs.testomat.io/management/project/settings/#purge-old-runs)**.
311
-
312
- When an automatic purge is triggered:
313
-
314
- - The **group itself** is **deleted**
315
- - All **nested Runs** are moved to the **[Archive](https://docs.testomat.io/project/runs/archive-runs-and-groups/)** page
316
- - All **nested Runs** are marked with the **purged** badge for easy identification
317
-
318
- ### Limitations
319
-
320
- :::note
321
-
322
- Run Groups have a **limit of 20,000 Runs** for the purge process. If a group contains **more than 20,000 Runs**, only the **first 20,000** will be purged and moved to the Archive.
323
-
324
- All remaining Runs beyond this limit will be **permanently deleted** and **cannot be restored**.
325
-
326
- :::
327
-
328
- ### Permanent Deletion from Archive
329
-
330
- :::note
331
-
332
- If you remove a Run (that was part of a purged Run Group) from the Archive page, it is permanently deleted. This step is **irreversible** — once **deleted**, the Run cannot be restored.
333
-
334
- :::
335
-
336
- ![Permanent Deletion from Archive](./images/att3_6779.png)
337
-
338
- ### Summary
339
-
340
- By extending the **Purge** functionality to Run Groups, Testomat.io ensures that all grouped executions follow the same safe, transparent, and recoverable process as individual Runs.
341
-
342
- Teams can efficiently manage large-scale test executions while maintaining complete data traceability and control.