testeiya 0.2.1 → 0.3.10
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +284 -171
- package/dist/prompt/context.js +79 -0
- package/dist/prompt/context.js.map +1 -0
- package/dist/prompt/index.js +60 -0
- package/dist/prompt/index.js.map +1 -0
- package/dist/prompt/print.js +30 -0
- package/dist/prompt/print.js.map +1 -0
- package/dist/prompt/project-info.js +2 -0
- package/dist/prompt/project-info.js.map +1 -0
- package/{src/prompt/system-prompt.ts → dist/prompt/system-prompt.js} +38 -25
- package/dist/prompt/system-prompt.js.map +1 -0
- package/dist/prompt/testomatio.js +264 -0
- package/dist/prompt/testomatio.js.map +1 -0
- package/dist/prompt/tools.js +57 -0
- package/dist/prompt/tools.js.map +1 -0
- package/dist/prompt/vocab.js +8 -0
- package/dist/prompt/vocab.js.map +1 -0
- package/dist/src/args.js +172 -0
- package/dist/src/args.js.map +1 -0
- package/dist/src/cli.js +97 -0
- package/dist/src/cli.js.map +1 -0
- package/dist/src/doctor.js +186 -0
- package/dist/src/doctor.js.map +1 -0
- package/dist/src/env.js +93 -0
- package/dist/src/env.js.map +1 -0
- package/dist/src/mcp-extension.js +9 -0
- package/dist/src/mcp-extension.js.map +1 -0
- package/dist/src/mcp.js +139 -0
- package/dist/src/mcp.js.map +1 -0
- package/dist/src/model.js +71 -0
- package/dist/src/model.js.map +1 -0
- package/dist/src/models.js +38 -0
- package/dist/src/models.js.map +1 -0
- package/dist/src/output.js +185 -0
- package/dist/src/output.js.map +1 -0
- package/dist/src/result.js +41 -0
- package/dist/src/result.js.map +1 -0
- package/dist/src/run.js +291 -0
- package/dist/src/run.js.map +1 -0
- package/dist/src/session.js +92 -0
- package/dist/src/session.js.map +1 -0
- package/dist/src/sessions.js +61 -0
- package/dist/src/sessions.js.map +1 -0
- package/dist/vendor/mcp.js +14801 -0
- package/mcp-tools.json +2967 -0
- package/package.json +34 -43
- package/prompt/context.ts +101 -0
- package/prompt/index.ts +102 -0
- package/prompt/print.ts +32 -0
- package/prompt/project-info.ts +30 -0
- package/prompt/system-prompt.ts +187 -0
- package/{src/prompt → prompt}/testomatio.ts +120 -9
- package/prompt/tools.ts +57 -0
- package/prompt/vocab.ts +7 -0
- package/skills/codeceptjs/ci-fix-tests/SKILL.md +7 -8
- package/skills/codeceptjs/codeceptjs-auth/SKILL.md +47 -64
- package/skills/codeceptjs/codeceptjs-exploration/SKILL.md +56 -47
- package/skills/codeceptjs/codeceptjs-fundamentals/SKILL.md +144 -106
- package/skills/codeceptjs/codeceptjs-run-analysis/SKILL.md +50 -57
- package/skills/codeceptjs/debugging-codeceptjs-tests/SKILL.md +88 -196
- package/skills/codeceptjs/migrate-codeceptjs-4/SKILL.md +4 -5
- package/skills/codeceptjs/migrate-cypress-to-codeceptjs/SKILL.md +25 -36
- package/skills/codeceptjs/migrate-protractor-to-codeceptjs/SKILL.md +23 -36
- package/skills/codeceptjs/migrate-selenium-java-to-codeceptjs/SKILL.md +23 -36
- package/skills/codeceptjs/migrate-testcafe-to-codeceptjs/SKILL.md +23 -35
- package/skills/codeceptjs/refactoring-codeceptjs-tests/SKILL.md +50 -53
- package/skills/codeceptjs/writing-codeceptjs-tests/SKILL.md +87 -191
- package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/SKILL.md +1 -1
- package/{skills.lock.json → skills/skills.lock.json} +43 -36
- package/skills/skills.yaml +34 -0
- package/skills/testomatio/explorbot/explorbot-fundamentals/SKILL.md +49 -61
- package/skills/testomatio/explorbot/explorbot-fundamentals/references/no-install.md +36 -0
- package/skills/testomatio/explorbot/explorbot-plan/SKILL.md +20 -7
- package/skills/testomatio/explorbot/explorbot-setup/SKILL.md +41 -289
- package/skills/testomatio/qa-process/qa-explain-behavior/SKILL.md +111 -0
- package/skills/testomatio/qa-process/qa-explain-behavior/references/answer-shapes.md +116 -0
- package/skills/testomatio/qa-process/qa-explain-behavior/references/examples.md +157 -0
- package/skills/testomatio/qa-process/qa-lead-strategy-advisor/SKILL.md +91 -0
- package/skills/testomatio/qa-process/qa-lead-strategy-advisor/references/output-format.md +71 -0
- package/skills/testomatio/qa-process/testing-workflow/SKILL.md +227 -0
- package/skills/testomatio/test-automation/qa-data-seeder/SKILL.md +83 -0
- package/skills/testomatio/test-automation/run-tests-with-testomatio-reporter/SKILL.md +5 -3
- package/skills/testomatio/test-automation/setup-change-aware-pr-testing/SKILL.md +190 -0
- package/skills/testomatio/test-automation/setup-ci-automation/SKILL.md +76 -0
- package/skills/testomatio/test-management/qa-split-testing-levels-pyramid/SKILL.md +55 -0
- package/skills/testomatio/test-management/qa-sprint-report-by-testomatio/SKILL.md +220 -0
- package/skills/testomatio/test-management/qa-sprint-report-by-testomatio/references/qa-sprint-report.md +126 -0
- package/skills/testomatio/test-management/qa-test-code-coverage/SKILL.md +1 -1
- package/skills/testomatio/test-management/qa-thinking/SKILL.md +25 -25
- package/skills/testomatio/test-management/testomatio-mcp/SKILL.md +14 -1
- package/skills/testomatio/test-management/testomatio-mcp/references/MCP_SETUP.md +8 -0
- package/assets/testeiya-logo-dark.svg +0 -16
- package/assets/testeiya-logo.svg +0 -15
- package/models.catalog.json +0 -6963
- package/skills/playwright-best-practices-skill/LICENSE.md +0 -7
- package/skills/playwright-best-practices-skill/README.md +0 -147
- package/skills/testeiya/README.md +0 -29
- package/skills/testeiya/answer-formatting/SKILL.md +0 -30
- package/skills/testeiya/check-cucumber/SKILL.md +0 -200
- package/skills/testeiya/check-tests/SKILL.md +0 -260
- package/skills/testeiya/manual-run-assistant/SKILL.md +0 -61
- package/skills/testeiya/testomatio-docs/INDEX.md +0 -175
- package/skills/testeiya/testomatio-docs/SKILL.md +0 -102
- package/skills/testeiya/testomatio-docs/docs/advanced/ai-powered-features/ai-agents.md +0 -144
- package/skills/testeiya/testomatio-docs/docs/advanced/ai-powered-features/ai-powered-features.md +0 -531
- package/skills/testeiya/testomatio-docs/docs/advanced/ai-powered-features/ai-requirements.md +0 -437
- package/skills/testeiya/testomatio-docs/docs/advanced/api-access/index.md +0 -191
- package/skills/testeiya/testomatio-docs/docs/advanced/branches-folder/index.md +0 -277
- package/skills/testeiya/testomatio-docs/docs/advanced/bulk-edit-folder/bulk-edit-demos.mdx +0 -39
- package/skills/testeiya/testomatio-docs/docs/advanced/bulk-edit-folder/bulk-edit-on-suite-and-test-level.md +0 -122
- package/skills/testeiya/testomatio-docs/docs/advanced/bulk-edit-folder/what-is-bulk-edit.md +0 -170
- package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/branches.md +0 -53
- package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/index.mdx +0 -98
- package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/jira-issue.md +0 -151
- package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/runs-and-reports.md +0 -377
- package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/work-in-jira.md +0 -267
- package/skills/testeiya/testomatio-docs/docs/advanced/living-doc/index.md +0 -138
- package/skills/testeiya/testomatio-docs/docs/advanced/milestones/index.md +0 -220
- package/skills/testeiya/testomatio-docs/docs/advanced/shortcuts/index.md +0 -48
- package/skills/testeiya/testomatio-docs/docs/advanced/tags-labels/labels-and-custom-fields.md +0 -201
- package/skills/testeiya/testomatio-docs/docs/advanced/tags-labels/tags-or-labels.md +0 -94
- package/skills/testeiya/testomatio-docs/docs/advanced/tags-labels/tags.md +0 -243
- package/skills/testeiya/testomatio-docs/docs/advanced/test-artifacts/index.md +0 -135
- package/skills/testeiya/testomatio-docs/docs/advanced/tql/index.md +0 -394
- package/skills/testeiya/testomatio-docs/docs/getting-started/index.md +0 -290
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/azure.md +0 -96
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/bamboo.md +0 -94
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/bitbucket.md +0 -106
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/circle.md +0 -115
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/github.md +0 -120
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/gitlab.md +0 -88
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/index.mdx +0 -151
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/jenkins.md +0 -121
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/teamcity.md +0 -96
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/azure.md +0 -31
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/clickup.md +0 -54
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/confluence.md +0 -34
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/github.md +0 -31
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/gitlab.md +0 -93
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/index.md +0 -131
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/jira.md +0 -164
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/linear.md +0 -29
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/shortcut.md +0 -67
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/youtrack.md +0 -94
- package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/azure-devops.md +0 -53
- package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/email.md +0 -71
- package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/jira.md +0 -52
- package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/ms-teams.md +0 -60
- package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/rules.md +0 -267
- package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/slack.md +0 -82
- package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/telegram.md +0 -73
- package/skills/testeiya/testomatio-docs/docs/integrations/scim-provisioning/index.mdx +0 -207
- package/skills/testeiya/testomatio-docs/docs/integrations/scim-provisioning/okta.md +0 -204
- package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/azure.md +0 -95
- package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/google.md +0 -75
- package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/index.mdx +0 -33
- package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/okta.md +0 -108
- package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/saml.md +0 -66
- package/skills/testeiya/testomatio-docs/docs/legal/compliance/gdpr.md +0 -95
- package/skills/testeiya/testomatio-docs/docs/legal/compliance/index.mdx +0 -94
- package/skills/testeiya/testomatio-docs/docs/legal/compliance/privacy.md +0 -26
- package/skills/testeiya/testomatio-docs/docs/legal/compliance/terms.md +0 -32
- package/skills/testeiya/testomatio-docs/docs/legal/security/index.mdx +0 -242
- package/skills/testeiya/testomatio-docs/docs/legal/security/jira.md +0 -140
- package/skills/testeiya/testomatio-docs/docs/legal/security/subprocessors.md +0 -76
- package/skills/testeiya/testomatio-docs/docs/management/company/administration.md +0 -363
- package/skills/testeiya/testomatio-docs/docs/management/company/audit-log.md +0 -41
- package/skills/testeiya/testomatio-docs/docs/management/company/plan-features-comparison.md +0 -240
- package/skills/testeiya/testomatio-docs/docs/management/company/subscriptions.md +0 -326
- package/skills/testeiya/testomatio-docs/docs/management/company/trials.md +0 -75
- package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/accountant.md +0 -49
- package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/index.mdx +0 -285
- package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/read-only-user.md +0 -67
- package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/teams.md +0 -138
- package/skills/testeiya/testomatio-docs/docs/management/project/settings.md +0 -87
- package/skills/testeiya/testomatio-docs/docs/management/project/templates.md +0 -546
- package/skills/testeiya/testomatio-docs/docs/project/analytics/index.md +0 -326
- package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/classical-tests-markdown-format.md +0 -386
- package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/download-manual-tests-as-files.md +0 -378
- package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/export-to-obsidian.md +0 -112
- package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/export-to-spreadsheet.md +0 -127
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/auto-import.md +0 -63
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-Allure-TestOps.md +0 -49
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-QTest.md +0 -41
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-Qase.md +0 -39
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-Zephyr.md +0 -37
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-csv-xlsx.md +0 -117
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-cucumber.md +0 -93
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-qmetry.md +0 -44
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-source-code.md +0 -117
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-testcaselabs.md +0 -44
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-testmo.md +0 -47
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-testrail.md +0 -82
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-xray.md +0 -55
- package/skills/testeiya/testomatio-docs/docs/project/plans/index.md +0 -334
- package/skills/testeiya/testomatio-docs/docs/project/pulse/index.md +0 -104
- package/skills/testeiya/testomatio-docs/docs/project/runs/archive-runs-and-groups.md +0 -176
- package/skills/testeiya/testomatio-docs/docs/project/runs/custom-statuses.md +0 -195
- package/skills/testeiya/testomatio-docs/docs/project/runs/environments.md +0 -203
- package/skills/testeiya/testomatio-docs/docs/project/runs/managing-runs.md +0 -455
- package/skills/testeiya/testomatio-docs/docs/project/runs/merge-strategies.md +0 -137
- package/skills/testeiya/testomatio-docs/docs/project/runs/reports.md +0 -329
- package/skills/testeiya/testomatio-docs/docs/project/runs/rungroups.md +0 -342
- package/skills/testeiya/testomatio-docs/docs/project/runs/running-automated-tests.md +0 -256
- package/skills/testeiya/testomatio-docs/docs/project/runs/running-manual-and-automated-tests.md +0 -331
- package/skills/testeiya/testomatio-docs/docs/project/runs/running-tests-manually.md +0 -633
- package/skills/testeiya/testomatio-docs/docs/project/runs/temporary-tests-notes.md +0 -223
- package/skills/testeiya/testomatio-docs/docs/project/steps-snippets/snippets.md +0 -68
- package/skills/testeiya/testomatio-docs/docs/project/steps-snippets/steps.md +0 -181
- package/skills/testeiya/testomatio-docs/docs/project/tests/bdd-test-case-editor.md +0 -98
- package/skills/testeiya/testomatio-docs/docs/project/tests/classical-test-case-editor.md +0 -354
- package/skills/testeiya/testomatio-docs/docs/project/tests/classical-vs-bdd.md +0 -169
- package/skills/testeiya/testomatio-docs/docs/project/tests/copy-and-move-your-tests.md +0 -156
- package/skills/testeiya/testomatio-docs/docs/project/tests/other-features-for-test-case-design.md +0 -412
- package/skills/testeiya/testomatio-docs/docs/project/tests/test-case-creation-and-editing.md +0 -527
- package/skills/testeiya/testomatio-docs/docs/support/index.md +0 -75
- package/skills/testeiya/testomatio-docs/docs/test-reporting/artifacts.md +0 -388
- package/skills/testeiya/testomatio-docs/docs/test-reporting/cli.md +0 -215
- package/skills/testeiya/testomatio-docs/docs/test-reporting/configuration.md +0 -247
- package/skills/testeiya/testomatio-docs/docs/test-reporting/frameworks.md +0 -490
- package/skills/testeiya/testomatio-docs/docs/test-reporting/functions.md +0 -144
- package/skills/testeiya/testomatio-docs/docs/test-reporting/index.mdx +0 -48
- package/skills/testeiya/testomatio-docs/docs/test-reporting/junit.md +0 -248
- package/skills/testeiya/testomatio-docs/docs/test-reporting/logger.md +0 -138
- package/skills/testeiya/testomatio-docs/docs/test-reporting/php.md +0 -77
- package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/bitbucket.md +0 -72
- package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/csv.md +0 -23
- package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/debug.md +0 -18
- package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/github.md +0 -63
- package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/gitlab.md +0 -29
- package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/html.md +0 -80
- package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/index.mdx +0 -57
- package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/testomatio.md +0 -331
- package/skills/testeiya/testomatio-docs/docs/test-reporting/python.md +0 -229
- package/skills/testeiya/testomatio-docs/docs/test-reporting/workflows.md +0 -325
- package/skills/testeiya/testomatio-docs/docs/tutorials/Interacting-via-API.md +0 -159
- package/skills/testeiya/testomatio-docs/docs/tutorials/Manual-Testing-Classic.md +0 -164
- package/skills/testeiya/testomatio-docs/docs/tutorials/Set-Up-Read-Only-Access.md +0 -85
- package/skills/testeiya/testomatio-docs/docs/tutorials/Using-AI-in-Testing.md +0 -163
- package/skills/testeiya/testomatio-docs/docs/tutorials/Working-with-Jira.md +0 -125
- package/skills/testeiya/testomatio-docs/docs/tutorials/java.md +0 -478
- package/skills/testeiya/testomatio-docs/docs/tutorials/playwright.md +0 -207
- package/skills/testeiya/testomatio-docs/docs/tutorials/webdriver.md +0 -218
- package/skills/testeiya/testomatio-docs/docs.lock.json +0 -6
- package/skills/testeiya/testomatio-reporter/SKILL.md +0 -307
- package/skills/testomatio/test-automation/setup-pr-testing/SKILL.md +0 -199
- package/skills.yaml +0 -21
- package/src/ai-debug.ts +0 -188
- package/src/api/agent-get.ts +0 -22
- package/src/api/agent-start.ts +0 -384
- package/src/api/client-log.ts +0 -29
- package/src/api/context.ts +0 -332
- package/src/api/debug-layout.ts +0 -47
- package/src/api/debug-report.ts +0 -29
- package/src/api/debug-snapshot.ts +0 -30
- package/src/api/debug-stream.ts +0 -52
- package/src/api/files-delete.ts +0 -184
- package/src/api/files-read.ts +0 -41
- package/src/api/files-rename.ts +0 -99
- package/src/api/files-tree.ts +0 -292
- package/src/api/files-write.ts +0 -41
- package/src/api/mcp.ts +0 -603
- package/src/api/memory.ts +0 -69
- package/src/api/open-external.ts +0 -39
- package/src/api/playwright-cli.ts +0 -606
- package/src/api/providers.ts +0 -457
- package/src/api/read-env-token.ts +0 -33
- package/src/api/sessions.ts +0 -79
- package/src/api/settings.ts +0 -120
- package/src/api/skills.ts +0 -58
- package/src/api/testomatio-attachment.ts +0 -89
- package/src/api/testomatio-auth.ts +0 -284
- package/src/api/testomatio-proxy.ts +0 -197
- package/src/api/testomatio-run-stats.ts +0 -81
- package/src/api/testomatio-target.ts +0 -225
- package/src/api/testomatio-transcription.ts +0 -82
- package/src/api/workspace-search.ts +0 -96
- package/src/api/workspace-sync.ts +0 -151
- package/src/api/workspace.ts +0 -165
- package/src/app-server.ts +0 -556
- package/src/bridge.ts +0 -251
- package/src/check-tests.ts +0 -156
- package/src/cli.ts +0 -17
- package/src/commands.ts +0 -244
- package/src/config.ts +0 -183
- package/src/connection.ts +0 -658
- package/src/context-store.ts +0 -265
- package/src/debug-bus.ts +0 -405
- package/src/extensions/tool-gate.ts +0 -220
- package/src/extensions/webui/ask-channel.ts +0 -52
- package/src/extensions/webui/index.ts +0 -184
- package/src/extensions/webui/tools/ask-question.ts +0 -88
- package/src/extensions/webui/tools/query-result.ts +0 -79
- package/src/extensions/webui/tools/render-chart.ts +0 -53
- package/src/extensions/webui/tools/render-item.ts +0 -61
- package/src/extensions/webui/tools/render-list.ts +0 -170
- package/src/extensions/webui/tools/render-result.ts +0 -85
- package/src/extensions/webui/tools/render-tree.ts +0 -71
- package/src/extensions/webui/tools/ui-widget.ts +0 -54
- package/src/extensions/webui/tools/widget-result.ts +0 -13
- package/src/extensions/webui/widget-channel.ts +0 -50
- package/src/file-log.ts +0 -196
- package/src/git-tracked.ts +0 -68
- package/src/gitignore.ts +0 -53
- package/src/json-store.ts +0 -43
- package/src/load-env.ts +0 -70
- package/src/loader-style.ts +0 -96
- package/src/main.ts +0 -140
- package/src/mcp-catalog.ts +0 -96
- package/src/models-catalog.ts +0 -72
- package/src/permissions.ts +0 -120
- package/src/project-dir.ts +0 -189
- package/src/project-info.ts +0 -108
- package/src/prompt/app-ui.ts +0 -59
- package/src/prompt/browser.ts +0 -27
- package/src/prompt/index.ts +0 -62
- package/src/prompt/tools.ts +0 -27
- package/src/server-info.ts +0 -62
- package/src/session-factory.ts +0 -442
- package/src/session-store.ts +0 -115
- package/src/shell-env.ts +0 -45
- package/src/skills.ts +0 -163
- package/src/sync-snapshot.ts +0 -140
- package/src/telemetry.ts +0 -239
- package/src/testomatio-auth.ts +0 -299
- package/src/testomatio.ts +0 -98
- package/src/theme.ts +0 -30
- package/src/user-env.ts +0 -94
- package/src/welcome.ts +0 -194
- package/src/workspace/safe-path.ts +0 -85
- package/src/workspace/test-md.ts +0 -70
- package/src/workspace-model.ts +0 -440
- package/testeiya.config.json +0 -14
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/authentication-flows.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/authentication.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/clock-mocking.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/mobile-testing.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/multi-context.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/multi-user.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/network-advanced.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/third-party.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/architecture/pom-vs-fixtures.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/architecture/test-architecture.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/architecture/when-to-mock.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/browser-apis/browser-apis.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/browser-apis/iframes.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/browser-apis/service-workers.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/browser-apis/websockets.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/annotations.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/assertions-waiting.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/configuration.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/fixtures-hooks.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/global-setup.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/locators.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/page-object-model.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/projects-dependencies.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/test-data.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/test-suite-structure.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/test-tags.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/debugging/console-errors.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/debugging/debugging.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/debugging/error-testing.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/debugging/flaky-tests.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/frameworks/angular.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/frameworks/nextjs.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/frameworks/react.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/frameworks/vue.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/ci-cd.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/docker.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/github-actions.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/gitlab.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/other-providers.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/parallel-sharding.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/performance.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/reporting.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/test-coverage.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/accessibility.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/api-testing.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/browser-extensions.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/canvas-webgl.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/component-testing.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/drag-drop.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/electron.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/file-operations.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/file-upload-download.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/forms-validation.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/graphql-testing.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/i18n.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/performance-testing.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/security-testing.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/visual-regression.md +0 -0
- /package/skills/{playwright-cli → playwright/playwright-cli}/SKILL.md +0 -0
- /package/skills/{playwright-cli → playwright/playwright-cli}/references/element-attributes.md +0 -0
- /package/skills/{playwright-cli → playwright/playwright-cli}/references/playwright-tests.md +0 -0
- /package/skills/{playwright-cli → playwright/playwright-cli}/references/request-mocking.md +0 -0
- /package/skills/{playwright-cli → playwright/playwright-cli}/references/running-code.md +0 -0
- /package/skills/{playwright-cli → playwright/playwright-cli}/references/session-management.md +0 -0
- /package/skills/{playwright-cli → playwright/playwright-cli}/references/storage-state.md +0 -0
- /package/skills/{playwright-cli → playwright/playwright-cli}/references/test-generation.md +0 -0
- /package/skills/{playwright-cli → playwright/playwright-cli}/references/tracing.md +0 -0
- /package/skills/{playwright-cli → playwright/playwright-cli}/references/video-recording.md +0 -0
|
@@ -1,329 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
title: Run Reports
|
|
3
|
-
description: Learn how Testomat.io generates comprehensive Run Reports for both manual and automated tests, offering basic and extended views. This guide covers features like filtering, sorting, customizing report layouts, viewing test case history, and accessing run statistics. It also details options for downloading reports as spreadsheets or PDFs, sharing reports via email or public links, and comparing test runs for in-depth analysis.
|
|
4
|
-
type: article
|
|
5
|
-
url: https://docs.testomat.io/project/runs/reports
|
|
6
|
-
head:
|
|
7
|
-
- tag: meta
|
|
8
|
-
attrs:
|
|
9
|
-
name: keywords
|
|
10
|
-
content: managing test runs, run reports, rungroups, test management, Testomat.io,multi-selection, test reporting, sharing reports, test run statistics, customize test cases view, download run report, run report in Spreadsheet, export run report, PDF format, Share Run Report, Share Report by Email, Share Report Publicly, Public Report, Combined Report.
|
|
11
|
-
---
|
|
12
|
-
|
|
13
|
-
When you run tests with Testomat.io, the system generates a **Run Report** displaying your test results. You can view these reports in two ways: **basic view** and **extended view**. Let's explore each option.
|
|
14
|
-
|
|
15
|
-
## Run Report Basic View
|
|
16
|
-
|
|
17
|
-
Clicking on the Test Run will toggle Run Report screen in basic view.
|
|
18
|
-
|
|
19
|
-

|
|
20
|
-
|
|
21
|
-
**In this basic Run Report, you can:**
|
|
22
|
-
|
|
23
|
-
1. View a list of test cases.
|
|
24
|
-
2. Check their statuses (Passed, Failed, Skipped, Pending, Custom statuses).
|
|
25
|
-
3. Review general information on the Run Result.
|
|
26
|
-
|
|
27
|
-

|
|
28
|
-
|
|
29
|
-
4. Sort test cases by suite, test case names or failure status.
|
|
30
|
-
5. Filter test cases by status, type, added messages, custom statuses, or assignees.
|
|
31
|
-
|
|
32
|
-

|
|
33
|
-
|
|
34
|
-
6. Search test cases by title or message.
|
|
35
|
-
|
|
36
|
-

|
|
37
|
-
|
|
38
|
-
7. Customize test cases view - (learn more in the [How to Customize Test Cases View](https://docs.testomat.io/project/runs/reports/#how-to-customize-test-cases-view) section).
|
|
39
|
-
|
|
40
|
-

|
|
41
|
-
|
|
42
|
-
Additionally, click on any test case within the run report to open it and view its details.
|
|
43
|
-
|
|
44
|
-

|
|
45
|
-
|
|
46
|
-
Moreover, you can view the history of run results for selected test case by filtering them by group name, defects, labels or environment.
|
|
47
|
-
|
|
48
|
-

|
|
49
|
-
|
|
50
|
-
You can also check overall **Statistics** by suites/tags/labels/assignees/priorities/custom statuses for the selected test run.
|
|
51
|
-
|
|
52
|
-

|
|
53
|
-
|
|
54
|
-
### How to Customize Test Cases View
|
|
55
|
-
|
|
56
|
-
Testomat.io allows you to customize the dispalay of test cases within a Run Report.
|
|
57
|
-
|
|
58
|
-
**This feature helps you:**
|
|
59
|
-
|
|
60
|
-
- **Show or hide columns** such as test status, duration, labels, tags, priority, etc.
|
|
61
|
-
- **Focus only on the data** most reelvant to you or your team.
|
|
62
|
-
- **Tailor the report layout** for different review needs.
|
|
63
|
-
|
|
64
|
-
To cusmozite your Run Report view, follow next steps:
|
|
65
|
-
|
|
66
|
-
1. Select any Run.
|
|
67
|
-
2. Click the **'Custom view'** button to open the configurable view instantly.
|
|
68
|
-
|
|
69
|
-

|
|
70
|
-
|
|
71
|
-
3. Click the **'Settings'** icon to access **'Runs List Settings'**.
|
|
72
|
-
4. Adjust Run view:
|
|
73
|
-
|
|
74
|
-
- Select/deselect columns to show only the data you need.
|
|
75
|
-
- Set each column width (px), to suit your preferences (or leave empty for automatic sizing).
|
|
76
|
-
|
|
77
|
-
5. Click the **'Save'** button to apply your changes.
|
|
78
|
-
|
|
79
|
-

|
|
80
|
-
|
|
81
|
-
You can also adjust each column width manually, in this case changes will save automatically.
|
|
82
|
-
|
|
83
|
-

|
|
84
|
-
|
|
85
|
-
:::note
|
|
86
|
-
|
|
87
|
-
Customized Test Cases view within a Run Report automatically applies to all Run Reports.
|
|
88
|
-
|
|
89
|
-
:::
|
|
90
|
-
|
|
91
|
-
### Tree View for Runs
|
|
92
|
-
|
|
93
|
-
Runs can be displayed in a hierarchical tree structure that mirrors the test organization. To switch to tree view, open a run and click the tree view icon in the top right corner of the test list.
|
|
94
|
-
|
|
95
|
-
You can:
|
|
96
|
-
|
|
97
|
-
* Expand folders
|
|
98
|
-
* Collapse sections
|
|
99
|
-
* Navigate large test sets faster
|
|
100
|
-
* Focus on specific areas of a project
|
|
101
|
-
|
|
102
|
-
:::note
|
|
103
|
-
|
|
104
|
-
This is particularly useful for large projects containing hundreds or thousands of tests.
|
|
105
|
-
|
|
106
|
-
:::
|
|
107
|
-
|
|
108
|
-
## Run Report Extended View
|
|
109
|
-
|
|
110
|
-
If you click the **'Report'** button from Basic Run Report view, you will see the extended view of the Run Report.
|
|
111
|
-
|
|
112
|
-

|
|
113
|
-
|
|
114
|
-
**In the extended report view, you can:**
|
|
115
|
-
|
|
116
|
-
1. View a list of tests.
|
|
117
|
-
2. Check their status (Passed, Failed, Skipped, Pending, Custom statuses).
|
|
118
|
-
3. See overview of all executed tests grouped by suites, tags, labels, assignees, or priorities.
|
|
119
|
-
4. Review the Run Report Summary.
|
|
120
|
-
|
|
121
|
-

|
|
122
|
-
|
|
123
|
-
5. Check **'Flaky Tests'** Analytics, if applicable.
|
|
124
|
-
|
|
125
|
-

|
|
126
|
-
|
|
127
|
-
6. Sort suites by Name and Failed status (as well works for Tags, Labels, Assignees, Priorities, Custom Statuses) in the Overview widget.
|
|
128
|
-
|
|
129
|
-

|
|
130
|
-
|
|
131
|
-
7. Filter test cases by status, type, added messages, custom statuses, or assignees.
|
|
132
|
-
|
|
133
|
-

|
|
134
|
-
|
|
135
|
-
8. Search test cases by title or message.
|
|
136
|
-
|
|
137
|
-

|
|
138
|
-
|
|
139
|
-
:::note
|
|
140
|
-
|
|
141
|
-
Use ↓ and ↑ keys to navigate tests and quickly view details for each test.
|
|
142
|
-
|
|
143
|
-

|
|
144
|
-
|
|
145
|
-
:::
|
|
146
|
-
|
|
147
|
-
Similar to the basic run report view, click on a test case to see its details or the history of run results.
|
|
148
|
-
|
|
149
|
-

|
|
150
|
-
|
|
151
|
-
## RunGroup Report
|
|
152
|
-
|
|
153
|
-
The same as for Runs, you can also view RunGroups Reportes.
|
|
154
|
-
To open **RunGroup Report basic view**, simply click on its name - RunGroup Report window will be displayed with general information and runs summary.
|
|
155
|
-
|
|
156
|
-

|
|
157
|
-
|
|
158
|
-
For more information go to the [RunGroup Report Basic View](https://docs.testomat.io/project/runs/rungroups#rungroup-report-basic-view) section **'RunGroups'** page.
|
|
159
|
-
|
|
160
|
-
From RunGroup Report Basic View you can open **Combined Report for RunGroups**, by clicking on the **'Combined Report'** button. This Report is designed to help you aggregate and analyze the results of multiple test runs within a single view. You can find more information about this report in [Combined Report for RunGroups](https://docs.testomat.io/project/runs/rungroups/#combined-report-for-rungroups) section on **'RunGroups'** page.
|
|
161
|
-
|
|
162
|
-
## How to Download Run Report as Spreadsheet
|
|
163
|
-
|
|
164
|
-
Testomat.io allows you to export Run Reports as a spreadsheet (XLSX file) to obtain your test data.
|
|
165
|
-
|
|
166
|
-
**Download Multiple Run Reports in One File**:
|
|
167
|
-
|
|
168
|
-
1. Go to Runs.
|
|
169
|
-
2. Enable **'Multi-select'**.
|
|
170
|
-
3. Select the Runs you want to download.
|
|
171
|
-
4. Click the **'Extra menu'** button.
|
|
172
|
-
5. Select **'Download'** option.
|
|
173
|
-
|
|
174
|
-

|
|
175
|
-
|
|
176
|
-
**Download a Separate Run Report**:
|
|
177
|
-
|
|
178
|
-
1. Go to Runs.
|
|
179
|
-
2. Select a Run Report.
|
|
180
|
-
3. Click the **'Report'** button.
|
|
181
|
-
|
|
182
|
-

|
|
183
|
-
|
|
184
|
-
4. Click the **'Extra menu'** button.
|
|
185
|
-
5. Select **'Download as Spreadsheet'** option.
|
|
186
|
-
|
|
187
|
-

|
|
188
|
-
|
|
189
|
-
## How to Export Run Report in PDF Format
|
|
190
|
-
|
|
191
|
-
Testomat.io also allows you to export your test run results as a PDF report. This feature enables easy sharing and archiving of test outcomes outside of Testomat.io.
|
|
192
|
-
|
|
193
|
-
**You can generate a structured PDF report with test execution details in two ways:**
|
|
194
|
-
|
|
195
|
-
1. Go to Runs.
|
|
196
|
-
2. Select a Run Report.
|
|
197
|
-
3. Click the **'Extra menu'** button.
|
|
198
|
-
4. Select **'Export as PDF'** option.
|
|
199
|
-
|
|
200
|
-

|
|
201
|
-
|
|
202
|
-
**OR**
|
|
203
|
-
|
|
204
|
-
1. Go to Runs.
|
|
205
|
-
2. Select a Run Report.
|
|
206
|
-
3. Click the **'Report'** button.
|
|
207
|
-
|
|
208
|
-

|
|
209
|
-
|
|
210
|
-
4. Click the **'Extra menu'** button.
|
|
211
|
-
5. Select **'Export as PDF'** option.
|
|
212
|
-
|
|
213
|
-

|
|
214
|
-
|
|
215
|
-
**Example of Test Run Report in PDF format**
|
|
216
|
-
|
|
217
|
-

|
|
218
|
-
|
|
219
|
-
**Use cases for PDF Reports:**
|
|
220
|
-
|
|
221
|
-
- **Sharing results with stakeholders:** Quickly generate a portable summary of test outcomes for product owners, QA managers, or external partners who may not have access to the platform.
|
|
222
|
-
|
|
223
|
-
- **Audit and compliance documentation:** Maintain a formal record of test results in a fixed format for compliance, certification, or audit purposes.
|
|
224
|
-
|
|
225
|
-
- **Team retrospectives and reviews:** Use the PDF report to review test coverage and outcomes during retrospectives or sprint reviews without relying on live access to the system.
|
|
226
|
-
|
|
227
|
-
## How to Share Run Report
|
|
228
|
-
|
|
229
|
-
You can share your Run Report for external access from the Extended report view using two options: **'Share Report by Email'** or **'Share Report Publicly'**.
|
|
230
|
-
|
|
231
|
-
### Share Report by Email
|
|
232
|
-
|
|
233
|
-
1. Open Extended report view.
|
|
234
|
-
|
|
235
|
-

|
|
236
|
-
|
|
237
|
-
2. Click the **'Extra menu'** button.
|
|
238
|
-
3. Select **'Share Report by Email'** option.
|
|
239
|
-
|
|
240
|
-

|
|
241
|
-
|
|
242
|
-
4. Enter one or multiple emails, separated with commas, in the displayed modal.
|
|
243
|
-
5. Click the **'Send'** button.
|
|
244
|
-
|
|
245
|
-

|
|
246
|
-
|
|
247
|
-
:::note
|
|
248
|
-
|
|
249
|
-
This action also creates public report without passcode for external access.
|
|
250
|
-
|
|
251
|
-
:::
|
|
252
|
-
|
|
253
|
-
**Example of the email with Run Report**
|
|
254
|
-
|
|
255
|
-

|
|
256
|
-
|
|
257
|
-
### Share Report Publicly
|
|
258
|
-
|
|
259
|
-
:::note
|
|
260
|
-
|
|
261
|
-
To enable the **'Share Report Publicly'** option, first enable this feature at the **Company level** and then enable it on **Project level** (Project Settings -> Project -> ‘Sharing’ section).
|
|
262
|
-
Check the instruction in the [Share Options](https://docs.testomat.io/management/company/administration/#share-options) section on **'Administration'** page.
|
|
263
|
-
|
|
264
|
-
:::
|
|
265
|
-
|
|
266
|
-
1. Open Extended report view.
|
|
267
|
-
|
|
268
|
-

|
|
269
|
-
|
|
270
|
-
2. Click the **'Extra menu'** button.
|
|
271
|
-
3. Select **'Share Report Publicly'** option.
|
|
272
|
-
|
|
273
|
-

|
|
274
|
-
|
|
275
|
-
4. Select **'Expiration Date'** (7 days is set by default).
|
|
276
|
-
5. **'Protect by passcode'** is enabled by default for better security (you can disable it if needed).
|
|
277
|
-
6. Click the **'Share'** button in the displayed modal.
|
|
278
|
-
|
|
279
|
-

|
|
280
|
-
|
|
281
|
-
7. Copy the **URL** for sharing the report.
|
|
282
|
-
8. Copy and share the **Passcode** with the link.
|
|
283
|
-
|
|
284
|
-
:::note
|
|
285
|
-
|
|
286
|
-
Copy and save the **Passcode** in a secure place. For security reasons, you will not be able to see it again after the Run Report Public URL is generated.
|
|
287
|
-
|
|
288
|
-
If a shared run report is protected with a **Passcode**, only users with the valid **Passcode** will have access to it.
|
|
289
|
-
|
|
290
|
-
:::
|
|
291
|
-
|
|
292
|
-
9. Copy and share the **Expiration date**, if needed.
|
|
293
|
-
|
|
294
|
-

|
|
295
|
-
|
|
296
|
-
After the Run Report Public URL is generated, **'Share'** button will be displayed on the Extended run report page.
|
|
297
|
-
|
|
298
|
-

|
|
299
|
-
|
|
300
|
-
:::note
|
|
301
|
-
|
|
302
|
-
You can stop sharing at any time by clicking the **'Stop Sharing'** button.
|
|
303
|
-
|
|
304
|
-

|
|
305
|
-
|
|
306
|
-
:::
|
|
307
|
-
|
|
308
|
-
Users with access to **'Public Run Report'** can:
|
|
309
|
-
|
|
310
|
-
1. Filter test cases by statuses.
|
|
311
|
-
2. Search test cases by name.
|
|
312
|
-
3. Copy the URL for sharing the Run report.
|
|
313
|
-
4. Check **'Full Report'** (only users with project access).
|
|
314
|
-
|
|
315
|
-

|
|
316
|
-
|
|
317
|
-
## Compare Test Runs
|
|
318
|
-
|
|
319
|
-
Select test runs to see a visual display of their similarities and differences. Compare your run reports and analyze them by parameters such as Flaky, Reviewed, Degraded, and more.
|
|
320
|
-
|
|
321
|
-
To compare Runs:
|
|
322
|
-
|
|
323
|
-
1. Enable Multiselection.
|
|
324
|
-
2. Select the Runs you want to compare.
|
|
325
|
-
3. Click the **'Compare'** button.
|
|
326
|
-
|
|
327
|
-

|
|
328
|
-
|
|
329
|
-
This feature is also available for RunGroups, allowing you to compare test results from different runs within a single RunGroup. You can find more information about it in the [Combined Report for RunGroups](https://docs.testomat.io/project/runs/rungroups/#combined-report-for-rungroups) section on **'RunGroups'** page.
|
|
@@ -1,342 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
title: RunGroups
|
|
3
|
-
description: Testomat.io's 'RunGroups' feature allows users to organize multiple test runs based on criteria like sprint or release, providing a consolidated view of testing efforts. This guide explains how to create, customize, move runs into, and copy RunGroups, along with options for managing their visibility, such as pinning and archiving, to streamline test management.
|
|
4
|
-
type: article
|
|
5
|
-
url: https://docs.testomat.io/project/runs/rungroups
|
|
6
|
-
head:
|
|
7
|
-
- tag: meta
|
|
8
|
-
attrs:
|
|
9
|
-
name: keywords
|
|
10
|
-
content: managing test runs, run reports, merge strategy, rungroups, relaunch runs, test management, Testomat.io, multi-selection, archiving, purge run groups, purge runs, purge logic, test reporting, runGroup flow, runs dashboard flow, RunGroup chart, copy RunGroup, pin a RunGroup, archive RunGroup, unarchive RunGroup, delete run group, purge flow.
|
|
11
|
-
---
|
|
12
|
-
|
|
13
|
-
First, let's clarify what **'RunGroups'** are in Testomat.io.
|
|
14
|
-
**RunGroups** allow you to organize and group multiple test runs together based on criteria like sprint, release, functional area, or any other logical grouping. This helps in analyzing aggregated results and providing a consolidated view of your testing efforts.
|
|
15
|
-
|
|
16
|
-
## How to Create a RunGroup
|
|
17
|
-
|
|
18
|
-
**To create a new RunGroup:**
|
|
19
|
-
|
|
20
|
-
1. Open **'Runs'** page.
|
|
21
|
-
2. Click the **'Additional options'** arrow button next to the 'Manual run' button.
|
|
22
|
-
3. Select **'New group'** option from the dropdown menu.
|
|
23
|
-
|
|
24
|
-

|
|
25
|
-
|
|
26
|
-
On New RunGroup screen you need:
|
|
27
|
-
|
|
28
|
-
1. Choose a **'Group Type'** (optional).
|
|
29
|
-
2. Add a **'Name of a Group'**.
|
|
30
|
-
3. Select **'Merge strategy'** (For more details, refer to the [Merge Strategies](https://docs.testomat.io/project/runs/merge-strategies) page in documentation).
|
|
31
|
-
4. Add **'Description of a group'**, if needed.
|
|
32
|
-
5. Click the **'Save'** button.
|
|
33
|
-
|
|
34
|
-

|
|
35
|
-
|
|
36
|
-
The new RunGroup will appear on Runs page and will open automatically after creation.
|
|
37
|
-
|
|
38
|
-

|
|
39
|
-
|
|
40
|
-
After RunGroup is added, you can create new Manual/Automated/Mixed Runs inside it.
|
|
41
|
-
|
|
42
|
-
## How to Create a Run in RunGroup
|
|
43
|
-
|
|
44
|
-
- To create a new **Manual Run**, open the RunGroup and click on **'Manual Run'** button.
|
|
45
|
-
|
|
46
|
-

|
|
47
|
-
|
|
48
|
-
After this in the displayed window select a test plan from the list or create a new one, and fill in other data that you need for your run (like: assignee, title, environment, description).
|
|
49
|
-
|
|
50
|
-
:::note
|
|
51
|
-
|
|
52
|
-
When you create a manual run from RunGroup, it will be automatically pre-populated on the **'New Manual Run'** window, but you can change it if needed.
|
|
53
|
-
|
|
54
|
-
:::
|
|
55
|
-
|
|
56
|
-

|
|
57
|
-
|
|
58
|
-
- To add a new **Automated Run** to a RunGroup, execute your tests with Testomat.io, providing TESTOMATIO_RUNGROUP_TITLE="Build ${BUILD_ID}".
|
|
59
|
-
|
|
60
|
-
Now, by clicking on expand arrow, you can view Test Runs and their results within created RunGroup.
|
|
61
|
-
|
|
62
|
-

|
|
63
|
-
|
|
64
|
-
Clicking on RunGroup name will toggle RunGroup Report screen in basic view. Here you can view more detailed information about the runs inside selected RunGroup and total results.
|
|
65
|
-
|
|
66
|
-

|
|
67
|
-
|
|
68
|
-
For more details go to the relevant section - [RunGroup Report in Basic View](https://docs.testomat.io/project/runs/reports/#rungroup-report-in-basic-view) on **'Reports'** page.
|
|
69
|
-
|
|
70
|
-
## How to Move a Run to a RunGroup
|
|
71
|
-
|
|
72
|
-
You may need to move a Run to a specific RunGroup (e.g., to associate it with a particular release or build). There are two ways to do this in Testomat.io:
|
|
73
|
-
|
|
74
|
-
Method 1: Move a Run to a RunGroup
|
|
75
|
-
|
|
76
|
-
1. Navigate to the **'Runs'** page
|
|
77
|
-
2. Select the Run you want to move
|
|
78
|
-
3. Click the **'Extra menu'** button
|
|
79
|
-
4. Select the **'Move'** from the dropdown menu
|
|
80
|
-
|
|
81
|
-

|
|
82
|
-
|
|
83
|
-
5. In the pop-up window, choose the **destination RunGroup**
|
|
84
|
-
6. Click the **'Move'** button to confirm the action
|
|
85
|
-
|
|
86
|
-

|
|
87
|
-
|
|
88
|
-
Method 2: Add Existing Runs to the RunGroup
|
|
89
|
-
|
|
90
|
-
1. Navigate to the **'Runs'** page
|
|
91
|
-
2. Click the **'Extra menu'** next to the RunGroup you want to move Runs into
|
|
92
|
-
3. Select the **'Add Existing Run'** option
|
|
93
|
-
|
|
94
|
-

|
|
95
|
-
|
|
96
|
-
4. In the modal window, select the Run(s) you want to move
|
|
97
|
-
5. Click the **Move Runs to Group** button to confirm
|
|
98
|
-
|
|
99
|
-

|
|
100
|
-
|
|
101
|
-
Moved or added Runs will appear in the selected RunGroup.
|
|
102
|
-
|
|
103
|
-

|
|
104
|
-
|
|
105
|
-
## RunGroup Report Basic View
|
|
106
|
-
|
|
107
|
-
You can view RunGroup Report by clicking on its name.
|
|
108
|
-
|
|
109
|
-

|
|
110
|
-
|
|
111
|
-
**In this basic RunGroup Report, you can:**
|
|
112
|
-
|
|
113
|
-
1. View the [RunGroup Chart](https://docs.testomat.io/project/runs/rungroups/#rungroup-chart).
|
|
114
|
-
2. Review **'RunGroup summary section'** with general information based on the selected [Merge Strategy](https://docs.testomat.io/project/runs/merge-strategies/).
|
|
115
|
-
3. View a list of test runs, their statuses with the number of Passed, Failed, Skipped tests, along with the run assignee and executed time.
|
|
116
|
-
4. Customize your RunGroup Report view (Read more in the [How to Customize Runs List View ](https://docs.testomat.io/project/runs/rungroups/#how-to-customize-runs-list-view) section below).
|
|
117
|
-
5. Check **'Combined Report'** (Read more in the [Combined Report for RunGroups](https://docs.testomat.io/project/runs/rungroups/#combined-report-for-rungroups) section below).
|
|
118
|
-
6. Add a new Manual Run to the selected RunGroup.
|
|
119
|
-
7. Generate a [Rungroup Statistic Report](https://docs.testomat.io/advanced/ai-powered-features/ai-powered-features/#rungroup-statistic-report)(AI-feature should be enabled on **'Company Settings'** page).
|
|
120
|
-
|
|
121
|
-

|
|
122
|
-
|
|
123
|
-
The **'RunGroup summary section'** displays a counter with the number of included and analyzed runs in the RunGroup. Archived runs are excluded from this counter and do not affect the total RunGroup result. You can see how many runs were archived, and by clicking on archived counter, you will be redirected to the **'Groups Archive'** page, where runs are already filtered by selected RunGroup.
|
|
124
|
-
|
|
125
|
-

|
|
126
|
-
|
|
127
|
-
## How to Customize Runs List View
|
|
128
|
-
|
|
129
|
-
When working with test runs inside RunGroup or Runs Dashboard Flow, you can adjust the table layout to fit your needs. Instead of using the default view, you can customize the runs table layout within **RunGroup** page or directly from the main **Runs Dashboard**.
|
|
130
|
-
|
|
131
|
-
**RunGroup Flow**
|
|
132
|
-
|
|
133
|
-
1. Go to **'Runs'** page.
|
|
134
|
-
2. Select RunGroup.
|
|
135
|
-
3. Click the **'Custom view'** button.
|
|
136
|
-
|
|
137
|
-

|
|
138
|
-
|
|
139
|
-
You can use the default custom view or personalize the table layout:
|
|
140
|
-
|
|
141
|
-
4. Click the **'Settings'** icon to access **'Runs List Settings'**.
|
|
142
|
-
|
|
143
|
-

|
|
144
|
-
|
|
145
|
-
5. Select or deselect columns to show only the data you need.
|
|
146
|
-
6. Set the column width (px), if needed, to improve readability.
|
|
147
|
-
7. Click the **'Save'** button to apply your changes.
|
|
148
|
-
|
|
149
|
-

|
|
150
|
-
|
|
151
|
-
Here's how your customized table within a RunGroup will appear after customization:
|
|
152
|
-
|
|
153
|
-

|
|
154
|
-
|
|
155
|
-
**Runs Dashboard Flow**
|
|
156
|
-
|
|
157
|
-
Similarly, customize the runs view from the main Runs Dashboard:
|
|
158
|
-
|
|
159
|
-
1. Go to **'Runs'** page.
|
|
160
|
-
2. Click the **'Custom view'** button.
|
|
161
|
-
|
|
162
|
-

|
|
163
|
-
|
|
164
|
-
3. Click the **'Settings'** icon to access **'Runs List Settings'**.
|
|
165
|
-
4. Customize your view in the **'Runs list settings'**.
|
|
166
|
-
5. Click the **'Save'** button to apply your changes.
|
|
167
|
-
|
|
168
|
-

|
|
169
|
-
|
|
170
|
-
Here's how your customized table will appear.
|
|
171
|
-
|
|
172
|
-

|
|
173
|
-

|
|
174
|
-
These customization options allow you to tailor the Runs View to display only the most relevant details, improving clarity and efficiency.
|
|
175
|
-
|
|
176
|
-
:::note
|
|
177
|
-
|
|
178
|
-
A customized runs list view automatically applies to all your RunGroups and main Runs Dashboard.
|
|
179
|
-
|
|
180
|
-
:::
|
|
181
|
-
|
|
182
|
-
## RunGroup Chart
|
|
183
|
-
|
|
184
|
-
The chart displays up to 50 of the latest test runs belonging to the group. If you have more runs, use the pagination arrows to view the results of the previous runs.
|
|
185
|
-
|
|
186
|
-

|
|
187
|
-
|
|
188
|
-
## Combined Report for RunGroups
|
|
189
|
-
|
|
190
|
-
Testomat.io provides a **'Combined Report'** feature for RunGroups. This report is designed to help you aggregate and analyze the results of all test runs within the RunGroup in a single view.
|
|
191
|
-
|
|
192
|
-
You can open **'Combined Report'** for a RunGroup by clicking on **'Combined Report'** button from the RunGroup Report Basic View window.
|
|
193
|
-
|
|
194
|
-

|
|
195
|
-
|
|
196
|
-
The comparison of runs is based on the first (main) launch. The statuses counters on the left side are calculated based on the main run.
|
|
197
|
-
|
|
198
|
-
To change the main run, click the **'Compare To'** button. Note that the counters of loaded tests is above them. The final summary on the right side is calculated based on the loaded tests.
|
|
199
|
-
|
|
200
|
-

|
|
201
|
-
|
|
202
|
-
:::note
|
|
203
|
-
|
|
204
|
-
The **'Combined Report'** feature for RunGroups does **not** combine test results of all runs inside one RunGroup into a single report. It only allows you **to compare** test results of all runs within this RunGroup.
|
|
205
|
-
|
|
206
|
-
:::
|
|
207
|
-
|
|
208
|
-
Inside the **'Combined Report'**, you can get a quick overview of the pass/fail rates for all tests within the Group. You can also see the total number of tests present in all compared runs, as well as the total number of flaky (unstable tests that passed or failed in different runs), reviewed (previously failing that passed in next runs), and degraded (previously passed that failed in next runs) tests.
|
|
209
|
-
|
|
210
|
-
Additionally, you can use filters and search features to view the data most relevant to your needs.
|
|
211
|
-
|
|
212
|
-

|
|
213
|
-
|
|
214
|
-
**Key benefits:**
|
|
215
|
-
|
|
216
|
-
- **Analyze Trends:** By seeing the results from multiple runs in one place, you can identify trends in test performance and stability over time.
|
|
217
|
-
|
|
218
|
-
- **Consolidate Data:** The combined report is especially useful for "mixed runs" (runs that include both manual and automated tests), as it consolidates all results into a single, cohesive report.
|
|
219
|
-
|
|
220
|
-
- **Compare Runs:** Testomat.io also has a separate **'Compare Test Runs'** feature that can be used within a RunGroup. This allows you to side-by-side compare the results of different runs to pinpoint changes, regressions, or improvements. (Go to [Compare Test Runs](https://docs.testomat.io/project/runs/reports/#compare-test-runs) section on **'Run reports'** page to read more about this feature.)
|
|
221
|
-
|
|
222
|
-
This feature is a powerful tool for project managers, QA leads, and stakeholders who need to make informed decisions based on a holistic view of testing activities, without having to manually compile data from individual reports.
|
|
223
|
-
|
|
224
|
-
## How to Copy RunGroup
|
|
225
|
-
|
|
226
|
-
You can easily create a new RunGroup, completely independent of any previous runs, by copying all relevant tests exactly as they are. You can configure what data should be copied, namely:
|
|
227
|
-
|
|
228
|
-
- **Assignee**: Define assignee details separately, preventing them from being copied.
|
|
229
|
-
- **Issues**: Choose whether to include or exclude linked issues during duplication.
|
|
230
|
-
- **Labels**: Decide whether labels should be duplicated or omitted in the new test run.
|
|
231
|
-
- **Environments**: Control the duplication of environment settings based on your requirements.
|
|
232
|
-
- **Nested Structure**: Preserve or exclude the nested structure of your test groups as you duplicate them.
|
|
233
|
-
|
|
234
|
-
**Follow these steps:**
|
|
235
|
-
|
|
236
|
-
1. Open the RunGroup.
|
|
237
|
-
2. Click the **'Extra menu'** button.
|
|
238
|
-
3. Select **'Copy'** option form the dropdown menu.
|
|
239
|
-
|
|
240
|
-

|
|
241
|
-
|
|
242
|
-
4. Select configuration options.
|
|
243
|
-
5. Click the **'Copy'** button.
|
|
244
|
-
|
|
245
|
-

|
|
246
|
-
|
|
247
|
-
6. Verify the created RunGroup.
|
|
248
|
-
|
|
249
|
-

|
|
250
|
-
|
|
251
|
-
:::note
|
|
252
|
-
|
|
253
|
-
Only manual runs will be copied with the RunGroup. If a copied RunGroup contains a mixed test run, automated test cases will be marked as skipped by default.
|
|
254
|
-
|
|
255
|
-
:::
|
|
256
|
-
|
|
257
|
-
## How to Pin a RunGroup
|
|
258
|
-
|
|
259
|
-
Testomat.io allows users to pin RunGroups to the top of the Dashboard. This provides quick access to critical or frequently used tests or test runs, helping teams stay focused on the most relevant tasks. Pinning supports faster navigation, improved focus, and customizable workflows—ideal for monitoring regression tests, environment-specific runs, or production hotfixes.
|
|
260
|
-
|
|
261
|
-
For more details, refer to the [How to Pin a Run or RunGroup](https://docs.testomat.io/project/runs/managing-runs/#how-to-pin-a-run-or-rungroup) section.
|
|
262
|
-
|
|
263
|
-
## How to Archive RunGroup
|
|
264
|
-
|
|
265
|
-
Testomat.io gives you an opportunity to archive a RunGroup, including all its contained Test Runs. This helps maintain better visibility on your main Run Dashboard.
|
|
266
|
-
|
|
267
|
-
For more information on archiving, visit the [Archive Runs & RunGroups](https://docs.testomat.io/project/runs/archive-runs-and-groups) page.
|
|
268
|
-
|
|
269
|
-
## How to Unarchive RunGroup
|
|
270
|
-
|
|
271
|
-
Archived RunGroups can be unarchived and moved back to the main Run Dashboard. Restoring a RunGroup also restores all its archived runs.
|
|
272
|
-
|
|
273
|
-
Read more in the [How to Unarchive Runs & RunGroups](https://docs.testomat.io/project/runs/archive-runs-and-groups/#how-to-unarchive-runs-&-groups) section.
|
|
274
|
-
|
|
275
|
-
## Purge Run Groups
|
|
276
|
-
|
|
277
|
-
Starting from **October 2025**, Testomat.io introduces the new **Purge** approach for **Run Groups**. This update extends the same **two-step purge-to-archive logic** that was introduced for individual [Runs](https://docs.testomat.io/project/runs/managing-runs/#purge-runs) in September 2025, ensuring data safety and consistency across all Run types.
|
|
278
|
-
|
|
279
|
-
The **Delete** option for Run Groups is now replaced with a **Purge** option.
|
|
280
|
-
|
|
281
|
-

|
|
282
|
-
|
|
283
|
-
### Compress and Move to Archive (Purge)
|
|
284
|
-
|
|
285
|
-
When a **Run Group** is purged (either manually or automatically):
|
|
286
|
-
|
|
287
|
-
- The **group itself** is **deleted**
|
|
288
|
-
- All **nested Runs** are moved to the **[Archive](https://docs.testomat.io/project/runs/archive-runs-and-groups/)** page
|
|
289
|
-
- All **nested Runs** are marked with the **purged** badge for easy identification
|
|
290
|
-
|
|
291
|
-

|
|
292
|
-
|
|
293
|
-
During this step:
|
|
294
|
-
|
|
295
|
-
- Stack traces are removed to reduce storage size
|
|
296
|
-
- Essential data is preserved, including:
|
|
297
|
-
- Test results
|
|
298
|
-
- Artifacts (attachments, logs, screenshots)
|
|
299
|
-
- Custom statuses and metadata
|
|
300
|
-
- Purged Runs remain available in the Archive page and can be restored at any time
|
|
301
|
-
|
|
302
|
-
:::note
|
|
303
|
-
|
|
304
|
-
The purge logic for **ongoing (pending) Runs** within a Run Group follows the **same rules** as described for individual Runs. For detailed information about the purge flow of ongoing Runs, see [**Purge Logic for Ongoing Runs**](https://docs.testomat.io/project/runs/managing-runs/#purge-logic-for-ongoing-runs).
|
|
305
|
-
|
|
306
|
-
:::
|
|
307
|
-
|
|
308
|
-
### Automatic Purge Flow
|
|
309
|
-
|
|
310
|
-
The same **two-step purge-to-archive logic** applies to automatically purged **Run Groups** via **Project Settings → [Purge Old Runs](https://docs.testomat.io/management/project/settings/#purge-old-runs)**.
|
|
311
|
-
|
|
312
|
-
When an automatic purge is triggered:
|
|
313
|
-
|
|
314
|
-
- The **group itself** is **deleted**
|
|
315
|
-
- All **nested Runs** are moved to the **[Archive](https://docs.testomat.io/project/runs/archive-runs-and-groups/)** page
|
|
316
|
-
- All **nested Runs** are marked with the **purged** badge for easy identification
|
|
317
|
-
|
|
318
|
-
### Limitations
|
|
319
|
-
|
|
320
|
-
:::note
|
|
321
|
-
|
|
322
|
-
Run Groups have a **limit of 20,000 Runs** for the purge process. If a group contains **more than 20,000 Runs**, only the **first 20,000** will be purged and moved to the Archive.
|
|
323
|
-
|
|
324
|
-
All remaining Runs beyond this limit will be **permanently deleted** and **cannot be restored**.
|
|
325
|
-
|
|
326
|
-
:::
|
|
327
|
-
|
|
328
|
-
### Permanent Deletion from Archive
|
|
329
|
-
|
|
330
|
-
:::note
|
|
331
|
-
|
|
332
|
-
If you remove a Run (that was part of a purged Run Group) from the Archive page, it is permanently deleted. This step is **irreversible** — once **deleted**, the Run cannot be restored.
|
|
333
|
-
|
|
334
|
-
:::
|
|
335
|
-
|
|
336
|
-

|
|
337
|
-
|
|
338
|
-
### Summary
|
|
339
|
-
|
|
340
|
-
By extending the **Purge** functionality to Run Groups, Testomat.io ensures that all grouped executions follow the same safe, transparent, and recoverable process as individual Runs.
|
|
341
|
-
|
|
342
|
-
Teams can efficiently manage large-scale test executions while maintaining complete data traceability and control.
|