testeiya 0.2.0 → 0.3.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +37 -175
- package/dist/prompt/context.js +79 -0
- package/dist/prompt/context.js.map +1 -0
- package/dist/prompt/index.js +56 -0
- package/dist/prompt/index.js.map +1 -0
- package/dist/prompt/print.js +22 -0
- package/dist/prompt/print.js.map +1 -0
- package/dist/prompt/project-info.js +2 -0
- package/dist/prompt/project-info.js.map +1 -0
- package/{src/prompt/system-prompt.ts → dist/prompt/system-prompt.js} +24 -21
- package/dist/prompt/system-prompt.js.map +1 -0
- package/dist/prompt/testomatio.js +264 -0
- package/dist/prompt/testomatio.js.map +1 -0
- package/dist/prompt/tools.js +49 -0
- package/dist/prompt/tools.js.map +1 -0
- package/dist/prompt/vocab.js +8 -0
- package/dist/prompt/vocab.js.map +1 -0
- package/dist/src/args.js +72 -0
- package/dist/src/args.js.map +1 -0
- package/dist/src/cli.js +61 -0
- package/dist/src/cli.js.map +1 -0
- package/dist/src/env.js +69 -0
- package/dist/src/env.js.map +1 -0
- package/dist/src/mcp-extension.js +9 -0
- package/dist/src/mcp-extension.js.map +1 -0
- package/dist/src/mcp.js +139 -0
- package/dist/src/mcp.js.map +1 -0
- package/dist/src/model.js +66 -0
- package/dist/src/model.js.map +1 -0
- package/dist/src/result.js +41 -0
- package/dist/src/result.js.map +1 -0
- package/dist/src/run.js +199 -0
- package/dist/src/run.js.map +1 -0
- package/dist/src/session.js +77 -0
- package/dist/src/session.js.map +1 -0
- package/dist/vendor/mcp.js +14801 -0
- package/mcp-tools.json +2967 -0
- package/package.json +29 -41
- package/prompt/context.ts +101 -0
- package/prompt/index.ts +93 -0
- package/prompt/print.ts +23 -0
- package/prompt/project-info.ts +30 -0
- package/prompt/system-prompt.ts +173 -0
- package/{src/prompt → prompt}/testomatio.ts +119 -8
- package/{src/prompt → prompt}/tools.ts +30 -8
- package/prompt/vocab.ts +7 -0
- package/{skills.lock.json → skills/skills.lock.json} +11 -7
- package/skills/skills.yaml +34 -0
- package/models.catalog.json +0 -6963
- package/skills/codeceptjs/ci-fix-tests/SKILL.md +0 -120
- package/skills/codeceptjs/codeceptjs-auth/SKILL.md +0 -158
- package/skills/codeceptjs/codeceptjs-exploration/SKILL.md +0 -91
- package/skills/codeceptjs/codeceptjs-fundamentals/SKILL.md +0 -158
- package/skills/codeceptjs/codeceptjs-run-analysis/SKILL.md +0 -90
- package/skills/codeceptjs/debugging-codeceptjs-tests/SKILL.md +0 -234
- package/skills/codeceptjs/migrate-codeceptjs-4/SKILL.md +0 -110
- package/skills/codeceptjs/migrate-cypress-to-codeceptjs/SKILL.md +0 -306
- package/skills/codeceptjs/migrate-protractor-to-codeceptjs/SKILL.md +0 -352
- package/skills/codeceptjs/migrate-selenium-java-to-codeceptjs/SKILL.md +0 -472
- package/skills/codeceptjs/migrate-testcafe-to-codeceptjs/SKILL.md +0 -350
- package/skills/codeceptjs/refactoring-codeceptjs-tests/SKILL.md +0 -81
- package/skills/codeceptjs/writing-codeceptjs-tests/SKILL.md +0 -206
- package/skills/playwright-best-practices-skill/LICENSE.md +0 -7
- package/skills/playwright-best-practices-skill/README.md +0 -147
- package/skills/playwright-best-practices-skill/SKILL.md +0 -303
- package/skills/playwright-best-practices-skill/advanced/authentication-flows.md +0 -360
- package/skills/playwright-best-practices-skill/advanced/authentication.md +0 -871
- package/skills/playwright-best-practices-skill/advanced/clock-mocking.md +0 -364
- package/skills/playwright-best-practices-skill/advanced/mobile-testing.md +0 -409
- package/skills/playwright-best-practices-skill/advanced/multi-context.md +0 -288
- package/skills/playwright-best-practices-skill/advanced/multi-user.md +0 -393
- package/skills/playwright-best-practices-skill/advanced/network-advanced.md +0 -452
- package/skills/playwright-best-practices-skill/advanced/third-party.md +0 -464
- package/skills/playwright-best-practices-skill/architecture/pom-vs-fixtures.md +0 -363
- package/skills/playwright-best-practices-skill/architecture/test-architecture.md +0 -369
- package/skills/playwright-best-practices-skill/architecture/when-to-mock.md +0 -383
- package/skills/playwright-best-practices-skill/browser-apis/browser-apis.md +0 -391
- package/skills/playwright-best-practices-skill/browser-apis/iframes.md +0 -403
- package/skills/playwright-best-practices-skill/browser-apis/service-workers.md +0 -504
- package/skills/playwright-best-practices-skill/browser-apis/websockets.md +0 -403
- package/skills/playwright-best-practices-skill/core/annotations.md +0 -424
- package/skills/playwright-best-practices-skill/core/assertions-waiting.md +0 -361
- package/skills/playwright-best-practices-skill/core/configuration.md +0 -452
- package/skills/playwright-best-practices-skill/core/fixtures-hooks.md +0 -417
- package/skills/playwright-best-practices-skill/core/global-setup.md +0 -434
- package/skills/playwright-best-practices-skill/core/locators.md +0 -242
- package/skills/playwright-best-practices-skill/core/page-object-model.md +0 -315
- package/skills/playwright-best-practices-skill/core/projects-dependencies.md +0 -453
- package/skills/playwright-best-practices-skill/core/test-data.md +0 -492
- package/skills/playwright-best-practices-skill/core/test-suite-structure.md +0 -361
- package/skills/playwright-best-practices-skill/core/test-tags.md +0 -298
- package/skills/playwright-best-practices-skill/debugging/console-errors.md +0 -420
- package/skills/playwright-best-practices-skill/debugging/debugging.md +0 -504
- package/skills/playwright-best-practices-skill/debugging/error-testing.md +0 -360
- package/skills/playwright-best-practices-skill/debugging/flaky-tests.md +0 -496
- package/skills/playwright-best-practices-skill/frameworks/angular.md +0 -530
- package/skills/playwright-best-practices-skill/frameworks/nextjs.md +0 -469
- package/skills/playwright-best-practices-skill/frameworks/react.md +0 -531
- package/skills/playwright-best-practices-skill/frameworks/vue.md +0 -574
- package/skills/playwright-best-practices-skill/infrastructure-ci-cd/ci-cd.md +0 -468
- package/skills/playwright-best-practices-skill/infrastructure-ci-cd/docker.md +0 -283
- package/skills/playwright-best-practices-skill/infrastructure-ci-cd/github-actions.md +0 -546
- package/skills/playwright-best-practices-skill/infrastructure-ci-cd/gitlab.md +0 -397
- package/skills/playwright-best-practices-skill/infrastructure-ci-cd/other-providers.md +0 -521
- package/skills/playwright-best-practices-skill/infrastructure-ci-cd/parallel-sharding.md +0 -371
- package/skills/playwright-best-practices-skill/infrastructure-ci-cd/performance.md +0 -453
- package/skills/playwright-best-practices-skill/infrastructure-ci-cd/reporting.md +0 -424
- package/skills/playwright-best-practices-skill/infrastructure-ci-cd/test-coverage.md +0 -497
- package/skills/playwright-best-practices-skill/testing-patterns/accessibility.md +0 -359
- package/skills/playwright-best-practices-skill/testing-patterns/api-testing.md +0 -719
- package/skills/playwright-best-practices-skill/testing-patterns/browser-extensions.md +0 -506
- package/skills/playwright-best-practices-skill/testing-patterns/canvas-webgl.md +0 -493
- package/skills/playwright-best-practices-skill/testing-patterns/component-testing.md +0 -500
- package/skills/playwright-best-practices-skill/testing-patterns/drag-drop.md +0 -576
- package/skills/playwright-best-practices-skill/testing-patterns/electron.md +0 -509
- package/skills/playwright-best-practices-skill/testing-patterns/file-operations.md +0 -377
- package/skills/playwright-best-practices-skill/testing-patterns/file-upload-download.md +0 -562
- package/skills/playwright-best-practices-skill/testing-patterns/forms-validation.md +0 -561
- package/skills/playwright-best-practices-skill/testing-patterns/graphql-testing.md +0 -331
- package/skills/playwright-best-practices-skill/testing-patterns/i18n.md +0 -508
- package/skills/playwright-best-practices-skill/testing-patterns/performance-testing.md +0 -476
- package/skills/playwright-best-practices-skill/testing-patterns/security-testing.md +0 -430
- package/skills/playwright-best-practices-skill/testing-patterns/visual-regression.md +0 -634
- package/skills/playwright-cli/SKILL.md +0 -420
- package/skills/playwright-cli/references/element-attributes.md +0 -23
- package/skills/playwright-cli/references/playwright-tests.md +0 -39
- package/skills/playwright-cli/references/request-mocking.md +0 -87
- package/skills/playwright-cli/references/running-code.md +0 -241
- package/skills/playwright-cli/references/session-management.md +0 -225
- package/skills/playwright-cli/references/storage-state.md +0 -275
- package/skills/playwright-cli/references/test-generation.md +0 -433
- package/skills/playwright-cli/references/tracing.md +0 -139
- package/skills/playwright-cli/references/video-recording.md +0 -143
- package/skills/testeiya/README.md +0 -29
- package/skills/testeiya/answer-formatting/SKILL.md +0 -30
- package/skills/testeiya/check-cucumber/SKILL.md +0 -200
- package/skills/testeiya/check-tests/SKILL.md +0 -260
- package/skills/testeiya/manual-run-assistant/SKILL.md +0 -61
- package/skills/testeiya/testomatio-docs/INDEX.md +0 -175
- package/skills/testeiya/testomatio-docs/SKILL.md +0 -102
- package/skills/testeiya/testomatio-docs/docs/advanced/ai-powered-features/ai-agents.md +0 -144
- package/skills/testeiya/testomatio-docs/docs/advanced/ai-powered-features/ai-powered-features.md +0 -531
- package/skills/testeiya/testomatio-docs/docs/advanced/ai-powered-features/ai-requirements.md +0 -437
- package/skills/testeiya/testomatio-docs/docs/advanced/api-access/index.md +0 -191
- package/skills/testeiya/testomatio-docs/docs/advanced/branches-folder/index.md +0 -277
- package/skills/testeiya/testomatio-docs/docs/advanced/bulk-edit-folder/bulk-edit-demos.mdx +0 -39
- package/skills/testeiya/testomatio-docs/docs/advanced/bulk-edit-folder/bulk-edit-on-suite-and-test-level.md +0 -122
- package/skills/testeiya/testomatio-docs/docs/advanced/bulk-edit-folder/what-is-bulk-edit.md +0 -170
- package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/branches.md +0 -53
- package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/index.mdx +0 -98
- package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/jira-issue.md +0 -151
- package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/runs-and-reports.md +0 -377
- package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/work-in-jira.md +0 -267
- package/skills/testeiya/testomatio-docs/docs/advanced/living-doc/index.md +0 -138
- package/skills/testeiya/testomatio-docs/docs/advanced/milestones/index.md +0 -220
- package/skills/testeiya/testomatio-docs/docs/advanced/shortcuts/index.md +0 -48
- package/skills/testeiya/testomatio-docs/docs/advanced/tags-labels/labels-and-custom-fields.md +0 -201
- package/skills/testeiya/testomatio-docs/docs/advanced/tags-labels/tags-or-labels.md +0 -94
- package/skills/testeiya/testomatio-docs/docs/advanced/tags-labels/tags.md +0 -243
- package/skills/testeiya/testomatio-docs/docs/advanced/test-artifacts/index.md +0 -135
- package/skills/testeiya/testomatio-docs/docs/advanced/tql/index.md +0 -394
- package/skills/testeiya/testomatio-docs/docs/getting-started/index.md +0 -290
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/azure.md +0 -96
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/bamboo.md +0 -94
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/bitbucket.md +0 -106
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/circle.md +0 -115
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/github.md +0 -120
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/gitlab.md +0 -88
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/index.mdx +0 -151
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/jenkins.md +0 -121
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/teamcity.md +0 -96
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/azure.md +0 -31
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/clickup.md +0 -54
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/confluence.md +0 -34
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/github.md +0 -31
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/gitlab.md +0 -93
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/index.md +0 -131
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/jira.md +0 -164
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/linear.md +0 -29
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/shortcut.md +0 -67
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/youtrack.md +0 -94
- package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/azure-devops.md +0 -53
- package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/email.md +0 -71
- package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/jira.md +0 -52
- package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/ms-teams.md +0 -60
- package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/rules.md +0 -267
- package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/slack.md +0 -82
- package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/telegram.md +0 -73
- package/skills/testeiya/testomatio-docs/docs/integrations/scim-provisioning/index.mdx +0 -207
- package/skills/testeiya/testomatio-docs/docs/integrations/scim-provisioning/okta.md +0 -204
- package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/azure.md +0 -95
- package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/google.md +0 -75
- package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/index.mdx +0 -33
- package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/okta.md +0 -108
- package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/saml.md +0 -66
- package/skills/testeiya/testomatio-docs/docs/legal/compliance/gdpr.md +0 -95
- package/skills/testeiya/testomatio-docs/docs/legal/compliance/index.mdx +0 -94
- package/skills/testeiya/testomatio-docs/docs/legal/compliance/privacy.md +0 -26
- package/skills/testeiya/testomatio-docs/docs/legal/compliance/terms.md +0 -32
- package/skills/testeiya/testomatio-docs/docs/legal/security/index.mdx +0 -242
- package/skills/testeiya/testomatio-docs/docs/legal/security/jira.md +0 -140
- package/skills/testeiya/testomatio-docs/docs/legal/security/subprocessors.md +0 -76
- package/skills/testeiya/testomatio-docs/docs/management/company/administration.md +0 -363
- package/skills/testeiya/testomatio-docs/docs/management/company/audit-log.md +0 -41
- package/skills/testeiya/testomatio-docs/docs/management/company/plan-features-comparison.md +0 -240
- package/skills/testeiya/testomatio-docs/docs/management/company/subscriptions.md +0 -326
- package/skills/testeiya/testomatio-docs/docs/management/company/trials.md +0 -75
- package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/accountant.md +0 -49
- package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/index.mdx +0 -285
- package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/read-only-user.md +0 -67
- package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/teams.md +0 -138
- package/skills/testeiya/testomatio-docs/docs/management/project/settings.md +0 -87
- package/skills/testeiya/testomatio-docs/docs/management/project/templates.md +0 -546
- package/skills/testeiya/testomatio-docs/docs/project/analytics/index.md +0 -326
- package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/classical-tests-markdown-format.md +0 -386
- package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/download-manual-tests-as-files.md +0 -378
- package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/export-to-obsidian.md +0 -112
- package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/export-to-spreadsheet.md +0 -127
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/auto-import.md +0 -63
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-Allure-TestOps.md +0 -49
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-QTest.md +0 -41
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-Qase.md +0 -39
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-Zephyr.md +0 -37
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-csv-xlsx.md +0 -117
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-cucumber.md +0 -93
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-qmetry.md +0 -44
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-source-code.md +0 -117
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-testcaselabs.md +0 -44
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-testmo.md +0 -47
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-testrail.md +0 -82
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-xray.md +0 -55
- package/skills/testeiya/testomatio-docs/docs/project/plans/index.md +0 -334
- package/skills/testeiya/testomatio-docs/docs/project/pulse/index.md +0 -104
- package/skills/testeiya/testomatio-docs/docs/project/runs/archive-runs-and-groups.md +0 -176
- package/skills/testeiya/testomatio-docs/docs/project/runs/custom-statuses.md +0 -195
- package/skills/testeiya/testomatio-docs/docs/project/runs/environments.md +0 -203
- package/skills/testeiya/testomatio-docs/docs/project/runs/managing-runs.md +0 -455
- package/skills/testeiya/testomatio-docs/docs/project/runs/merge-strategies.md +0 -137
- package/skills/testeiya/testomatio-docs/docs/project/runs/reports.md +0 -329
- package/skills/testeiya/testomatio-docs/docs/project/runs/rungroups.md +0 -342
- package/skills/testeiya/testomatio-docs/docs/project/runs/running-automated-tests.md +0 -256
- package/skills/testeiya/testomatio-docs/docs/project/runs/running-manual-and-automated-tests.md +0 -331
- package/skills/testeiya/testomatio-docs/docs/project/runs/running-tests-manually.md +0 -633
- package/skills/testeiya/testomatio-docs/docs/project/runs/temporary-tests-notes.md +0 -223
- package/skills/testeiya/testomatio-docs/docs/project/steps-snippets/snippets.md +0 -68
- package/skills/testeiya/testomatio-docs/docs/project/steps-snippets/steps.md +0 -181
- package/skills/testeiya/testomatio-docs/docs/project/tests/bdd-test-case-editor.md +0 -98
- package/skills/testeiya/testomatio-docs/docs/project/tests/classical-test-case-editor.md +0 -354
- package/skills/testeiya/testomatio-docs/docs/project/tests/classical-vs-bdd.md +0 -169
- package/skills/testeiya/testomatio-docs/docs/project/tests/copy-and-move-your-tests.md +0 -156
- package/skills/testeiya/testomatio-docs/docs/project/tests/other-features-for-test-case-design.md +0 -412
- package/skills/testeiya/testomatio-docs/docs/project/tests/test-case-creation-and-editing.md +0 -527
- package/skills/testeiya/testomatio-docs/docs/support/index.md +0 -75
- package/skills/testeiya/testomatio-docs/docs/test-reporting/artifacts.md +0 -388
- package/skills/testeiya/testomatio-docs/docs/test-reporting/cli.md +0 -215
- package/skills/testeiya/testomatio-docs/docs/test-reporting/configuration.md +0 -247
- package/skills/testeiya/testomatio-docs/docs/test-reporting/frameworks.md +0 -490
- package/skills/testeiya/testomatio-docs/docs/test-reporting/functions.md +0 -144
- package/skills/testeiya/testomatio-docs/docs/test-reporting/index.mdx +0 -48
- package/skills/testeiya/testomatio-docs/docs/test-reporting/junit.md +0 -248
- package/skills/testeiya/testomatio-docs/docs/test-reporting/logger.md +0 -138
- package/skills/testeiya/testomatio-docs/docs/test-reporting/php.md +0 -77
- package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/bitbucket.md +0 -72
- package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/csv.md +0 -23
- package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/debug.md +0 -18
- package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/github.md +0 -63
- package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/gitlab.md +0 -29
- package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/html.md +0 -80
- package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/index.mdx +0 -57
- package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/testomatio.md +0 -331
- package/skills/testeiya/testomatio-docs/docs/test-reporting/python.md +0 -229
- package/skills/testeiya/testomatio-docs/docs/test-reporting/workflows.md +0 -325
- package/skills/testeiya/testomatio-docs/docs/tutorials/Interacting-via-API.md +0 -159
- package/skills/testeiya/testomatio-docs/docs/tutorials/Manual-Testing-Classic.md +0 -164
- package/skills/testeiya/testomatio-docs/docs/tutorials/Set-Up-Read-Only-Access.md +0 -85
- package/skills/testeiya/testomatio-docs/docs/tutorials/Using-AI-in-Testing.md +0 -163
- package/skills/testeiya/testomatio-docs/docs/tutorials/Working-with-Jira.md +0 -125
- package/skills/testeiya/testomatio-docs/docs/tutorials/java.md +0 -478
- package/skills/testeiya/testomatio-docs/docs/tutorials/playwright.md +0 -207
- package/skills/testeiya/testomatio-docs/docs/tutorials/webdriver.md +0 -218
- package/skills/testeiya/testomatio-docs/docs.lock.json +0 -6
- package/skills/testeiya/testomatio-reporter/SKILL.md +0 -307
- package/skills/testomatio/explorbot/explorbot-fundamentals/SKILL.md +0 -88
- package/skills/testomatio/explorbot/explorbot-plan/SKILL.md +0 -95
- package/skills/testomatio/explorbot/explorbot-setup/SKILL.md +0 -334
- package/skills/testomatio/test-automation/automate-manual-test-cases/CLAUDE.md +0 -56
- package/skills/testomatio/test-automation/automate-manual-test-cases/SKILL.md +0 -242
- package/skills/testomatio/test-automation/automate-manual-test-cases/references/CODECEPTJS_BEST_PRACTICES.md +0 -182
- package/skills/testomatio/test-automation/automate-manual-test-cases/references/FINAL_SUMMARY_TEMPLATE.md +0 -22
- package/skills/testomatio/test-automation/automate-manual-test-cases/references/PLAYWRIGHT_BEST_PRACTICES.md +0 -90
- package/skills/testomatio/test-automation/automate-manual-test-cases/references/POM_BEST_PRACTICES.md +0 -53
- package/skills/testomatio/test-automation/automate-manual-test-cases/references/TEST_DATA_MANAGEMENT.md +0 -52
- package/skills/testomatio/test-automation/debug-fix-failed-flaky-autotests/SKILL.md +0 -107
- package/skills/testomatio/test-automation/debug-fix-failed-flaky-autotests/references/DEBUGGING_QUICK_REFERENCE.md +0 -113
- package/skills/testomatio/test-automation/qa-automation-test-consolidation/SKILL.md +0 -29
- package/skills/testomatio/test-automation/run-tests-with-testomatio-reporter/SKILL.md +0 -151
- package/skills/testomatio/test-automation/setup-pr-testing/SKILL.md +0 -199
- package/skills/testomatio/test-automation/testomat-allure-adapter/SKILL.md +0 -289
- package/skills/testomatio/test-management/detect-duplicate-test-cases/SKILL.md +0 -86
- package/skills/testomatio/test-management/detect-duplicate-test-cases/references/DUPLICATE_INSTRUCTIONS.md +0 -54
- package/skills/testomatio/test-management/improve-test-cases/SKILL.md +0 -114
- package/skills/testomatio/test-management/improve-test-cases/references/TESTOMAT_MARKDOWN_EXAMPLE.md +0 -66
- package/skills/testomatio/test-management/pull-request-diff-analyzer/SKILL.md +0 -141
- package/skills/testomatio/test-management/qa-e2e-tests-reporting/SKILL.md +0 -354
- package/skills/testomatio/test-management/qa-e2e-tests-reporting/references/TESTOMATIO_ARTIFACTS.md +0 -82
- package/skills/testomatio/test-management/qa-e2e-tests-reporting/references/TESTOMATIO_HTML_REPORT.md +0 -37
- package/skills/testomatio/test-management/qa-e2e-tests-reporting/references/TESTOMATIO_REPORTERS_CONFIG.md +0 -118
- package/skills/testomatio/test-management/qa-pr-requirements-analyzer/SKILL.md +0 -149
- package/skills/testomatio/test-management/qa-pr-requirements-analyzer/references/summary-example.md +0 -54
- package/skills/testomatio/test-management/qa-requirement-reviewer/SKILL.md +0 -126
- package/skills/testomatio/test-management/qa-requirement-reviewer/references/requirements_reviewer_examples.md +0 -128
- package/skills/testomatio/test-management/qa-test-code-coverage/SKILL.md +0 -174
- package/skills/testomatio/test-management/qa-test-code-coverage/references/COVERAGE_FILE_FORMAT.md +0 -138
- package/skills/testomatio/test-management/qa-test-code-coverage/references/E2E_FRAMEWORKS.md +0 -65
- package/skills/testomatio/test-management/qa-test-code-coverage/scripts/check-coverage.mjs +0 -40
- package/skills/testomatio/test-management/qa-thinking/SKILL.md +0 -36
- package/skills/testomatio/test-management/qa-write-test-cases/SKILL.md +0 -267
- package/skills/testomatio/test-management/qa-write-test-cases/references/test-case-format.md +0 -387
- package/skills/testomatio/test-management/qa-write-test-cases/references/testomat-tms-guide.md +0 -72
- package/skills/testomatio/test-management/qa-write-test-cases/references/writing-rule.md +0 -159
- package/skills/testomatio/test-management/scan-automation-project/SKILL.md +0 -152
- package/skills/testomatio/test-management/sync-test-cases-with-tms/SKILL.md +0 -107
- package/skills/testomatio/test-management/sync-test-cases-with-tms/references/TESTOMATIO_CLI.md +0 -140
- package/skills/testomatio/test-management/testomatio-mcp/SKILL.md +0 -182
- package/skills/testomatio/test-management/testomatio-mcp/references/MCP_SETUP.md +0 -169
- package/skills.yaml +0 -21
- package/src/ai-debug.ts +0 -188
- package/src/api/agent-get.ts +0 -22
- package/src/api/agent-start.ts +0 -384
- package/src/api/client-log.ts +0 -29
- package/src/api/context.ts +0 -332
- package/src/api/debug-layout.ts +0 -47
- package/src/api/debug-report.ts +0 -29
- package/src/api/debug-snapshot.ts +0 -30
- package/src/api/debug-stream.ts +0 -52
- package/src/api/files-delete.ts +0 -184
- package/src/api/files-read.ts +0 -41
- package/src/api/files-rename.ts +0 -99
- package/src/api/files-tree.ts +0 -292
- package/src/api/files-write.ts +0 -41
- package/src/api/mcp.ts +0 -603
- package/src/api/memory.ts +0 -69
- package/src/api/open-external.ts +0 -39
- package/src/api/playwright-cli.ts +0 -606
- package/src/api/providers.ts +0 -457
- package/src/api/read-env-token.ts +0 -33
- package/src/api/sessions.ts +0 -79
- package/src/api/settings.ts +0 -120
- package/src/api/skills.ts +0 -58
- package/src/api/testomatio-attachment.ts +0 -89
- package/src/api/testomatio-auth.ts +0 -284
- package/src/api/testomatio-proxy.ts +0 -197
- package/src/api/testomatio-run-stats.ts +0 -81
- package/src/api/testomatio-target.ts +0 -225
- package/src/api/testomatio-transcription.ts +0 -82
- package/src/api/workspace-search.ts +0 -96
- package/src/api/workspace-sync.ts +0 -151
- package/src/api/workspace.ts +0 -165
- package/src/app-server.ts +0 -556
- package/src/bridge.ts +0 -251
- package/src/check-tests.ts +0 -156
- package/src/cli.ts +0 -17
- package/src/commands.ts +0 -244
- package/src/config.ts +0 -183
- package/src/connection.ts +0 -658
- package/src/context-store.ts +0 -265
- package/src/debug-bus.ts +0 -405
- package/src/extensions/tool-gate.ts +0 -220
- package/src/extensions/webui/ask-channel.ts +0 -52
- package/src/extensions/webui/index.ts +0 -184
- package/src/extensions/webui/tools/ask-question.ts +0 -88
- package/src/extensions/webui/tools/query-result.ts +0 -79
- package/src/extensions/webui/tools/render-chart.ts +0 -53
- package/src/extensions/webui/tools/render-item.ts +0 -61
- package/src/extensions/webui/tools/render-list.ts +0 -170
- package/src/extensions/webui/tools/render-result.ts +0 -85
- package/src/extensions/webui/tools/render-tree.ts +0 -71
- package/src/extensions/webui/tools/ui-widget.ts +0 -54
- package/src/extensions/webui/tools/widget-result.ts +0 -13
- package/src/extensions/webui/widget-channel.ts +0 -50
- package/src/file-log.ts +0 -196
- package/src/git-tracked.ts +0 -68
- package/src/gitignore.ts +0 -53
- package/src/json-store.ts +0 -43
- package/src/load-env.ts +0 -70
- package/src/loader-style.ts +0 -96
- package/src/main.ts +0 -140
- package/src/mcp-catalog.ts +0 -96
- package/src/models-catalog.ts +0 -72
- package/src/permissions.ts +0 -120
- package/src/project-dir.ts +0 -189
- package/src/project-info.ts +0 -108
- package/src/prompt/app-ui.ts +0 -59
- package/src/prompt/browser.ts +0 -27
- package/src/prompt/index.ts +0 -62
- package/src/server-info.ts +0 -62
- package/src/session-factory.ts +0 -442
- package/src/session-store.ts +0 -115
- package/src/shell-env.ts +0 -45
- package/src/skills.ts +0 -163
- package/src/sync-snapshot.ts +0 -140
- package/src/telemetry.ts +0 -239
- package/src/testomatio-auth.ts +0 -299
- package/src/testomatio.ts +0 -98
- package/src/theme.ts +0 -30
- package/src/user-env.ts +0 -94
- package/src/welcome.ts +0 -127
- package/src/workspace/safe-path.ts +0 -85
- package/src/workspace/test-md.ts +0 -70
- package/src/workspace-model.ts +0 -440
- package/testeiya.config.json +0 -14
|
@@ -1,455 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
title: Managing Runs
|
|
3
|
-
description: Learn how to manage test runs effectively in Testomat.io. This guide covers generating detailed run reports, organizing runs into groups, applying merge strategies, relaunching runs, copying runs, and filtering results. Testomat.io also supports creating new test cases directly within runs, managing test case history, pinning important runs, and bulk actions for archiving, merging, and downloading reports for efficient test management.
|
|
4
|
-
|
|
5
|
-
type: article
|
|
6
|
-
url: https://docs.testomat.io/project/runs/managing-runs
|
|
7
|
-
head:
|
|
8
|
-
- tag: meta
|
|
9
|
-
attrs:
|
|
10
|
-
name: keywords
|
|
11
|
-
content: managing test runs, purge runs, purge logic, purge old runs, delete runs, purge flow, run reports, merge strategy, rungroups, relaunch runs, copy run, test management, Testomat.io, run comparison, multi-selection, archiving, test reporting, filter test runs, share reports, pin a run, test case history, mixed runs, manual runs, TQL.
|
|
12
|
-
---
|
|
13
|
-
|
|
14
|
-
Testomat.io offers comprehensive features for managing test runs, catering to both manual and automated testing. Here's how to manage runs within the system:
|
|
15
|
-
|
|
16
|
-
1. **Update Test Runs:** Add new test cases directly from Test Run, **Relaunch** finished Test Runs to update tests statuses, or **Launch a copy**. Find more information in the sections below.
|
|
17
|
-
|
|
18
|
-
2. **Filter Test Runs:** Filter runs and **share the filter parameters** with your teammates directly via URL. find more information added in the sections below.
|
|
19
|
-
|
|
20
|
-
3. **Group Runs:** Organize your runs into **RunGroups** based on criteria like sprint, release, functional area, or any other logical grouping. Read more on the [RunGroups](https://docs.testomat.io/project/runs/rungroups) page.
|
|
21
|
-
|
|
22
|
-
4. **Select a Merge Strategy:** Aggregate the results of the individual test runs within a RunGroup. Find more information on the [Merge Strategies](https://docs.testomat.io/project/runs/merge-strategies) page.
|
|
23
|
-
|
|
24
|
-
5. **Archive Runs & RunGroups:** Maintain a clean and organized main board tailored to your ongoing testing priorities. Read how to archive Runs & RunGroups on the [Archive Runs & RunGroups](https://docs.testomat.io/project/runs/archive-runs-and-groups) page.
|
|
25
|
-
|
|
26
|
-
6. **Share Run Reports:** Share test results with your teammates or stakeholders. You can find detail information about Reports on the [Run Reports](https://docs.testomat.io/project/runs/reports) page.
|
|
27
|
-
|
|
28
|
-
## How to Create a Test Case within a Run
|
|
29
|
-
|
|
30
|
-
You can create test cases directly within a test Run, avoiding additional navigation and ensuring the new tests are automatically included in the associated test plan.
|
|
31
|
-
|
|
32
|
-
1. Go to the **'Runs'** page.
|
|
33
|
-
2. Start the desired test run.
|
|
34
|
-
3. Click the **'Add test to suite'** button.
|
|
35
|
-
|
|
36
|
-

|
|
37
|
-
|
|
38
|
-
4. Enter the test case title in the **'Test Title'** field on **'Create manual test in suite'** window.
|
|
39
|
-
5. Click **'Add description'** button if you want to add more details to a test case.
|
|
40
|
-
6. Click **'Save test'** button to add the new test case.
|
|
41
|
-
|
|
42
|
-
OR
|
|
43
|
-
|
|
44
|
-
7. Click the **'Cancel'** button to discard it.
|
|
45
|
-
|
|
46
|
-

|
|
47
|
-
|
|
48
|
-
:::note
|
|
49
|
-
|
|
50
|
-
The **'Add test to suite'** button can be hidden via the **'Hide Creation Buttons'** option in the **'Extra menu'** dropdown menu.
|
|
51
|
-
|
|
52
|
-

|
|
53
|
-
|
|
54
|
-
:::
|
|
55
|
-
|
|
56
|
-
**Use Cases for Creating Test Cases Within a Run:**
|
|
57
|
-
|
|
58
|
-
- **Identifying gaps in test coverage during execution:** While performing manual testing, a tester discovers an untested scenario. They can instantly add it to the run and repository without switching context.
|
|
59
|
-
- **Capturing exploratory test cases:** Testers conducting exploratory testing can log and formalize new test cases as they go, ensuring useful insights are not lost.
|
|
60
|
-
- **Collaborative test planning in real time:** During team sessions or test reviews, testers can collectively identify and add new tests based on discussion or observed issues.
|
|
61
|
-
|
|
62
|
-
## Tags and Labels on Manual Run Page
|
|
63
|
-
|
|
64
|
-
On Manual Run page you can enable/disable the view of tags and labels for your test. To do that, follow next steps:
|
|
65
|
-
|
|
66
|
-
1. Go to the **'Runs'** page.
|
|
67
|
-
2. Start the desired test run.
|
|
68
|
-
3. Click the **'Extra menu'** button.
|
|
69
|
-
4. Select desired option for labels 'Show/hide Labels'.
|
|
70
|
-
5. Select desired option for tags 'Show/hide Tags'.
|
|
71
|
-
|
|
72
|
-

|
|
73
|
-
|
|
74
|
-
## Test Case History
|
|
75
|
-
|
|
76
|
-
When you launch a test run and make changes to test case title or description, a new entry is added to the test case's history. This entry records who made the edit and when it occurred.
|
|
77
|
-
|
|
78
|
-
1. Navigate to the **'Runs'** page.
|
|
79
|
-
2. Launch a new Run or continue an existing Run.
|
|
80
|
-
|
|
81
|
-

|
|
82
|
-
|
|
83
|
-
3. Click the **'Extra menu'** button next to the test case you want to edit.
|
|
84
|
-
4. Select **Edit**.
|
|
85
|
-
|
|
86
|
-

|
|
87
|
-
|
|
88
|
-
5. Modify the test case title or description as needed.
|
|
89
|
-
6. Click **Save** to apply the changes.
|
|
90
|
-
|
|
91
|
-

|
|
92
|
-
|
|
93
|
-
7. The new history entry will appear directly in the test case block, showing who edited the test case and the date and time of the change.
|
|
94
|
-
|
|
95
|
-

|
|
96
|
-
|
|
97
|
-
:::note
|
|
98
|
-
|
|
99
|
-
The test case history is only available while the test run is in progress.
|
|
100
|
-
|
|
101
|
-
:::
|
|
102
|
-
|
|
103
|
-
## How to Pin a Run or RunGroup
|
|
104
|
-
|
|
105
|
-
Testomat.io allows users to pin RunGroups or individual Runs to the top of the Dashboard for quick access to critical or frequently used tests or test runs, helping teams stay focused on the most relevant tasks. This supports faster navigation, improved focus, and customizable workflows — ideal for monitoring regression tests, environment-specific runs, or production hotfixes.
|
|
106
|
-
|
|
107
|
-
**Use Cases for Pinning:**
|
|
108
|
-
|
|
109
|
-
- **Faster Access:** Quickly access important or active test runs without searching through lists. Example: Pin ongoing regression tests for easy monitoring during a sprint.
|
|
110
|
-
|
|
111
|
-
- **Improved Focus:** Keeps high-priority RunGroups or Runs in view, reducing distractions. Example: Pin runs for specific environments or features during critical phases.
|
|
112
|
-
|
|
113
|
-
- **Customizable Dashboard:** Tailors the Dashboard to individual team workflows and priorities. Example: Pin runs related to production hotfixes or high-impact bugs.
|
|
114
|
-
|
|
115
|
-
**To Pin Run or RunGroup:**
|
|
116
|
-
|
|
117
|
-
1. Go to the **'Runs'** page.
|
|
118
|
-
2. Find the Run or RunGroup you want to pin.
|
|
119
|
-
3. Click the **'Extra menu'** button aligned next to the item.
|
|
120
|
-
4. Choose **'Pin'** from the dropdown options.
|
|
121
|
-
|
|
122
|
-

|
|
123
|
-
|
|
124
|
-
The **'Pin'** option is also available in the detailed run view menu.
|
|
125
|
-
|
|
126
|
-

|
|
127
|
-
|
|
128
|
-
The selected Run or RunGroup will now appear **pinned at the top** of the list for quick and easy access.
|
|
129
|
-
|
|
130
|
-

|
|
131
|
-
|
|
132
|
-
## How to Relaunch Runs
|
|
133
|
-
|
|
134
|
-
For some reason, you may need to re-check some tests in your Run. Testomat.io offers several options to relaunch your Test Run, depending on the run type and your needs.
|
|
135
|
-
|
|
136
|
-
### Basic Relaunch for Manual Run
|
|
137
|
-
|
|
138
|
-
Select the basic relauch option to restart your manual test run with all test cases and their current statuses:
|
|
139
|
-
|
|
140
|
-
1. On the **'Runs'** page, select the run that you need to re-check.
|
|
141
|
-
2. Click the **'Extra menu'** button.
|
|
142
|
-
3. Select **'Relaunch'** option from the dropdown menu.
|
|
143
|
-
|
|
144
|
-

|
|
145
|
-
|
|
146
|
-
The **'Relaunch'** option is also available on the main Dashboard.
|
|
147
|
-
|
|
148
|
-

|
|
149
|
-
|
|
150
|
-
Your Test Run will open in Manual Run window, allowing you to re-check tests and change their statuses:
|
|
151
|
-
|
|
152
|
-

|
|
153
|
-
|
|
154
|
-
When finished, click the **'Finish'** button. You will see tests with new statuses and updated Run Report.
|
|
155
|
-
|
|
156
|
-

|
|
157
|
-
|
|
158
|
-
Your Test Run will also contain updated information on its results, including a changed tag and Run status.
|
|
159
|
-
|
|
160
|
-

|
|
161
|
-
|
|
162
|
-
### Relaunch Failed Test Cases for Automated Runs
|
|
163
|
-
|
|
164
|
-
For your automated or mixed runs, select **'Relaunch Failed on CI'** option to relaunch only failed test cases from the selected run.
|
|
165
|
-
|
|
166
|
-
1. On the **'Runs'** page, select the automated or mixed run that you need to re-check.
|
|
167
|
-
2. Click the **'Extra menu'** button.
|
|
168
|
-
3. Select **'Relaunch Failed on CI'** option from the dropdown menu.
|
|
169
|
-
|
|
170
|
-

|
|
171
|
-
|
|
172
|
-
After selecting **'Relaunch Failed on CI'** option, failed automated test cases will automatically re-run on CI and Manual Run window will open for re-checking manual failed test cases. The same test run with updated statuses will be displayed after run is finished.
|
|
173
|
-
|
|
174
|
-
### Relaunch All Test Cases for Automated Runs
|
|
175
|
-
|
|
176
|
-
For your automated or mixed runs, you can also select **'Relaunch All on CI'** option to relaunch all test cases from the selected run.
|
|
177
|
-
|
|
178
|
-
1. On the **'Runs'** page, select the automated or mixed run that you need to re-check.
|
|
179
|
-
2. Click the **'Extra menu'** button.
|
|
180
|
-
3. Select **'Relaunch All on CI'** option from the dropdown menu.
|
|
181
|
-
|
|
182
|
-

|
|
183
|
-
|
|
184
|
-
All automated tests cases will automatically re-run on CI.
|
|
185
|
-
If you relauch mixed run, your Test Run will open in Manual Run window and you will be able to re-check manual tests and change their statuses, while automated test cases re-run on CI simultaneously.
|
|
186
|
-
|
|
187
|
-
### Relaunch Automated Runs Manually
|
|
188
|
-
|
|
189
|
-
With **'Relaunch Manually'** option you can as well re-check your automated test cases manually.
|
|
190
|
-
|
|
191
|
-
1. On the **'Runs'** page, select the automated or mixed run that you need to re-check.
|
|
192
|
-
2. Click the **'Extra menu'** button.
|
|
193
|
-
3. Select **'Relaunch Manually'** option from the dropdown menu.
|
|
194
|
-
|
|
195
|
-

|
|
196
|
-
|
|
197
|
-
In this case, your Test Run will open in Manual Run window, allowing you to manually re-check all tests (manual and automated) and change their statuses if needed.
|
|
198
|
-
|
|
199
|
-
### Advanced Relaunch
|
|
200
|
-
|
|
201
|
-
The **'Advanced Relaunch'** option offers more flexibility when relaunching tests.
|
|
202
|
-
It allows you to select specific tests to relaunch, instead of repeating the full run. This enables more targeted, efficient workflows — whether you’re rerunning failed tests or just need to retest a subset of scenarios.
|
|
203
|
-
|
|
204
|
-
**'Advanced Relaunch'** option is available for all types of test runs.
|
|
205
|
-
|
|
206
|
-
1. On the **'Runs'** page, select the run that you need to re-check.
|
|
207
|
-
2. Click the **'Extra menu'** button.
|
|
208
|
-
3. Select **'Advanced Relaunch'** option from the dropdown menu.
|
|
209
|
-
|
|
210
|
-

|
|
211
|
-
|
|
212
|
-
In the opened sidebar window, configure your relaunch (optional):
|
|
213
|
-
|
|
214
|
-
1. Enter a custom run title.
|
|
215
|
-
2. Enable **'Create new run'** option (if needed).
|
|
216
|
-
3. Enable **'Keep values'** option (available only if **'Create new run'** is enabled; this shows previous statuses for test cases).
|
|
217
|
-
4. Select the tests that you want to include in your relaunched run one by one (use filtering, sorting, or search options).
|
|
218
|
-
5. Click the **'Relaunch'** button.
|
|
219
|
-
|
|
220
|
-

|
|
221
|
-
|
|
222
|
-
#### Create New Run
|
|
223
|
-
|
|
224
|
-
The **'Create new run'** checkbox controls whether the relaunch creates a separate copy or updates the original run.
|
|
225
|
-
|
|
226
|
-
| | **Create new run ON** | **Create new run OFF** |
|
|
227
|
-
| --------------------------------- | ---------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- |
|
|
228
|
-
| **Run ID** | A new run is created with a new ID. The original run remains unchanged. | The original run is reused — same run ID. |
|
|
229
|
-
| **Selected tests** | Only selected tests are included in the new run and reset to Pending status (unless **Keep values** is enabled). | All tests are included with their current statuses kept. Only selected tests are reset to **Pending** status. |
|
|
230
|
-
| **Unselected tests / Select all** | All tests are reset to Pending status in the new run (unless **Keep values** is enabled). | All tests retain their existing results. |
|
|
231
|
-
|
|
232
|
-
:::note
|
|
233
|
-
|
|
234
|
-
When re-running tests, selecting **Select all** or leaving no tests selected produces the same result: all tests in the run are included automatically.
|
|
235
|
-
|
|
236
|
-
:::
|
|
237
|
-
|
|
238
|
-
#### Scheduled Runs
|
|
239
|
-
|
|
240
|
-
Create a run test in advance without launching it immediately. After creating and configuring the run:
|
|
241
|
-
|
|
242
|
-
1. Click **Save** to store the Run in a **Scheduled** state.
|
|
243
|
-
2. At execution time, open the run and click **Start**.
|
|
244
|
-
|
|
245
|
-
Scheduled Runs are useful for:
|
|
246
|
-
|
|
247
|
-
* Release regression testing
|
|
248
|
-
* Planned testing activities
|
|
249
|
-
* Recurring QA cycles
|
|
250
|
-
* Coordinating testing across multiple team members
|
|
251
|
-
|
|
252
|
-
#### Keep Values
|
|
253
|
-
|
|
254
|
-
The **'Keep values'** option is only available when **'Create new run'** is enabled. It controls whether test results are carried over into the new run.
|
|
255
|
-
|
|
256
|
-
| | **Keep values ON** | **Keep values OFF** |
|
|
257
|
-
| -------------------- | --------------------------------------------------------------- | ------------------------------------ |
|
|
258
|
-
| **Selected tests** | Results are preserved in the new run (Passed, Failed, Skipped). | Results are reset to Pending status. |
|
|
259
|
-
| **Unselected tests** | Results are preserved in the new run. | Results are reset to Pending status. |
|
|
260
|
-
|
|
261
|
-
#### Select All Tests
|
|
262
|
-
|
|
263
|
-
There are two ways to select all tests for relaunch — the top-level **Checkbox** and the **Select All** option from the dropdown.
|
|
264
|
-
|
|
265
|
-
| **Scenario** | **Checkbox** (top-level) | **Select All** (dropdown) |
|
|
266
|
-
| ------------------------------------- | ------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------- |
|
|
267
|
-
| **No filter applied** | Selects all tests in the run — passed, failed, and skipped. | Selects all tests in the run — passed, failed, and skipped. |
|
|
268
|
-
| **Filter applied (e.g. failed only)** | Selects only tests matching the active filter. Tests outside the filter are not included. | Selects only tests matching the active filter. Tests outside the filter are not included. |
|
|
269
|
-
| **Switch to another filter** | The previous checkbox selection is cleared only when tests are selected in the new filter. | Selection updates automatically to the new filter. Previous selection is cleared. |
|
|
270
|
-
| **Add individual tests manually** | Individual tests from other statuses can be added manually. All selected tests are included in the relaunch. | Individual tests cannot be added or removed manually. |
|
|
271
|
-
| **Clear selection** | Uncheck the checkbox. | Use **'None'** from the dropdown. |
|
|
272
|
-
|
|
273
|
-

|
|
274
|
-
|
|
275
|
-
:::note
|
|
276
|
-
|
|
277
|
-
When using the Checkbox, only the tests that are loaded on the page are selected — scroll through the full list first to ensure all tests are included.
|
|
278
|
-
|
|
279
|
-
:::
|
|
280
|
-
|
|
281
|
-
**Key benefits of Advanced Relaunch:**
|
|
282
|
-
|
|
283
|
-
- Customize relaunches to better fit your team's workflow
|
|
284
|
-
- Simplify recovery after CI failures or flaky test results
|
|
285
|
-
- Avoid redundant test executions and manual reconfiguration
|
|
286
|
-
|
|
287
|
-
## How to Launch a Run Copy
|
|
288
|
-
|
|
289
|
-
If you need need to re-check tests in your Run, but for some reason, you want to save this Test Run separately, use the **'Launch a copy'** option.
|
|
290
|
-
|
|
291
|
-
**To Lauch a Run Copy you need:**
|
|
292
|
-
|
|
293
|
-
1. On the **'Runs'** page, select the run that you need to copy and re-check.
|
|
294
|
-
2. Click the **'Extra menu'** button.
|
|
295
|
-
3. Select **'Launch a Copy'** option from the dropdown menu.
|
|
296
|
-
|
|
297
|
-

|
|
298
|
-
|
|
299
|
-
The **'Launch a Copy'** option is also available on the main Dashboard.
|
|
300
|
-
|
|
301
|
-

|
|
302
|
-
|
|
303
|
-
Your Test Run will open in Manual Run window, allowing you to re-check tests and change their statuses:
|
|
304
|
-
|
|
305
|
-

|
|
306
|
-
|
|
307
|
-
When finished, click on **'Finish'** button. A new Test Run with new test statuses and a new Run Report will display:
|
|
308
|
-
|
|
309
|
-

|
|
310
|
-
|
|
311
|
-
On the main Runs Dashboard, you will see both Tests Runs: the original run with old test results and the copy with new test results.
|
|
312
|
-
|
|
313
|
-

|
|
314
|
-
|
|
315
|
-
## Multiselection For Runs
|
|
316
|
-
|
|
317
|
-
Multiselection allows the bulk application options to many runs with one click. Click the **'Multi-select'** icon and select a few runs to see multi-select menu panel.
|
|
318
|
-
|
|
319
|
-
**Multi-select Options Overview:**
|
|
320
|
-
|
|
321
|
-
1. **'Select all'** - select all runs on the Runs page.
|
|
322
|
-
2. **'Archive'** - archive selected runs.
|
|
323
|
-
3. **'Labels'** - apply/remove [labels](https://docs.testomat.io/usage/labels-and-custom-fields/#how-to-setup-a-label) from runs.
|
|
324
|
-
4. **'Compare'** - open Compare runs view.
|
|
325
|
-
5. **'Extra menu'** - access additional options.
|
|
326
|
-
6. **'Purge icon'** - purge selected runs [see more details here](https://docs.testomat.io/project/runs/managing-runs/#purge-runs).
|
|
327
|
-
|
|
328
|
-

|
|
329
|
-
|
|
330
|
-
Click the **'Extra menu'** button on multi-select menu panel to see additonal options:
|
|
331
|
-
|
|
332
|
-
1. **'Link'** - link selected runs to [JIRA](https://docs.testomat.io/integration/jira/#connecting-to-jira-project), [GitHub](https://docs.testomat.io/integration/issues-management-systems/#github-issues-configuration), [Azure](https://docs.testomat.io/integration/issues-management-systems/#azure-devops-configuration),[Linear](https://docs.testomat.io/integration/issues-management-systems/#linear-configuration) or other [issues management system](https://docs.testomat.io/integrations/issues-management/).
|
|
333
|
-
2. **'Download'** - export selected run reports to spreadsheet.
|
|
334
|
-
3. **'Merge'** - merge selected runs (read more in the [Merge runs](https://docs.testomat.io/project/runs/merge-strategies/#merge-runs) section).
|
|
335
|
-
4. **'Move'** - move Runs to/from RunGroups.
|
|
336
|
-
|
|
337
|
-

|
|
338
|
-
|
|
339
|
-
## Filtering Test Runs
|
|
340
|
-
|
|
341
|
-
Testomat.io provides filtering capabilities for Runs and RunGroups. On the Runs page, you can find **Basic filters** and **Additional filters**.
|
|
342
|
-
|
|
343
|
-

|
|
344
|
-
|
|
345
|
-
## Filter Runs by TQL
|
|
346
|
-
|
|
347
|
-
Testomat.io Query Language (TQL) for Runs enables more flexible searches, allowing users to filter Test Runs by criteria like `finished_at`, `passed_count`, `has_test_label` and many others. This helps quickly find relevant runs and improves test analysis.
|
|
348
|
-
|
|
349
|
-

|
|
350
|
-
|
|
351
|
-
Visit a separate section of the documentation - [Runs Variables](https://docs.testomat.io/usage/query-language/#runs-variables), if you want to find out more about the Runs Variable.
|
|
352
|
-
|
|
353
|
-

|
|
354
|
-
|
|
355
|
-
## Share Runs Filter
|
|
356
|
-
|
|
357
|
-
This feature allows you to pass **Runs** and **Runs Archive** filter parameters directly through the URL. This improves collaboration by enabling teams to quickly access relevant test results without manually adjusting filters.
|
|
358
|
-
|
|
359
|
-
Specify the necessary filtering parameters on the **Runs** or **Runs Archive** page, then copy the URL and share it with interested members.
|
|
360
|
-
|
|
361
|
-

|
|
362
|
-
|
|
363
|
-
## Purge Runs
|
|
364
|
-
|
|
365
|
-
Starting from September 2025, Testomat.io introduces a new approach to managing old and unused Runs.
|
|
366
|
-
|
|
367
|
-
The **Delete** option for Runs is now replaced with a **Purge** option.
|
|
368
|
-
|
|
369
|
-

|
|
370
|
-
|
|
371
|
-
This change provides a safer and more flexible way to manage Run data by introducing a two-step deletion strategy instead of immediate and irreversible removal.
|
|
372
|
-
|
|
373
|
-
### Compress and Move to Archive (Purge)
|
|
374
|
-
|
|
375
|
-
When a Run is purged (either manually or automatically), it is first compressed and moved to the [Archive](https://docs.testomat.io/project/runs/archive-runs-and-groups/).
|
|
376
|
-
|
|
377
|
-

|
|
378
|
-
|
|
379
|
-
Runs that are moved to the Archive receive a specific badge:
|
|
380
|
-
|
|
381
|
-
- **Purged** – when a run is purged (manually or automatically)
|
|
382
|
-
- **Archived** – when a run is archived manually
|
|
383
|
-
|
|
384
|
-
During this step:
|
|
385
|
-
|
|
386
|
-
- Stack traces are removed to reduce storage size.
|
|
387
|
-
- Essential data is preserved, including:
|
|
388
|
-
- Test results
|
|
389
|
-
- Artifacts (attachments, logs, screenshots)
|
|
390
|
-
- Custom statuses
|
|
391
|
-
- Archived Runs remain available for as long as needed and can be restored at any time.
|
|
392
|
-
|
|
393
|
-
### Purge Logic for Ongoing Runs
|
|
394
|
-
|
|
395
|
-
The purge process also applies to **ongoing (in-progress) Runs**, ensuring consistency across all run states.
|
|
396
|
-
|
|
397
|
-
When a Run (either inside a Run Group or standalone) is **purged while still ongoing**, the following logic applies:
|
|
398
|
-
|
|
399
|
-
- The Run is automatically **terminated**
|
|
400
|
-
- Test results that were already recorded (e.g., **Passed**, **Failed**, **Skipped**) remain preserved
|
|
401
|
-
- All **Pending** test cases are automatically marked as **Skipped**
|
|
402
|
-
- The Run receives a **terminated** status flag, which remains visible even after restoration
|
|
403
|
-
- The Run is then **compressed and moved to the Archive**, following the same purge-to-archive flow as completed runs
|
|
404
|
-
|
|
405
|
-
:::note
|
|
406
|
-
|
|
407
|
-
Restored Runs that were terminated during purge **cannot be resumed**. They retain the **terminated** state but can still be viewed and analyzed in the Archive or after restoration.
|
|
408
|
-
|
|
409
|
-
:::
|
|
410
|
-
|
|
411
|
-
This ensures that no partial or inconsistent test data remains in the system and that all purged Runs — even those in progress — are preserved in a clean, final state.
|
|
412
|
-
|
|
413
|
-
### Automatic Purge Flow
|
|
414
|
-
|
|
415
|
-
The same **two-step purge-to-archive logic** applies to automatically purged **Runs** via **Project Settings → [Purge Old Runs](https://docs.testomat.io/management/project/settings/#purge-old-runs)**.
|
|
416
|
-
|
|
417
|
-
When an automatic purge is triggered:
|
|
418
|
-
|
|
419
|
-
- The **Run** is compressed and moved to the **Archive** page
|
|
420
|
-
- It receives the **purged** badge
|
|
421
|
-
- The Run remains restorable from the Archive, but permanent deletion must still be performed **manually**
|
|
422
|
-
|
|
423
|
-
### Permanent Deletion from Archive
|
|
424
|
-
|
|
425
|
-
:::note
|
|
426
|
-
|
|
427
|
-
If you remove a Run from the Archive, it is permanently deleted. This step is irreversible — once **deleted**, the Run cannot be restored.
|
|
428
|
-
|
|
429
|
-
:::
|
|
430
|
-
|
|
431
|
-

|
|
432
|
-
|
|
433
|
-
### Runs Activity Tracked in Pulse
|
|
434
|
-
|
|
435
|
-
All actions related to purging or permanently deleting Runs are recorded in [Pulse](https://docs.testomat.io/project/pulse/) under the **Deleted Run** activity type. This ensures full transparency and traceability of Run lifecycle events.
|
|
436
|
-
|
|
437
|
-
Pulse displays different activity messages depending on how the Run was removed:
|
|
438
|
-
|
|
439
|
-
1. **Automatic purge**
|
|
440
|
-
When a Run is purged automatically by the system due to expiration, Pulse records an entry similar to:
|
|
441
|
-
_“Manual tests at 20 Nov 2025 11:56 have been purged by system on expiration (moved to Archive).”_
|
|
442
|
-
2. **User-initiated purge**
|
|
443
|
-
When a Run is purged manually, Pulse records the event:
|
|
444
|
-
_“Manual run has been purged by user (moved to Archive).”_
|
|
445
|
-
|
|
446
|
-
3. **Permanent deletion from Archive**
|
|
447
|
-
When a Run is permanently removed from the Archive, Pulse displays the event indicating that the manual run has been permanently deleted.
|
|
448
|
-
|
|
449
|
-

|
|
450
|
-
|
|
451
|
-
These Pulse records provide a clear audit trail for all purge-related actions, mirroring the two-step deletion flow and ensuring accountability across both manual and automated processes.
|
|
452
|
-
|
|
453
|
-
### Summary
|
|
454
|
-
|
|
455
|
-
By introducing the Purge option, Testomat.io ensures that Run data is handled with more safety and flexibility. You can keep your workspace clean and optimize storage, while still maintaining the ability to restore important Runs from the Archive when needed.
|
|
@@ -1,137 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
title: Merge Strategies
|
|
3
|
-
description: Learn how 'Merge Strategies' in Testomat.io can help you to consolidate test results from multiple runs within a RunGroup, providing a unified view of your testing outcomes. This guide covers how to select and change these strategies (Realistic, Optimistic, Pessimistic) when creating new RunGroups or merging existing runs, enabling effective analysis of tests executed across various scenarios.
|
|
4
|
-
url: https://docs.testomat.io/project/runs/merge-strategies
|
|
5
|
-
head:
|
|
6
|
-
- tag: meta
|
|
7
|
-
attrs:
|
|
8
|
-
name: keywords
|
|
9
|
-
content: managing test runs, run reports, merge strategy, merge runs, rungroups, test management, Testomat.io, run comparison, realistic merge strategy, optimistic merge strategy, pessimistic merge strategy.
|
|
10
|
-
---
|
|
11
|
-
|
|
12
|
-
## Merge Strategies
|
|
13
|
-
|
|
14
|
-
Testomat.io provides different **'Merge Strategies'** to aggregate the results of the individual test runs within a **RunGroup**. This is crucial when you have multiple executions of the same tests (e.g., retries, parallel execution, runs on different environments) and need a single, representative status for tests within that group.
|
|
15
|
-
|
|
16
|
-
You need to select a **Merge Strategy** when you create a new RunGroup:
|
|
17
|
-
|
|
18
|
-
1. Go to **'Runs'** page.
|
|
19
|
-
2. Click the **'New Group'** button.
|
|
20
|
-
|
|
21
|
-

|
|
22
|
-
|
|
23
|
-
3. Select **Merge strategy**.
|
|
24
|
-
|
|
25
|
-

|
|
26
|
-
|
|
27
|
-
:::note
|
|
28
|
-
|
|
29
|
-
You can change **Merge Strategy** at any time after the RunGroup is created.
|
|
30
|
-
|
|
31
|
-
:::
|
|
32
|
-
|
|
33
|
-
**Change an Existing RunGroup's Merge Strategy:**
|
|
34
|
-
|
|
35
|
-
1. Select the RunGroup.
|
|
36
|
-
2. Click the **'Extra menu'** button.
|
|
37
|
-
3. Click the **'Edit'** option.
|
|
38
|
-
|
|
39
|
-

|
|
40
|
-
|
|
41
|
-
You also need to select the **Merge Strategy** when merging multiple runs into one:
|
|
42
|
-
|
|
43
|
-
1. Go to **'Runs'** page.
|
|
44
|
-
2. Click the **'Multi-select'** icon.
|
|
45
|
-
3. Select a few runs that you want to merge.
|
|
46
|
-
4. Click the **'Extra menu'** button in the bottom menu.
|
|
47
|
-
5. Select **'Merge'** option.
|
|
48
|
-
|
|
49
|
-

|
|
50
|
-
|
|
51
|
-
6. Select a **Merge Strategy**.
|
|
52
|
-
|
|
53
|
-

|
|
54
|
-
|
|
55
|
-
**Testomat.io offers three different Merge strategies:**
|
|
56
|
-
|
|
57
|
-
For instance, let's say we have 2 runs inside a Rungroup with the same tests A and B:
|
|
58
|
-
|
|
59
|
-
- Run 1: A - failed, B - failed
|
|
60
|
-
- Run 2: A - passed, B - passed
|
|
61
|
-
|
|
62
|
-
How should the RunGroup's counters display results? Both answers: "Passed: 2, Failed: 0" or "Failed 2, Passed 0" are absolutely valid depending on how you calculate those numbers. That's why Testomat.io provides customizable Merge strategies:
|
|
63
|
-
|
|
64
|
-
- **Realistic:** this strategy is based on test results from the **last executed run** (excluding skipped tests). If your RunGroup includes test runs with different test cases, it will summarize and display the results from the last executed runs with unique test cases.
|
|
65
|
-
|
|
66
|
-
In our example, results will be next: **Passed: 2, Failed: 0**, as Run 2 was executed last.
|
|
67
|
-
|
|
68
|
-
- **Optimistic:** this strategy will mark a test as **passed if it passed in at least one run** within the group, even if it failed in others. This approach provides a more lenient view, focusing on the positive outcomes.
|
|
69
|
-
|
|
70
|
-
In other words, it prefers passed tests over failed ones: **Passed: 2, Failed: 0** in our example.
|
|
71
|
-
|
|
72
|
-
- **Pessimistic:** unlike the optimistic strategy, this strategy will mark a test as **failed if it failed in at least one run** within the group. It prioritizes identifying potential issues.
|
|
73
|
-
|
|
74
|
-
Simply to say, it prefers failed tests over passed ones: **Passed: 0, Failed: 2**.
|
|
75
|
-
|
|
76
|
-
### Merge Strategy Examples
|
|
77
|
-
|
|
78
|
-
**Example #1: Same Test Cases Across Different Environments**
|
|
79
|
-
|
|
80
|
-
Let's imagine that you have 2 Test Runs with the same test cases for different environments. In one Test Run, the first 3 test cases failed while the others passed, and in another Test Run last 2 test cases failed and the others - passed.
|
|
81
|
-
In this case, the results with the different strategies will be as follows:
|
|
82
|
-
|
|
83
|
-
1. **Realistic** - the result will match the result of the last executed test run, in this case 2 test cases - failed, and the others - passed.
|
|
84
|
-
|
|
85
|
-

|
|
86
|
-
|
|
87
|
-
2. **Optimistic** - all test cases will show as passed, as each test case passed in at least one test run.
|
|
88
|
-
|
|
89
|
-

|
|
90
|
-
|
|
91
|
-
3. **Pessimistic** - by this strategy 5 test cases will show as failed, as a total of five unique test cases failed across these 2 test runs.
|
|
92
|
-
|
|
93
|
-

|
|
94
|
-
|
|
95
|
-
**Example #2: Adding Runs with Unique Test Cases**
|
|
96
|
-
|
|
97
|
-
In this example, you added 2 more Test Runs with unique test cases (for instance, to test another feature in the module under test) to your previous 2 Test Runs. In one added Test Runs, the last 2 test cases failed, and in the other, the first 3 test cases failed. In these 2 runs, each test case failed and passed at least once.
|
|
98
|
-
As a result, the outcome with the different strategies will be as follows:
|
|
99
|
-
|
|
100
|
-
1. **Realistic** - 5 test cases will show as failed, as it summarizes the results of the last two runs with unique test cases.
|
|
101
|
-
|
|
102
|
-

|
|
103
|
-
|
|
104
|
-
2. **Optimistic** - all test cases will show as passed, as each test case passed in at least one test run.
|
|
105
|
-
|
|
106
|
-

|
|
107
|
-
|
|
108
|
-
3. **Pessimistic** - by this strategy 10 test cases will show as failed, because a total of 10 unique test cases failed across all 4 test runs.
|
|
109
|
-
|
|
110
|
-

|
|
111
|
-
|
|
112
|
-
As you can see, **Merge Strategies** for RunGroups in Testomat.io help you aggregate and analyze test results from multiple runs. They define how the system determines the final status of a test case within a RunGroup, providing a unified view of your overall testing outcomes.
|
|
113
|
-
|
|
114
|
-
## Merge Runs
|
|
115
|
-
|
|
116
|
-
With Testomat.io you can merge your Test Runs. To do this you need:
|
|
117
|
-
|
|
118
|
-
1. Enable multi-selection.
|
|
119
|
-
2. Choose the Test Runs you want to merge.
|
|
120
|
-
3. Click the **'Extra menu'** button in the menu panel at the bottom of the page.
|
|
121
|
-
4. Select **'Merge'** option.
|
|
122
|
-
|
|
123
|
-

|
|
124
|
-
|
|
125
|
-
On the displayed modal you will need to:
|
|
126
|
-
|
|
127
|
-
1. Select testing Environment.
|
|
128
|
-
2. Enter a name for merged run.
|
|
129
|
-
3. Choose a **Merging Strategy**.
|
|
130
|
-
4. Decide whether to keep original runs or not.
|
|
131
|
-
5. Click the **'Merge'** button.
|
|
132
|
-
|
|
133
|
-

|
|
134
|
-
|
|
135
|
-
Your merged run will appear in the list of runs on the **Runs** page and will combine unique test cases from the merged runs.
|
|
136
|
-
|
|
137
|
-

|