testeiya 0.2.1 → 0.3.10
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +284 -171
- package/dist/prompt/context.js +79 -0
- package/dist/prompt/context.js.map +1 -0
- package/dist/prompt/index.js +60 -0
- package/dist/prompt/index.js.map +1 -0
- package/dist/prompt/print.js +30 -0
- package/dist/prompt/print.js.map +1 -0
- package/dist/prompt/project-info.js +2 -0
- package/dist/prompt/project-info.js.map +1 -0
- package/{src/prompt/system-prompt.ts → dist/prompt/system-prompt.js} +38 -25
- package/dist/prompt/system-prompt.js.map +1 -0
- package/dist/prompt/testomatio.js +264 -0
- package/dist/prompt/testomatio.js.map +1 -0
- package/dist/prompt/tools.js +57 -0
- package/dist/prompt/tools.js.map +1 -0
- package/dist/prompt/vocab.js +8 -0
- package/dist/prompt/vocab.js.map +1 -0
- package/dist/src/args.js +172 -0
- package/dist/src/args.js.map +1 -0
- package/dist/src/cli.js +97 -0
- package/dist/src/cli.js.map +1 -0
- package/dist/src/doctor.js +186 -0
- package/dist/src/doctor.js.map +1 -0
- package/dist/src/env.js +93 -0
- package/dist/src/env.js.map +1 -0
- package/dist/src/mcp-extension.js +9 -0
- package/dist/src/mcp-extension.js.map +1 -0
- package/dist/src/mcp.js +139 -0
- package/dist/src/mcp.js.map +1 -0
- package/dist/src/model.js +71 -0
- package/dist/src/model.js.map +1 -0
- package/dist/src/models.js +38 -0
- package/dist/src/models.js.map +1 -0
- package/dist/src/output.js +185 -0
- package/dist/src/output.js.map +1 -0
- package/dist/src/result.js +41 -0
- package/dist/src/result.js.map +1 -0
- package/dist/src/run.js +291 -0
- package/dist/src/run.js.map +1 -0
- package/dist/src/session.js +92 -0
- package/dist/src/session.js.map +1 -0
- package/dist/src/sessions.js +61 -0
- package/dist/src/sessions.js.map +1 -0
- package/dist/vendor/mcp.js +14801 -0
- package/mcp-tools.json +2967 -0
- package/package.json +34 -43
- package/prompt/context.ts +101 -0
- package/prompt/index.ts +102 -0
- package/prompt/print.ts +32 -0
- package/prompt/project-info.ts +30 -0
- package/prompt/system-prompt.ts +187 -0
- package/{src/prompt → prompt}/testomatio.ts +120 -9
- package/prompt/tools.ts +57 -0
- package/prompt/vocab.ts +7 -0
- package/skills/codeceptjs/ci-fix-tests/SKILL.md +7 -8
- package/skills/codeceptjs/codeceptjs-auth/SKILL.md +47 -64
- package/skills/codeceptjs/codeceptjs-exploration/SKILL.md +56 -47
- package/skills/codeceptjs/codeceptjs-fundamentals/SKILL.md +144 -106
- package/skills/codeceptjs/codeceptjs-run-analysis/SKILL.md +50 -57
- package/skills/codeceptjs/debugging-codeceptjs-tests/SKILL.md +88 -196
- package/skills/codeceptjs/migrate-codeceptjs-4/SKILL.md +4 -5
- package/skills/codeceptjs/migrate-cypress-to-codeceptjs/SKILL.md +25 -36
- package/skills/codeceptjs/migrate-protractor-to-codeceptjs/SKILL.md +23 -36
- package/skills/codeceptjs/migrate-selenium-java-to-codeceptjs/SKILL.md +23 -36
- package/skills/codeceptjs/migrate-testcafe-to-codeceptjs/SKILL.md +23 -35
- package/skills/codeceptjs/refactoring-codeceptjs-tests/SKILL.md +50 -53
- package/skills/codeceptjs/writing-codeceptjs-tests/SKILL.md +87 -191
- package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/SKILL.md +1 -1
- package/{skills.lock.json → skills/skills.lock.json} +43 -36
- package/skills/skills.yaml +34 -0
- package/skills/testomatio/explorbot/explorbot-fundamentals/SKILL.md +49 -61
- package/skills/testomatio/explorbot/explorbot-fundamentals/references/no-install.md +36 -0
- package/skills/testomatio/explorbot/explorbot-plan/SKILL.md +20 -7
- package/skills/testomatio/explorbot/explorbot-setup/SKILL.md +41 -289
- package/skills/testomatio/qa-process/qa-explain-behavior/SKILL.md +111 -0
- package/skills/testomatio/qa-process/qa-explain-behavior/references/answer-shapes.md +116 -0
- package/skills/testomatio/qa-process/qa-explain-behavior/references/examples.md +157 -0
- package/skills/testomatio/qa-process/qa-lead-strategy-advisor/SKILL.md +91 -0
- package/skills/testomatio/qa-process/qa-lead-strategy-advisor/references/output-format.md +71 -0
- package/skills/testomatio/qa-process/testing-workflow/SKILL.md +227 -0
- package/skills/testomatio/test-automation/qa-data-seeder/SKILL.md +83 -0
- package/skills/testomatio/test-automation/run-tests-with-testomatio-reporter/SKILL.md +5 -3
- package/skills/testomatio/test-automation/setup-change-aware-pr-testing/SKILL.md +190 -0
- package/skills/testomatio/test-automation/setup-ci-automation/SKILL.md +76 -0
- package/skills/testomatio/test-management/qa-split-testing-levels-pyramid/SKILL.md +55 -0
- package/skills/testomatio/test-management/qa-sprint-report-by-testomatio/SKILL.md +220 -0
- package/skills/testomatio/test-management/qa-sprint-report-by-testomatio/references/qa-sprint-report.md +126 -0
- package/skills/testomatio/test-management/qa-test-code-coverage/SKILL.md +1 -1
- package/skills/testomatio/test-management/qa-thinking/SKILL.md +25 -25
- package/skills/testomatio/test-management/testomatio-mcp/SKILL.md +14 -1
- package/skills/testomatio/test-management/testomatio-mcp/references/MCP_SETUP.md +8 -0
- package/assets/testeiya-logo-dark.svg +0 -16
- package/assets/testeiya-logo.svg +0 -15
- package/models.catalog.json +0 -6963
- package/skills/playwright-best-practices-skill/LICENSE.md +0 -7
- package/skills/playwright-best-practices-skill/README.md +0 -147
- package/skills/testeiya/README.md +0 -29
- package/skills/testeiya/answer-formatting/SKILL.md +0 -30
- package/skills/testeiya/check-cucumber/SKILL.md +0 -200
- package/skills/testeiya/check-tests/SKILL.md +0 -260
- package/skills/testeiya/manual-run-assistant/SKILL.md +0 -61
- package/skills/testeiya/testomatio-docs/INDEX.md +0 -175
- package/skills/testeiya/testomatio-docs/SKILL.md +0 -102
- package/skills/testeiya/testomatio-docs/docs/advanced/ai-powered-features/ai-agents.md +0 -144
- package/skills/testeiya/testomatio-docs/docs/advanced/ai-powered-features/ai-powered-features.md +0 -531
- package/skills/testeiya/testomatio-docs/docs/advanced/ai-powered-features/ai-requirements.md +0 -437
- package/skills/testeiya/testomatio-docs/docs/advanced/api-access/index.md +0 -191
- package/skills/testeiya/testomatio-docs/docs/advanced/branches-folder/index.md +0 -277
- package/skills/testeiya/testomatio-docs/docs/advanced/bulk-edit-folder/bulk-edit-demos.mdx +0 -39
- package/skills/testeiya/testomatio-docs/docs/advanced/bulk-edit-folder/bulk-edit-on-suite-and-test-level.md +0 -122
- package/skills/testeiya/testomatio-docs/docs/advanced/bulk-edit-folder/what-is-bulk-edit.md +0 -170
- package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/branches.md +0 -53
- package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/index.mdx +0 -98
- package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/jira-issue.md +0 -151
- package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/runs-and-reports.md +0 -377
- package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/work-in-jira.md +0 -267
- package/skills/testeiya/testomatio-docs/docs/advanced/living-doc/index.md +0 -138
- package/skills/testeiya/testomatio-docs/docs/advanced/milestones/index.md +0 -220
- package/skills/testeiya/testomatio-docs/docs/advanced/shortcuts/index.md +0 -48
- package/skills/testeiya/testomatio-docs/docs/advanced/tags-labels/labels-and-custom-fields.md +0 -201
- package/skills/testeiya/testomatio-docs/docs/advanced/tags-labels/tags-or-labels.md +0 -94
- package/skills/testeiya/testomatio-docs/docs/advanced/tags-labels/tags.md +0 -243
- package/skills/testeiya/testomatio-docs/docs/advanced/test-artifacts/index.md +0 -135
- package/skills/testeiya/testomatio-docs/docs/advanced/tql/index.md +0 -394
- package/skills/testeiya/testomatio-docs/docs/getting-started/index.md +0 -290
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/azure.md +0 -96
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/bamboo.md +0 -94
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/bitbucket.md +0 -106
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/circle.md +0 -115
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/github.md +0 -120
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/gitlab.md +0 -88
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/index.mdx +0 -151
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/jenkins.md +0 -121
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/teamcity.md +0 -96
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/azure.md +0 -31
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/clickup.md +0 -54
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/confluence.md +0 -34
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/github.md +0 -31
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/gitlab.md +0 -93
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/index.md +0 -131
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/jira.md +0 -164
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/linear.md +0 -29
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/shortcut.md +0 -67
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/youtrack.md +0 -94
- package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/azure-devops.md +0 -53
- package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/email.md +0 -71
- package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/jira.md +0 -52
- package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/ms-teams.md +0 -60
- package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/rules.md +0 -267
- package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/slack.md +0 -82
- package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/telegram.md +0 -73
- package/skills/testeiya/testomatio-docs/docs/integrations/scim-provisioning/index.mdx +0 -207
- package/skills/testeiya/testomatio-docs/docs/integrations/scim-provisioning/okta.md +0 -204
- package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/azure.md +0 -95
- package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/google.md +0 -75
- package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/index.mdx +0 -33
- package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/okta.md +0 -108
- package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/saml.md +0 -66
- package/skills/testeiya/testomatio-docs/docs/legal/compliance/gdpr.md +0 -95
- package/skills/testeiya/testomatio-docs/docs/legal/compliance/index.mdx +0 -94
- package/skills/testeiya/testomatio-docs/docs/legal/compliance/privacy.md +0 -26
- package/skills/testeiya/testomatio-docs/docs/legal/compliance/terms.md +0 -32
- package/skills/testeiya/testomatio-docs/docs/legal/security/index.mdx +0 -242
- package/skills/testeiya/testomatio-docs/docs/legal/security/jira.md +0 -140
- package/skills/testeiya/testomatio-docs/docs/legal/security/subprocessors.md +0 -76
- package/skills/testeiya/testomatio-docs/docs/management/company/administration.md +0 -363
- package/skills/testeiya/testomatio-docs/docs/management/company/audit-log.md +0 -41
- package/skills/testeiya/testomatio-docs/docs/management/company/plan-features-comparison.md +0 -240
- package/skills/testeiya/testomatio-docs/docs/management/company/subscriptions.md +0 -326
- package/skills/testeiya/testomatio-docs/docs/management/company/trials.md +0 -75
- package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/accountant.md +0 -49
- package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/index.mdx +0 -285
- package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/read-only-user.md +0 -67
- package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/teams.md +0 -138
- package/skills/testeiya/testomatio-docs/docs/management/project/settings.md +0 -87
- package/skills/testeiya/testomatio-docs/docs/management/project/templates.md +0 -546
- package/skills/testeiya/testomatio-docs/docs/project/analytics/index.md +0 -326
- package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/classical-tests-markdown-format.md +0 -386
- package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/download-manual-tests-as-files.md +0 -378
- package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/export-to-obsidian.md +0 -112
- package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/export-to-spreadsheet.md +0 -127
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/auto-import.md +0 -63
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-Allure-TestOps.md +0 -49
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-QTest.md +0 -41
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-Qase.md +0 -39
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-Zephyr.md +0 -37
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-csv-xlsx.md +0 -117
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-cucumber.md +0 -93
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-qmetry.md +0 -44
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-source-code.md +0 -117
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-testcaselabs.md +0 -44
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-testmo.md +0 -47
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-testrail.md +0 -82
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-xray.md +0 -55
- package/skills/testeiya/testomatio-docs/docs/project/plans/index.md +0 -334
- package/skills/testeiya/testomatio-docs/docs/project/pulse/index.md +0 -104
- package/skills/testeiya/testomatio-docs/docs/project/runs/archive-runs-and-groups.md +0 -176
- package/skills/testeiya/testomatio-docs/docs/project/runs/custom-statuses.md +0 -195
- package/skills/testeiya/testomatio-docs/docs/project/runs/environments.md +0 -203
- package/skills/testeiya/testomatio-docs/docs/project/runs/managing-runs.md +0 -455
- package/skills/testeiya/testomatio-docs/docs/project/runs/merge-strategies.md +0 -137
- package/skills/testeiya/testomatio-docs/docs/project/runs/reports.md +0 -329
- package/skills/testeiya/testomatio-docs/docs/project/runs/rungroups.md +0 -342
- package/skills/testeiya/testomatio-docs/docs/project/runs/running-automated-tests.md +0 -256
- package/skills/testeiya/testomatio-docs/docs/project/runs/running-manual-and-automated-tests.md +0 -331
- package/skills/testeiya/testomatio-docs/docs/project/runs/running-tests-manually.md +0 -633
- package/skills/testeiya/testomatio-docs/docs/project/runs/temporary-tests-notes.md +0 -223
- package/skills/testeiya/testomatio-docs/docs/project/steps-snippets/snippets.md +0 -68
- package/skills/testeiya/testomatio-docs/docs/project/steps-snippets/steps.md +0 -181
- package/skills/testeiya/testomatio-docs/docs/project/tests/bdd-test-case-editor.md +0 -98
- package/skills/testeiya/testomatio-docs/docs/project/tests/classical-test-case-editor.md +0 -354
- package/skills/testeiya/testomatio-docs/docs/project/tests/classical-vs-bdd.md +0 -169
- package/skills/testeiya/testomatio-docs/docs/project/tests/copy-and-move-your-tests.md +0 -156
- package/skills/testeiya/testomatio-docs/docs/project/tests/other-features-for-test-case-design.md +0 -412
- package/skills/testeiya/testomatio-docs/docs/project/tests/test-case-creation-and-editing.md +0 -527
- package/skills/testeiya/testomatio-docs/docs/support/index.md +0 -75
- package/skills/testeiya/testomatio-docs/docs/test-reporting/artifacts.md +0 -388
- package/skills/testeiya/testomatio-docs/docs/test-reporting/cli.md +0 -215
- package/skills/testeiya/testomatio-docs/docs/test-reporting/configuration.md +0 -247
- package/skills/testeiya/testomatio-docs/docs/test-reporting/frameworks.md +0 -490
- package/skills/testeiya/testomatio-docs/docs/test-reporting/functions.md +0 -144
- package/skills/testeiya/testomatio-docs/docs/test-reporting/index.mdx +0 -48
- package/skills/testeiya/testomatio-docs/docs/test-reporting/junit.md +0 -248
- package/skills/testeiya/testomatio-docs/docs/test-reporting/logger.md +0 -138
- package/skills/testeiya/testomatio-docs/docs/test-reporting/php.md +0 -77
- package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/bitbucket.md +0 -72
- package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/csv.md +0 -23
- package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/debug.md +0 -18
- package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/github.md +0 -63
- package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/gitlab.md +0 -29
- package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/html.md +0 -80
- package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/index.mdx +0 -57
- package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/testomatio.md +0 -331
- package/skills/testeiya/testomatio-docs/docs/test-reporting/python.md +0 -229
- package/skills/testeiya/testomatio-docs/docs/test-reporting/workflows.md +0 -325
- package/skills/testeiya/testomatio-docs/docs/tutorials/Interacting-via-API.md +0 -159
- package/skills/testeiya/testomatio-docs/docs/tutorials/Manual-Testing-Classic.md +0 -164
- package/skills/testeiya/testomatio-docs/docs/tutorials/Set-Up-Read-Only-Access.md +0 -85
- package/skills/testeiya/testomatio-docs/docs/tutorials/Using-AI-in-Testing.md +0 -163
- package/skills/testeiya/testomatio-docs/docs/tutorials/Working-with-Jira.md +0 -125
- package/skills/testeiya/testomatio-docs/docs/tutorials/java.md +0 -478
- package/skills/testeiya/testomatio-docs/docs/tutorials/playwright.md +0 -207
- package/skills/testeiya/testomatio-docs/docs/tutorials/webdriver.md +0 -218
- package/skills/testeiya/testomatio-docs/docs.lock.json +0 -6
- package/skills/testeiya/testomatio-reporter/SKILL.md +0 -307
- package/skills/testomatio/test-automation/setup-pr-testing/SKILL.md +0 -199
- package/skills.yaml +0 -21
- package/src/ai-debug.ts +0 -188
- package/src/api/agent-get.ts +0 -22
- package/src/api/agent-start.ts +0 -384
- package/src/api/client-log.ts +0 -29
- package/src/api/context.ts +0 -332
- package/src/api/debug-layout.ts +0 -47
- package/src/api/debug-report.ts +0 -29
- package/src/api/debug-snapshot.ts +0 -30
- package/src/api/debug-stream.ts +0 -52
- package/src/api/files-delete.ts +0 -184
- package/src/api/files-read.ts +0 -41
- package/src/api/files-rename.ts +0 -99
- package/src/api/files-tree.ts +0 -292
- package/src/api/files-write.ts +0 -41
- package/src/api/mcp.ts +0 -603
- package/src/api/memory.ts +0 -69
- package/src/api/open-external.ts +0 -39
- package/src/api/playwright-cli.ts +0 -606
- package/src/api/providers.ts +0 -457
- package/src/api/read-env-token.ts +0 -33
- package/src/api/sessions.ts +0 -79
- package/src/api/settings.ts +0 -120
- package/src/api/skills.ts +0 -58
- package/src/api/testomatio-attachment.ts +0 -89
- package/src/api/testomatio-auth.ts +0 -284
- package/src/api/testomatio-proxy.ts +0 -197
- package/src/api/testomatio-run-stats.ts +0 -81
- package/src/api/testomatio-target.ts +0 -225
- package/src/api/testomatio-transcription.ts +0 -82
- package/src/api/workspace-search.ts +0 -96
- package/src/api/workspace-sync.ts +0 -151
- package/src/api/workspace.ts +0 -165
- package/src/app-server.ts +0 -556
- package/src/bridge.ts +0 -251
- package/src/check-tests.ts +0 -156
- package/src/cli.ts +0 -17
- package/src/commands.ts +0 -244
- package/src/config.ts +0 -183
- package/src/connection.ts +0 -658
- package/src/context-store.ts +0 -265
- package/src/debug-bus.ts +0 -405
- package/src/extensions/tool-gate.ts +0 -220
- package/src/extensions/webui/ask-channel.ts +0 -52
- package/src/extensions/webui/index.ts +0 -184
- package/src/extensions/webui/tools/ask-question.ts +0 -88
- package/src/extensions/webui/tools/query-result.ts +0 -79
- package/src/extensions/webui/tools/render-chart.ts +0 -53
- package/src/extensions/webui/tools/render-item.ts +0 -61
- package/src/extensions/webui/tools/render-list.ts +0 -170
- package/src/extensions/webui/tools/render-result.ts +0 -85
- package/src/extensions/webui/tools/render-tree.ts +0 -71
- package/src/extensions/webui/tools/ui-widget.ts +0 -54
- package/src/extensions/webui/tools/widget-result.ts +0 -13
- package/src/extensions/webui/widget-channel.ts +0 -50
- package/src/file-log.ts +0 -196
- package/src/git-tracked.ts +0 -68
- package/src/gitignore.ts +0 -53
- package/src/json-store.ts +0 -43
- package/src/load-env.ts +0 -70
- package/src/loader-style.ts +0 -96
- package/src/main.ts +0 -140
- package/src/mcp-catalog.ts +0 -96
- package/src/models-catalog.ts +0 -72
- package/src/permissions.ts +0 -120
- package/src/project-dir.ts +0 -189
- package/src/project-info.ts +0 -108
- package/src/prompt/app-ui.ts +0 -59
- package/src/prompt/browser.ts +0 -27
- package/src/prompt/index.ts +0 -62
- package/src/prompt/tools.ts +0 -27
- package/src/server-info.ts +0 -62
- package/src/session-factory.ts +0 -442
- package/src/session-store.ts +0 -115
- package/src/shell-env.ts +0 -45
- package/src/skills.ts +0 -163
- package/src/sync-snapshot.ts +0 -140
- package/src/telemetry.ts +0 -239
- package/src/testomatio-auth.ts +0 -299
- package/src/testomatio.ts +0 -98
- package/src/theme.ts +0 -30
- package/src/user-env.ts +0 -94
- package/src/welcome.ts +0 -194
- package/src/workspace/safe-path.ts +0 -85
- package/src/workspace/test-md.ts +0 -70
- package/src/workspace-model.ts +0 -440
- package/testeiya.config.json +0 -14
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/authentication-flows.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/authentication.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/clock-mocking.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/mobile-testing.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/multi-context.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/multi-user.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/network-advanced.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/third-party.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/architecture/pom-vs-fixtures.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/architecture/test-architecture.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/architecture/when-to-mock.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/browser-apis/browser-apis.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/browser-apis/iframes.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/browser-apis/service-workers.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/browser-apis/websockets.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/annotations.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/assertions-waiting.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/configuration.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/fixtures-hooks.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/global-setup.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/locators.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/page-object-model.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/projects-dependencies.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/test-data.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/test-suite-structure.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/test-tags.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/debugging/console-errors.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/debugging/debugging.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/debugging/error-testing.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/debugging/flaky-tests.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/frameworks/angular.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/frameworks/nextjs.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/frameworks/react.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/frameworks/vue.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/ci-cd.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/docker.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/github-actions.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/gitlab.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/other-providers.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/parallel-sharding.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/performance.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/reporting.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/test-coverage.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/accessibility.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/api-testing.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/browser-extensions.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/canvas-webgl.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/component-testing.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/drag-drop.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/electron.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/file-operations.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/file-upload-download.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/forms-validation.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/graphql-testing.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/i18n.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/performance-testing.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/security-testing.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/visual-regression.md +0 -0
- /package/skills/{playwright-cli → playwright/playwright-cli}/SKILL.md +0 -0
- /package/skills/{playwright-cli → playwright/playwright-cli}/references/element-attributes.md +0 -0
- /package/skills/{playwright-cli → playwright/playwright-cli}/references/playwright-tests.md +0 -0
- /package/skills/{playwright-cli → playwright/playwright-cli}/references/request-mocking.md +0 -0
- /package/skills/{playwright-cli → playwright/playwright-cli}/references/running-code.md +0 -0
- /package/skills/{playwright-cli → playwright/playwright-cli}/references/session-management.md +0 -0
- /package/skills/{playwright-cli → playwright/playwright-cli}/references/storage-state.md +0 -0
- /package/skills/{playwright-cli → playwright/playwright-cli}/references/test-generation.md +0 -0
- /package/skills/{playwright-cli → playwright/playwright-cli}/references/tracing.md +0 -0
- /package/skills/{playwright-cli → playwright/playwright-cli}/references/video-recording.md +0 -0
|
@@ -1,16 +1,16 @@
|
|
|
1
1
|
import dedent from "dedent";
|
|
2
|
-
import type { TestomatioProjectInfo } from "
|
|
2
|
+
import type { TestomatioProjectInfo } from "./project-info.js";
|
|
3
3
|
|
|
4
4
|
const MAX_PROMPT_LABELS = 20;
|
|
5
5
|
const MAX_PROMPT_TAGS = 30;
|
|
6
6
|
const MAX_PROMPT_ENVS = 15;
|
|
7
7
|
|
|
8
|
-
|
|
8
|
+
const tmsViaDirectTools = dedent`
|
|
9
9
|
# Testomat.io-first operating rules (highest priority)
|
|
10
10
|
|
|
11
|
-
This agent
|
|
11
|
+
This agent works on Testomat.io projects. You have TWO sources of truth and they answer different kinds of questions:
|
|
12
12
|
|
|
13
|
-
1. **The pulled markdown workspace (\`cwd\`)** — for **tests** and **suites**. Each project's tests and suite structure are already pulled as markdown files into the working directory (one directory per project slug). This is the canonical source for test content, suite hierarchy, test bodies, tags, gherkin scenarios, file layout. Use \`read\`, \`find\`, \`grep\`, \`ls\`
|
|
13
|
+
1. **The pulled markdown workspace (\`cwd\`)** — for **tests** and **suites**. Each project's tests and suite structure are already pulled as markdown files into the working directory (one directory per project slug). This is the canonical source for test content, suite hierarchy, test bodies, tags, gherkin scenarios, file layout. Use \`read\`, \`find\`, \`grep\`, \`ls\` — they're instant and don't hit the network.
|
|
14
14
|
2. **MCP tools under \`testomatio-<slug>\`** — for **dynamic Testomat.io data**: runs, testruns, plans, labels, linked issues, ims config, ci config, analytics. These things don't live as files and MUST come from MCP.
|
|
15
15
|
|
|
16
16
|
**Prefer the filesystem for test/suite queries.** Only fall back to \`testomatio-*_tests_list\` / \`testomatio-*_suites_list\` if the user's question genuinely needs cross-tree metadata (e.g. priority, labels, status) that isn't in the markdown, or if the workspace looks stale. Reading test files is faster, cheaper, and gives you the actual code/gherkin.
|
|
@@ -30,21 +30,103 @@ export const testomatioTms = dedent`
|
|
|
30
30
|
| Question is about… | First action |
|
|
31
31
|
|---|---|
|
|
32
32
|
| Test content (code, steps, gherkin, description, file path, tags) | \`read\` / \`find\` / \`grep\` in \`cwd\` |
|
|
33
|
-
| Suite hierarchy / structure / which files exist | \`ls\` / \`find\`
|
|
33
|
+
| Suite hierarchy / structure / which files exist | \`ls\` / \`find\` |
|
|
34
34
|
| Individual test metadata (priority, status flag, labels) | try filesystem; if missing, \`testomatio-*_tests_get\` |
|
|
35
35
|
| Runs, testruns, plans, labels, issues, analytics | MCP \`testomatio-*\` tools |
|
|
36
36
|
| Creating/updating tests or suites | edit the markdown file, then \`npx check-tests push\` |
|
|
37
37
|
| Creating/updating runs, linking issues | MCP create/update tools |
|
|
38
|
-
| Launching automated or mixed Run remotely on CI | Use npx @testomatio/reporter run --remote <profile>, raise error if no CI profiles configured
|
|
38
|
+
| Launching automated or mixed Run remotely on CI | Use npx -y @testomatio/reporter@latest run --remote <profile>, raise error if no CI profiles configured
|
|
39
39
|
| Running automated Run | Use local test runner with testomatio reporter attached to report to current project
|
|
40
40
|
|
|
41
41
|
---
|
|
42
42
|
`;
|
|
43
43
|
|
|
44
|
+
const tmsViaProxy = dedent`
|
|
45
|
+
# Testomat.io-first operating rules (highest priority)
|
|
46
|
+
|
|
47
|
+
This agent works on a Testomat.io project. You have TWO sources of truth and they answer different kinds of questions:
|
|
48
|
+
|
|
49
|
+
1. **The pulled markdown workspace (\`cwd\`)** — for **tests** and **suites**. Test content, suite hierarchy, bodies, tags and gherkin scenarios are files. Use \`read\`, \`find\`, \`grep\`, \`ls\` — they are instant and hit no network.
|
|
50
|
+
2. **The \`mcp\` tool** — for **dynamic Testomat.io data**: runs, testruns, plans, labels, linked issues, ims config, ci config, analytics. These do not live as files and MUST come from there.
|
|
51
|
+
|
|
52
|
+
Testomat.io is reached through **one \`mcp\` tool**, not a tool per operation. Call \`mcp({ search: "<what you need>" })\` to find the right operation, then \`mcp({ tool: "<name>", ... })\` to run it. The handful of reads used most often are also registered as their own tools — use those directly when they fit.
|
|
53
|
+
|
|
54
|
+
**Prefer the filesystem for test/suite queries.** Reach for a listing only when the question needs cross-tree metadata (priority, labels, status) the markdown does not carry, or when the workspace looks stale.
|
|
55
|
+
|
|
56
|
+
**Never** ask the user for the **Testomat.io** API token — it's configured. (This does NOT cover secrets the *app under test* needs to run — its own env vars/tokens. If a test run is blocked by one of those, report it.)
|
|
57
|
+
|
|
58
|
+
**Statuses and counts are live** — runs change them at any time. Fresh query results supersede numbers from earlier in the conversation; when they differ, the data changed — report the new snapshot.
|
|
59
|
+
|
|
60
|
+
## Banned moves
|
|
61
|
+
|
|
62
|
+
- Do NOT search for an operation as your first action for a test/suite question — **read the filesystem first** (\`find\`, \`ls\`, \`grep\`, \`read\`). Fall back to a listing only if the metadata you need isn't in the markdown.
|
|
63
|
+
- Do NOT ask the user for the **Testomat.io** API token — it's already configured. (Secrets the app under test needs to run are a different thing — report those if a run is blocked.)
|
|
64
|
+
- Do NOT call \`bash\` / \`find\` / \`grep\` to answer questions about **runs, testruns, plans, labels, linked issues, analytics** — those don't exist as files.
|
|
65
|
+
|
|
66
|
+
## Filesystem vs. Testomat.io cheat sheet
|
|
67
|
+
|
|
68
|
+
| Question is about… | First action |
|
|
69
|
+
|---|---|
|
|
70
|
+
| Test content (code, steps, gherkin, description, file path, tags) | \`read\` / \`find\` / \`grep\` in \`cwd\` |
|
|
71
|
+
| Suite hierarchy / structure / which files exist | \`ls\` / \`find\` |
|
|
72
|
+
| Individual test metadata (priority, status flag, labels) | try filesystem; if missing, \`mcp({ search: "get test" })\` |
|
|
73
|
+
| Runs, testruns, plans, labels, issues, analytics | \`mcp({ search: … })\`, then \`mcp({ tool: … })\` |
|
|
74
|
+
| Creating/updating tests or suites | edit the markdown file, then \`npx check-tests push\` |
|
|
75
|
+
| Creating/updating runs, linking issues | the matching \`mcp\` create/update operation |
|
|
76
|
+
| Launching automated or mixed Run remotely on CI | Use npx -y @testomatio/reporter@latest run --remote <profile>, raise error if no CI profiles configured |
|
|
77
|
+
| Running automated Run | Use local test runner with testomatio reporter attached to report to current project |
|
|
78
|
+
|
|
79
|
+
---
|
|
80
|
+
`;
|
|
81
|
+
|
|
82
|
+
const tmsViaCli = dedent`
|
|
83
|
+
# Testomat.io-first operating rules (highest priority)
|
|
84
|
+
|
|
85
|
+
This agent works on a Testomat.io project through the **shell only** — there are no Testomat.io tools in this session.
|
|
86
|
+
|
|
87
|
+
1. **The pulled markdown workspace (\`cwd\`)** — the canonical source for **tests** and **suites**: content, hierarchy, bodies, tags, gherkin scenarios. Use \`read\`, \`find\`, \`grep\`, \`ls\`.
|
|
88
|
+
2. **\`npx check-tests\`** — moves test cases between the markdown and Testomat.io (\`pull\`, \`push\`). It reads its credentials from the environment.
|
|
89
|
+
3. **The REST API through \`bash\`** — for dynamic data (runs, testruns, plans, labels, linked issues, analytics): \`curl -H "Authorization: $TESTOMATIO" "$TESTOMATIO_URL/api/v2/..."\`.
|
|
90
|
+
|
|
91
|
+
**Never** ask the user for the **Testomat.io** API token — it's configured as \`TESTOMATIO\`. (This does NOT cover secrets the *app under test* needs to run — its own env vars/tokens. If a test run is blocked by one of those, report it.)
|
|
92
|
+
|
|
93
|
+
**Statuses and counts are live** — runs change them at any time. Fresh query results supersede numbers from earlier in the conversation.
|
|
94
|
+
|
|
95
|
+
## Banned moves
|
|
96
|
+
|
|
97
|
+
- Do NOT shell out for test content that is already a file in \`cwd\` — read the file.
|
|
98
|
+
- Do NOT invent REST paths or parameters. Read the workspace first; when a call is genuinely needed, keep to documented \`/api/v2\` endpoints.
|
|
99
|
+
- Do NOT ask the user for the **Testomat.io** API token.
|
|
100
|
+
|
|
101
|
+
## Filesystem vs. shell cheat sheet
|
|
102
|
+
|
|
103
|
+
| Question is about… | First action |
|
|
104
|
+
|---|---|
|
|
105
|
+
| Test content (code, steps, gherkin, description, file path, tags) | \`read\` / \`find\` / \`grep\` in \`cwd\` |
|
|
106
|
+
| Suite hierarchy / structure / which files exist | \`ls\` / \`find\` |
|
|
107
|
+
| Runs, testruns, plans, labels, issues, analytics | \`curl\` against \`/api/v2\` |
|
|
108
|
+
| Creating/updating tests or suites | edit the markdown file, then \`npx check-tests push\` |
|
|
109
|
+
| Launching automated or mixed Run remotely on CI | Use npx -y @testomatio/reporter@latest run --remote <profile>, raise error if no CI profiles configured |
|
|
110
|
+
| Running automated Run | Use local test runner with testomatio reporter attached to report to current project |
|
|
111
|
+
|
|
112
|
+
---
|
|
113
|
+
`;
|
|
114
|
+
|
|
115
|
+
/**
|
|
116
|
+
* How the agent reaches Testomat.io in this session. The rules are the same;
|
|
117
|
+
* the way to act on them is not, so the harness picks the matching wording.
|
|
118
|
+
*/
|
|
119
|
+
export function testomatioTms(access: TmsAccess): string {
|
|
120
|
+
if (access === "mcp-proxy") return tmsViaProxy;
|
|
121
|
+
if (access === "cli-only") return tmsViaCli;
|
|
122
|
+
return tmsViaDirectTools;
|
|
123
|
+
}
|
|
124
|
+
|
|
44
125
|
export function testomatioConnection(
|
|
45
126
|
tokenSlugs: string[],
|
|
46
127
|
backendUrl?: string,
|
|
47
|
-
connection?: { projectId?: string; title?: string }
|
|
128
|
+
connection?: { projectId?: string; title?: string },
|
|
129
|
+
access: TmsAccess = "mcp-direct"
|
|
48
130
|
): string {
|
|
49
131
|
let urlSuffix = "";
|
|
50
132
|
if (backendUrl) urlSuffix = ` (\`${backendUrl}\`)`;
|
|
@@ -60,7 +142,7 @@ export function testomatioConnection(
|
|
|
60
142
|
|
|
61
143
|
The project API key is **already set** as \`TESTOMATIO\` in the environment of every \`bash\` command you run, along with \`TESTOMATIO_URL\`${urlSuffix}. **Never ask the user for a Testomat.io token or API key** — it is already in scope.
|
|
62
144
|
|
|
63
|
-
Push or pull manual test cases with \`npx check-tests\` (or other \`npx @testomatio/*\` commands) — they pick the credentials up from the environment.
|
|
145
|
+
Push or pull manual test cases with \`npx check-tests\` (or other \`npx @testomatio/*\` commands) — they pick the credentials up from the environment. ${envTokenActions(access)}
|
|
64
146
|
|
|
65
147
|
`;
|
|
66
148
|
}
|
|
@@ -89,7 +171,7 @@ export function testomatioConnection(
|
|
|
89
171
|
Projects pulled into this working directory (one dir per project):
|
|
90
172
|
${projects}
|
|
91
173
|
|
|
92
|
-
|
|
174
|
+
${perProjectActions(access)}
|
|
93
175
|
|
|
94
176
|
When you need to push changes back, run \`npx check-tests push\` (or similar \`npx @testomatio/*\` commands) directly — credentials are already in scope.
|
|
95
177
|
|
|
@@ -161,6 +243,28 @@ export function projectSettings(info: TestomatioProjectInfo): string {
|
|
|
161
243
|
`;
|
|
162
244
|
}
|
|
163
245
|
|
|
246
|
+
// How to act on a connection whose token comes from the environment.
|
|
247
|
+
function envTokenActions(access: TmsAccess): string {
|
|
248
|
+
if (access === "mcp-proxy") {
|
|
249
|
+
return "For structured actions (list/create/update tests, runs, labels, suites, etc.) prefer the `mcp` tool: `mcp({ search })` to find an operation, `mcp({ tool })` to run it.";
|
|
250
|
+
}
|
|
251
|
+
if (access === "cli-only") {
|
|
252
|
+
return "For runs, plans, labels and analytics use `curl` against `/api/v2` — there are no Testomat.io tools in this session.";
|
|
253
|
+
}
|
|
254
|
+
return "If `testomatio-*` MCP tools are available, prefer them for structured actions (list/create/update tests, runs, labels, suites, etc.).";
|
|
255
|
+
}
|
|
256
|
+
|
|
257
|
+
// The same, where the harness runs one MCP server per pulled project.
|
|
258
|
+
function perProjectActions(access: TmsAccess): string {
|
|
259
|
+
if (access === "mcp-proxy") {
|
|
260
|
+
return "The **Testomat.io MCP server** is also connected, reached through the `mcp` tool (`mcp({ search })` to find an operation, `mcp({ tool })` to run it). Prefer it for structured actions (list/create/update tests, runs, labels, suites, etc.) — those are typed, safer, and faster than shell calls. Fall back to `bash` + `check-tests` only for things it does not cover (e.g. pulling full markdown files).";
|
|
261
|
+
}
|
|
262
|
+
if (access === "cli-only") {
|
|
263
|
+
return "There are no Testomat.io tools in this session: use `npx check-tests` for test cases and `curl` against `/api/v2` for runs, plans and labels.";
|
|
264
|
+
}
|
|
265
|
+
return "The **Testomat.io MCP server** is also auto-connected, one instance per project (tool prefix: `testomatio-<slug>`). Prefer MCP tools for structured actions (list/create/update tests, runs, labels, suites, etc.) — they are typed, safer, and faster than shell calls. Fall back to `bash` + `check-tests` only for things the MCP does not cover (e.g. pulling full markdown files).";
|
|
266
|
+
}
|
|
267
|
+
|
|
164
268
|
function capped(shown: string[], total: number): string {
|
|
165
269
|
let suffix = "";
|
|
166
270
|
if (total > shown.length) {
|
|
@@ -168,3 +272,10 @@ function capped(shown: string[], total: number): string {
|
|
|
168
272
|
}
|
|
169
273
|
return shown.join(", ") + suffix;
|
|
170
274
|
}
|
|
275
|
+
|
|
276
|
+
/**
|
|
277
|
+
* `mcp-direct` — every Testomat.io operation is its own tool, prefixed
|
|
278
|
+
* `testomatio-<slug>`. `mcp-proxy` — one `mcp` tool with search and call.
|
|
279
|
+
* `cli-only` — no tools at all: `check-tests` and REST through the shell.
|
|
280
|
+
*/
|
|
281
|
+
export type TmsAccess = "mcp-direct" | "mcp-proxy" | "cli-only";
|
package/prompt/tools.ts
ADDED
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
import { existsSync } from 'node:fs';
|
|
2
|
+
import { delimiter, join } from 'node:path';
|
|
3
|
+
|
|
4
|
+
import dedent from 'dedent';
|
|
5
|
+
|
|
6
|
+
/**
|
|
7
|
+
* The tools every harness gives the agent. Anything a particular harness adds —
|
|
8
|
+
* a question tool, a browser it can drive for a watching user — arrives as an
|
|
9
|
+
* extra bullet from that harness, since the agent must not be told about a tool
|
|
10
|
+
* it does not have.
|
|
11
|
+
*/
|
|
12
|
+
export function tools(options?: { extra?: string[] }): string {
|
|
13
|
+
const extra = (options?.extra ?? []).join('');
|
|
14
|
+
let routing = ' - GitHub: issues and projects through MCP.';
|
|
15
|
+
if (onPath('gh')) {
|
|
16
|
+
routing +=
|
|
17
|
+
'\n - Pull requests and repositories: `gh` in `bash`. When a PR question needs issue detail, query the issue through MCP — never infer it from a PR title.';
|
|
18
|
+
}
|
|
19
|
+
if (onPath('acli')) {
|
|
20
|
+
routing +=
|
|
21
|
+
'\n - Jira in bulk (many issues at once), boards, sprints and filters: `acli` in `bash`, per the `atlassian` skill. Confluence and attachments have no `acli` command and stay on MCP.';
|
|
22
|
+
}
|
|
23
|
+
if (onPath('glab')) {
|
|
24
|
+
routing +=
|
|
25
|
+
'\n - GitLab releases, labels, milestones, snippets, CI artifacts, and any endpoint with no MCP tool: `glab` in `bash`. Issues, merge requests, pipelines and code search stay on MCP, which covers them well.';
|
|
26
|
+
}
|
|
27
|
+
return dedent`
|
|
28
|
+
<available-tools>
|
|
29
|
+
You have these tools available:
|
|
30
|
+
* **Read operations (use freely):** \`read\`, \`grep\`, \`ls\`.
|
|
31
|
+
- Use these aggressively to understand the system under test before proposing changes.
|
|
32
|
+
* **Write operations (restricted):** \`write\`, \`edit\`, \`bash\`.
|
|
33
|
+
- Requires explicit user confirmation for destructive or shared-state actions.
|
|
34
|
+
${extra} * **Internal Skills:** For QA-related tasks, check available skills first and prefer using them when applicable.
|
|
35
|
+
* **External Integrations:** Use MCP tools when they provide superior data or specialized integrations.
|
|
36
|
+
- Primary MCP (Testomat.io): Use Testomat.io MCP tools to extend functionality.
|
|
37
|
+
- Secondary MCP (Jira, GitHub, etc.): Invoke only when user explicitly asks or when remote context exploration is required.
|
|
38
|
+
${routing}
|
|
39
|
+
|
|
40
|
+
<tool-governance>
|
|
41
|
+
* **Prioritize Specificity:** Use \`read\` instead of \`cat\`, and \`edit\` instead of \`sed\`. Dedicated tools provide better error tracking and safety.
|
|
42
|
+
* **Parallel Execution:** Call independent tools in parallel to minimize latency. Sequential calls are reserved for operations with data dependencies.
|
|
43
|
+
* **Atomic Updates:** Mark tasks as complete in your internal state immediately upon execution. Do not batch status updates.
|
|
44
|
+
* **Analysis-First:** Use your own reasoning to guide tool selection. Do not wait for step-by-step instructions if discovery tools can provide the answer.
|
|
45
|
+
</tool-governance>
|
|
46
|
+
</available-tools>
|
|
47
|
+
`;
|
|
48
|
+
}
|
|
49
|
+
|
|
50
|
+
/** Is this CLI on PATH? Never tell the agent to use a binary this machine lacks. */
|
|
51
|
+
function onPath(bin: string): boolean {
|
|
52
|
+
for (const dir of (process.env.PATH ?? '').split(delimiter)) {
|
|
53
|
+
if (!dir) continue;
|
|
54
|
+
if (existsSync(join(dir, bin)) || existsSync(join(dir, `${bin}.exe`))) return true;
|
|
55
|
+
}
|
|
56
|
+
return false;
|
|
57
|
+
}
|
package/prompt/vocab.ts
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* The per-project config directory name, as the agent knows it. Part of the
|
|
3
|
+
* vocabulary the system prompt teaches — every path the prompt names below it
|
|
4
|
+
* (`.testeiya/manual-tests`, `.testeiya/project-info.json`, …) is built from
|
|
5
|
+
* this constant, so the prompt and the harness can never disagree about it.
|
|
6
|
+
*/
|
|
7
|
+
export const TESTEIYA_DIR_NAME = ".testeiya";
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: ci-fix-tests
|
|
3
|
-
description: "Use on CI in non-interactive mode
|
|
3
|
+
description: "Use on CI in non-interactive mode after a failed CodeceptJS run — attempts safe fixes only (locator drift, missing waits), reruns failing scenarios, rolls back edits that didn't help, reports at `output/ci-fix.md`. No refactors, no config edits, no flake-masking. Trigger on \"ci fix tests\", \"auto-fix failing tests\", \"attempt repair on CI\", or as a CI step after a failed run."
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Auto-fix CodeceptJS Tests on CI
|
|
@@ -25,7 +25,7 @@ Save this as the **baseline** — `count`, `failing_set`, `clusters`.
|
|
|
25
25
|
### 2. Pick safe fix candidates
|
|
26
26
|
Only attempt fixes from this allowed list:
|
|
27
27
|
|
|
28
|
-
- **Locator drift** — the failed locator no longer matches anything but a similar element exists. Use the **codeceptjs-exploration** skill (headless) to find candidates; pick
|
|
28
|
+
- **Locator drift** — the failed locator no longer matches anything but a similar element exists. Use the **codeceptjs-exploration** skill (headless) to find candidates; pick a short locator scoped to a stable region — `I.click('Save', '.toolbar')` — in this order: visible text / accessible name → ARIA `{ role, name }` → `$name` (`customLocator`) → composed CSS. Replace the locator at the failing step only.
|
|
29
29
|
- **Missing wait for a spinner / loader / modal** — the failed step's ARIA snapshot shows a spinner or skeleton present, or the target element appears later. Add a single matching `I.waitFor*` immediately before the failing step.
|
|
30
30
|
- **`I.wait(N)` replacement** — when a hardcoded sleep is the only thing between a failing assertion and a passing one and the gating element is identifiable, replace the sleep with a specific `waitFor*`.
|
|
31
31
|
|
|
@@ -81,7 +81,7 @@ Failing scenarios: N
|
|
|
81
81
|
- ...
|
|
82
82
|
|
|
83
83
|
## Attempted fixes
|
|
84
|
-
- `tests/foo_test.js:42` — locator drift: `'Save'` → `
|
|
84
|
+
- `tests/foo_test.js:42` — locator drift: `I.click('Save')` → `I.click('Save', '.toolbar')`
|
|
85
85
|
- `tests/bar_test.js:15` — added `I.waitForInvisible('.spinner')` before checkout click
|
|
86
86
|
- `tests/baz_test.js:7` — replaced `I.wait(3)` with `I.waitForVisible('.confirmation-dialog', 10)`
|
|
87
87
|
|
|
@@ -112,9 +112,8 @@ The first line of `Status:` is the machine-parseable signal. The rest is for the
|
|
|
112
112
|
- Writing a partial or missing `output/ci-fix.md`. CI depends on it; the absence of a report is itself a failure mode.
|
|
113
113
|
- Running the **whole** suite for the verification step — only the originally-failing scenarios, to keep CI time bounded.
|
|
114
114
|
|
|
115
|
-
##
|
|
115
|
+
## Related skills
|
|
116
116
|
|
|
117
|
-
- `codeceptjs-run-analysis` —
|
|
118
|
-
- `codeceptjs-exploration` —
|
|
119
|
-
- `codeceptjs-fundamentals` —
|
|
120
|
-
- `node_modules/codeceptjs/docs/aitrace.md` — trace format.
|
|
117
|
+
- `codeceptjs-run-analysis` — baseline, clustering, post-fix comparison
|
|
118
|
+
- `codeceptjs-exploration` — replacement locators for drift
|
|
119
|
+
- `codeceptjs-fundamentals` — config, helper, CI environment
|
|
@@ -1,51 +1,45 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: codeceptjs-auth
|
|
3
|
-
description:
|
|
3
|
+
description: >
|
|
4
|
+
Use when a CodeceptJS test needs login, user roles, or session reuse — or
|
|
5
|
+
when another CodeceptJS skill identifies authorization is required. Also on
|
|
6
|
+
auth-related test failures. Trigger on login, sign-in, sign-up,
|
|
7
|
+
authentication, sessions, "logged in", admin/editor/user roles.
|
|
4
8
|
---
|
|
5
9
|
|
|
6
10
|
# CodeceptJS Auth Plugin
|
|
7
11
|
|
|
8
|
-
The `auth` plugin logs each user in once, captures cookies (or
|
|
12
|
+
The `auth` plugin logs each user in once, captures cookies (or localStorage/token via overrides), and restores the session for subsequent tests. Stale sessions trigger a fresh login automatically.
|
|
9
13
|
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
## When to add it
|
|
13
|
-
|
|
14
|
-
Suggested if tests have repeatable authentication and session is needed to be persisted accross tests. Also optimizes start time of tests by saving previous sessision cookies in files.
|
|
15
|
-
|
|
16
|
-
Suggested as performance optimization, refactoring measure.
|
|
17
|
-
|
|
18
|
-
If `auth` plugin already exists in the project (check fundamentals' output), reuse the existing `inject` name and user keys.
|
|
14
|
+
If the project already has `auth` configured (fundamentals' discovery output), **reuse** its existing inject name and user keys — don't reconfigure.
|
|
19
15
|
|
|
20
16
|
## Decide first — ask the user
|
|
21
17
|
|
|
22
|
-
Four answers shape the plugin. Don't guess any.
|
|
18
|
+
Four answers shape the plugin. Don't guess any.
|
|
23
19
|
|
|
24
|
-
1. **Is a session needed at all?** Public flows (landing, signup)
|
|
25
|
-
2. **One user or many?** Default
|
|
26
|
-
3. **If many — what splits them?**
|
|
27
|
-
4. **What's the auth type?**
|
|
28
|
-
- **Form** —
|
|
29
|
-
- **OAuth / SSO** — click provider button, drive the IdP page (often
|
|
30
|
-
- **Magic link / passwordless** — UI flow
|
|
31
|
-
- **API token** — skip the form;
|
|
32
|
-
- **2FA / OTP** —
|
|
20
|
+
1. **Is a session needed at all?** Public flows (landing, signup) → skip the plugin.
|
|
21
|
+
2. **One user or many?** Default one; add more only when actually exercised.
|
|
22
|
+
3. **If many — what splits them?** Role, workspace/tenant, plan tier, sign-in provider, per-test fixture — real systems vary. Ask; use the answer to name `users.<key>` entries.
|
|
23
|
+
4. **What's the auth type?**
|
|
24
|
+
- **Form** — default; canonical shape below
|
|
25
|
+
- **OAuth / SSO** — click provider button, drive the IdP page (often separate origin)
|
|
26
|
+
- **Magic link / passwordless** — UI flow rarely worth automating; prefer an API mint or reading the link from a test mailbox
|
|
27
|
+
- **API token** — skip the form; write the token into `localStorage` via `executeScript`, or `I.setCookie(...)`
|
|
28
|
+
- **2FA / OTP** — async `login`; fetch the code from a test mailbox / backdoor before submitting
|
|
33
29
|
|
|
34
30
|
## Rules
|
|
35
31
|
|
|
36
|
-
1. **Login flow
|
|
37
|
-
2. **Credentials
|
|
38
|
-
3. **`.env` is gitignored; `.env.example` is committed** with
|
|
32
|
+
1. **Login flow never lives in the conf.** Put it in `steps_file.js` (if included) or a page object; conf only references it: `login: (I) => I.login()`.
|
|
33
|
+
2. **Credentials from env only** — `.env` loaded via `process.loadEnvFile()` (no dotenv dependency). Passwords wrapped with `secret(...)`. No literals anywhere — conf, steps file, test, git history.
|
|
34
|
+
3. **`.env` is gitignored; `.env.example` is committed** with names, no values. Gitignore `output/*_session.json` too.
|
|
39
35
|
|
|
40
36
|
## Canonical shape
|
|
41
37
|
|
|
42
38
|
```js
|
|
43
39
|
// codecept.conf.js — first line of the file
|
|
44
|
-
process.loadEnvFile()
|
|
45
|
-
// ...
|
|
40
|
+
process.loadEnvFile()
|
|
46
41
|
|
|
47
42
|
export const config = {
|
|
48
|
-
// ...
|
|
49
43
|
include: { I: './steps_file.js' },
|
|
50
44
|
plugins: {
|
|
51
45
|
auth: {
|
|
@@ -79,56 +73,46 @@ export default function () {
|
|
|
79
73
|
}
|
|
80
74
|
```
|
|
81
75
|
|
|
82
|
-
```
|
|
83
|
-
# .env (gitignored)
|
|
84
|
-
USER_EMAIL
|
|
85
|
-
USER_PASSWORD=<secret>
|
|
76
|
+
```sh
|
|
77
|
+
# .env (gitignored) # .env.example (committed)
|
|
78
|
+
USER_EMAIL=... USER_EMAIL=
|
|
79
|
+
USER_PASSWORD=<secret> USER_PASSWORD=
|
|
86
80
|
```
|
|
87
81
|
|
|
88
82
|
## Pre-flight (before writing config)
|
|
89
83
|
|
|
90
|
-
1. **Read the real login page
|
|
91
|
-
2. **Pick a role-specific post-login marker
|
|
92
|
-
3. **Confirm session storage
|
|
84
|
+
1. **Read the real login page** — MCP `run_code` to `/login`, inspect the ARIA snapshot (`codeceptjs-exploration`). Field labels / `name` / `id` / submit control from the actual page, not guesses. Unclear authorization mechanism → ask the user.
|
|
85
|
+
2. **Pick a role-specific post-login marker** — something rendered only for *this* user (navbar username, `data-user-role`).
|
|
86
|
+
3. **Confirm session storage** — cookies (default) for server-rendered apps; localStorage/sessionStorage for SPAs. Verify after a manual login with `I.executeScript(() => Object.keys(localStorage))`. Cookie fetch/restore silently no-op against token storage.
|
|
93
87
|
|
|
94
88
|
## Verify
|
|
95
89
|
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
Enable `saveToFile: true` only after the verification round-trip succeeds — a bad saved session masks a broken `login`.
|
|
99
|
-
|
|
100
|
-
## Refactor
|
|
101
|
-
|
|
102
|
-
Add to before hooks (if applied to all tests in suite) or to exact tests in syute
|
|
103
|
-
|
|
104
|
-
`Before(({ login }) => login())`
|
|
105
|
-
|
|
106
|
-
`login` was declared in auth plugin configuration
|
|
107
|
-
|
|
108
|
-
Run real, not dry:
|
|
90
|
+
Run a one-Scenario file that calls `login(<role>)` then asserts on the post-login marker:
|
|
109
91
|
|
|
110
92
|
```bash
|
|
111
|
-
npx codeceptjs run --grep '<scenario>' --debug
|
|
93
|
+
npx codeceptjs run --grep '<scenario>' --debug # real run, not dry — dry-run doesn't init plugins
|
|
112
94
|
```
|
|
113
95
|
|
|
96
|
+
Enable `saveToFile: true` only after this round-trip succeeds — a bad saved session masks a broken `login`.
|
|
97
|
+
|
|
98
|
+
Then wire into hooks/tests: `Before(({ login }) => login())` for suite-wide, or per-test as needed.
|
|
114
99
|
|
|
115
|
-
## Multi-
|
|
100
|
+
## Multi-role shape
|
|
116
101
|
|
|
117
|
-
Only after question 3 is answered.
|
|
102
|
+
Only after question 3 is answered. Keys named after whatever splits users *in this system*; one matching actor method per key:
|
|
118
103
|
|
|
119
104
|
```js
|
|
120
|
-
// keys named after the dimension (role / workspace / provider / …)
|
|
121
105
|
users: {
|
|
122
106
|
admin: { login: (I) => I.loginAsAdmin() },
|
|
123
107
|
workspaceB: { login: (I) => I.loginToWorkspaceB() },
|
|
124
108
|
}
|
|
125
109
|
```
|
|
126
110
|
|
|
127
|
-
Don't parameterise into a single `login(key)` — the plugin keys sessions by name
|
|
111
|
+
Don't parameterise into a single `login(key)` — the plugin keys sessions by name, explicit methods read better. Switch mid-Scenario: `session('<key>')` opens a parallel browser context (fundamentals § Writing tests).
|
|
128
112
|
|
|
129
|
-
## Token /
|
|
113
|
+
## Token / localStorage auth
|
|
130
114
|
|
|
131
|
-
|
|
115
|
+
Override `fetch` / `restore` when sessions live outside cookies:
|
|
132
116
|
|
|
133
117
|
```js
|
|
134
118
|
admin: {
|
|
@@ -142,17 +126,16 @@ admin: {
|
|
|
142
126
|
}
|
|
143
127
|
```
|
|
144
128
|
|
|
145
|
-
`check(I, session)` receives whatever `fetch` returned — throw inside `check` to force
|
|
146
|
-
|
|
147
|
-
## Common pitfalls
|
|
129
|
+
`check(I, session)` receives whatever `fetch` returned — throw inside `check` to force fresh login (e.g. `/me` endpoint shows wrong user).
|
|
148
130
|
|
|
149
|
-
|
|
150
|
-
- **Forgetting to gitignore `.env` and `output/*_session.json`.** Both leak credentials.
|
|
131
|
+
## Pitfalls
|
|
151
132
|
|
|
133
|
+
- Credentials inlined in conf/test — always env-driven + `secret()`.
|
|
134
|
+
- Forgetting to gitignore `.env` and `output/*_session.json` — both leak credentials.
|
|
152
135
|
|
|
153
|
-
##
|
|
136
|
+
## Related skills
|
|
154
137
|
|
|
155
|
-
- `
|
|
156
|
-
- `
|
|
157
|
-
- `
|
|
158
|
-
- `
|
|
138
|
+
- `codeceptjs-fundamentals` — secrets rule, sessions, config mutation trap
|
|
139
|
+
- `codeceptjs-exploration` — reading the live login page
|
|
140
|
+
- `writing-codeceptjs-tests` / `refactoring-codeceptjs-tests` — invoke this skill when auth is identified
|
|
141
|
+
- `debugging-codeceptjs-tests` — auth-related failure patterns
|