testeiya 0.2.1 → 0.3.1-5.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +325 -171
- package/dist/prompt/context.js +79 -0
- package/dist/prompt/context.js.map +1 -0
- package/dist/prompt/index.js +60 -0
- package/dist/prompt/index.js.map +1 -0
- package/dist/prompt/print.js +30 -0
- package/dist/prompt/print.js.map +1 -0
- package/dist/prompt/project-info.js +2 -0
- package/dist/prompt/project-info.js.map +1 -0
- package/{src/prompt/system-prompt.ts → dist/prompt/system-prompt.js} +38 -25
- package/dist/prompt/system-prompt.js.map +1 -0
- package/dist/prompt/testomatio.js +264 -0
- package/dist/prompt/testomatio.js.map +1 -0
- package/dist/prompt/tools.js +70 -0
- package/dist/prompt/tools.js.map +1 -0
- package/dist/prompt/vocab.js +8 -0
- package/dist/prompt/vocab.js.map +1 -0
- package/dist/src/args.js +224 -0
- package/dist/src/args.js.map +1 -0
- package/dist/src/cli.js +105 -0
- package/dist/src/cli.js.map +1 -0
- package/dist/src/doctor.js +171 -0
- package/dist/src/doctor.js.map +1 -0
- package/dist/src/env.js +103 -0
- package/dist/src/env.js.map +1 -0
- package/dist/src/mcp-extension.js +9 -0
- package/dist/src/mcp-extension.js.map +1 -0
- package/dist/src/mcp.js +139 -0
- package/dist/src/mcp.js.map +1 -0
- package/dist/src/model.js +71 -0
- package/dist/src/model.js.map +1 -0
- package/dist/src/models.js +38 -0
- package/dist/src/models.js.map +1 -0
- package/dist/src/output.js +216 -0
- package/dist/src/output.js.map +1 -0
- package/dist/src/result.js +41 -0
- package/dist/src/result.js.map +1 -0
- package/dist/src/run.js +313 -0
- package/dist/src/run.js.map +1 -0
- package/dist/src/session.js +98 -0
- package/dist/src/session.js.map +1 -0
- package/dist/src/sessions.js +72 -0
- package/dist/src/sessions.js.map +1 -0
- package/dist/src/skills.js +130 -0
- package/dist/src/skills.js.map +1 -0
- package/dist/vendor/mcp.js +14801 -0
- package/mcp-tools.json +2967 -0
- package/package.json +34 -43
- package/prompt/context.ts +101 -0
- package/prompt/index.ts +102 -0
- package/prompt/print.ts +32 -0
- package/prompt/project-info.ts +30 -0
- package/prompt/system-prompt.ts +187 -0
- package/{src/prompt → prompt}/testomatio.ts +120 -9
- package/prompt/tools.ts +72 -0
- package/prompt/vocab.ts +7 -0
- package/skills/codeceptjs/ci-fix-tests/SKILL.md +7 -8
- package/skills/codeceptjs/codeceptjs-auth/SKILL.md +47 -64
- package/skills/codeceptjs/codeceptjs-exploration/SKILL.md +56 -47
- package/skills/codeceptjs/codeceptjs-fundamentals/SKILL.md +144 -106
- package/skills/codeceptjs/codeceptjs-run-analysis/SKILL.md +50 -57
- package/skills/codeceptjs/debugging-codeceptjs-tests/SKILL.md +88 -196
- package/skills/codeceptjs/migrate-codeceptjs-4/SKILL.md +4 -5
- package/skills/codeceptjs/migrate-cypress-to-codeceptjs/SKILL.md +25 -36
- package/skills/codeceptjs/migrate-protractor-to-codeceptjs/SKILL.md +23 -36
- package/skills/codeceptjs/migrate-selenium-java-to-codeceptjs/SKILL.md +23 -36
- package/skills/codeceptjs/migrate-testcafe-to-codeceptjs/SKILL.md +23 -35
- package/skills/codeceptjs/refactoring-codeceptjs-tests/SKILL.md +50 -53
- package/skills/codeceptjs/writing-codeceptjs-tests/SKILL.md +87 -191
- package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/SKILL.md +1 -1
- package/{skills.lock.json → skills/skills.lock.json} +45 -37
- package/skills/skills.yaml +35 -0
- package/skills/testomatio/explorbot/explorbot-fundamentals/SKILL.md +49 -61
- package/skills/testomatio/explorbot/explorbot-fundamentals/references/no-install.md +36 -0
- package/skills/testomatio/explorbot/explorbot-plan/SKILL.md +20 -7
- package/skills/testomatio/explorbot/explorbot-setup/SKILL.md +41 -289
- package/skills/testomatio/explorbot/prima/SKILL.md +87 -0
- package/skills/testomatio/qa-process/qa-explain-behavior/SKILL.md +111 -0
- package/skills/testomatio/qa-process/qa-explain-behavior/references/answer-shapes.md +116 -0
- package/skills/testomatio/qa-process/qa-explain-behavior/references/examples.md +157 -0
- package/skills/testomatio/qa-process/qa-lead-strategy-advisor/SKILL.md +91 -0
- package/skills/testomatio/qa-process/qa-lead-strategy-advisor/references/output-format.md +71 -0
- package/skills/testomatio/qa-process/testing-workflow/SKILL.md +227 -0
- package/skills/testomatio/test-automation/qa-data-seeder/SKILL.md +83 -0
- package/skills/testomatio/test-automation/run-tests-with-testomatio-reporter/SKILL.md +5 -3
- package/skills/testomatio/test-automation/setup-change-aware-pr-testing/SKILL.md +190 -0
- package/skills/testomatio/test-automation/setup-ci-automation/SKILL.md +76 -0
- package/skills/testomatio/test-management/qa-split-testing-levels-pyramid/SKILL.md +55 -0
- package/skills/testomatio/test-management/qa-sprint-report-by-testomatio/SKILL.md +220 -0
- package/skills/testomatio/test-management/qa-sprint-report-by-testomatio/references/qa-sprint-report.md +126 -0
- package/skills/testomatio/test-management/qa-test-code-coverage/SKILL.md +1 -1
- package/skills/testomatio/test-management/qa-thinking/SKILL.md +25 -25
- package/skills/testomatio/test-management/testomatio-mcp/SKILL.md +14 -1
- package/skills/testomatio/test-management/testomatio-mcp/references/MCP_SETUP.md +8 -0
- package/assets/testeiya-logo-dark.svg +0 -16
- package/assets/testeiya-logo.svg +0 -15
- package/models.catalog.json +0 -6963
- package/skills/playwright-best-practices-skill/LICENSE.md +0 -7
- package/skills/playwright-best-practices-skill/README.md +0 -147
- package/skills/testeiya/README.md +0 -29
- package/skills/testeiya/answer-formatting/SKILL.md +0 -30
- package/skills/testeiya/check-cucumber/SKILL.md +0 -200
- package/skills/testeiya/check-tests/SKILL.md +0 -260
- package/skills/testeiya/manual-run-assistant/SKILL.md +0 -61
- package/skills/testeiya/testomatio-docs/INDEX.md +0 -175
- package/skills/testeiya/testomatio-docs/SKILL.md +0 -102
- package/skills/testeiya/testomatio-docs/docs/advanced/ai-powered-features/ai-agents.md +0 -144
- package/skills/testeiya/testomatio-docs/docs/advanced/ai-powered-features/ai-powered-features.md +0 -531
- package/skills/testeiya/testomatio-docs/docs/advanced/ai-powered-features/ai-requirements.md +0 -437
- package/skills/testeiya/testomatio-docs/docs/advanced/api-access/index.md +0 -191
- package/skills/testeiya/testomatio-docs/docs/advanced/branches-folder/index.md +0 -277
- package/skills/testeiya/testomatio-docs/docs/advanced/bulk-edit-folder/bulk-edit-demos.mdx +0 -39
- package/skills/testeiya/testomatio-docs/docs/advanced/bulk-edit-folder/bulk-edit-on-suite-and-test-level.md +0 -122
- package/skills/testeiya/testomatio-docs/docs/advanced/bulk-edit-folder/what-is-bulk-edit.md +0 -170
- package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/branches.md +0 -53
- package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/index.mdx +0 -98
- package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/jira-issue.md +0 -151
- package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/runs-and-reports.md +0 -377
- package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/work-in-jira.md +0 -267
- package/skills/testeiya/testomatio-docs/docs/advanced/living-doc/index.md +0 -138
- package/skills/testeiya/testomatio-docs/docs/advanced/milestones/index.md +0 -220
- package/skills/testeiya/testomatio-docs/docs/advanced/shortcuts/index.md +0 -48
- package/skills/testeiya/testomatio-docs/docs/advanced/tags-labels/labels-and-custom-fields.md +0 -201
- package/skills/testeiya/testomatio-docs/docs/advanced/tags-labels/tags-or-labels.md +0 -94
- package/skills/testeiya/testomatio-docs/docs/advanced/tags-labels/tags.md +0 -243
- package/skills/testeiya/testomatio-docs/docs/advanced/test-artifacts/index.md +0 -135
- package/skills/testeiya/testomatio-docs/docs/advanced/tql/index.md +0 -394
- package/skills/testeiya/testomatio-docs/docs/getting-started/index.md +0 -290
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/azure.md +0 -96
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/bamboo.md +0 -94
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/bitbucket.md +0 -106
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/circle.md +0 -115
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/github.md +0 -120
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/gitlab.md +0 -88
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/index.mdx +0 -151
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/jenkins.md +0 -121
- package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/teamcity.md +0 -96
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/azure.md +0 -31
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/clickup.md +0 -54
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/confluence.md +0 -34
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/github.md +0 -31
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/gitlab.md +0 -93
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/index.md +0 -131
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/jira.md +0 -164
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/linear.md +0 -29
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/shortcut.md +0 -67
- package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/youtrack.md +0 -94
- package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/azure-devops.md +0 -53
- package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/email.md +0 -71
- package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/jira.md +0 -52
- package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/ms-teams.md +0 -60
- package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/rules.md +0 -267
- package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/slack.md +0 -82
- package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/telegram.md +0 -73
- package/skills/testeiya/testomatio-docs/docs/integrations/scim-provisioning/index.mdx +0 -207
- package/skills/testeiya/testomatio-docs/docs/integrations/scim-provisioning/okta.md +0 -204
- package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/azure.md +0 -95
- package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/google.md +0 -75
- package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/index.mdx +0 -33
- package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/okta.md +0 -108
- package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/saml.md +0 -66
- package/skills/testeiya/testomatio-docs/docs/legal/compliance/gdpr.md +0 -95
- package/skills/testeiya/testomatio-docs/docs/legal/compliance/index.mdx +0 -94
- package/skills/testeiya/testomatio-docs/docs/legal/compliance/privacy.md +0 -26
- package/skills/testeiya/testomatio-docs/docs/legal/compliance/terms.md +0 -32
- package/skills/testeiya/testomatio-docs/docs/legal/security/index.mdx +0 -242
- package/skills/testeiya/testomatio-docs/docs/legal/security/jira.md +0 -140
- package/skills/testeiya/testomatio-docs/docs/legal/security/subprocessors.md +0 -76
- package/skills/testeiya/testomatio-docs/docs/management/company/administration.md +0 -363
- package/skills/testeiya/testomatio-docs/docs/management/company/audit-log.md +0 -41
- package/skills/testeiya/testomatio-docs/docs/management/company/plan-features-comparison.md +0 -240
- package/skills/testeiya/testomatio-docs/docs/management/company/subscriptions.md +0 -326
- package/skills/testeiya/testomatio-docs/docs/management/company/trials.md +0 -75
- package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/accountant.md +0 -49
- package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/index.mdx +0 -285
- package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/read-only-user.md +0 -67
- package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/teams.md +0 -138
- package/skills/testeiya/testomatio-docs/docs/management/project/settings.md +0 -87
- package/skills/testeiya/testomatio-docs/docs/management/project/templates.md +0 -546
- package/skills/testeiya/testomatio-docs/docs/project/analytics/index.md +0 -326
- package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/classical-tests-markdown-format.md +0 -386
- package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/download-manual-tests-as-files.md +0 -378
- package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/export-to-obsidian.md +0 -112
- package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/export-to-spreadsheet.md +0 -127
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/auto-import.md +0 -63
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-Allure-TestOps.md +0 -49
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-QTest.md +0 -41
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-Qase.md +0 -39
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-Zephyr.md +0 -37
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-csv-xlsx.md +0 -117
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-cucumber.md +0 -93
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-qmetry.md +0 -44
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-source-code.md +0 -117
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-testcaselabs.md +0 -44
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-testmo.md +0 -47
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-testrail.md +0 -82
- package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-xray.md +0 -55
- package/skills/testeiya/testomatio-docs/docs/project/plans/index.md +0 -334
- package/skills/testeiya/testomatio-docs/docs/project/pulse/index.md +0 -104
- package/skills/testeiya/testomatio-docs/docs/project/runs/archive-runs-and-groups.md +0 -176
- package/skills/testeiya/testomatio-docs/docs/project/runs/custom-statuses.md +0 -195
- package/skills/testeiya/testomatio-docs/docs/project/runs/environments.md +0 -203
- package/skills/testeiya/testomatio-docs/docs/project/runs/managing-runs.md +0 -455
- package/skills/testeiya/testomatio-docs/docs/project/runs/merge-strategies.md +0 -137
- package/skills/testeiya/testomatio-docs/docs/project/runs/reports.md +0 -329
- package/skills/testeiya/testomatio-docs/docs/project/runs/rungroups.md +0 -342
- package/skills/testeiya/testomatio-docs/docs/project/runs/running-automated-tests.md +0 -256
- package/skills/testeiya/testomatio-docs/docs/project/runs/running-manual-and-automated-tests.md +0 -331
- package/skills/testeiya/testomatio-docs/docs/project/runs/running-tests-manually.md +0 -633
- package/skills/testeiya/testomatio-docs/docs/project/runs/temporary-tests-notes.md +0 -223
- package/skills/testeiya/testomatio-docs/docs/project/steps-snippets/snippets.md +0 -68
- package/skills/testeiya/testomatio-docs/docs/project/steps-snippets/steps.md +0 -181
- package/skills/testeiya/testomatio-docs/docs/project/tests/bdd-test-case-editor.md +0 -98
- package/skills/testeiya/testomatio-docs/docs/project/tests/classical-test-case-editor.md +0 -354
- package/skills/testeiya/testomatio-docs/docs/project/tests/classical-vs-bdd.md +0 -169
- package/skills/testeiya/testomatio-docs/docs/project/tests/copy-and-move-your-tests.md +0 -156
- package/skills/testeiya/testomatio-docs/docs/project/tests/other-features-for-test-case-design.md +0 -412
- package/skills/testeiya/testomatio-docs/docs/project/tests/test-case-creation-and-editing.md +0 -527
- package/skills/testeiya/testomatio-docs/docs/support/index.md +0 -75
- package/skills/testeiya/testomatio-docs/docs/test-reporting/artifacts.md +0 -388
- package/skills/testeiya/testomatio-docs/docs/test-reporting/cli.md +0 -215
- package/skills/testeiya/testomatio-docs/docs/test-reporting/configuration.md +0 -247
- package/skills/testeiya/testomatio-docs/docs/test-reporting/frameworks.md +0 -490
- package/skills/testeiya/testomatio-docs/docs/test-reporting/functions.md +0 -144
- package/skills/testeiya/testomatio-docs/docs/test-reporting/index.mdx +0 -48
- package/skills/testeiya/testomatio-docs/docs/test-reporting/junit.md +0 -248
- package/skills/testeiya/testomatio-docs/docs/test-reporting/logger.md +0 -138
- package/skills/testeiya/testomatio-docs/docs/test-reporting/php.md +0 -77
- package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/bitbucket.md +0 -72
- package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/csv.md +0 -23
- package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/debug.md +0 -18
- package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/github.md +0 -63
- package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/gitlab.md +0 -29
- package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/html.md +0 -80
- package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/index.mdx +0 -57
- package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/testomatio.md +0 -331
- package/skills/testeiya/testomatio-docs/docs/test-reporting/python.md +0 -229
- package/skills/testeiya/testomatio-docs/docs/test-reporting/workflows.md +0 -325
- package/skills/testeiya/testomatio-docs/docs/tutorials/Interacting-via-API.md +0 -159
- package/skills/testeiya/testomatio-docs/docs/tutorials/Manual-Testing-Classic.md +0 -164
- package/skills/testeiya/testomatio-docs/docs/tutorials/Set-Up-Read-Only-Access.md +0 -85
- package/skills/testeiya/testomatio-docs/docs/tutorials/Using-AI-in-Testing.md +0 -163
- package/skills/testeiya/testomatio-docs/docs/tutorials/Working-with-Jira.md +0 -125
- package/skills/testeiya/testomatio-docs/docs/tutorials/java.md +0 -478
- package/skills/testeiya/testomatio-docs/docs/tutorials/playwright.md +0 -207
- package/skills/testeiya/testomatio-docs/docs/tutorials/webdriver.md +0 -218
- package/skills/testeiya/testomatio-docs/docs.lock.json +0 -6
- package/skills/testeiya/testomatio-reporter/SKILL.md +0 -307
- package/skills/testomatio/test-automation/setup-pr-testing/SKILL.md +0 -199
- package/skills.yaml +0 -21
- package/src/ai-debug.ts +0 -188
- package/src/api/agent-get.ts +0 -22
- package/src/api/agent-start.ts +0 -384
- package/src/api/client-log.ts +0 -29
- package/src/api/context.ts +0 -332
- package/src/api/debug-layout.ts +0 -47
- package/src/api/debug-report.ts +0 -29
- package/src/api/debug-snapshot.ts +0 -30
- package/src/api/debug-stream.ts +0 -52
- package/src/api/files-delete.ts +0 -184
- package/src/api/files-read.ts +0 -41
- package/src/api/files-rename.ts +0 -99
- package/src/api/files-tree.ts +0 -292
- package/src/api/files-write.ts +0 -41
- package/src/api/mcp.ts +0 -603
- package/src/api/memory.ts +0 -69
- package/src/api/open-external.ts +0 -39
- package/src/api/playwright-cli.ts +0 -606
- package/src/api/providers.ts +0 -457
- package/src/api/read-env-token.ts +0 -33
- package/src/api/sessions.ts +0 -79
- package/src/api/settings.ts +0 -120
- package/src/api/skills.ts +0 -58
- package/src/api/testomatio-attachment.ts +0 -89
- package/src/api/testomatio-auth.ts +0 -284
- package/src/api/testomatio-proxy.ts +0 -197
- package/src/api/testomatio-run-stats.ts +0 -81
- package/src/api/testomatio-target.ts +0 -225
- package/src/api/testomatio-transcription.ts +0 -82
- package/src/api/workspace-search.ts +0 -96
- package/src/api/workspace-sync.ts +0 -151
- package/src/api/workspace.ts +0 -165
- package/src/app-server.ts +0 -556
- package/src/bridge.ts +0 -251
- package/src/check-tests.ts +0 -156
- package/src/cli.ts +0 -17
- package/src/commands.ts +0 -244
- package/src/config.ts +0 -183
- package/src/connection.ts +0 -658
- package/src/context-store.ts +0 -265
- package/src/debug-bus.ts +0 -405
- package/src/extensions/tool-gate.ts +0 -220
- package/src/extensions/webui/ask-channel.ts +0 -52
- package/src/extensions/webui/index.ts +0 -184
- package/src/extensions/webui/tools/ask-question.ts +0 -88
- package/src/extensions/webui/tools/query-result.ts +0 -79
- package/src/extensions/webui/tools/render-chart.ts +0 -53
- package/src/extensions/webui/tools/render-item.ts +0 -61
- package/src/extensions/webui/tools/render-list.ts +0 -170
- package/src/extensions/webui/tools/render-result.ts +0 -85
- package/src/extensions/webui/tools/render-tree.ts +0 -71
- package/src/extensions/webui/tools/ui-widget.ts +0 -54
- package/src/extensions/webui/tools/widget-result.ts +0 -13
- package/src/extensions/webui/widget-channel.ts +0 -50
- package/src/file-log.ts +0 -196
- package/src/git-tracked.ts +0 -68
- package/src/gitignore.ts +0 -53
- package/src/json-store.ts +0 -43
- package/src/load-env.ts +0 -70
- package/src/loader-style.ts +0 -96
- package/src/main.ts +0 -140
- package/src/mcp-catalog.ts +0 -96
- package/src/models-catalog.ts +0 -72
- package/src/permissions.ts +0 -120
- package/src/project-dir.ts +0 -189
- package/src/project-info.ts +0 -108
- package/src/prompt/app-ui.ts +0 -59
- package/src/prompt/browser.ts +0 -27
- package/src/prompt/index.ts +0 -62
- package/src/prompt/tools.ts +0 -27
- package/src/server-info.ts +0 -62
- package/src/session-factory.ts +0 -442
- package/src/session-store.ts +0 -115
- package/src/shell-env.ts +0 -45
- package/src/skills.ts +0 -163
- package/src/sync-snapshot.ts +0 -140
- package/src/telemetry.ts +0 -239
- package/src/testomatio-auth.ts +0 -299
- package/src/testomatio.ts +0 -98
- package/src/theme.ts +0 -30
- package/src/user-env.ts +0 -94
- package/src/welcome.ts +0 -194
- package/src/workspace/safe-path.ts +0 -85
- package/src/workspace/test-md.ts +0 -70
- package/src/workspace-model.ts +0 -440
- package/testeiya.config.json +0 -14
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/authentication-flows.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/authentication.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/clock-mocking.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/mobile-testing.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/multi-context.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/multi-user.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/network-advanced.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/advanced/third-party.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/architecture/pom-vs-fixtures.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/architecture/test-architecture.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/architecture/when-to-mock.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/browser-apis/browser-apis.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/browser-apis/iframes.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/browser-apis/service-workers.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/browser-apis/websockets.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/annotations.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/assertions-waiting.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/configuration.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/fixtures-hooks.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/global-setup.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/locators.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/page-object-model.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/projects-dependencies.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/test-data.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/test-suite-structure.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/core/test-tags.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/debugging/console-errors.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/debugging/debugging.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/debugging/error-testing.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/debugging/flaky-tests.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/frameworks/angular.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/frameworks/nextjs.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/frameworks/react.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/frameworks/vue.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/ci-cd.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/docker.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/github-actions.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/gitlab.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/other-providers.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/parallel-sharding.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/performance.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/reporting.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/infrastructure-ci-cd/test-coverage.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/accessibility.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/api-testing.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/browser-extensions.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/canvas-webgl.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/component-testing.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/drag-drop.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/electron.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/file-operations.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/file-upload-download.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/forms-validation.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/graphql-testing.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/i18n.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/performance-testing.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/security-testing.md +0 -0
- /package/skills/{playwright-best-practices-skill → playwright/playwright-best-practices}/testing-patterns/visual-regression.md +0 -0
- /package/skills/{playwright-cli → playwright/playwright-cli}/SKILL.md +0 -0
- /package/skills/{playwright-cli → playwright/playwright-cli}/references/element-attributes.md +0 -0
- /package/skills/{playwright-cli → playwright/playwright-cli}/references/playwright-tests.md +0 -0
- /package/skills/{playwright-cli → playwright/playwright-cli}/references/request-mocking.md +0 -0
- /package/skills/{playwright-cli → playwright/playwright-cli}/references/running-code.md +0 -0
- /package/skills/{playwright-cli → playwright/playwright-cli}/references/session-management.md +0 -0
- /package/skills/{playwright-cli → playwright/playwright-cli}/references/storage-state.md +0 -0
- /package/skills/{playwright-cli → playwright/playwright-cli}/references/test-generation.md +0 -0
- /package/skills/{playwright-cli → playwright/playwright-cli}/references/tracing.md +0 -0
- /package/skills/{playwright-cli → playwright/playwright-cli}/references/video-recording.md +0 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Testomat.io
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,246 +1,400 @@
|
|
|
1
1
|
<p align="center">
|
|
2
|
-
<
|
|
3
|
-
<picture>
|
|
4
|
-
<source media="(prefers-color-scheme: dark)" srcset="https://cdn.jsdelivr.net/npm/testeiya/assets/testeiya-logo-dark.svg">
|
|
5
|
-
<img src="https://cdn.jsdelivr.net/npm/testeiya/assets/testeiya-logo.svg" alt="Testeiya logo" width="140">
|
|
6
|
-
</picture>
|
|
7
|
-
</a>
|
|
2
|
+
<img src="https://raw.githubusercontent.com/testomatio/testeiya-app/main/assets/testeiya-cli-logo.png" alt="Testeiya" width="220">
|
|
8
3
|
</p>
|
|
9
4
|
|
|
10
|
-
|
|
5
|
+
# Testeiya
|
|
11
6
|
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
Built on the <a href="https://github.com/can1357/oh-my-pi">oh-my-pi</a> harness (a fork of <a href="https://github.com/badlogic/pi-mono">Pi</a>).
|
|
15
|
-
</p>
|
|
7
|
+
[](https://www.npmjs.com/package/testeiya)
|
|
8
|
+
[](LICENSE)
|
|
16
9
|
|
|
17
|
-
|
|
18
|
-
<a href="https://testomat.ai/testeiya/">Website</a> ·
|
|
19
|
-
<a href="https://www.npmjs.com/package/testeiya">npm</a> ·
|
|
20
|
-
<a href="https://testomat.io">Testomat.io</a>
|
|
21
|
-
</p>
|
|
10
|
+
An agent, app, and the goddess of testing.
|
|
22
11
|
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
██ ███████ ███████ ██ ███████ ██████ ██ ██ ██
|
|
12
|
+
Testeiya is an autonomous QA agent. It reacts to triggers — a pull request, a new
|
|
13
|
+
issue, a failed test run, a deploy — runs exactly one analysis, and delivers its
|
|
14
|
+
verdict where your team works: a PR comment, a markdown report, or a Testomat.io
|
|
15
|
+
project. There is no interactive mode and no human in the loop. That is the point:
|
|
16
|
+
you wire it into CI once, and every event that should get QA thinking gets it.
|
|
29
17
|
|
|
30
|
-
|
|
31
|
-
```
|
|
18
|
+
It ships as a desktop app, a web app, and the command-line agent in this repository.
|
|
32
19
|
|
|
33
|
-
|
|
20
|
+
| Folder | What it is |
|
|
21
|
+
|---|---|
|
|
22
|
+
| `prompt/` | System-prompt fragments: the agent's role, rules, tool guidance, Testomat.io operating rules, and the report contract |
|
|
23
|
+
| `skills/` | The manifest of skills the agent can invoke. Every folder is fetched from its own upstream repository |
|
|
24
|
+
| `src/` | The `testeiya` command-line agent (Node) |
|
|
34
25
|
|
|
35
|
-
|
|
26
|
+
The desktop and web harness is not open source. That covers the servers, session management, sync, and UI.
|
|
36
27
|
|
|
37
|
-
|
|
28
|
+
## Install
|
|
38
29
|
|
|
39
|
-
|
|
40
|
-
- **run in non-interactive environments like CI** — everything is configurable through environment variables and config files, with no GUI required
|
|
30
|
+
Requires Node 22.19 or newer, an LLM provider key, and a model.
|
|
41
31
|
|
|
42
|
-
|
|
32
|
+
```bash
|
|
33
|
+
npx testeiya doctor
|
|
34
|
+
```
|
|
43
35
|
|
|
44
|
-
|
|
36
|
+
`doctor` reports which key won, which skills loaded, and whether Testomat.io is reachable — without spending a token.
|
|
45
37
|
|
|
46
|
-
|
|
38
|
+
The key comes from the environment (`OPENROUTER_API_KEY`, `ANTHROPIC_API_KEY`,
|
|
39
|
+
`OPENAI_API_KEY`, `GEMINI_API_KEY`), from `~/.testeiya/.env`, or from
|
|
40
|
+
`~/.testeiya/auth.json`. That last file is the one the desktop app's Settings
|
|
41
|
+
dialog writes, so configuring it once covers both.
|
|
47
42
|
|
|
48
|
-
|
|
43
|
+
There is no default model. Name one with `--model <provider>/<id>` or
|
|
44
|
+
`TESTEIYA_MODEL`. CI usually has neither set, so a run that resolves no model
|
|
45
|
+
exits `2` rather than picking one for you.
|
|
49
46
|
|
|
50
47
|
```bash
|
|
51
|
-
|
|
48
|
+
export TESTEIYA_MODEL=openrouter/anthropic/claude-sonnet-5
|
|
49
|
+
testeiya models anthropic # list what your key can reach
|
|
52
50
|
```
|
|
53
51
|
|
|
54
|
-
|
|
52
|
+
## How it runs
|
|
55
53
|
|
|
56
54
|
```bash
|
|
57
|
-
|
|
58
|
-
testeiya
|
|
55
|
+
testeiya task "<task>" # run one task, deliver a report, exit
|
|
56
|
+
testeiya ask "<question>" # answer a question, no report
|
|
57
|
+
testeiya skills # list the skills bundled with this package
|
|
58
|
+
testeiya sessions # list saved sessions for this folder
|
|
59
59
|
```
|
|
60
60
|
|
|
61
|
-
|
|
61
|
+
The agent runs one task and exits. Progress goes to stderr, so a run drops into CI as-is:
|
|
62
|
+
|
|
63
|
+
| Exit code | Meaning |
|
|
64
|
+
|---|---|
|
|
65
|
+
| `0` | pass, or a positive verdict from the agent |
|
|
66
|
+
| `1` | failed run, or a negative verdict — findings that need a human |
|
|
67
|
+
| `2` | bad usage |
|
|
68
|
+
| `130` | interrupted |
|
|
62
69
|
|
|
63
|
-
|
|
70
|
+
Pass `--exit-zero` when a negative verdict must not fail the job. A broken run
|
|
71
|
+
still exits `1`, bad usage still exits `2`, and the verdict is still in the
|
|
72
|
+
report and in the run envelope for anything that wants to gate on it.
|
|
64
73
|
|
|
65
|
-
|
|
66
|
-
- "Find untested code paths in src/controllers"
|
|
67
|
-
- "Generate test cases for the auth module"
|
|
68
|
-
- "Find duplicate test cases and suggest which to remove"
|
|
74
|
+
A task can come from stdin too:
|
|
69
75
|
|
|
70
|
-
|
|
76
|
+
```bash
|
|
77
|
+
cat issue-42.md | testeiya task --output report.md
|
|
78
|
+
```
|
|
71
79
|
|
|
72
|
-
|
|
80
|
+
Every command takes `--json` for machine-readable output of the run envelope
|
|
81
|
+
(verdict, reason, tokens, session id). `testeiya --help` lists all options;
|
|
82
|
+
`testeiya help` is the full guide.
|
|
73
83
|
|
|
74
|
-
|
|
84
|
+
## Scenarios
|
|
75
85
|
|
|
76
|
-
|
|
86
|
+
Each scenario below is a single unattended run. Nobody answers questions; the
|
|
87
|
+
agent reads what the trigger gives it and acts on its own judgement. If its
|
|
88
|
+
verdict is negative, the job fails — that failure *is* the signal.
|
|
77
89
|
|
|
78
|
-
|
|
90
|
+
### Grill a pull request
|
|
79
91
|
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
| OpenAI | `openai` | `OPENAI_API_KEY` |
|
|
84
|
-
| Anthropic | `anthropic` | `ANTHROPIC_API_KEY` |
|
|
85
|
-
| Google Gemini | `google` | `GEMINI_API_KEY` |
|
|
86
|
-
| Groq | `groq` | `GROQ_API_KEY` |
|
|
87
|
-
| Mistral | `mistral` | `MISTRAL_API_KEY` |
|
|
88
|
-
| xAI | `xai` | `XAI_API_KEY` |
|
|
89
|
-
| Azure OpenAI | `azure-openai-responses` | `AZURE_OPENAI_API_KEY` + `AZURE_OPENAI_BASE_URL` |
|
|
90
|
-
| Ollama (local) | `ollama` | not required |
|
|
92
|
+
Every PR gets a QA review before merge. The agent loads the branch's diff,
|
|
93
|
+
analyzes it through the `qa-thinking` skill — edge cases, negative flows,
|
|
94
|
+
abuses, data-consistency risks — and posts the findings as a PR comment:
|
|
91
95
|
|
|
92
|
-
|
|
96
|
+
```bash
|
|
97
|
+
git fetch origin "$PR_BRANCH"
|
|
98
|
+
git checkout "$PR_BRANCH"
|
|
99
|
+
testeiya task "Review this pull request as a QA engineer. What could go wrong?" \
|
|
100
|
+
--output gh:pr-comment --output review.json
|
|
101
|
+
```
|
|
93
102
|
|
|
94
|
-
|
|
103
|
+
Exit code `1` means the agent found real risk, so you can gate the merge on it.
|
|
95
104
|
|
|
96
|
-
###
|
|
105
|
+
### Write test cases from a pull request
|
|
97
106
|
|
|
98
|
-
|
|
107
|
+
Same trigger, different deliverable: the `qa-write-test-cases` skill turns the
|
|
108
|
+
change into test cases in Testomat.io markdown format, written as a build
|
|
109
|
+
artifact ready to commit or import:
|
|
99
110
|
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
| GitHub Copilot | `github-copilot` |
|
|
105
|
-
| Gemini (Google account) | `google-gemini-cli` |
|
|
106
|
-
| Cursor | `cursor` |
|
|
107
|
-
| Kimi Code | `kimi-code` |
|
|
111
|
+
```bash
|
|
112
|
+
testeiya task "Create test cases covering the changes in this pull request" \
|
|
113
|
+
--output testcases/
|
|
114
|
+
```
|
|
108
115
|
|
|
109
|
-
|
|
116
|
+
With `TESTOMATIO` and the project id set, add
|
|
117
|
+
`sync them to Testomat.io` to the task and they land straight in the TMS via
|
|
118
|
+
`sync-test-cases-with-tms`.
|
|
110
119
|
|
|
111
|
-
|
|
120
|
+
### Create test cases from a new issue
|
|
112
121
|
|
|
113
|
-
|
|
122
|
+
A requirements text arrives from the issue tracker — piped in, no human
|
|
123
|
+
summarizing it first. The agent writes a checklist and test cases from it:
|
|
114
124
|
|
|
115
|
-
|
|
125
|
+
```bash
|
|
126
|
+
gh issue view 57 --json title,body -q '.title + "\n\n" + .body' \
|
|
127
|
+
| testeiya task --output testcases/issue-57.md
|
|
128
|
+
```
|
|
116
129
|
|
|
117
|
-
|
|
130
|
+
### Explore a deployed app with Explorbot
|
|
118
131
|
|
|
119
|
-
|
|
132
|
+
After a deploy, point [Explorbot](https://github.com/testomatio/explorbot) — the
|
|
133
|
+
autonomous browser-testing CLI the agent drives through the `explorbot-*`
|
|
134
|
+
skills — at the staging URL. It researches, plans, and tests the live app in
|
|
135
|
+
its own browser, then reports what broke:
|
|
120
136
|
|
|
121
137
|
```bash
|
|
122
|
-
|
|
138
|
+
testeiya task "Run explorbot against https://staging.example.com, max 10 tests, report failures" \
|
|
139
|
+
--output explorbot-report.md
|
|
123
140
|
```
|
|
124
141
|
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
Testeiya ships with a curated skill library, loaded automatically:
|
|
142
|
+
### Repair failing tests after a red build
|
|
128
143
|
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
- **Testomat.io docs** — the full product documentation, bundled and searchable, so platform questions are answered from the docs
|
|
144
|
+
On a failed CodeceptJS run, the `ci-fix-tests` skill attempts safe fixes only —
|
|
145
|
+
locator drift, missing waits — reruns just the failing scenarios, rolls back any
|
|
146
|
+
edit that did not help, and always writes `output/ci-fix.md` for the next job:
|
|
133
147
|
|
|
134
|
-
|
|
148
|
+
```bash
|
|
149
|
+
testeiya task "Fix the failed CodeceptJS tests using ci-fix-tests. No refactors." \
|
|
150
|
+
--no-session
|
|
151
|
+
```
|
|
135
152
|
|
|
136
|
-
|
|
153
|
+
### Audit the test suite on a schedule
|
|
137
154
|
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
| `/connect` | Link a Testomat.io project and load its MCP tools |
|
|
141
|
-
| `/project` | Switch the active Testomat.io project |
|
|
142
|
-
| `/model` | Switch the LLM provider and model |
|
|
143
|
-
| `/login` | Sign in with an AI subscription account |
|
|
144
|
-
| `/skills` | List loaded skills |
|
|
145
|
-
| `/help` | Show all commands |
|
|
155
|
+
Nightly, the agent walks the Testomat.io project and reports gaps — suites with
|
|
156
|
+
no tests, cases gone stale against the current code, coverage holes:
|
|
146
157
|
|
|
147
|
-
|
|
158
|
+
```bash
|
|
159
|
+
TESTOMATIO=tstmt_xxx testeiya task \
|
|
160
|
+
"Audit this project: which suites have no automated tests, and which manual cases look automatable?" \
|
|
161
|
+
--output audits/$(date +%F).md
|
|
162
|
+
```
|
|
148
163
|
|
|
149
|
-
|
|
164
|
+
## CI setup
|
|
150
165
|
|
|
151
|
-
|
|
166
|
+
Testeiya needs three things in any CI system: a provider key, a model name, and
|
|
167
|
+
the checkout of whatever the task reads. Everything else is standard.
|
|
152
168
|
|
|
153
169
|
### GitHub Actions
|
|
154
170
|
|
|
171
|
+
GitHub runners ship the [GitHub CLI](https://cli.github.com), so posting back to
|
|
172
|
+
the PR is one flag. `GITHUB_TOKEN` authenticates it; keep the LLM key in
|
|
173
|
+
repository secrets.
|
|
174
|
+
|
|
155
175
|
```yaml
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
176
|
+
name: qa-review
|
|
177
|
+
on:
|
|
178
|
+
pull_request:
|
|
179
|
+
types: [opened, synchronize]
|
|
180
|
+
|
|
181
|
+
permissions:
|
|
182
|
+
pull-requests: write
|
|
183
|
+
contents: read
|
|
184
|
+
|
|
185
|
+
jobs:
|
|
186
|
+
grill:
|
|
187
|
+
runs-on: ubuntu-latest
|
|
188
|
+
env:
|
|
189
|
+
GH_TOKEN: ${{ github.token }}
|
|
190
|
+
OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
|
|
191
|
+
TESTEIYA_MODEL: openrouter/anthropic/claude-sonnet-5
|
|
192
|
+
# optional, for scenarios that touch Testomat.io
|
|
193
|
+
TESTOMATIO: ${{ secrets.TESTOMATIO }}
|
|
194
|
+
TESTOMATIO_PROJECT_ID: my-project
|
|
195
|
+
steps:
|
|
196
|
+
- uses: actions/checkout@v4
|
|
197
|
+
with:
|
|
198
|
+
fetch-depth: 0
|
|
199
|
+
- uses: actions/setup-node@v4
|
|
200
|
+
with:
|
|
201
|
+
node-version: 22
|
|
202
|
+
- run: npx testeiya@latest doctor
|
|
203
|
+
- run: |
|
|
204
|
+
npx testeiya@latest task \
|
|
205
|
+
"Review this pull request as a QA engineer. What could go wrong?" \
|
|
206
|
+
--output gh:pr-comment --output review.json
|
|
207
|
+
- uses: actions/upload-artifact@v4
|
|
208
|
+
if: always()
|
|
209
|
+
with:
|
|
210
|
+
name: review
|
|
211
|
+
path: review.json
|
|
161
212
|
```
|
|
162
213
|
|
|
214
|
+
The same shape works for the other scenarios: trigger on `issues` and pipe the
|
|
215
|
+
body in for test-case generation, or run on a `schedule` for the nightly audit.
|
|
216
|
+
|
|
163
217
|
### GitLab CI
|
|
164
218
|
|
|
219
|
+
GitLab runners do not ship `gh`, so the report goes to job artifacts — visible
|
|
220
|
+
in the merge request pipeline page. Exit code `1` fails the job and blocks the
|
|
221
|
+
merge when you mark it required.
|
|
222
|
+
|
|
165
223
|
```yaml
|
|
166
|
-
|
|
167
|
-
image:
|
|
168
|
-
|
|
169
|
-
-
|
|
224
|
+
qa-review:
|
|
225
|
+
image: node:22
|
|
226
|
+
rules:
|
|
227
|
+
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
|
|
170
228
|
variables:
|
|
171
|
-
|
|
172
|
-
|
|
229
|
+
TESTEIYA_MODEL: openrouter/anthropic/claude-sonnet-5
|
|
230
|
+
TESTOMATIO_PROJECT_ID: my-project
|
|
231
|
+
script:
|
|
232
|
+
- git fetch origin $CI_MERGE_REQUEST_TARGET_BRANCH_NAME
|
|
233
|
+
- npx testeiya@latest doctor
|
|
234
|
+
- npx testeiya@latest task
|
|
235
|
+
"Review this merge request as a QA engineer. What could go wrong?"
|
|
236
|
+
--output review/report.md --output review/envelope.json
|
|
237
|
+
artifacts:
|
|
238
|
+
when: always
|
|
239
|
+
paths:
|
|
240
|
+
- review/
|
|
241
|
+
parallel:
|
|
242
|
+
# secrets go to Settings > CI/CD > Variables:
|
|
243
|
+
# OPENROUTER_API_KEY (masked), TESTOMATIO (masked, protected)
|
|
173
244
|
```
|
|
174
245
|
|
|
175
|
-
|
|
246
|
+
For posting back to the merge request, hand the envelope's report to GitLab's
|
|
247
|
+
API in a follow-up step, or install `gh` plus a token mirror if the project is
|
|
248
|
+
also on GitHub.
|
|
176
249
|
|
|
177
|
-
|
|
178
|
-
pipelines:
|
|
179
|
-
default:
|
|
180
|
-
- step:
|
|
181
|
-
name: testeiya
|
|
182
|
-
image: oven/bun:1
|
|
183
|
-
script:
|
|
184
|
-
# ANTHROPIC_API_KEY and TESTOMATIO are set as secured repository variables
|
|
185
|
-
- bunx testeiya
|
|
186
|
-
```
|
|
187
|
-
|
|
188
|
-
### Jenkins
|
|
189
|
-
|
|
190
|
-
```groovy
|
|
191
|
-
pipeline {
|
|
192
|
-
agent { docker { image 'oven/bun:1' } }
|
|
193
|
-
environment {
|
|
194
|
-
MISTRAL_API_KEY = credentials('mistral-api-key')
|
|
195
|
-
TESTOMATIO = credentials('testomatio-token')
|
|
196
|
-
}
|
|
197
|
-
stages {
|
|
198
|
-
stage('testeiya') {
|
|
199
|
-
steps {
|
|
200
|
-
sh 'bunx testeiya'
|
|
201
|
-
}
|
|
202
|
-
}
|
|
203
|
-
}
|
|
204
|
-
}
|
|
205
|
-
```
|
|
206
|
-
|
|
207
|
-
### Azure Pipelines
|
|
250
|
+
### Anywhere else
|
|
208
251
|
|
|
209
|
-
|
|
210
|
-
|
|
211
|
-
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
252
|
+
Any scheduler that can run a container works — the contract is just stdin,
|
|
253
|
+
stdout, and an exit code:
|
|
254
|
+
|
|
255
|
+
```bash
|
|
256
|
+
echo "Audit the checkout suite for gaps" | testeiya task --output audit.md
|
|
257
|
+
case $? in
|
|
258
|
+
0) echo "clean" ;;
|
|
259
|
+
1) echo "findings need review" ;;
|
|
260
|
+
esac
|
|
261
|
+
```
|
|
262
|
+
|
|
263
|
+
## Where the report goes
|
|
264
|
+
|
|
265
|
+
`--output` names a destination and can repeat. Without one the report goes to stdout.
|
|
266
|
+
|
|
267
|
+
```bash
|
|
268
|
+
testeiya task "<task>" --output report.md --output run.json --output gh:pr-comment
|
|
269
|
+
```
|
|
270
|
+
|
|
271
|
+
| Destination | What happens |
|
|
272
|
+
|---|---|
|
|
273
|
+
| `report.md` | the agent writes the report there |
|
|
274
|
+
| `run.json` | the run envelope: verdict, reason, report, tokens, session id |
|
|
275
|
+
| `gh:pr-comment` | posted on this branch's pull request |
|
|
276
|
+
| `gh:pr#123` | posted on that pull request |
|
|
277
|
+
|
|
278
|
+
Posting uses the GitHub CLI and resolves the PR number from `GITHUB_EVENT_PATH`,
|
|
279
|
+
`GITHUB_REF`, or `gh pr view` — so inside Actions it just works. Every
|
|
280
|
+
destination is checked before the run starts, so a missing `gh` costs no tokens.
|
|
281
|
+
|
|
282
|
+
`--footer "<text>"` adds a line under the report and `--header "<text>"` adds
|
|
283
|
+
one above. Both go wherever the report goes: stdout, the file, the posted
|
|
284
|
+
comment. A footer is what turns a posted report into a conversation: tell the
|
|
285
|
+
reader how to answer, and let the workflow feed their reply back into the same
|
|
286
|
+
session.
|
|
287
|
+
|
|
288
|
+
```bash
|
|
289
|
+
testeiya task "Review this pull request" \
|
|
290
|
+
--output gh:pr-comment \
|
|
291
|
+
--footer "> You can reply to this comment by typing /testeiya"
|
|
292
|
+
```
|
|
293
|
+
|
|
294
|
+
Write no footer of your own and the report is signed:
|
|
295
|
+
|
|
296
|
+
> *🧚🏻♀️ Provided by [Testeiya QA Agent](https://testomat.ai/testeiya) & claude-sonnet-5*
|
|
297
|
+
|
|
298
|
+
Pass `--no-default-footer`, or set `TESTEIYA_NO_DEFAULT_FOOTER`, to drop it.
|
|
299
|
+
|
|
300
|
+
## Sessions
|
|
301
|
+
|
|
302
|
+
Runs are saved under `~/.testeiya`, so a follow-up picks up where the last one
|
|
303
|
+
stopped. A resumed run reuses its session's model.
|
|
304
|
+
|
|
305
|
+
```bash
|
|
306
|
+
testeiya task "Review the checkout suite" --name checkout-review --output report.md
|
|
307
|
+
testeiya task "Now write the missing cases" -c
|
|
308
|
+
testeiya sessions
|
|
309
|
+
testeiya task "<task>" --resume <id>
|
|
310
|
+
```
|
|
311
|
+
|
|
312
|
+
Pass `--name` to label a session and `--no-session` to save nothing.
|
|
313
|
+
|
|
314
|
+
In CI, `--session <label>` is the one to reach for: it continues the session
|
|
315
|
+
with that label, and starts it the first time, so a job that runs again and
|
|
316
|
+
again needs no "does it exist yet" branch. Give each thread its own label, and
|
|
317
|
+
carry `~/.testeiya` between rounds with the runner's cache. `--no-session` keeps
|
|
318
|
+
runners stateless when continuity is not wanted.
|
|
319
|
+
|
|
320
|
+
```bash
|
|
321
|
+
testeiya task "Review the new commits" --session "pr-42" --output gh:pr-comment
|
|
322
|
+
```
|
|
323
|
+
|
|
324
|
+
## Testomat.io
|
|
325
|
+
|
|
326
|
+
Set `TESTOMATIO` to a project API key. The agent can then read and write that
|
|
327
|
+
project's tests, suites, runs and plans through `check-tests` and the REST API.
|
|
328
|
+
|
|
329
|
+
```bash
|
|
330
|
+
TESTOMATIO=tstmt_xxx testeiya task --project my-project "Which suites have no tests?"
|
|
331
|
+
```
|
|
332
|
+
|
|
333
|
+
Add the project id and the agent also gets the Testomat.io MCP tools. The id
|
|
334
|
+
comes from `--project` or from `TESTOMATIO_PROJECT_ID`; the MCP server needs it,
|
|
335
|
+
because a token alone does not say which project to talk to. `TESTOMATIO_URL`
|
|
336
|
+
points at a self-hosted instance.
|
|
337
|
+
|
|
338
|
+
## Skills
|
|
339
|
+
|
|
340
|
+
A skill is a folder with a `SKILL.md`. The agent sees them all and reaches for
|
|
341
|
+
the ones a task calls for. Name one with a slash to make it certain:
|
|
342
|
+
|
|
343
|
+
```bash
|
|
344
|
+
testeiya task "Review this pull request as a QA engineer /qa-thinking"
|
|
345
|
+
```
|
|
346
|
+
|
|
347
|
+
That skill is loaded in front of the task before the run starts, so it does not
|
|
348
|
+
depend on the model deciding to open it. A name the package does not ship is
|
|
349
|
+
left as plain text, which keeps a task safe to build from someone else's words —
|
|
350
|
+
a `/word` in a pull request comment stays a word.
|
|
351
|
+
|
|
352
|
+
The set is vendored from upstream repositories and moves with every release, so
|
|
353
|
+
ask your own install rather than a list in a README:
|
|
354
|
+
|
|
355
|
+
```bash
|
|
356
|
+
testeiya skills # every bundled skill: name and what it is for
|
|
357
|
+
testeiya skills playwright # filter by name, category or description
|
|
358
|
+
testeiya skills --json # [{name, group, description}]
|
|
219
359
|
```
|
|
220
360
|
|
|
221
|
-
|
|
361
|
+
Categories today: QA process, test management, test automation, Explorbot,
|
|
362
|
+
Playwright, CodeceptJS.
|
|
363
|
+
|
|
364
|
+
Sources are declared in `skills/skills.yaml` and pinned in
|
|
365
|
+
`skills/skills.lock.json`. The vendored folders are deliberately not committed —
|
|
366
|
+
they belong to their authors, under their own licences. A clone has the manifest
|
|
367
|
+
and nothing else; `node scripts/vendor-skills.js` fills the tree. Every release
|
|
368
|
+
runs it, so the published `testeiya` package ships each skill as current on
|
|
369
|
+
release day.
|
|
370
|
+
|
|
371
|
+
`skillsOverride` in `src/session.ts` keeps only what is found under that tree,
|
|
372
|
+
so an arbitrary clone cannot hand the model its own skills. To add yours, point
|
|
373
|
+
`additionalSkillPaths` at your folder. [EXTENDING.md](EXTENDING.md) covers both hooks.
|
|
374
|
+
|
|
375
|
+
To propose a new source, add its line to `skills/skills.yaml`. See [CONTRIBUTING.md](CONTRIBUTING.md).
|
|
376
|
+
|
|
377
|
+
## Building your own agent
|
|
378
|
+
|
|
379
|
+
The CLI is a thin composition over [pi](https://pi.dev): about 1,500 lines wiring
|
|
380
|
+
the SDK to the prompt and skills here. A fork can add pi extensions and custom
|
|
381
|
+
tools, or swap the one-shot run loop for pi's full interactive TUI.
|
|
382
|
+
[EXTENDING.md](EXTENDING.md) walks through both.
|
|
383
|
+
|
|
384
|
+
## Contributing
|
|
222
385
|
|
|
223
|
-
|
|
386
|
+
Prompt wording is exactly what an outside contributor can improve, and a change
|
|
387
|
+
to it changes how the agent behaves for everyone. Read
|
|
388
|
+
[CONTRIBUTING.md](CONTRIBUTING.md) first. It covers what belongs here and what
|
|
389
|
+
belongs upstream, in the repository that owns a given skill.
|
|
224
390
|
|
|
225
|
-
|
|
391
|
+
## Issues
|
|
226
392
|
|
|
227
|
-
|
|
228
|
-
| ------------------------- | ------------ | ------- |
|
|
229
|
-
| `provider.name` | `openrouter` | LLM provider id (see [Providers](#providers)) |
|
|
230
|
-
| `provider.model` | — | Model id, as your provider names it |
|
|
231
|
-
| `provider.baseUrl` | provider's own | API endpoint override, for gateways and self-hosted OpenAI-compatible servers |
|
|
232
|
-
| `provider.contextWindow` | `200000` | Context window size the agent budgets against |
|
|
233
|
-
| `provider.maxTokens` | `16384` | Maximum output tokens per response |
|
|
234
|
-
| `thinkingLevel` | `medium` | Reasoning effort: `off`, `minimal`, `low`, `medium`, `high`, `xhigh` |
|
|
235
|
-
| `permissions.autoAllowRead` | `true` | Auto-approve read-only tools |
|
|
236
|
-
| `permissions.blockWrites` | `true` | Block edit and write tools |
|
|
237
|
-
| `permissions.blockBash` | `true` | Allow only read-only bash commands |
|
|
238
|
-
| `memoryEnabled` | `true` | Per-project agent memory |
|
|
239
|
-
| `testomatioHost` | `https://app.testomat.io` | Testomat.io base URL, for self-hosted instances |
|
|
240
|
-
| `toolGate.activeTools` | — | Per-MCP-server glob patterns of tool names to keep active; the rest load on demand |
|
|
393
|
+
This repository is also the public issue tracker for both surfaces:
|
|
241
394
|
|
|
242
|
-
|
|
395
|
+
- Testeiya Desktop app, the packaged desktop application
|
|
396
|
+
- Testeiya CLI, the command-line agent in `src/`
|
|
243
397
|
|
|
244
|
-
##
|
|
398
|
+
## Licence
|
|
245
399
|
|
|
246
|
-
|
|
400
|
+
[MIT](LICENSE).
|