testeiya 0.2.1 → 0.3.9

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (414) hide show
  1. package/LICENSE +21 -0
  2. package/README.md +41 -213
  3. package/dist/prompt/context.js +79 -0
  4. package/dist/prompt/context.js.map +1 -0
  5. package/dist/prompt/index.js +56 -0
  6. package/dist/prompt/index.js.map +1 -0
  7. package/dist/prompt/print.js +22 -0
  8. package/dist/prompt/print.js.map +1 -0
  9. package/dist/prompt/project-info.js +2 -0
  10. package/dist/prompt/project-info.js.map +1 -0
  11. package/{src/prompt/system-prompt.ts → dist/prompt/system-prompt.js} +24 -21
  12. package/dist/prompt/system-prompt.js.map +1 -0
  13. package/dist/prompt/testomatio.js +264 -0
  14. package/dist/prompt/testomatio.js.map +1 -0
  15. package/dist/prompt/tools.js +49 -0
  16. package/dist/prompt/tools.js.map +1 -0
  17. package/dist/prompt/vocab.js +8 -0
  18. package/dist/prompt/vocab.js.map +1 -0
  19. package/dist/src/args.js +72 -0
  20. package/dist/src/args.js.map +1 -0
  21. package/dist/src/cli.js +61 -0
  22. package/dist/src/cli.js.map +1 -0
  23. package/dist/src/env.js +69 -0
  24. package/dist/src/env.js.map +1 -0
  25. package/dist/src/mcp-extension.js +9 -0
  26. package/dist/src/mcp-extension.js.map +1 -0
  27. package/dist/src/mcp.js +139 -0
  28. package/dist/src/mcp.js.map +1 -0
  29. package/dist/src/model.js +66 -0
  30. package/dist/src/model.js.map +1 -0
  31. package/dist/src/result.js +41 -0
  32. package/dist/src/result.js.map +1 -0
  33. package/dist/src/run.js +199 -0
  34. package/dist/src/run.js.map +1 -0
  35. package/dist/src/session.js +77 -0
  36. package/dist/src/session.js.map +1 -0
  37. package/dist/vendor/mcp.js +14801 -0
  38. package/mcp-tools.json +2967 -0
  39. package/package.json +29 -42
  40. package/prompt/context.ts +101 -0
  41. package/prompt/index.ts +93 -0
  42. package/prompt/print.ts +23 -0
  43. package/prompt/project-info.ts +30 -0
  44. package/prompt/system-prompt.ts +173 -0
  45. package/{src/prompt → prompt}/testomatio.ts +119 -8
  46. package/{src/prompt → prompt}/tools.ts +30 -8
  47. package/prompt/vocab.ts +7 -0
  48. package/{skills.lock.json → skills/skills.lock.json} +11 -7
  49. package/skills/skills.yaml +34 -0
  50. package/assets/testeiya-logo-dark.svg +0 -16
  51. package/assets/testeiya-logo.svg +0 -15
  52. package/models.catalog.json +0 -6963
  53. package/skills/codeceptjs/ci-fix-tests/SKILL.md +0 -120
  54. package/skills/codeceptjs/codeceptjs-auth/SKILL.md +0 -158
  55. package/skills/codeceptjs/codeceptjs-exploration/SKILL.md +0 -91
  56. package/skills/codeceptjs/codeceptjs-fundamentals/SKILL.md +0 -158
  57. package/skills/codeceptjs/codeceptjs-run-analysis/SKILL.md +0 -90
  58. package/skills/codeceptjs/debugging-codeceptjs-tests/SKILL.md +0 -234
  59. package/skills/codeceptjs/migrate-codeceptjs-4/SKILL.md +0 -110
  60. package/skills/codeceptjs/migrate-cypress-to-codeceptjs/SKILL.md +0 -306
  61. package/skills/codeceptjs/migrate-protractor-to-codeceptjs/SKILL.md +0 -352
  62. package/skills/codeceptjs/migrate-selenium-java-to-codeceptjs/SKILL.md +0 -472
  63. package/skills/codeceptjs/migrate-testcafe-to-codeceptjs/SKILL.md +0 -350
  64. package/skills/codeceptjs/refactoring-codeceptjs-tests/SKILL.md +0 -81
  65. package/skills/codeceptjs/writing-codeceptjs-tests/SKILL.md +0 -206
  66. package/skills/playwright-best-practices-skill/LICENSE.md +0 -7
  67. package/skills/playwright-best-practices-skill/README.md +0 -147
  68. package/skills/playwright-best-practices-skill/SKILL.md +0 -303
  69. package/skills/playwright-best-practices-skill/advanced/authentication-flows.md +0 -360
  70. package/skills/playwright-best-practices-skill/advanced/authentication.md +0 -871
  71. package/skills/playwright-best-practices-skill/advanced/clock-mocking.md +0 -364
  72. package/skills/playwright-best-practices-skill/advanced/mobile-testing.md +0 -409
  73. package/skills/playwright-best-practices-skill/advanced/multi-context.md +0 -288
  74. package/skills/playwright-best-practices-skill/advanced/multi-user.md +0 -393
  75. package/skills/playwright-best-practices-skill/advanced/network-advanced.md +0 -452
  76. package/skills/playwright-best-practices-skill/advanced/third-party.md +0 -464
  77. package/skills/playwright-best-practices-skill/architecture/pom-vs-fixtures.md +0 -363
  78. package/skills/playwright-best-practices-skill/architecture/test-architecture.md +0 -369
  79. package/skills/playwright-best-practices-skill/architecture/when-to-mock.md +0 -383
  80. package/skills/playwright-best-practices-skill/browser-apis/browser-apis.md +0 -391
  81. package/skills/playwright-best-practices-skill/browser-apis/iframes.md +0 -403
  82. package/skills/playwright-best-practices-skill/browser-apis/service-workers.md +0 -504
  83. package/skills/playwright-best-practices-skill/browser-apis/websockets.md +0 -403
  84. package/skills/playwright-best-practices-skill/core/annotations.md +0 -424
  85. package/skills/playwright-best-practices-skill/core/assertions-waiting.md +0 -361
  86. package/skills/playwright-best-practices-skill/core/configuration.md +0 -452
  87. package/skills/playwright-best-practices-skill/core/fixtures-hooks.md +0 -417
  88. package/skills/playwright-best-practices-skill/core/global-setup.md +0 -434
  89. package/skills/playwright-best-practices-skill/core/locators.md +0 -242
  90. package/skills/playwright-best-practices-skill/core/page-object-model.md +0 -315
  91. package/skills/playwright-best-practices-skill/core/projects-dependencies.md +0 -453
  92. package/skills/playwright-best-practices-skill/core/test-data.md +0 -492
  93. package/skills/playwright-best-practices-skill/core/test-suite-structure.md +0 -361
  94. package/skills/playwright-best-practices-skill/core/test-tags.md +0 -298
  95. package/skills/playwright-best-practices-skill/debugging/console-errors.md +0 -420
  96. package/skills/playwright-best-practices-skill/debugging/debugging.md +0 -504
  97. package/skills/playwright-best-practices-skill/debugging/error-testing.md +0 -360
  98. package/skills/playwright-best-practices-skill/debugging/flaky-tests.md +0 -496
  99. package/skills/playwright-best-practices-skill/frameworks/angular.md +0 -530
  100. package/skills/playwright-best-practices-skill/frameworks/nextjs.md +0 -469
  101. package/skills/playwright-best-practices-skill/frameworks/react.md +0 -531
  102. package/skills/playwright-best-practices-skill/frameworks/vue.md +0 -574
  103. package/skills/playwright-best-practices-skill/infrastructure-ci-cd/ci-cd.md +0 -468
  104. package/skills/playwright-best-practices-skill/infrastructure-ci-cd/docker.md +0 -283
  105. package/skills/playwright-best-practices-skill/infrastructure-ci-cd/github-actions.md +0 -546
  106. package/skills/playwright-best-practices-skill/infrastructure-ci-cd/gitlab.md +0 -397
  107. package/skills/playwright-best-practices-skill/infrastructure-ci-cd/other-providers.md +0 -521
  108. package/skills/playwright-best-practices-skill/infrastructure-ci-cd/parallel-sharding.md +0 -371
  109. package/skills/playwright-best-practices-skill/infrastructure-ci-cd/performance.md +0 -453
  110. package/skills/playwright-best-practices-skill/infrastructure-ci-cd/reporting.md +0 -424
  111. package/skills/playwright-best-practices-skill/infrastructure-ci-cd/test-coverage.md +0 -497
  112. package/skills/playwright-best-practices-skill/testing-patterns/accessibility.md +0 -359
  113. package/skills/playwright-best-practices-skill/testing-patterns/api-testing.md +0 -719
  114. package/skills/playwright-best-practices-skill/testing-patterns/browser-extensions.md +0 -506
  115. package/skills/playwright-best-practices-skill/testing-patterns/canvas-webgl.md +0 -493
  116. package/skills/playwright-best-practices-skill/testing-patterns/component-testing.md +0 -500
  117. package/skills/playwright-best-practices-skill/testing-patterns/drag-drop.md +0 -576
  118. package/skills/playwright-best-practices-skill/testing-patterns/electron.md +0 -509
  119. package/skills/playwright-best-practices-skill/testing-patterns/file-operations.md +0 -377
  120. package/skills/playwright-best-practices-skill/testing-patterns/file-upload-download.md +0 -562
  121. package/skills/playwright-best-practices-skill/testing-patterns/forms-validation.md +0 -561
  122. package/skills/playwright-best-practices-skill/testing-patterns/graphql-testing.md +0 -331
  123. package/skills/playwright-best-practices-skill/testing-patterns/i18n.md +0 -508
  124. package/skills/playwright-best-practices-skill/testing-patterns/performance-testing.md +0 -476
  125. package/skills/playwright-best-practices-skill/testing-patterns/security-testing.md +0 -430
  126. package/skills/playwright-best-practices-skill/testing-patterns/visual-regression.md +0 -634
  127. package/skills/playwright-cli/SKILL.md +0 -420
  128. package/skills/playwright-cli/references/element-attributes.md +0 -23
  129. package/skills/playwright-cli/references/playwright-tests.md +0 -39
  130. package/skills/playwright-cli/references/request-mocking.md +0 -87
  131. package/skills/playwright-cli/references/running-code.md +0 -241
  132. package/skills/playwright-cli/references/session-management.md +0 -225
  133. package/skills/playwright-cli/references/storage-state.md +0 -275
  134. package/skills/playwright-cli/references/test-generation.md +0 -433
  135. package/skills/playwright-cli/references/tracing.md +0 -139
  136. package/skills/playwright-cli/references/video-recording.md +0 -143
  137. package/skills/testeiya/README.md +0 -29
  138. package/skills/testeiya/answer-formatting/SKILL.md +0 -30
  139. package/skills/testeiya/check-cucumber/SKILL.md +0 -200
  140. package/skills/testeiya/check-tests/SKILL.md +0 -260
  141. package/skills/testeiya/manual-run-assistant/SKILL.md +0 -61
  142. package/skills/testeiya/testomatio-docs/INDEX.md +0 -175
  143. package/skills/testeiya/testomatio-docs/SKILL.md +0 -102
  144. package/skills/testeiya/testomatio-docs/docs/advanced/ai-powered-features/ai-agents.md +0 -144
  145. package/skills/testeiya/testomatio-docs/docs/advanced/ai-powered-features/ai-powered-features.md +0 -531
  146. package/skills/testeiya/testomatio-docs/docs/advanced/ai-powered-features/ai-requirements.md +0 -437
  147. package/skills/testeiya/testomatio-docs/docs/advanced/api-access/index.md +0 -191
  148. package/skills/testeiya/testomatio-docs/docs/advanced/branches-folder/index.md +0 -277
  149. package/skills/testeiya/testomatio-docs/docs/advanced/bulk-edit-folder/bulk-edit-demos.mdx +0 -39
  150. package/skills/testeiya/testomatio-docs/docs/advanced/bulk-edit-folder/bulk-edit-on-suite-and-test-level.md +0 -122
  151. package/skills/testeiya/testomatio-docs/docs/advanced/bulk-edit-folder/what-is-bulk-edit.md +0 -170
  152. package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/branches.md +0 -53
  153. package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/index.mdx +0 -98
  154. package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/jira-issue.md +0 -151
  155. package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/runs-and-reports.md +0 -377
  156. package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/work-in-jira.md +0 -267
  157. package/skills/testeiya/testomatio-docs/docs/advanced/living-doc/index.md +0 -138
  158. package/skills/testeiya/testomatio-docs/docs/advanced/milestones/index.md +0 -220
  159. package/skills/testeiya/testomatio-docs/docs/advanced/shortcuts/index.md +0 -48
  160. package/skills/testeiya/testomatio-docs/docs/advanced/tags-labels/labels-and-custom-fields.md +0 -201
  161. package/skills/testeiya/testomatio-docs/docs/advanced/tags-labels/tags-or-labels.md +0 -94
  162. package/skills/testeiya/testomatio-docs/docs/advanced/tags-labels/tags.md +0 -243
  163. package/skills/testeiya/testomatio-docs/docs/advanced/test-artifacts/index.md +0 -135
  164. package/skills/testeiya/testomatio-docs/docs/advanced/tql/index.md +0 -394
  165. package/skills/testeiya/testomatio-docs/docs/getting-started/index.md +0 -290
  166. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/azure.md +0 -96
  167. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/bamboo.md +0 -94
  168. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/bitbucket.md +0 -106
  169. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/circle.md +0 -115
  170. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/github.md +0 -120
  171. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/gitlab.md +0 -88
  172. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/index.mdx +0 -151
  173. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/jenkins.md +0 -121
  174. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/teamcity.md +0 -96
  175. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/azure.md +0 -31
  176. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/clickup.md +0 -54
  177. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/confluence.md +0 -34
  178. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/github.md +0 -31
  179. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/gitlab.md +0 -93
  180. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/index.md +0 -131
  181. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/jira.md +0 -164
  182. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/linear.md +0 -29
  183. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/shortcut.md +0 -67
  184. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/youtrack.md +0 -94
  185. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/azure-devops.md +0 -53
  186. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/email.md +0 -71
  187. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/jira.md +0 -52
  188. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/ms-teams.md +0 -60
  189. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/rules.md +0 -267
  190. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/slack.md +0 -82
  191. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/telegram.md +0 -73
  192. package/skills/testeiya/testomatio-docs/docs/integrations/scim-provisioning/index.mdx +0 -207
  193. package/skills/testeiya/testomatio-docs/docs/integrations/scim-provisioning/okta.md +0 -204
  194. package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/azure.md +0 -95
  195. package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/google.md +0 -75
  196. package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/index.mdx +0 -33
  197. package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/okta.md +0 -108
  198. package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/saml.md +0 -66
  199. package/skills/testeiya/testomatio-docs/docs/legal/compliance/gdpr.md +0 -95
  200. package/skills/testeiya/testomatio-docs/docs/legal/compliance/index.mdx +0 -94
  201. package/skills/testeiya/testomatio-docs/docs/legal/compliance/privacy.md +0 -26
  202. package/skills/testeiya/testomatio-docs/docs/legal/compliance/terms.md +0 -32
  203. package/skills/testeiya/testomatio-docs/docs/legal/security/index.mdx +0 -242
  204. package/skills/testeiya/testomatio-docs/docs/legal/security/jira.md +0 -140
  205. package/skills/testeiya/testomatio-docs/docs/legal/security/subprocessors.md +0 -76
  206. package/skills/testeiya/testomatio-docs/docs/management/company/administration.md +0 -363
  207. package/skills/testeiya/testomatio-docs/docs/management/company/audit-log.md +0 -41
  208. package/skills/testeiya/testomatio-docs/docs/management/company/plan-features-comparison.md +0 -240
  209. package/skills/testeiya/testomatio-docs/docs/management/company/subscriptions.md +0 -326
  210. package/skills/testeiya/testomatio-docs/docs/management/company/trials.md +0 -75
  211. package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/accountant.md +0 -49
  212. package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/index.mdx +0 -285
  213. package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/read-only-user.md +0 -67
  214. package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/teams.md +0 -138
  215. package/skills/testeiya/testomatio-docs/docs/management/project/settings.md +0 -87
  216. package/skills/testeiya/testomatio-docs/docs/management/project/templates.md +0 -546
  217. package/skills/testeiya/testomatio-docs/docs/project/analytics/index.md +0 -326
  218. package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/classical-tests-markdown-format.md +0 -386
  219. package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/download-manual-tests-as-files.md +0 -378
  220. package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/export-to-obsidian.md +0 -112
  221. package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/export-to-spreadsheet.md +0 -127
  222. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/auto-import.md +0 -63
  223. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-Allure-TestOps.md +0 -49
  224. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-QTest.md +0 -41
  225. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-Qase.md +0 -39
  226. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-Zephyr.md +0 -37
  227. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-csv-xlsx.md +0 -117
  228. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-cucumber.md +0 -93
  229. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-qmetry.md +0 -44
  230. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-source-code.md +0 -117
  231. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-testcaselabs.md +0 -44
  232. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-testmo.md +0 -47
  233. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-testrail.md +0 -82
  234. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-xray.md +0 -55
  235. package/skills/testeiya/testomatio-docs/docs/project/plans/index.md +0 -334
  236. package/skills/testeiya/testomatio-docs/docs/project/pulse/index.md +0 -104
  237. package/skills/testeiya/testomatio-docs/docs/project/runs/archive-runs-and-groups.md +0 -176
  238. package/skills/testeiya/testomatio-docs/docs/project/runs/custom-statuses.md +0 -195
  239. package/skills/testeiya/testomatio-docs/docs/project/runs/environments.md +0 -203
  240. package/skills/testeiya/testomatio-docs/docs/project/runs/managing-runs.md +0 -455
  241. package/skills/testeiya/testomatio-docs/docs/project/runs/merge-strategies.md +0 -137
  242. package/skills/testeiya/testomatio-docs/docs/project/runs/reports.md +0 -329
  243. package/skills/testeiya/testomatio-docs/docs/project/runs/rungroups.md +0 -342
  244. package/skills/testeiya/testomatio-docs/docs/project/runs/running-automated-tests.md +0 -256
  245. package/skills/testeiya/testomatio-docs/docs/project/runs/running-manual-and-automated-tests.md +0 -331
  246. package/skills/testeiya/testomatio-docs/docs/project/runs/running-tests-manually.md +0 -633
  247. package/skills/testeiya/testomatio-docs/docs/project/runs/temporary-tests-notes.md +0 -223
  248. package/skills/testeiya/testomatio-docs/docs/project/steps-snippets/snippets.md +0 -68
  249. package/skills/testeiya/testomatio-docs/docs/project/steps-snippets/steps.md +0 -181
  250. package/skills/testeiya/testomatio-docs/docs/project/tests/bdd-test-case-editor.md +0 -98
  251. package/skills/testeiya/testomatio-docs/docs/project/tests/classical-test-case-editor.md +0 -354
  252. package/skills/testeiya/testomatio-docs/docs/project/tests/classical-vs-bdd.md +0 -169
  253. package/skills/testeiya/testomatio-docs/docs/project/tests/copy-and-move-your-tests.md +0 -156
  254. package/skills/testeiya/testomatio-docs/docs/project/tests/other-features-for-test-case-design.md +0 -412
  255. package/skills/testeiya/testomatio-docs/docs/project/tests/test-case-creation-and-editing.md +0 -527
  256. package/skills/testeiya/testomatio-docs/docs/support/index.md +0 -75
  257. package/skills/testeiya/testomatio-docs/docs/test-reporting/artifacts.md +0 -388
  258. package/skills/testeiya/testomatio-docs/docs/test-reporting/cli.md +0 -215
  259. package/skills/testeiya/testomatio-docs/docs/test-reporting/configuration.md +0 -247
  260. package/skills/testeiya/testomatio-docs/docs/test-reporting/frameworks.md +0 -490
  261. package/skills/testeiya/testomatio-docs/docs/test-reporting/functions.md +0 -144
  262. package/skills/testeiya/testomatio-docs/docs/test-reporting/index.mdx +0 -48
  263. package/skills/testeiya/testomatio-docs/docs/test-reporting/junit.md +0 -248
  264. package/skills/testeiya/testomatio-docs/docs/test-reporting/logger.md +0 -138
  265. package/skills/testeiya/testomatio-docs/docs/test-reporting/php.md +0 -77
  266. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/bitbucket.md +0 -72
  267. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/csv.md +0 -23
  268. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/debug.md +0 -18
  269. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/github.md +0 -63
  270. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/gitlab.md +0 -29
  271. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/html.md +0 -80
  272. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/index.mdx +0 -57
  273. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/testomatio.md +0 -331
  274. package/skills/testeiya/testomatio-docs/docs/test-reporting/python.md +0 -229
  275. package/skills/testeiya/testomatio-docs/docs/test-reporting/workflows.md +0 -325
  276. package/skills/testeiya/testomatio-docs/docs/tutorials/Interacting-via-API.md +0 -159
  277. package/skills/testeiya/testomatio-docs/docs/tutorials/Manual-Testing-Classic.md +0 -164
  278. package/skills/testeiya/testomatio-docs/docs/tutorials/Set-Up-Read-Only-Access.md +0 -85
  279. package/skills/testeiya/testomatio-docs/docs/tutorials/Using-AI-in-Testing.md +0 -163
  280. package/skills/testeiya/testomatio-docs/docs/tutorials/Working-with-Jira.md +0 -125
  281. package/skills/testeiya/testomatio-docs/docs/tutorials/java.md +0 -478
  282. package/skills/testeiya/testomatio-docs/docs/tutorials/playwright.md +0 -207
  283. package/skills/testeiya/testomatio-docs/docs/tutorials/webdriver.md +0 -218
  284. package/skills/testeiya/testomatio-docs/docs.lock.json +0 -6
  285. package/skills/testeiya/testomatio-reporter/SKILL.md +0 -307
  286. package/skills/testomatio/explorbot/explorbot-fundamentals/SKILL.md +0 -88
  287. package/skills/testomatio/explorbot/explorbot-plan/SKILL.md +0 -95
  288. package/skills/testomatio/explorbot/explorbot-setup/SKILL.md +0 -334
  289. package/skills/testomatio/test-automation/automate-manual-test-cases/CLAUDE.md +0 -56
  290. package/skills/testomatio/test-automation/automate-manual-test-cases/SKILL.md +0 -242
  291. package/skills/testomatio/test-automation/automate-manual-test-cases/references/CODECEPTJS_BEST_PRACTICES.md +0 -182
  292. package/skills/testomatio/test-automation/automate-manual-test-cases/references/FINAL_SUMMARY_TEMPLATE.md +0 -22
  293. package/skills/testomatio/test-automation/automate-manual-test-cases/references/PLAYWRIGHT_BEST_PRACTICES.md +0 -90
  294. package/skills/testomatio/test-automation/automate-manual-test-cases/references/POM_BEST_PRACTICES.md +0 -53
  295. package/skills/testomatio/test-automation/automate-manual-test-cases/references/TEST_DATA_MANAGEMENT.md +0 -52
  296. package/skills/testomatio/test-automation/debug-fix-failed-flaky-autotests/SKILL.md +0 -107
  297. package/skills/testomatio/test-automation/debug-fix-failed-flaky-autotests/references/DEBUGGING_QUICK_REFERENCE.md +0 -113
  298. package/skills/testomatio/test-automation/qa-automation-test-consolidation/SKILL.md +0 -29
  299. package/skills/testomatio/test-automation/run-tests-with-testomatio-reporter/SKILL.md +0 -151
  300. package/skills/testomatio/test-automation/setup-pr-testing/SKILL.md +0 -199
  301. package/skills/testomatio/test-automation/testomat-allure-adapter/SKILL.md +0 -289
  302. package/skills/testomatio/test-management/detect-duplicate-test-cases/SKILL.md +0 -86
  303. package/skills/testomatio/test-management/detect-duplicate-test-cases/references/DUPLICATE_INSTRUCTIONS.md +0 -54
  304. package/skills/testomatio/test-management/improve-test-cases/SKILL.md +0 -114
  305. package/skills/testomatio/test-management/improve-test-cases/references/TESTOMAT_MARKDOWN_EXAMPLE.md +0 -66
  306. package/skills/testomatio/test-management/pull-request-diff-analyzer/SKILL.md +0 -141
  307. package/skills/testomatio/test-management/qa-e2e-tests-reporting/SKILL.md +0 -354
  308. package/skills/testomatio/test-management/qa-e2e-tests-reporting/references/TESTOMATIO_ARTIFACTS.md +0 -82
  309. package/skills/testomatio/test-management/qa-e2e-tests-reporting/references/TESTOMATIO_HTML_REPORT.md +0 -37
  310. package/skills/testomatio/test-management/qa-e2e-tests-reporting/references/TESTOMATIO_REPORTERS_CONFIG.md +0 -118
  311. package/skills/testomatio/test-management/qa-pr-requirements-analyzer/SKILL.md +0 -149
  312. package/skills/testomatio/test-management/qa-pr-requirements-analyzer/references/summary-example.md +0 -54
  313. package/skills/testomatio/test-management/qa-requirement-reviewer/SKILL.md +0 -126
  314. package/skills/testomatio/test-management/qa-requirement-reviewer/references/requirements_reviewer_examples.md +0 -128
  315. package/skills/testomatio/test-management/qa-test-code-coverage/SKILL.md +0 -174
  316. package/skills/testomatio/test-management/qa-test-code-coverage/references/COVERAGE_FILE_FORMAT.md +0 -138
  317. package/skills/testomatio/test-management/qa-test-code-coverage/references/E2E_FRAMEWORKS.md +0 -65
  318. package/skills/testomatio/test-management/qa-test-code-coverage/scripts/check-coverage.mjs +0 -40
  319. package/skills/testomatio/test-management/qa-thinking/SKILL.md +0 -36
  320. package/skills/testomatio/test-management/qa-write-test-cases/SKILL.md +0 -267
  321. package/skills/testomatio/test-management/qa-write-test-cases/references/test-case-format.md +0 -387
  322. package/skills/testomatio/test-management/qa-write-test-cases/references/testomat-tms-guide.md +0 -72
  323. package/skills/testomatio/test-management/qa-write-test-cases/references/writing-rule.md +0 -159
  324. package/skills/testomatio/test-management/scan-automation-project/SKILL.md +0 -152
  325. package/skills/testomatio/test-management/sync-test-cases-with-tms/SKILL.md +0 -107
  326. package/skills/testomatio/test-management/sync-test-cases-with-tms/references/TESTOMATIO_CLI.md +0 -140
  327. package/skills/testomatio/test-management/testomatio-mcp/SKILL.md +0 -182
  328. package/skills/testomatio/test-management/testomatio-mcp/references/MCP_SETUP.md +0 -169
  329. package/skills.yaml +0 -21
  330. package/src/ai-debug.ts +0 -188
  331. package/src/api/agent-get.ts +0 -22
  332. package/src/api/agent-start.ts +0 -384
  333. package/src/api/client-log.ts +0 -29
  334. package/src/api/context.ts +0 -332
  335. package/src/api/debug-layout.ts +0 -47
  336. package/src/api/debug-report.ts +0 -29
  337. package/src/api/debug-snapshot.ts +0 -30
  338. package/src/api/debug-stream.ts +0 -52
  339. package/src/api/files-delete.ts +0 -184
  340. package/src/api/files-read.ts +0 -41
  341. package/src/api/files-rename.ts +0 -99
  342. package/src/api/files-tree.ts +0 -292
  343. package/src/api/files-write.ts +0 -41
  344. package/src/api/mcp.ts +0 -603
  345. package/src/api/memory.ts +0 -69
  346. package/src/api/open-external.ts +0 -39
  347. package/src/api/playwright-cli.ts +0 -606
  348. package/src/api/providers.ts +0 -457
  349. package/src/api/read-env-token.ts +0 -33
  350. package/src/api/sessions.ts +0 -79
  351. package/src/api/settings.ts +0 -120
  352. package/src/api/skills.ts +0 -58
  353. package/src/api/testomatio-attachment.ts +0 -89
  354. package/src/api/testomatio-auth.ts +0 -284
  355. package/src/api/testomatio-proxy.ts +0 -197
  356. package/src/api/testomatio-run-stats.ts +0 -81
  357. package/src/api/testomatio-target.ts +0 -225
  358. package/src/api/testomatio-transcription.ts +0 -82
  359. package/src/api/workspace-search.ts +0 -96
  360. package/src/api/workspace-sync.ts +0 -151
  361. package/src/api/workspace.ts +0 -165
  362. package/src/app-server.ts +0 -556
  363. package/src/bridge.ts +0 -251
  364. package/src/check-tests.ts +0 -156
  365. package/src/cli.ts +0 -17
  366. package/src/commands.ts +0 -244
  367. package/src/config.ts +0 -183
  368. package/src/connection.ts +0 -658
  369. package/src/context-store.ts +0 -265
  370. package/src/debug-bus.ts +0 -405
  371. package/src/extensions/tool-gate.ts +0 -220
  372. package/src/extensions/webui/ask-channel.ts +0 -52
  373. package/src/extensions/webui/index.ts +0 -184
  374. package/src/extensions/webui/tools/ask-question.ts +0 -88
  375. package/src/extensions/webui/tools/query-result.ts +0 -79
  376. package/src/extensions/webui/tools/render-chart.ts +0 -53
  377. package/src/extensions/webui/tools/render-item.ts +0 -61
  378. package/src/extensions/webui/tools/render-list.ts +0 -170
  379. package/src/extensions/webui/tools/render-result.ts +0 -85
  380. package/src/extensions/webui/tools/render-tree.ts +0 -71
  381. package/src/extensions/webui/tools/ui-widget.ts +0 -54
  382. package/src/extensions/webui/tools/widget-result.ts +0 -13
  383. package/src/extensions/webui/widget-channel.ts +0 -50
  384. package/src/file-log.ts +0 -196
  385. package/src/git-tracked.ts +0 -68
  386. package/src/gitignore.ts +0 -53
  387. package/src/json-store.ts +0 -43
  388. package/src/load-env.ts +0 -70
  389. package/src/loader-style.ts +0 -96
  390. package/src/main.ts +0 -140
  391. package/src/mcp-catalog.ts +0 -96
  392. package/src/models-catalog.ts +0 -72
  393. package/src/permissions.ts +0 -120
  394. package/src/project-dir.ts +0 -189
  395. package/src/project-info.ts +0 -108
  396. package/src/prompt/app-ui.ts +0 -59
  397. package/src/prompt/browser.ts +0 -27
  398. package/src/prompt/index.ts +0 -62
  399. package/src/server-info.ts +0 -62
  400. package/src/session-factory.ts +0 -442
  401. package/src/session-store.ts +0 -115
  402. package/src/shell-env.ts +0 -45
  403. package/src/skills.ts +0 -163
  404. package/src/sync-snapshot.ts +0 -140
  405. package/src/telemetry.ts +0 -239
  406. package/src/testomatio-auth.ts +0 -299
  407. package/src/testomatio.ts +0 -98
  408. package/src/theme.ts +0 -30
  409. package/src/user-env.ts +0 -94
  410. package/src/welcome.ts +0 -194
  411. package/src/workspace/safe-path.ts +0 -85
  412. package/src/workspace/test-md.ts +0 -70
  413. package/src/workspace-model.ts +0 -440
  414. package/testeiya.config.json +0 -14
@@ -1,80 +0,0 @@
1
- ---
2
- title: HTML
3
- description: Discover how to use Testomat.io's HTML Pipe to generate detailed HTML reports from your test runs. Learn how to configure environment variables to save reports, customize output paths, and view essential test data like execution time and status.
4
- type: article
5
- url: https://docs.testomat.io/test-reporting/pipes/html
6
- head:
7
- - tag: meta
8
- attrs:
9
- name: keywords
10
- content: Testomat.io, HTML reports, test reporting, test automation, environment variables, CI/CD, CodeceptJS, test result visualization, report generation
11
- ---
12
- ## Testomat.io HTML Pipe
13
-
14
- The Testomat.io HTML Pipe is a feature that allows you to generate and save HTML reports based on your test results. This functionality is part of the Testomat.io testing ecosystem.
15
-
16
- ### Prerequisites
17
-
18
- **🔌 TESTOMATIO users now have the option to generate an HTML report based on their test execution results. If a user hasn't SET `TESTOMATIO` environment, the report will still be generated**
19
-
20
- ```
21
- TESTOMATIO_HTML_REPORT_SAVE=1 <actual run command>
22
- ```
23
-
24
- ### Enabling HTML Reports
25
-
26
- To enable HTML reports, set the TESTOMATIO_HTML_REPORT_SAVE environment variable to 1. Additionally, you can customize the HTML report folder and filename using the following variables:
27
-
28
- - TESTOMATIO_HTML_REPORT_SAVE=1
29
- - TESTOMATIO_HTML_REPORT_FOLDER: Specify the folder for HTML reports
30
- - TESTOMATIO_HTML_FILENAME: Set the desired filename for the HTML report
31
-
32
- _!!!Please note that the name must include the extension ".html". If the extension is missing, the report will be saved with the default name = testomatio-report.html_
33
-
34
- ### Usage
35
-
36
- The HTML Pipe operates as part of the test execution process. When running your tests with the specified environment variables, the HTML Pipe will generate reports based on the test results.
37
-
38
- ### Report Generation
39
-
40
- Once the test run is complete, the HTML Pipe compiles the test results and generates an HTML report. The generated report is saved to the specified HTML output path.
41
-
42
- The HTML report includes essential information, such as run ID, status, parallel execution indication, run URL, execution time, execution date, and details of individual tests.
43
-
44
- #### Example Command
45
-
46
- 📊 Generate a report without triggering the TESTOMATIO pipe (no data sent to the client)
47
-
48
- _Execute the following command to generate a report without sending data to TESTOMATIO:_
49
-
50
- ```
51
- TESTOMATIO_HTML_REPORT_SAVE=1 npx codeceptjs run
52
-
53
- ```
54
-
55
- 📝 Generate a report based on the default folder/html-file name: This command triggers the test execution process and enables the HTML Pipe to generate reports based on the test results:
56
-
57
- - Output folder: html-report/
58
- - Output file: testomatio-report.html
59
-
60
- _Execute the following command to generate a report with default settings:_
61
-
62
- ```
63
- TESTOMATIO_HTML_REPORT_SAVE=1 TESTOMATIO={API_KEY} npx codeceptjs run
64
-
65
- ```
66
-
67
- 📂 Generate a report based on a custom HTML folder/html-file name.
68
- **!!!You can SET only one of the following parameters if needed: folder or file name**
69
-
70
- _Execute the following command to generate a report with both folder & file names:_
71
-
72
- ```
73
- TESTOMATIO_HTML_REPORT_SAVE=1 TESTOMATIO_HTML_REPORT_FOLDER=user-html-reporter TESTOMATIO_HTML_FILENAME=custom.html TESTOMATIO={API_KEY} npx codeceptjs run
74
-
75
- ```
76
-
77
- 🖥️ An example of a report can be seen in the screenshot below:
78
-
79
- ![HTML example](./images/html-pipe.png)
80
-
@@ -1,57 +0,0 @@
1
- ---
2
- title: Pipes
3
- description: Testomat.io's Pipes feature allows you to forward test data from your test framework to other services like GitHub, GitLab, or CSV reports. Pipes are configured via environment variables and can be customized to send data to analytics, databases, or notifications. You can also create custom pipes for specific needs, ensuring compatibility across different test frameworks.
4
- type: article
5
- url: https://docs.testomat.io/test-reporting/pipes/overview
6
- head:
7
- - tag: meta
8
- attrs:
9
- name: keywords
10
- content: Testomat.io, Pipes, test reporting, GitHub integration, GitLab, CSV reports, CI pipelines, custom pipes, environment variables, test frameworks
11
- ---
12
-
13
- import DirectoryLinks from '/src/components/DirectoryLinks.astro';
14
-
15
- <DirectoryLinks directory="test-reporting/pipes" />
16
-
17
- ## 📯 Pipes
18
-
19
- ### What are Pipes?
20
-
21
- Unlike other reporters, Testomat.io Reporter doesn't force you to report data only to Testomat.io application. Testomat.io Reporter collects data from test frameworks and submits it via pipes to other services.
22
-
23
- ![](./images/reporter-pipes.png)
24
-
25
- For instance, you can enable GitHub Pipe to create a comment on a GitHub Pull Request and CSV Export Pipe to write a report in CSV format. To report data to Testomat.io App, Testomat.io Pipe should be enabled, but not required if you don't plan to use Testomat.io App. You can also write your own pipe to:
26
-
27
- - 📈 send data to analytics storage
28
- - 🗄 store data into database
29
- - 📧 send emails
30
- - 💬 send notifications to chats
31
- - etc....
32
-
33
- Pipes work in the very same manner for all supported test frameworks, as well as for JUnit reports. So writing a pipe instead of a custom reporter for a particular framework, saves your time as you can be sure, this pipe will work with any attached test result.
34
-
35
- Testomat.io Reporter doesn't store test data in any internal format. Test data is processed in memory. While a test is running, all pipes are notified of changes.
36
-
37
- Pipes Concepts:
38
-
39
- - **Pipes are configured via environment variables** - each test framework has its own configuration (or not). To ensure that a pipe receives its configuration it is independent from a running test framework.
40
- - **Pipes are enabled if a corresponding environment variables are set** - for instance, if `TESTOMATIO` variable is present, Testomat.io Pipe is enabled, if `GH_PAT` variable is set, GitHub Pipe is enabled.
41
- - **Custom Pipes** can be created as modules and added to `package.json` file.
42
- - When used locally environment variables are expected to be set in `.env` file. When running on CI server use Environment Configuration a CI server provides to pass environment variables into the pipeline.
43
-
44
- ### Built-in Pipes
45
-
46
- - [Testomat.io](https://docs.testomat.io/test-reporting/pipes/testomatio)
47
- - [GitHub](https://docs.testomat.io/test-reporting/pipes/github)
48
- - [Gitlab](https://docs.testomat.io/test-reporting/pipes/gitlab)
49
- - [CSV](https://docs.testomat.io/test-reporting/pipes/csv)
50
- - [Bitbucket](https://docs.testomat.io/test-reporting/pipes/bitbucket)
51
- - [Debugging](https://docs.testomat.io/test-reporting/pipes/debug)
52
-
53
- ## Custom Pipe
54
-
55
- _to be updated_
56
-
57
- ....
@@ -1,331 +0,0 @@
1
- ---
2
- title: Advanced Options
3
- description: Learn how to use the Testomat.io Pipe for advanced reporting configurations in test automation. The guide covers setting environment variables, adding reports to test runs, managing parallel executions, filtering tests, and integrating with JIRA for issue tracking.
4
- type: article
5
- url: https://docs.testomat.io/test-reporting/pipes/testomatio
6
- head:
7
- - tag: meta
8
- attrs:
9
- name: keywords
10
- content: Test reporting, test automation, Testomat.io Pipe, environment variables, CI integration, parallel execution, test filtering, JIRA integration, build URL, test management
11
- ---
12
- ## Testomat.io Pipe
13
-
14
- ![](./images/testomatio.png)
15
-
16
- Testomat.io Pipe sends data to [Testomat.io Application](https://app.testomat.io). Testomat.io provides free projects forever and unlimited test runs. Even it is a cloud application you can use it for free.
17
-
18
- **🔌 To enable Testomat.io pipe set `TESTOMATIO` environment variable with API key provided by Testomatio.**
19
-
20
- ```
21
- TESTOMATIO={API_KEY} <actual run command>
22
- ```
23
-
24
- Here are some possible use cases where you can use additional configuration on reporter:
25
-
26
- - [Create Unmatched Tests](#create-unmatched-tests)
27
- - [Add Report to Run by ID](#add-report-to-run-by-id)
28
- - [Do Not Finalize Run](#do-not-finalize-run)
29
- - [Setting Report Title](#setting-report-title)
30
- - [Reporting Parallel Execution to To Same Run](#reporting-parallel-execution-to-to-same-run)
31
- - [Adding Report to RunGroup](#adding-report-to-rungroup)
32
- - [Adding Environments to Run](#adding-environments-to-run)
33
- - [Starting an Empty Run](#starting-an-empty-run)
34
- - [Manually Finishing Run](#manually-finishing-run)
35
- - [Setting Build URL](#setting-build-url)
36
- - [Publish Run](#publish-run)
37
- - [Assign Label To Run](#assign-label-to-run)
38
- - [Create Jira Issue for Run](#create-jira-issue-for-run)
39
- - [Filter Tests](#filter-tests)
40
-
41
- ### Create Unmatched Tests
42
-
43
- Testomat.io will not create tests from the report if they have not been previously imported. To create tests during the report `TESTOMATIO_CREATE` option can be used:
44
-
45
- ```bash
46
- TESTOMATIO={API_KEY} TESTOMATIO_CREATE=1 <actual run command>
47
- ```
48
-
49
- ### Add Report to Run by ID
50
-
51
- This feature is widely used when a run is executed on CI.
52
- A run is created before the test is started and it is marked as `scheduled`. Then
53
- a report is assigned to that run using `TESTOMATIO_RUN` environment variable and `{RUN_ID}` of a run:
54
-
55
- ```bash
56
- TESTOMATIO={API_KEY} TESTOMATIO_RUN={RUN_ID} <actual run command>
57
- ```
58
-
59
- ### Do Not Finalize Run
60
-
61
- If multiple reports are added to the same run, each of them should not finalize the run.
62
- In this case use `TESTOMATIO_PROCEED=1` environment variable, so the Run will be shown as `Running`
63
-
64
- ```
65
- TESTOMATIO={API_KEY} TESTOMATIO_PROCEED=1 TESTOMATIO_RUN={RUN_ID} <actual run command>
66
- ```
67
-
68
- After all reports were attached and run can be execute the following command:
69
-
70
- ```
71
- TESTOMATIO={API_KEY} TESTOMATIO_RUN={RUN_ID} npx @testomatio/reporter finish
72
- ```
73
-
74
- ### Setting Report Title
75
-
76
- Give a title to your reports by passing it as environment variable to `TESTOMATIO_TITLE`.
77
-
78
- ```bash
79
- TESTOMATIO={API_KEY} TESTOMATIO_TITLE="title for the report" <actual run command>
80
- ```
81
-
82
- ### Reporting Parallel Execution to To Same Run
83
-
84
- Provide a shared unique title to all runs that will be running in parallel, and add `TESTOMATIO_SHARED_RUN` environment var. So all reports will be sent to this run.
85
-
86
- ```bash
87
- TESTOMATIO={API_KEY} TESTOMATIO_TITLE="report for commit ${GIT_COMMIT}" TESTOMATIO_SHARED_RUN=1 <actual run command>
88
- ```
89
-
90
- We recommend using a commit hash as a title to create a new Run. In this case we ensure that run title is unique and will be the same for all parallel jobs running exactly for this commit.
91
-
92
- By default you can report to the same run if a run was created no more than 20 minutes ago. If you want to increase this time you can use `TESTOMATIO_SHARED_RUN_TIMEOUT` environment variable. `TESTOMATIO_SHARED_RUN_TIMEOUT`accepts value in minutes, so to increase timeout to 2 hours (120 minutes) use the following command:
93
-
94
- ```bash
95
- TESTOMATIO={API_KEY} TESTOMATIO_TITLE="report for commit ${GIT_COMMIT}" TESTOMATIO_SHARED_RUN=1 TESTOMATIO_SHARED_RUN_TIMEOUT=120 <actual run command>
96
- ```
97
-
98
- ### Adding Report to RunGroup
99
-
100
- Create/Add run to group by providing `TESTOMATIO_RUNGROUP_TITLE`:
101
-
102
- ```sh
103
- TESTOMATIO={API_KEY} TESTOMATIO_RUNGROUP_TITLE="Build ${BUILD_ID}" <actual run command>
104
- ```
105
-
106
- ### Adding Environments to Run
107
-
108
- Add environments to run by providing `TESTOMATIO_ENV` as comma seperated values:
109
-
110
- ```bash
111
- TESTOMATIO={API_KEY} TESTOMATIO_ENV="Windows, Chrome" <actual run command>
112
- ```
113
-
114
- ### Starting an Empty Run
115
-
116
- If you want to create a run and obtain its `{RUN_ID}` from [testomat.io](https://testomat.io) you can use `--launch` option:
117
-
118
- ```bash
119
- TESTOMATIO={API_KEY} npx @testomatio/reporter start
120
- ```
121
-
122
- This command will return `{RUN_ID}` which you can pass to other jobs in a workflow.
123
-
124
- > When executed with `--launch` a command provided by `-c` flag is ignored
125
-
126
- ### Manually Finishing Run
127
-
128
- If you want to finish a run started by `--launch` use `--finish` option. `TESTOMATIO_RUN` environment variable is required:
129
-
130
- ```bash
131
- TESTOMATIO={API_KEY} TESTOMATIO_RUN={RUN_ID} npx @testomatio/reporter finish
132
- ```
133
-
134
- ### Setting Build URL
135
-
136
- When running on CI reporter tries to detect automatically the URL of the current build. This URL will be set to Run report. Reporter automatically detects build URL for following CI services:
137
-
138
- - GitHub Actions
139
- - Azure Devops
140
- - Jenkins
141
- - CircleCi
142
- - Gitlab CI
143
-
144
- If you are using a different CI or you want to override build url run tests with `BUILD_URL` environment variable:
145
-
146
- ```bash
147
- TESTOMATIO={API_KEY} BUILD_URL=<build-url> <actual run command>
148
- ```
149
-
150
- ### Publish Run
151
-
152
- To make report publicly available pass `TESTOMATIO_PUBLISH=1` variable:
153
-
154
- ```
155
- TESTOMATIO={API_KEY} TESTOMATIO_PUBLISH=1 <actual run command>
156
- ```
157
-
158
- ### Assign Label To Run
159
-
160
- To assign a label to run, a **label with scope `runs` must exist**.
161
- Then a label can be passed via `TESTOMATIO_LABEL` variable:
162
-
163
- ```
164
- TESTOMATIO={API_KEY} TESTOMATIO_LABEL=release <actual run command>
165
- ```
166
-
167
- There is a way to pass a value to label via `:`:
168
-
169
- ```
170
- TESTOMATIO={API_KEY} TESTOMATIO_LABEL="module:checkout" <actual run command>
171
- ```
172
-
173
- Or you can set multiple labels separating them with comma:
174
-
175
- ```
176
- TESTOMATIO={API_KEY} TESTOMATIO_LABEL="release,module:checkout" <actual run command>
177
- ```
178
-
179
- ### Create Jira Issue for Run
180
-
181
- To automatically assign Run to Jira Issue pass the ID of that issue as `TESTOMATIO_JIRA_ID` variable
182
-
183
- ```
184
- TESTOMATIO={API_KEY} TESTOMATIO_JIRA_ID={Jira_Issue_ID} <actual run command>
185
- ```
186
-
187
- For example, if your Jira issue id is `TST-1`, command should be:
188
-
189
- ```
190
- TESTOMATIO={API_KEY} TESTOMATIO_JIRA_ID=TST-1 <actual run command>
191
- ```
192
-
193
- ### Filter Tests
194
-
195
- Testomatio allows you to enhance your test execution with the option to filter and manage test runs based on various criteria. This can help you efficiently organize and execute your tests. Below, we explain how to use the `--filter` option in conjunction with Testomatio for customized test runs.
196
-
197
- > The filtering functionality for test execution is currently available for the following frameworks: Playwright & CodeceptJS.
198
-
199
- To use the `--filter` option with Testomatio, follow the format:
200
-
201
- ```bash
202
- TESTOMATIO={API_KEY} npx @testomatio/reporter run <actual run command> --filter 'testomatio:{filter-type}={filter-value}'
203
- ```
204
-
205
- #### Available Filter Types
206
-
207
- **Tag Name Filter**
208
-
209
- Filter tests by tag names. Use this filter to select tests associated with specific tags.
210
-
211
- ![Tag Name Filter](./images/by-tag-name.png)
212
-
213
- _Command execution example_
214
-
215
- ```bash
216
- TESTOMATIO={API_KEY} npx @testomatio/reporter run 'actual run command' --filter 'testomatio:tag-name=smoke'
217
- ```
218
-
219
- **Plan ID Filter**
220
-
221
- Filter tests by Testomatio plan IDs. Use this filter to organize and execute tests based on plan IDs.
222
-
223
- ![Plan ID Filter](./images/by-plan-id.png)
224
-
225
- _Command execution example_
226
-
227
- ```bash
228
- TESTOMATIO={API_KEY} npx @testomatio/reporter run 'actual run command' --filter 'testomatio:plan-id=b62f3170'
229
- ```
230
-
231
- **Label Filter**
232
-
233
- Filter tests using custom labels. Labels allow you to categorize tests for better management and execution.
234
-
235
- Label Filter (by label name) Example 1:
236
- ![Label Filter Example 3](./images/by-label-1.png)
237
-
238
- Label Filter (by label id) Example 2:
239
- ![Label Filter Example 1](./images/by-label.png)
240
-
241
- Another Label Filter Example 3:
242
- ![Label Filter Example 2](./images/by-label-3.png)
243
-
244
- _Command execution example_
245
-
246
- ```bash
247
- TESTOMATIO={API_KEY} npx @testomatio/reporter run 'actual run command' --filter 'testomatio:label=new-label-test'
248
- ```
249
-
250
- OR
251
-
252
- ```bash
253
- TESTOMATIO={API_KEY} npx @testomatio/reporter run 'actual run command' --filter 'testomatio:label=NEW label TEST'
254
- ```
255
-
256
- **Label Filter based on the Severity type**
257
-
258
- If you want to use a specific severity type, you should join the "severity" label id + severity type(e.g.: `s1-id2:⛔ Blocker`).
259
-
260
- Here's a list of available types that you can copy and add to the run command:
261
-
262
- - ⚠️ Critical
263
- - ⛔ Blocker
264
- - 🔥 Major
265
- - 👌 Normal
266
- - ❄️ Minor
267
- - 💤 Trivial
268
-
269
- Label Filter Example based on the Severity type:
270
- ![Label Filter Example 3](./images/by-label-severity.png)
271
-
272
- _Command execution example_
273
-
274
- ```bash
275
- TESTOMATIO={API_KEY} npx @testomatio/reporter run 'actual run command' --filter 'testomatio:label=severity-f1b11:🔥 Major'
276
- ```
277
-
278
- OR
279
-
280
- ```bash
281
- TESTOMATIO={API_KEY} npx @testomatio/reporter run 'actual run command' --filter 'testomatio:label=NEW label TEST:💤 Trivial'
282
- ```
283
-
284
- **JIRA Ticket Filter**
285
-
286
- Filter tests linked to JIRA tickets. Use this filter to coordinate testing efforts with your JIRA workflow.
287
-
288
- ![JIRA Ticket Filter](./images/by-jira-id.png)
289
-
290
- _Command execution example_
291
-
292
- ```bash
293
- TESTOMATIO={API_KEY} npx @testomatio/reporter run 'actual run command' --filter 'testomatio:jira-ticket=LMP-51'
294
- ```
295
-
296
- **📚 Example Usage**
297
-
298
- For instance, if you want to run tests with a specific label named "new-label-test," you can use the following command:
299
-
300
- ```bash
301
- TESTOMATIO={API_KEY} npx @testomatio/reporter run 'npx codeceptjs run' --filter 'testomatio:label=new-label'
302
- ```
303
-
304
- OR using Playwright:
305
-
306
- ```bash
307
- TESTOMATIO={API_KEY} npx @testomatio/reporter run 'npx playwright test' --filter 'testomatio:label=new-label'
308
- ```
309
-
310
- > Remember to replace {API_KEY} with your actual Testomatio API key and adjust the `<actual run command>` according to your testing setup
311
-
312
- > It's important to note that the provided filter value must match exactly the corresponding tag name, plan ID, label, or JIRA ticket for the desired tests
313
-
314
- Please note, that this functionality allows you to easily filter and execute tests based on specific criteria, enhancing your testing experience.
315
-
316
- ### Exclude Tests from Report by Glob Pattern
317
-
318
- To exclude tests from the report by [glob pattern](https://www.npmjs.com/package/glob) use `TESTOMATIO_EXCLUDE_FILES_FROM_REPORT_GLOB_PATTERN` environment variable:
319
-
320
- ```bash
321
- TESTOMATIO={API_KEY} TESTOMATIO_EXCLUDE_FILES_FROM_REPORT_GLOB_PATTERN="**/*.setup.ts" <actual run command>
322
- ```
323
-
324
- You can use multiple patterns separated by `;`:
325
-
326
- ```bash
327
- TESTOMATIO={API_KEY} TESTOMATIO_EXCLUDE_FILES_FROM_REPORT_GLOB_PATTERN="**/*.setup.ts;tests/*.auth.js" <actual run command>
328
- ```
329
-
330
- (Any files in `node_modules` will be ignored).
331
-
@@ -1,229 +0,0 @@
1
- ---
2
- title: Python Test Frameworks
3
- description: Learn how to integrate the Testomat.io reporter with Python test frameworks like Pytest. The documentation covers installation, synchronizing tests, configuring test environments, reporting results, and submitting test artifacts. It also includes detailed configuration options such as managing test artifacts through S3 and setting up environment variables for reporting.
4
- type: article
5
- url: https://docs.testomat.io/test-reporting/python
6
- head:
7
- - tag: meta
8
- attrs:
9
- name: keywords
10
- content: Python test frameworks, Pytest, Testomat.io, test synchronization, test reporting, test artifacts, automation, S3 storage, test configuration, API integration, continuous integration, environment setup
11
- ---
12
-
13
- :::note
14
- Taken from [Pytestomatio Reporter Readme](https://github.com/testomatio/pytestomatio)
15
- :::
16
-
17
- [![Support Ukraine Badge](https://bit.ly/support-ukraine-now)](https://github.com/support-ukraine/support-ukraine)
18
-
19
- # testomat.io plugin for pytest
20
-
21
- ## uses testomat.io API:
22
-
23
- - https://testomatio.github.io/check-tests/
24
- - https://testomatio.github.io/reporter/
25
-
26
- ## Installation
27
-
28
- ```bash
29
- pip install pytestomatio
30
- ```
31
-
32
- ## Usage
33
-
34
- Synchronize tests to testomat.io and get back test id.
35
- Tests will not be executed
36
-
37
- ```bash
38
- pytest --testomatio sync
39
- ```
40
-
41
- Remove all test ids from source code. Tests will not be executed
42
-
43
- ```bash
44
- pytest --testomatio remove
45
- ```
46
-
47
- Run pytest and send test results into testomat.io.
48
- Test can be sent to testomat.io without ids in your test code. If testomat.io failed to match tests by title, it will create
49
- new tests for the run
50
-
51
- ```bash
52
- pytest --testomatio report
53
- ```
54
-
55
- Run pytest with debug parameter to get test data collected in metadata.json file
56
-
57
- ```bash
58
- pytest --testomatio debug
59
- ```
60
-
61
- ## Configuration with environment variables
62
- You can use environment variable to control certain features of testomat.io
63
-
64
- | Env variable | What it does | Examples |
65
- |--------|--------|-------|
66
- | TESTOMATIO| Provides token for pytestomatio to access and push data to testomat.io | TESTOMATIO=tstmt_***** pytest --testomatio sync|
67
- |TESTOMATIO_RUN|Name of a test run to create on testomat.io|TESTOMATIO_RUN="Smoke test" pytest --testomatio report|
68
- |TESTOMATIO_RUN_ID| Id of existing test run to use for sending test results to | TESTOMATIO_RUN_ID=98dfas0 pytest --testomatio report |
69
- |TESTOMATIO_RUNGROUP_TITLE| Create a group (folder) for a test run | TESTOMATIO_RUNGROUP_TITLE="Release 2.0" pytest --testomatio report|
70
- |TESTOMATIO_ENV|Assign environment to a test run |TESTOMATIO_ENV="linux,chrome,1920x1080" pytest --testomatio report|
71
- |TESTOMATIO_LABELS|Assign labels to a test run |TESTOMATIO_ENV="smoke,regression" pytest --testomatio report|
72
- |TESTOMATIO_SYNC_LABELS|Assign labels to a test case when you synchronise test from code with testomat.io|TESTOMATIO_SYNC_LABELS="number:1,list:one,standalone" pytest --testomatio report|
73
- |TESTOMATIO_CODE_STYLE|If you are not sure, don't set this variable. Default value is 'default'|TESTOMATIO_CODE_STYLE=pep8 pytest --testomatio sync|
74
- |TESTOMATIO_CI_DOWNSTREAM|If set, pytestomatio will not set or update build url for a test run. This is useful in scenarios where build url is already set in the test run by Testomat.io for test runs that a created directly on Testomat.io.|TESTOMATIO_CI_DOWNSTREAM=true pytest --testomatio report|
75
-
76
-
77
- ### Run groups parameter
78
- There is environment variable `TESTOMATIO_RUNGROUP_TITLE` that can be used to specify run tests with specific group title.
79
-
80
- ### pytest.ini
81
- In case you are using private testomat.io service, create `pytest.ini` file in your project root directory. Specify
82
- testomat.io url in it
83
-
84
- ```ini
85
- [pytest]
86
- testomatio_url = https://app.testomat.io
87
-
88
- ```
89
-
90
- ### Submitting Test Run Environment
91
-
92
- to configure test environment, you can use additional option:
93
-
94
- ```bash
95
- pytest --testomatio report --testRunEnv "windows11,chrome,1920x1080"
96
- ```
97
-
98
- Environment values are comma separated, please use double quotation.
99
-
100
-
101
- ### Submitting Test Artifacts
102
-
103
- testomat.io does not store any screenshots, logs or other artifacts.
104
-
105
- In order to manage them it is advised to use S3 Buckets (GCP Storage).
106
- https://docs.testomat.io/usage/test-artifacts/
107
-
108
- Analyser needs to be aware of the cloud storage credentials.
109
- There are two options:
110
- 1. Enable **Share credentials with testomat.io Reporter** option in testomat.io Settings -> Artifacts.
111
- 2. Use environment variables `ACCESS_KEY_ID, SECRET_ACCESS_KEY, ENDPOINT, BUCKET, BUCKET_PATH`
112
-
113
- You would need to decide when you want to upload your test artifacts to cloud storage
114
-
115
- 1) Upload page screenshot when test fails, using fixtures [reference](https://docs.pytest.org/en/latest/example/simple.html#making-test-result-information-available-in-fixtures)
116
-
117
- ```python
118
- # content of conftest.py
119
- import pytest
120
- import random
121
- import os
122
- from typing import Dict
123
- from pytest import StashKey, CollectReport
124
- from playwright.sync_api import Page
125
-
126
- phase_report_key = StashKey[Dict[str, CollectReport]]()
127
-
128
- @pytest.hookimpl(wrapper=True, tryfirst=True)
129
- def pytest_runtest_makereport(item, call):
130
- rep = yield
131
- item.stash.setdefault(phase_report_key, {})[rep.when] = rep
132
- return rep
133
-
134
-
135
- @pytest.fixture(scope="function")
136
- def handle_artifacts(page: Page, request):
137
- yield
138
- report = request.node.stash[phase_report_key]
139
- if ("call" not in report) or report["setup"].failed or report["call"].failed:
140
- random_string = ''.join(random.choices(string.ascii_letters + string.digits, k=8))
141
-
142
- filename = f"{random_string}.png"
143
- screenshot_path = os.path.join(artifacts_dir, filename)
144
- page.screenshot(path=screenshot_path)
145
- # file_path - required, path to file to be uploaded
146
- # file_bytes - required, bytes of the file to be uploaded
147
- # key - required, file name in the s3 bucket
148
- # bucket_name - optional,name of the bucket to upload file to. Default value is taken from testomat.io
149
- artifact_url = pytest.testomatio.upload_file(screenshot_path, filename)
150
- # or
151
- # artifact_url = pytest.testomatio.upload_file_object(file_bytes, key, bucket_name)
152
- pytest.testomatio.add_artifacts(request.node, [artifact_url])
153
- page.close()
154
- ```
155
-
156
- ⚠️ Please take into account s3_connector available only after **pytest_collection_modifyitems()** hook is executed.
157
-
158
- 2) If you prefer to use pytest hooks - add `pytest_runtest_makereport` hook in your `conftest.py` file.
159
-
160
- ```python
161
- def pytest_runtest_makereport(item, call):
162
- artifact_url = pytest.testomatio.upload_file(screenshot_path, filename)
163
- pytest.testomatio.add_artifacts([artifact_url])
164
- ```
165
-
166
- ### Clarifications
167
-
168
- - tests can be synced even without `@patest.mark.testomatio('@T96c700e6')` decorator.
169
- - test title in testomat.io == test name in pytest
170
- - test suit title in testomat.io == test file name in pytest
171
-
172
- ## Example of test
173
-
174
- To make the experience more consistent, it uses standard pytest markers.
175
- testomat.io test id is a string value that starts with `@T` and has 8 symbols after.
176
-
177
- ```python
178
- from pytest import mark
179
-
180
-
181
- @mark.testomatio('@T96c700e6')
182
- def test_example():
183
- assert 2 + 2 == 4
184
- ```
185
-
186
- ### Compatibility table with [Testomatio check-tests](https://github.com/testomatio/check-tests)
187
-
188
- | Action | Compatibility | Method |
189
- |--------|--------|-------|
190
- | Report tests to testomat.io | complete | `pytest --testomatio report` |
191
- | Filter test by id to run | complete | `pytest --testomatio report --test-id="T00C73028\|T00C73029"` |
192
- | Importing test into testomat.io | complete | `pytest --testomatio sync` |
193
- | Exclude hook code of a test | N/A | N/A |
194
- | Include line number code of a test | N/A | N/A |
195
- | Import Parametrized Tests | complete | default behaviour |
196
- | Disable Detached Tests | complete | `pytest --testomatio sync --no-detach` |
197
- | Synchronous Import | complete | default behaviour |
198
- | Auto-assign Test IDs in Source Code | complete | default behaviour |
199
- | Keep Test IDs Between Projects | complete | `pytest --testomatio sync --create` |
200
- | Clean Test IDs | complete | `pytest --testomatio remove` |
201
- | Import Into a Branch | N/A | N/A |
202
- | Keep Structure of Source Code | complete | `pytest --testomatio sync --keep-structure` |
203
- | Delete Empty Suites | complete | `pytest --testomatio sync --no-empty` |
204
- | Import Into a Specific Folder | complete | `pytest --testomatio sync --directory "Windows\smoke"` |
205
- | Debugging | parity | `pytest --testomatio debug` |
206
-
207
-
208
- ## Test
209
- - import into empty project
210
- - updated test - (resync)
211
- - test run
212
- - test run into a folder
213
- - test run labels, tags
214
-
215
- ## TODO
216
- - retry test run update with less attributes, we get 500 from api
217
- - handler non configured s3 bucket error
218
- - Fix test duration
219
-
220
- ## Contribution
221
- Use python 3.12
222
-
223
- 1. `pip install ".[dev]"`
224
- 1. `python ./smoke.py`
225
- 1. Test things manually
226
- 1. Verify no regression bugs
227
- 1. `cz commit`
228
- 1. `cz bump`
229
- 1. `git push remoteName branchName --tags`