testeiya 0.2.0 → 0.3.9

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (412) hide show
  1. package/LICENSE +21 -0
  2. package/README.md +37 -175
  3. package/dist/prompt/context.js +79 -0
  4. package/dist/prompt/context.js.map +1 -0
  5. package/dist/prompt/index.js +56 -0
  6. package/dist/prompt/index.js.map +1 -0
  7. package/dist/prompt/print.js +22 -0
  8. package/dist/prompt/print.js.map +1 -0
  9. package/dist/prompt/project-info.js +2 -0
  10. package/dist/prompt/project-info.js.map +1 -0
  11. package/{src/prompt/system-prompt.ts → dist/prompt/system-prompt.js} +24 -21
  12. package/dist/prompt/system-prompt.js.map +1 -0
  13. package/dist/prompt/testomatio.js +264 -0
  14. package/dist/prompt/testomatio.js.map +1 -0
  15. package/dist/prompt/tools.js +49 -0
  16. package/dist/prompt/tools.js.map +1 -0
  17. package/dist/prompt/vocab.js +8 -0
  18. package/dist/prompt/vocab.js.map +1 -0
  19. package/dist/src/args.js +72 -0
  20. package/dist/src/args.js.map +1 -0
  21. package/dist/src/cli.js +61 -0
  22. package/dist/src/cli.js.map +1 -0
  23. package/dist/src/env.js +69 -0
  24. package/dist/src/env.js.map +1 -0
  25. package/dist/src/mcp-extension.js +9 -0
  26. package/dist/src/mcp-extension.js.map +1 -0
  27. package/dist/src/mcp.js +139 -0
  28. package/dist/src/mcp.js.map +1 -0
  29. package/dist/src/model.js +66 -0
  30. package/dist/src/model.js.map +1 -0
  31. package/dist/src/result.js +41 -0
  32. package/dist/src/result.js.map +1 -0
  33. package/dist/src/run.js +199 -0
  34. package/dist/src/run.js.map +1 -0
  35. package/dist/src/session.js +77 -0
  36. package/dist/src/session.js.map +1 -0
  37. package/dist/vendor/mcp.js +14801 -0
  38. package/mcp-tools.json +2967 -0
  39. package/package.json +29 -41
  40. package/prompt/context.ts +101 -0
  41. package/prompt/index.ts +93 -0
  42. package/prompt/print.ts +23 -0
  43. package/prompt/project-info.ts +30 -0
  44. package/prompt/system-prompt.ts +173 -0
  45. package/{src/prompt → prompt}/testomatio.ts +119 -8
  46. package/{src/prompt → prompt}/tools.ts +30 -8
  47. package/prompt/vocab.ts +7 -0
  48. package/{skills.lock.json → skills/skills.lock.json} +11 -7
  49. package/skills/skills.yaml +34 -0
  50. package/models.catalog.json +0 -6963
  51. package/skills/codeceptjs/ci-fix-tests/SKILL.md +0 -120
  52. package/skills/codeceptjs/codeceptjs-auth/SKILL.md +0 -158
  53. package/skills/codeceptjs/codeceptjs-exploration/SKILL.md +0 -91
  54. package/skills/codeceptjs/codeceptjs-fundamentals/SKILL.md +0 -158
  55. package/skills/codeceptjs/codeceptjs-run-analysis/SKILL.md +0 -90
  56. package/skills/codeceptjs/debugging-codeceptjs-tests/SKILL.md +0 -234
  57. package/skills/codeceptjs/migrate-codeceptjs-4/SKILL.md +0 -110
  58. package/skills/codeceptjs/migrate-cypress-to-codeceptjs/SKILL.md +0 -306
  59. package/skills/codeceptjs/migrate-protractor-to-codeceptjs/SKILL.md +0 -352
  60. package/skills/codeceptjs/migrate-selenium-java-to-codeceptjs/SKILL.md +0 -472
  61. package/skills/codeceptjs/migrate-testcafe-to-codeceptjs/SKILL.md +0 -350
  62. package/skills/codeceptjs/refactoring-codeceptjs-tests/SKILL.md +0 -81
  63. package/skills/codeceptjs/writing-codeceptjs-tests/SKILL.md +0 -206
  64. package/skills/playwright-best-practices-skill/LICENSE.md +0 -7
  65. package/skills/playwright-best-practices-skill/README.md +0 -147
  66. package/skills/playwright-best-practices-skill/SKILL.md +0 -303
  67. package/skills/playwright-best-practices-skill/advanced/authentication-flows.md +0 -360
  68. package/skills/playwright-best-practices-skill/advanced/authentication.md +0 -871
  69. package/skills/playwright-best-practices-skill/advanced/clock-mocking.md +0 -364
  70. package/skills/playwright-best-practices-skill/advanced/mobile-testing.md +0 -409
  71. package/skills/playwright-best-practices-skill/advanced/multi-context.md +0 -288
  72. package/skills/playwright-best-practices-skill/advanced/multi-user.md +0 -393
  73. package/skills/playwright-best-practices-skill/advanced/network-advanced.md +0 -452
  74. package/skills/playwright-best-practices-skill/advanced/third-party.md +0 -464
  75. package/skills/playwright-best-practices-skill/architecture/pom-vs-fixtures.md +0 -363
  76. package/skills/playwright-best-practices-skill/architecture/test-architecture.md +0 -369
  77. package/skills/playwright-best-practices-skill/architecture/when-to-mock.md +0 -383
  78. package/skills/playwright-best-practices-skill/browser-apis/browser-apis.md +0 -391
  79. package/skills/playwright-best-practices-skill/browser-apis/iframes.md +0 -403
  80. package/skills/playwright-best-practices-skill/browser-apis/service-workers.md +0 -504
  81. package/skills/playwright-best-practices-skill/browser-apis/websockets.md +0 -403
  82. package/skills/playwright-best-practices-skill/core/annotations.md +0 -424
  83. package/skills/playwright-best-practices-skill/core/assertions-waiting.md +0 -361
  84. package/skills/playwright-best-practices-skill/core/configuration.md +0 -452
  85. package/skills/playwright-best-practices-skill/core/fixtures-hooks.md +0 -417
  86. package/skills/playwright-best-practices-skill/core/global-setup.md +0 -434
  87. package/skills/playwright-best-practices-skill/core/locators.md +0 -242
  88. package/skills/playwright-best-practices-skill/core/page-object-model.md +0 -315
  89. package/skills/playwright-best-practices-skill/core/projects-dependencies.md +0 -453
  90. package/skills/playwright-best-practices-skill/core/test-data.md +0 -492
  91. package/skills/playwright-best-practices-skill/core/test-suite-structure.md +0 -361
  92. package/skills/playwright-best-practices-skill/core/test-tags.md +0 -298
  93. package/skills/playwright-best-practices-skill/debugging/console-errors.md +0 -420
  94. package/skills/playwright-best-practices-skill/debugging/debugging.md +0 -504
  95. package/skills/playwright-best-practices-skill/debugging/error-testing.md +0 -360
  96. package/skills/playwright-best-practices-skill/debugging/flaky-tests.md +0 -496
  97. package/skills/playwright-best-practices-skill/frameworks/angular.md +0 -530
  98. package/skills/playwright-best-practices-skill/frameworks/nextjs.md +0 -469
  99. package/skills/playwright-best-practices-skill/frameworks/react.md +0 -531
  100. package/skills/playwright-best-practices-skill/frameworks/vue.md +0 -574
  101. package/skills/playwright-best-practices-skill/infrastructure-ci-cd/ci-cd.md +0 -468
  102. package/skills/playwright-best-practices-skill/infrastructure-ci-cd/docker.md +0 -283
  103. package/skills/playwright-best-practices-skill/infrastructure-ci-cd/github-actions.md +0 -546
  104. package/skills/playwright-best-practices-skill/infrastructure-ci-cd/gitlab.md +0 -397
  105. package/skills/playwright-best-practices-skill/infrastructure-ci-cd/other-providers.md +0 -521
  106. package/skills/playwright-best-practices-skill/infrastructure-ci-cd/parallel-sharding.md +0 -371
  107. package/skills/playwright-best-practices-skill/infrastructure-ci-cd/performance.md +0 -453
  108. package/skills/playwright-best-practices-skill/infrastructure-ci-cd/reporting.md +0 -424
  109. package/skills/playwright-best-practices-skill/infrastructure-ci-cd/test-coverage.md +0 -497
  110. package/skills/playwright-best-practices-skill/testing-patterns/accessibility.md +0 -359
  111. package/skills/playwright-best-practices-skill/testing-patterns/api-testing.md +0 -719
  112. package/skills/playwright-best-practices-skill/testing-patterns/browser-extensions.md +0 -506
  113. package/skills/playwright-best-practices-skill/testing-patterns/canvas-webgl.md +0 -493
  114. package/skills/playwright-best-practices-skill/testing-patterns/component-testing.md +0 -500
  115. package/skills/playwright-best-practices-skill/testing-patterns/drag-drop.md +0 -576
  116. package/skills/playwright-best-practices-skill/testing-patterns/electron.md +0 -509
  117. package/skills/playwright-best-practices-skill/testing-patterns/file-operations.md +0 -377
  118. package/skills/playwright-best-practices-skill/testing-patterns/file-upload-download.md +0 -562
  119. package/skills/playwright-best-practices-skill/testing-patterns/forms-validation.md +0 -561
  120. package/skills/playwright-best-practices-skill/testing-patterns/graphql-testing.md +0 -331
  121. package/skills/playwright-best-practices-skill/testing-patterns/i18n.md +0 -508
  122. package/skills/playwright-best-practices-skill/testing-patterns/performance-testing.md +0 -476
  123. package/skills/playwright-best-practices-skill/testing-patterns/security-testing.md +0 -430
  124. package/skills/playwright-best-practices-skill/testing-patterns/visual-regression.md +0 -634
  125. package/skills/playwright-cli/SKILL.md +0 -420
  126. package/skills/playwright-cli/references/element-attributes.md +0 -23
  127. package/skills/playwright-cli/references/playwright-tests.md +0 -39
  128. package/skills/playwright-cli/references/request-mocking.md +0 -87
  129. package/skills/playwright-cli/references/running-code.md +0 -241
  130. package/skills/playwright-cli/references/session-management.md +0 -225
  131. package/skills/playwright-cli/references/storage-state.md +0 -275
  132. package/skills/playwright-cli/references/test-generation.md +0 -433
  133. package/skills/playwright-cli/references/tracing.md +0 -139
  134. package/skills/playwright-cli/references/video-recording.md +0 -143
  135. package/skills/testeiya/README.md +0 -29
  136. package/skills/testeiya/answer-formatting/SKILL.md +0 -30
  137. package/skills/testeiya/check-cucumber/SKILL.md +0 -200
  138. package/skills/testeiya/check-tests/SKILL.md +0 -260
  139. package/skills/testeiya/manual-run-assistant/SKILL.md +0 -61
  140. package/skills/testeiya/testomatio-docs/INDEX.md +0 -175
  141. package/skills/testeiya/testomatio-docs/SKILL.md +0 -102
  142. package/skills/testeiya/testomatio-docs/docs/advanced/ai-powered-features/ai-agents.md +0 -144
  143. package/skills/testeiya/testomatio-docs/docs/advanced/ai-powered-features/ai-powered-features.md +0 -531
  144. package/skills/testeiya/testomatio-docs/docs/advanced/ai-powered-features/ai-requirements.md +0 -437
  145. package/skills/testeiya/testomatio-docs/docs/advanced/api-access/index.md +0 -191
  146. package/skills/testeiya/testomatio-docs/docs/advanced/branches-folder/index.md +0 -277
  147. package/skills/testeiya/testomatio-docs/docs/advanced/bulk-edit-folder/bulk-edit-demos.mdx +0 -39
  148. package/skills/testeiya/testomatio-docs/docs/advanced/bulk-edit-folder/bulk-edit-on-suite-and-test-level.md +0 -122
  149. package/skills/testeiya/testomatio-docs/docs/advanced/bulk-edit-folder/what-is-bulk-edit.md +0 -170
  150. package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/branches.md +0 -53
  151. package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/index.mdx +0 -98
  152. package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/jira-issue.md +0 -151
  153. package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/runs-and-reports.md +0 -377
  154. package/skills/testeiya/testomatio-docs/docs/advanced/jira-plugin/work-in-jira.md +0 -267
  155. package/skills/testeiya/testomatio-docs/docs/advanced/living-doc/index.md +0 -138
  156. package/skills/testeiya/testomatio-docs/docs/advanced/milestones/index.md +0 -220
  157. package/skills/testeiya/testomatio-docs/docs/advanced/shortcuts/index.md +0 -48
  158. package/skills/testeiya/testomatio-docs/docs/advanced/tags-labels/labels-and-custom-fields.md +0 -201
  159. package/skills/testeiya/testomatio-docs/docs/advanced/tags-labels/tags-or-labels.md +0 -94
  160. package/skills/testeiya/testomatio-docs/docs/advanced/tags-labels/tags.md +0 -243
  161. package/skills/testeiya/testomatio-docs/docs/advanced/test-artifacts/index.md +0 -135
  162. package/skills/testeiya/testomatio-docs/docs/advanced/tql/index.md +0 -394
  163. package/skills/testeiya/testomatio-docs/docs/getting-started/index.md +0 -290
  164. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/azure.md +0 -96
  165. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/bamboo.md +0 -94
  166. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/bitbucket.md +0 -106
  167. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/circle.md +0 -115
  168. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/github.md +0 -120
  169. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/gitlab.md +0 -88
  170. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/index.mdx +0 -151
  171. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/jenkins.md +0 -121
  172. package/skills/testeiya/testomatio-docs/docs/integrations/continuous-integration/teamcity.md +0 -96
  173. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/azure.md +0 -31
  174. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/clickup.md +0 -54
  175. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/confluence.md +0 -34
  176. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/github.md +0 -31
  177. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/gitlab.md +0 -93
  178. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/index.md +0 -131
  179. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/jira.md +0 -164
  180. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/linear.md +0 -29
  181. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/shortcut.md +0 -67
  182. package/skills/testeiya/testomatio-docs/docs/integrations/issues-management/youtrack.md +0 -94
  183. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/azure-devops.md +0 -53
  184. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/email.md +0 -71
  185. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/jira.md +0 -52
  186. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/ms-teams.md +0 -60
  187. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/rules.md +0 -267
  188. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/slack.md +0 -82
  189. package/skills/testeiya/testomatio-docs/docs/integrations/report-notifications/telegram.md +0 -73
  190. package/skills/testeiya/testomatio-docs/docs/integrations/scim-provisioning/index.mdx +0 -207
  191. package/skills/testeiya/testomatio-docs/docs/integrations/scim-provisioning/okta.md +0 -204
  192. package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/azure.md +0 -95
  193. package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/google.md +0 -75
  194. package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/index.mdx +0 -33
  195. package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/okta.md +0 -108
  196. package/skills/testeiya/testomatio-docs/docs/integrations/single-sign-on/saml.md +0 -66
  197. package/skills/testeiya/testomatio-docs/docs/legal/compliance/gdpr.md +0 -95
  198. package/skills/testeiya/testomatio-docs/docs/legal/compliance/index.mdx +0 -94
  199. package/skills/testeiya/testomatio-docs/docs/legal/compliance/privacy.md +0 -26
  200. package/skills/testeiya/testomatio-docs/docs/legal/compliance/terms.md +0 -32
  201. package/skills/testeiya/testomatio-docs/docs/legal/security/index.mdx +0 -242
  202. package/skills/testeiya/testomatio-docs/docs/legal/security/jira.md +0 -140
  203. package/skills/testeiya/testomatio-docs/docs/legal/security/subprocessors.md +0 -76
  204. package/skills/testeiya/testomatio-docs/docs/management/company/administration.md +0 -363
  205. package/skills/testeiya/testomatio-docs/docs/management/company/audit-log.md +0 -41
  206. package/skills/testeiya/testomatio-docs/docs/management/company/plan-features-comparison.md +0 -240
  207. package/skills/testeiya/testomatio-docs/docs/management/company/subscriptions.md +0 -326
  208. package/skills/testeiya/testomatio-docs/docs/management/company/trials.md +0 -75
  209. package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/accountant.md +0 -49
  210. package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/index.mdx +0 -285
  211. package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/read-only-user.md +0 -67
  212. package/skills/testeiya/testomatio-docs/docs/management/company/users-and-permissions/teams.md +0 -138
  213. package/skills/testeiya/testomatio-docs/docs/management/project/settings.md +0 -87
  214. package/skills/testeiya/testomatio-docs/docs/management/project/templates.md +0 -546
  215. package/skills/testeiya/testomatio-docs/docs/project/analytics/index.md +0 -326
  216. package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/classical-tests-markdown-format.md +0 -386
  217. package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/download-manual-tests-as-files.md +0 -378
  218. package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/export-to-obsidian.md +0 -112
  219. package/skills/testeiya/testomatio-docs/docs/project/import-export/export-tests/export-to-spreadsheet.md +0 -127
  220. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/auto-import.md +0 -63
  221. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-Allure-TestOps.md +0 -49
  222. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-QTest.md +0 -41
  223. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-Qase.md +0 -39
  224. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-Zephyr.md +0 -37
  225. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-csv-xlsx.md +0 -117
  226. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-cucumber.md +0 -93
  227. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-qmetry.md +0 -44
  228. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-source-code.md +0 -117
  229. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-testcaselabs.md +0 -44
  230. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-testmo.md +0 -47
  231. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-testrail.md +0 -82
  232. package/skills/testeiya/testomatio-docs/docs/project/import-export/import/import-tests-from-xray.md +0 -55
  233. package/skills/testeiya/testomatio-docs/docs/project/plans/index.md +0 -334
  234. package/skills/testeiya/testomatio-docs/docs/project/pulse/index.md +0 -104
  235. package/skills/testeiya/testomatio-docs/docs/project/runs/archive-runs-and-groups.md +0 -176
  236. package/skills/testeiya/testomatio-docs/docs/project/runs/custom-statuses.md +0 -195
  237. package/skills/testeiya/testomatio-docs/docs/project/runs/environments.md +0 -203
  238. package/skills/testeiya/testomatio-docs/docs/project/runs/managing-runs.md +0 -455
  239. package/skills/testeiya/testomatio-docs/docs/project/runs/merge-strategies.md +0 -137
  240. package/skills/testeiya/testomatio-docs/docs/project/runs/reports.md +0 -329
  241. package/skills/testeiya/testomatio-docs/docs/project/runs/rungroups.md +0 -342
  242. package/skills/testeiya/testomatio-docs/docs/project/runs/running-automated-tests.md +0 -256
  243. package/skills/testeiya/testomatio-docs/docs/project/runs/running-manual-and-automated-tests.md +0 -331
  244. package/skills/testeiya/testomatio-docs/docs/project/runs/running-tests-manually.md +0 -633
  245. package/skills/testeiya/testomatio-docs/docs/project/runs/temporary-tests-notes.md +0 -223
  246. package/skills/testeiya/testomatio-docs/docs/project/steps-snippets/snippets.md +0 -68
  247. package/skills/testeiya/testomatio-docs/docs/project/steps-snippets/steps.md +0 -181
  248. package/skills/testeiya/testomatio-docs/docs/project/tests/bdd-test-case-editor.md +0 -98
  249. package/skills/testeiya/testomatio-docs/docs/project/tests/classical-test-case-editor.md +0 -354
  250. package/skills/testeiya/testomatio-docs/docs/project/tests/classical-vs-bdd.md +0 -169
  251. package/skills/testeiya/testomatio-docs/docs/project/tests/copy-and-move-your-tests.md +0 -156
  252. package/skills/testeiya/testomatio-docs/docs/project/tests/other-features-for-test-case-design.md +0 -412
  253. package/skills/testeiya/testomatio-docs/docs/project/tests/test-case-creation-and-editing.md +0 -527
  254. package/skills/testeiya/testomatio-docs/docs/support/index.md +0 -75
  255. package/skills/testeiya/testomatio-docs/docs/test-reporting/artifacts.md +0 -388
  256. package/skills/testeiya/testomatio-docs/docs/test-reporting/cli.md +0 -215
  257. package/skills/testeiya/testomatio-docs/docs/test-reporting/configuration.md +0 -247
  258. package/skills/testeiya/testomatio-docs/docs/test-reporting/frameworks.md +0 -490
  259. package/skills/testeiya/testomatio-docs/docs/test-reporting/functions.md +0 -144
  260. package/skills/testeiya/testomatio-docs/docs/test-reporting/index.mdx +0 -48
  261. package/skills/testeiya/testomatio-docs/docs/test-reporting/junit.md +0 -248
  262. package/skills/testeiya/testomatio-docs/docs/test-reporting/logger.md +0 -138
  263. package/skills/testeiya/testomatio-docs/docs/test-reporting/php.md +0 -77
  264. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/bitbucket.md +0 -72
  265. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/csv.md +0 -23
  266. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/debug.md +0 -18
  267. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/github.md +0 -63
  268. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/gitlab.md +0 -29
  269. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/html.md +0 -80
  270. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/index.mdx +0 -57
  271. package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/testomatio.md +0 -331
  272. package/skills/testeiya/testomatio-docs/docs/test-reporting/python.md +0 -229
  273. package/skills/testeiya/testomatio-docs/docs/test-reporting/workflows.md +0 -325
  274. package/skills/testeiya/testomatio-docs/docs/tutorials/Interacting-via-API.md +0 -159
  275. package/skills/testeiya/testomatio-docs/docs/tutorials/Manual-Testing-Classic.md +0 -164
  276. package/skills/testeiya/testomatio-docs/docs/tutorials/Set-Up-Read-Only-Access.md +0 -85
  277. package/skills/testeiya/testomatio-docs/docs/tutorials/Using-AI-in-Testing.md +0 -163
  278. package/skills/testeiya/testomatio-docs/docs/tutorials/Working-with-Jira.md +0 -125
  279. package/skills/testeiya/testomatio-docs/docs/tutorials/java.md +0 -478
  280. package/skills/testeiya/testomatio-docs/docs/tutorials/playwright.md +0 -207
  281. package/skills/testeiya/testomatio-docs/docs/tutorials/webdriver.md +0 -218
  282. package/skills/testeiya/testomatio-docs/docs.lock.json +0 -6
  283. package/skills/testeiya/testomatio-reporter/SKILL.md +0 -307
  284. package/skills/testomatio/explorbot/explorbot-fundamentals/SKILL.md +0 -88
  285. package/skills/testomatio/explorbot/explorbot-plan/SKILL.md +0 -95
  286. package/skills/testomatio/explorbot/explorbot-setup/SKILL.md +0 -334
  287. package/skills/testomatio/test-automation/automate-manual-test-cases/CLAUDE.md +0 -56
  288. package/skills/testomatio/test-automation/automate-manual-test-cases/SKILL.md +0 -242
  289. package/skills/testomatio/test-automation/automate-manual-test-cases/references/CODECEPTJS_BEST_PRACTICES.md +0 -182
  290. package/skills/testomatio/test-automation/automate-manual-test-cases/references/FINAL_SUMMARY_TEMPLATE.md +0 -22
  291. package/skills/testomatio/test-automation/automate-manual-test-cases/references/PLAYWRIGHT_BEST_PRACTICES.md +0 -90
  292. package/skills/testomatio/test-automation/automate-manual-test-cases/references/POM_BEST_PRACTICES.md +0 -53
  293. package/skills/testomatio/test-automation/automate-manual-test-cases/references/TEST_DATA_MANAGEMENT.md +0 -52
  294. package/skills/testomatio/test-automation/debug-fix-failed-flaky-autotests/SKILL.md +0 -107
  295. package/skills/testomatio/test-automation/debug-fix-failed-flaky-autotests/references/DEBUGGING_QUICK_REFERENCE.md +0 -113
  296. package/skills/testomatio/test-automation/qa-automation-test-consolidation/SKILL.md +0 -29
  297. package/skills/testomatio/test-automation/run-tests-with-testomatio-reporter/SKILL.md +0 -151
  298. package/skills/testomatio/test-automation/setup-pr-testing/SKILL.md +0 -199
  299. package/skills/testomatio/test-automation/testomat-allure-adapter/SKILL.md +0 -289
  300. package/skills/testomatio/test-management/detect-duplicate-test-cases/SKILL.md +0 -86
  301. package/skills/testomatio/test-management/detect-duplicate-test-cases/references/DUPLICATE_INSTRUCTIONS.md +0 -54
  302. package/skills/testomatio/test-management/improve-test-cases/SKILL.md +0 -114
  303. package/skills/testomatio/test-management/improve-test-cases/references/TESTOMAT_MARKDOWN_EXAMPLE.md +0 -66
  304. package/skills/testomatio/test-management/pull-request-diff-analyzer/SKILL.md +0 -141
  305. package/skills/testomatio/test-management/qa-e2e-tests-reporting/SKILL.md +0 -354
  306. package/skills/testomatio/test-management/qa-e2e-tests-reporting/references/TESTOMATIO_ARTIFACTS.md +0 -82
  307. package/skills/testomatio/test-management/qa-e2e-tests-reporting/references/TESTOMATIO_HTML_REPORT.md +0 -37
  308. package/skills/testomatio/test-management/qa-e2e-tests-reporting/references/TESTOMATIO_REPORTERS_CONFIG.md +0 -118
  309. package/skills/testomatio/test-management/qa-pr-requirements-analyzer/SKILL.md +0 -149
  310. package/skills/testomatio/test-management/qa-pr-requirements-analyzer/references/summary-example.md +0 -54
  311. package/skills/testomatio/test-management/qa-requirement-reviewer/SKILL.md +0 -126
  312. package/skills/testomatio/test-management/qa-requirement-reviewer/references/requirements_reviewer_examples.md +0 -128
  313. package/skills/testomatio/test-management/qa-test-code-coverage/SKILL.md +0 -174
  314. package/skills/testomatio/test-management/qa-test-code-coverage/references/COVERAGE_FILE_FORMAT.md +0 -138
  315. package/skills/testomatio/test-management/qa-test-code-coverage/references/E2E_FRAMEWORKS.md +0 -65
  316. package/skills/testomatio/test-management/qa-test-code-coverage/scripts/check-coverage.mjs +0 -40
  317. package/skills/testomatio/test-management/qa-thinking/SKILL.md +0 -36
  318. package/skills/testomatio/test-management/qa-write-test-cases/SKILL.md +0 -267
  319. package/skills/testomatio/test-management/qa-write-test-cases/references/test-case-format.md +0 -387
  320. package/skills/testomatio/test-management/qa-write-test-cases/references/testomat-tms-guide.md +0 -72
  321. package/skills/testomatio/test-management/qa-write-test-cases/references/writing-rule.md +0 -159
  322. package/skills/testomatio/test-management/scan-automation-project/SKILL.md +0 -152
  323. package/skills/testomatio/test-management/sync-test-cases-with-tms/SKILL.md +0 -107
  324. package/skills/testomatio/test-management/sync-test-cases-with-tms/references/TESTOMATIO_CLI.md +0 -140
  325. package/skills/testomatio/test-management/testomatio-mcp/SKILL.md +0 -182
  326. package/skills/testomatio/test-management/testomatio-mcp/references/MCP_SETUP.md +0 -169
  327. package/skills.yaml +0 -21
  328. package/src/ai-debug.ts +0 -188
  329. package/src/api/agent-get.ts +0 -22
  330. package/src/api/agent-start.ts +0 -384
  331. package/src/api/client-log.ts +0 -29
  332. package/src/api/context.ts +0 -332
  333. package/src/api/debug-layout.ts +0 -47
  334. package/src/api/debug-report.ts +0 -29
  335. package/src/api/debug-snapshot.ts +0 -30
  336. package/src/api/debug-stream.ts +0 -52
  337. package/src/api/files-delete.ts +0 -184
  338. package/src/api/files-read.ts +0 -41
  339. package/src/api/files-rename.ts +0 -99
  340. package/src/api/files-tree.ts +0 -292
  341. package/src/api/files-write.ts +0 -41
  342. package/src/api/mcp.ts +0 -603
  343. package/src/api/memory.ts +0 -69
  344. package/src/api/open-external.ts +0 -39
  345. package/src/api/playwright-cli.ts +0 -606
  346. package/src/api/providers.ts +0 -457
  347. package/src/api/read-env-token.ts +0 -33
  348. package/src/api/sessions.ts +0 -79
  349. package/src/api/settings.ts +0 -120
  350. package/src/api/skills.ts +0 -58
  351. package/src/api/testomatio-attachment.ts +0 -89
  352. package/src/api/testomatio-auth.ts +0 -284
  353. package/src/api/testomatio-proxy.ts +0 -197
  354. package/src/api/testomatio-run-stats.ts +0 -81
  355. package/src/api/testomatio-target.ts +0 -225
  356. package/src/api/testomatio-transcription.ts +0 -82
  357. package/src/api/workspace-search.ts +0 -96
  358. package/src/api/workspace-sync.ts +0 -151
  359. package/src/api/workspace.ts +0 -165
  360. package/src/app-server.ts +0 -556
  361. package/src/bridge.ts +0 -251
  362. package/src/check-tests.ts +0 -156
  363. package/src/cli.ts +0 -17
  364. package/src/commands.ts +0 -244
  365. package/src/config.ts +0 -183
  366. package/src/connection.ts +0 -658
  367. package/src/context-store.ts +0 -265
  368. package/src/debug-bus.ts +0 -405
  369. package/src/extensions/tool-gate.ts +0 -220
  370. package/src/extensions/webui/ask-channel.ts +0 -52
  371. package/src/extensions/webui/index.ts +0 -184
  372. package/src/extensions/webui/tools/ask-question.ts +0 -88
  373. package/src/extensions/webui/tools/query-result.ts +0 -79
  374. package/src/extensions/webui/tools/render-chart.ts +0 -53
  375. package/src/extensions/webui/tools/render-item.ts +0 -61
  376. package/src/extensions/webui/tools/render-list.ts +0 -170
  377. package/src/extensions/webui/tools/render-result.ts +0 -85
  378. package/src/extensions/webui/tools/render-tree.ts +0 -71
  379. package/src/extensions/webui/tools/ui-widget.ts +0 -54
  380. package/src/extensions/webui/tools/widget-result.ts +0 -13
  381. package/src/extensions/webui/widget-channel.ts +0 -50
  382. package/src/file-log.ts +0 -196
  383. package/src/git-tracked.ts +0 -68
  384. package/src/gitignore.ts +0 -53
  385. package/src/json-store.ts +0 -43
  386. package/src/load-env.ts +0 -70
  387. package/src/loader-style.ts +0 -96
  388. package/src/main.ts +0 -140
  389. package/src/mcp-catalog.ts +0 -96
  390. package/src/models-catalog.ts +0 -72
  391. package/src/permissions.ts +0 -120
  392. package/src/project-dir.ts +0 -189
  393. package/src/project-info.ts +0 -108
  394. package/src/prompt/app-ui.ts +0 -59
  395. package/src/prompt/browser.ts +0 -27
  396. package/src/prompt/index.ts +0 -62
  397. package/src/server-info.ts +0 -62
  398. package/src/session-factory.ts +0 -442
  399. package/src/session-store.ts +0 -115
  400. package/src/shell-env.ts +0 -45
  401. package/src/skills.ts +0 -163
  402. package/src/sync-snapshot.ts +0 -140
  403. package/src/telemetry.ts +0 -239
  404. package/src/testomatio-auth.ts +0 -299
  405. package/src/testomatio.ts +0 -98
  406. package/src/theme.ts +0 -30
  407. package/src/user-env.ts +0 -94
  408. package/src/welcome.ts +0 -127
  409. package/src/workspace/safe-path.ts +0 -85
  410. package/src/workspace/test-md.ts +0 -70
  411. package/src/workspace-model.ts +0 -440
  412. package/testeiya.config.json +0 -14
@@ -1,80 +0,0 @@
1
- ---
2
- title: HTML
3
- description: Discover how to use Testomat.io's HTML Pipe to generate detailed HTML reports from your test runs. Learn how to configure environment variables to save reports, customize output paths, and view essential test data like execution time and status.
4
- type: article
5
- url: https://docs.testomat.io/test-reporting/pipes/html
6
- head:
7
- - tag: meta
8
- attrs:
9
- name: keywords
10
- content: Testomat.io, HTML reports, test reporting, test automation, environment variables, CI/CD, CodeceptJS, test result visualization, report generation
11
- ---
12
- ## Testomat.io HTML Pipe
13
-
14
- The Testomat.io HTML Pipe is a feature that allows you to generate and save HTML reports based on your test results. This functionality is part of the Testomat.io testing ecosystem.
15
-
16
- ### Prerequisites
17
-
18
- **🔌 TESTOMATIO users now have the option to generate an HTML report based on their test execution results. If a user hasn't SET `TESTOMATIO` environment, the report will still be generated**
19
-
20
- ```
21
- TESTOMATIO_HTML_REPORT_SAVE=1 <actual run command>
22
- ```
23
-
24
- ### Enabling HTML Reports
25
-
26
- To enable HTML reports, set the TESTOMATIO_HTML_REPORT_SAVE environment variable to 1. Additionally, you can customize the HTML report folder and filename using the following variables:
27
-
28
- - TESTOMATIO_HTML_REPORT_SAVE=1
29
- - TESTOMATIO_HTML_REPORT_FOLDER: Specify the folder for HTML reports
30
- - TESTOMATIO_HTML_FILENAME: Set the desired filename for the HTML report
31
-
32
- _!!!Please note that the name must include the extension ".html". If the extension is missing, the report will be saved with the default name = testomatio-report.html_
33
-
34
- ### Usage
35
-
36
- The HTML Pipe operates as part of the test execution process. When running your tests with the specified environment variables, the HTML Pipe will generate reports based on the test results.
37
-
38
- ### Report Generation
39
-
40
- Once the test run is complete, the HTML Pipe compiles the test results and generates an HTML report. The generated report is saved to the specified HTML output path.
41
-
42
- The HTML report includes essential information, such as run ID, status, parallel execution indication, run URL, execution time, execution date, and details of individual tests.
43
-
44
- #### Example Command
45
-
46
- 📊 Generate a report without triggering the TESTOMATIO pipe (no data sent to the client)
47
-
48
- _Execute the following command to generate a report without sending data to TESTOMATIO:_
49
-
50
- ```
51
- TESTOMATIO_HTML_REPORT_SAVE=1 npx codeceptjs run
52
-
53
- ```
54
-
55
- 📝 Generate a report based on the default folder/html-file name: This command triggers the test execution process and enables the HTML Pipe to generate reports based on the test results:
56
-
57
- - Output folder: html-report/
58
- - Output file: testomatio-report.html
59
-
60
- _Execute the following command to generate a report with default settings:_
61
-
62
- ```
63
- TESTOMATIO_HTML_REPORT_SAVE=1 TESTOMATIO={API_KEY} npx codeceptjs run
64
-
65
- ```
66
-
67
- 📂 Generate a report based on a custom HTML folder/html-file name.
68
- **!!!You can SET only one of the following parameters if needed: folder or file name**
69
-
70
- _Execute the following command to generate a report with both folder & file names:_
71
-
72
- ```
73
- TESTOMATIO_HTML_REPORT_SAVE=1 TESTOMATIO_HTML_REPORT_FOLDER=user-html-reporter TESTOMATIO_HTML_FILENAME=custom.html TESTOMATIO={API_KEY} npx codeceptjs run
74
-
75
- ```
76
-
77
- 🖥️ An example of a report can be seen in the screenshot below:
78
-
79
- ![HTML example](./images/html-pipe.png)
80
-
@@ -1,57 +0,0 @@
1
- ---
2
- title: Pipes
3
- description: Testomat.io's Pipes feature allows you to forward test data from your test framework to other services like GitHub, GitLab, or CSV reports. Pipes are configured via environment variables and can be customized to send data to analytics, databases, or notifications. You can also create custom pipes for specific needs, ensuring compatibility across different test frameworks.
4
- type: article
5
- url: https://docs.testomat.io/test-reporting/pipes/overview
6
- head:
7
- - tag: meta
8
- attrs:
9
- name: keywords
10
- content: Testomat.io, Pipes, test reporting, GitHub integration, GitLab, CSV reports, CI pipelines, custom pipes, environment variables, test frameworks
11
- ---
12
-
13
- import DirectoryLinks from '/src/components/DirectoryLinks.astro';
14
-
15
- <DirectoryLinks directory="test-reporting/pipes" />
16
-
17
- ## 📯 Pipes
18
-
19
- ### What are Pipes?
20
-
21
- Unlike other reporters, Testomat.io Reporter doesn't force you to report data only to Testomat.io application. Testomat.io Reporter collects data from test frameworks and submits it via pipes to other services.
22
-
23
- ![](./images/reporter-pipes.png)
24
-
25
- For instance, you can enable GitHub Pipe to create a comment on a GitHub Pull Request and CSV Export Pipe to write a report in CSV format. To report data to Testomat.io App, Testomat.io Pipe should be enabled, but not required if you don't plan to use Testomat.io App. You can also write your own pipe to:
26
-
27
- - 📈 send data to analytics storage
28
- - 🗄 store data into database
29
- - 📧 send emails
30
- - 💬 send notifications to chats
31
- - etc....
32
-
33
- Pipes work in the very same manner for all supported test frameworks, as well as for JUnit reports. So writing a pipe instead of a custom reporter for a particular framework, saves your time as you can be sure, this pipe will work with any attached test result.
34
-
35
- Testomat.io Reporter doesn't store test data in any internal format. Test data is processed in memory. While a test is running, all pipes are notified of changes.
36
-
37
- Pipes Concepts:
38
-
39
- - **Pipes are configured via environment variables** - each test framework has its own configuration (or not). To ensure that a pipe receives its configuration it is independent from a running test framework.
40
- - **Pipes are enabled if a corresponding environment variables are set** - for instance, if `TESTOMATIO` variable is present, Testomat.io Pipe is enabled, if `GH_PAT` variable is set, GitHub Pipe is enabled.
41
- - **Custom Pipes** can be created as modules and added to `package.json` file.
42
- - When used locally environment variables are expected to be set in `.env` file. When running on CI server use Environment Configuration a CI server provides to pass environment variables into the pipeline.
43
-
44
- ### Built-in Pipes
45
-
46
- - [Testomat.io](https://docs.testomat.io/test-reporting/pipes/testomatio)
47
- - [GitHub](https://docs.testomat.io/test-reporting/pipes/github)
48
- - [Gitlab](https://docs.testomat.io/test-reporting/pipes/gitlab)
49
- - [CSV](https://docs.testomat.io/test-reporting/pipes/csv)
50
- - [Bitbucket](https://docs.testomat.io/test-reporting/pipes/bitbucket)
51
- - [Debugging](https://docs.testomat.io/test-reporting/pipes/debug)
52
-
53
- ## Custom Pipe
54
-
55
- _to be updated_
56
-
57
- ....
@@ -1,331 +0,0 @@
1
- ---
2
- title: Advanced Options
3
- description: Learn how to use the Testomat.io Pipe for advanced reporting configurations in test automation. The guide covers setting environment variables, adding reports to test runs, managing parallel executions, filtering tests, and integrating with JIRA for issue tracking.
4
- type: article
5
- url: https://docs.testomat.io/test-reporting/pipes/testomatio
6
- head:
7
- - tag: meta
8
- attrs:
9
- name: keywords
10
- content: Test reporting, test automation, Testomat.io Pipe, environment variables, CI integration, parallel execution, test filtering, JIRA integration, build URL, test management
11
- ---
12
- ## Testomat.io Pipe
13
-
14
- ![](./images/testomatio.png)
15
-
16
- Testomat.io Pipe sends data to [Testomat.io Application](https://app.testomat.io). Testomat.io provides free projects forever and unlimited test runs. Even it is a cloud application you can use it for free.
17
-
18
- **🔌 To enable Testomat.io pipe set `TESTOMATIO` environment variable with API key provided by Testomatio.**
19
-
20
- ```
21
- TESTOMATIO={API_KEY} <actual run command>
22
- ```
23
-
24
- Here are some possible use cases where you can use additional configuration on reporter:
25
-
26
- - [Create Unmatched Tests](#create-unmatched-tests)
27
- - [Add Report to Run by ID](#add-report-to-run-by-id)
28
- - [Do Not Finalize Run](#do-not-finalize-run)
29
- - [Setting Report Title](#setting-report-title)
30
- - [Reporting Parallel Execution to To Same Run](#reporting-parallel-execution-to-to-same-run)
31
- - [Adding Report to RunGroup](#adding-report-to-rungroup)
32
- - [Adding Environments to Run](#adding-environments-to-run)
33
- - [Starting an Empty Run](#starting-an-empty-run)
34
- - [Manually Finishing Run](#manually-finishing-run)
35
- - [Setting Build URL](#setting-build-url)
36
- - [Publish Run](#publish-run)
37
- - [Assign Label To Run](#assign-label-to-run)
38
- - [Create Jira Issue for Run](#create-jira-issue-for-run)
39
- - [Filter Tests](#filter-tests)
40
-
41
- ### Create Unmatched Tests
42
-
43
- Testomat.io will not create tests from the report if they have not been previously imported. To create tests during the report `TESTOMATIO_CREATE` option can be used:
44
-
45
- ```bash
46
- TESTOMATIO={API_KEY} TESTOMATIO_CREATE=1 <actual run command>
47
- ```
48
-
49
- ### Add Report to Run by ID
50
-
51
- This feature is widely used when a run is executed on CI.
52
- A run is created before the test is started and it is marked as `scheduled`. Then
53
- a report is assigned to that run using `TESTOMATIO_RUN` environment variable and `{RUN_ID}` of a run:
54
-
55
- ```bash
56
- TESTOMATIO={API_KEY} TESTOMATIO_RUN={RUN_ID} <actual run command>
57
- ```
58
-
59
- ### Do Not Finalize Run
60
-
61
- If multiple reports are added to the same run, each of them should not finalize the run.
62
- In this case use `TESTOMATIO_PROCEED=1` environment variable, so the Run will be shown as `Running`
63
-
64
- ```
65
- TESTOMATIO={API_KEY} TESTOMATIO_PROCEED=1 TESTOMATIO_RUN={RUN_ID} <actual run command>
66
- ```
67
-
68
- After all reports were attached and run can be execute the following command:
69
-
70
- ```
71
- TESTOMATIO={API_KEY} TESTOMATIO_RUN={RUN_ID} npx @testomatio/reporter finish
72
- ```
73
-
74
- ### Setting Report Title
75
-
76
- Give a title to your reports by passing it as environment variable to `TESTOMATIO_TITLE`.
77
-
78
- ```bash
79
- TESTOMATIO={API_KEY} TESTOMATIO_TITLE="title for the report" <actual run command>
80
- ```
81
-
82
- ### Reporting Parallel Execution to To Same Run
83
-
84
- Provide a shared unique title to all runs that will be running in parallel, and add `TESTOMATIO_SHARED_RUN` environment var. So all reports will be sent to this run.
85
-
86
- ```bash
87
- TESTOMATIO={API_KEY} TESTOMATIO_TITLE="report for commit ${GIT_COMMIT}" TESTOMATIO_SHARED_RUN=1 <actual run command>
88
- ```
89
-
90
- We recommend using a commit hash as a title to create a new Run. In this case we ensure that run title is unique and will be the same for all parallel jobs running exactly for this commit.
91
-
92
- By default you can report to the same run if a run was created no more than 20 minutes ago. If you want to increase this time you can use `TESTOMATIO_SHARED_RUN_TIMEOUT` environment variable. `TESTOMATIO_SHARED_RUN_TIMEOUT`accepts value in minutes, so to increase timeout to 2 hours (120 minutes) use the following command:
93
-
94
- ```bash
95
- TESTOMATIO={API_KEY} TESTOMATIO_TITLE="report for commit ${GIT_COMMIT}" TESTOMATIO_SHARED_RUN=1 TESTOMATIO_SHARED_RUN_TIMEOUT=120 <actual run command>
96
- ```
97
-
98
- ### Adding Report to RunGroup
99
-
100
- Create/Add run to group by providing `TESTOMATIO_RUNGROUP_TITLE`:
101
-
102
- ```sh
103
- TESTOMATIO={API_KEY} TESTOMATIO_RUNGROUP_TITLE="Build ${BUILD_ID}" <actual run command>
104
- ```
105
-
106
- ### Adding Environments to Run
107
-
108
- Add environments to run by providing `TESTOMATIO_ENV` as comma seperated values:
109
-
110
- ```bash
111
- TESTOMATIO={API_KEY} TESTOMATIO_ENV="Windows, Chrome" <actual run command>
112
- ```
113
-
114
- ### Starting an Empty Run
115
-
116
- If you want to create a run and obtain its `{RUN_ID}` from [testomat.io](https://testomat.io) you can use `--launch` option:
117
-
118
- ```bash
119
- TESTOMATIO={API_KEY} npx @testomatio/reporter start
120
- ```
121
-
122
- This command will return `{RUN_ID}` which you can pass to other jobs in a workflow.
123
-
124
- > When executed with `--launch` a command provided by `-c` flag is ignored
125
-
126
- ### Manually Finishing Run
127
-
128
- If you want to finish a run started by `--launch` use `--finish` option. `TESTOMATIO_RUN` environment variable is required:
129
-
130
- ```bash
131
- TESTOMATIO={API_KEY} TESTOMATIO_RUN={RUN_ID} npx @testomatio/reporter finish
132
- ```
133
-
134
- ### Setting Build URL
135
-
136
- When running on CI reporter tries to detect automatically the URL of the current build. This URL will be set to Run report. Reporter automatically detects build URL for following CI services:
137
-
138
- - GitHub Actions
139
- - Azure Devops
140
- - Jenkins
141
- - CircleCi
142
- - Gitlab CI
143
-
144
- If you are using a different CI or you want to override build url run tests with `BUILD_URL` environment variable:
145
-
146
- ```bash
147
- TESTOMATIO={API_KEY} BUILD_URL=<build-url> <actual run command>
148
- ```
149
-
150
- ### Publish Run
151
-
152
- To make report publicly available pass `TESTOMATIO_PUBLISH=1` variable:
153
-
154
- ```
155
- TESTOMATIO={API_KEY} TESTOMATIO_PUBLISH=1 <actual run command>
156
- ```
157
-
158
- ### Assign Label To Run
159
-
160
- To assign a label to run, a **label with scope `runs` must exist**.
161
- Then a label can be passed via `TESTOMATIO_LABEL` variable:
162
-
163
- ```
164
- TESTOMATIO={API_KEY} TESTOMATIO_LABEL=release <actual run command>
165
- ```
166
-
167
- There is a way to pass a value to label via `:`:
168
-
169
- ```
170
- TESTOMATIO={API_KEY} TESTOMATIO_LABEL="module:checkout" <actual run command>
171
- ```
172
-
173
- Or you can set multiple labels separating them with comma:
174
-
175
- ```
176
- TESTOMATIO={API_KEY} TESTOMATIO_LABEL="release,module:checkout" <actual run command>
177
- ```
178
-
179
- ### Create Jira Issue for Run
180
-
181
- To automatically assign Run to Jira Issue pass the ID of that issue as `TESTOMATIO_JIRA_ID` variable
182
-
183
- ```
184
- TESTOMATIO={API_KEY} TESTOMATIO_JIRA_ID={Jira_Issue_ID} <actual run command>
185
- ```
186
-
187
- For example, if your Jira issue id is `TST-1`, command should be:
188
-
189
- ```
190
- TESTOMATIO={API_KEY} TESTOMATIO_JIRA_ID=TST-1 <actual run command>
191
- ```
192
-
193
- ### Filter Tests
194
-
195
- Testomatio allows you to enhance your test execution with the option to filter and manage test runs based on various criteria. This can help you efficiently organize and execute your tests. Below, we explain how to use the `--filter` option in conjunction with Testomatio for customized test runs.
196
-
197
- > The filtering functionality for test execution is currently available for the following frameworks: Playwright & CodeceptJS.
198
-
199
- To use the `--filter` option with Testomatio, follow the format:
200
-
201
- ```bash
202
- TESTOMATIO={API_KEY} npx @testomatio/reporter run <actual run command> --filter 'testomatio:{filter-type}={filter-value}'
203
- ```
204
-
205
- #### Available Filter Types
206
-
207
- **Tag Name Filter**
208
-
209
- Filter tests by tag names. Use this filter to select tests associated with specific tags.
210
-
211
- ![Tag Name Filter](./images/by-tag-name.png)
212
-
213
- _Command execution example_
214
-
215
- ```bash
216
- TESTOMATIO={API_KEY} npx @testomatio/reporter run 'actual run command' --filter 'testomatio:tag-name=smoke'
217
- ```
218
-
219
- **Plan ID Filter**
220
-
221
- Filter tests by Testomatio plan IDs. Use this filter to organize and execute tests based on plan IDs.
222
-
223
- ![Plan ID Filter](./images/by-plan-id.png)
224
-
225
- _Command execution example_
226
-
227
- ```bash
228
- TESTOMATIO={API_KEY} npx @testomatio/reporter run 'actual run command' --filter 'testomatio:plan-id=b62f3170'
229
- ```
230
-
231
- **Label Filter**
232
-
233
- Filter tests using custom labels. Labels allow you to categorize tests for better management and execution.
234
-
235
- Label Filter (by label name) Example 1:
236
- ![Label Filter Example 3](./images/by-label-1.png)
237
-
238
- Label Filter (by label id) Example 2:
239
- ![Label Filter Example 1](./images/by-label.png)
240
-
241
- Another Label Filter Example 3:
242
- ![Label Filter Example 2](./images/by-label-3.png)
243
-
244
- _Command execution example_
245
-
246
- ```bash
247
- TESTOMATIO={API_KEY} npx @testomatio/reporter run 'actual run command' --filter 'testomatio:label=new-label-test'
248
- ```
249
-
250
- OR
251
-
252
- ```bash
253
- TESTOMATIO={API_KEY} npx @testomatio/reporter run 'actual run command' --filter 'testomatio:label=NEW label TEST'
254
- ```
255
-
256
- **Label Filter based on the Severity type**
257
-
258
- If you want to use a specific severity type, you should join the "severity" label id + severity type(e.g.: `s1-id2:⛔ Blocker`).
259
-
260
- Here's a list of available types that you can copy and add to the run command:
261
-
262
- - ⚠️ Critical
263
- - ⛔ Blocker
264
- - 🔥 Major
265
- - 👌 Normal
266
- - ❄️ Minor
267
- - 💤 Trivial
268
-
269
- Label Filter Example based on the Severity type:
270
- ![Label Filter Example 3](./images/by-label-severity.png)
271
-
272
- _Command execution example_
273
-
274
- ```bash
275
- TESTOMATIO={API_KEY} npx @testomatio/reporter run 'actual run command' --filter 'testomatio:label=severity-f1b11:🔥 Major'
276
- ```
277
-
278
- OR
279
-
280
- ```bash
281
- TESTOMATIO={API_KEY} npx @testomatio/reporter run 'actual run command' --filter 'testomatio:label=NEW label TEST:💤 Trivial'
282
- ```
283
-
284
- **JIRA Ticket Filter**
285
-
286
- Filter tests linked to JIRA tickets. Use this filter to coordinate testing efforts with your JIRA workflow.
287
-
288
- ![JIRA Ticket Filter](./images/by-jira-id.png)
289
-
290
- _Command execution example_
291
-
292
- ```bash
293
- TESTOMATIO={API_KEY} npx @testomatio/reporter run 'actual run command' --filter 'testomatio:jira-ticket=LMP-51'
294
- ```
295
-
296
- **📚 Example Usage**
297
-
298
- For instance, if you want to run tests with a specific label named "new-label-test," you can use the following command:
299
-
300
- ```bash
301
- TESTOMATIO={API_KEY} npx @testomatio/reporter run 'npx codeceptjs run' --filter 'testomatio:label=new-label'
302
- ```
303
-
304
- OR using Playwright:
305
-
306
- ```bash
307
- TESTOMATIO={API_KEY} npx @testomatio/reporter run 'npx playwright test' --filter 'testomatio:label=new-label'
308
- ```
309
-
310
- > Remember to replace {API_KEY} with your actual Testomatio API key and adjust the `<actual run command>` according to your testing setup
311
-
312
- > It's important to note that the provided filter value must match exactly the corresponding tag name, plan ID, label, or JIRA ticket for the desired tests
313
-
314
- Please note, that this functionality allows you to easily filter and execute tests based on specific criteria, enhancing your testing experience.
315
-
316
- ### Exclude Tests from Report by Glob Pattern
317
-
318
- To exclude tests from the report by [glob pattern](https://www.npmjs.com/package/glob) use `TESTOMATIO_EXCLUDE_FILES_FROM_REPORT_GLOB_PATTERN` environment variable:
319
-
320
- ```bash
321
- TESTOMATIO={API_KEY} TESTOMATIO_EXCLUDE_FILES_FROM_REPORT_GLOB_PATTERN="**/*.setup.ts" <actual run command>
322
- ```
323
-
324
- You can use multiple patterns separated by `;`:
325
-
326
- ```bash
327
- TESTOMATIO={API_KEY} TESTOMATIO_EXCLUDE_FILES_FROM_REPORT_GLOB_PATTERN="**/*.setup.ts;tests/*.auth.js" <actual run command>
328
- ```
329
-
330
- (Any files in `node_modules` will be ignored).
331
-
@@ -1,229 +0,0 @@
1
- ---
2
- title: Python Test Frameworks
3
- description: Learn how to integrate the Testomat.io reporter with Python test frameworks like Pytest. The documentation covers installation, synchronizing tests, configuring test environments, reporting results, and submitting test artifacts. It also includes detailed configuration options such as managing test artifacts through S3 and setting up environment variables for reporting.
4
- type: article
5
- url: https://docs.testomat.io/test-reporting/python
6
- head:
7
- - tag: meta
8
- attrs:
9
- name: keywords
10
- content: Python test frameworks, Pytest, Testomat.io, test synchronization, test reporting, test artifacts, automation, S3 storage, test configuration, API integration, continuous integration, environment setup
11
- ---
12
-
13
- :::note
14
- Taken from [Pytestomatio Reporter Readme](https://github.com/testomatio/pytestomatio)
15
- :::
16
-
17
- [![Support Ukraine Badge](https://bit.ly/support-ukraine-now)](https://github.com/support-ukraine/support-ukraine)
18
-
19
- # testomat.io plugin for pytest
20
-
21
- ## uses testomat.io API:
22
-
23
- - https://testomatio.github.io/check-tests/
24
- - https://testomatio.github.io/reporter/
25
-
26
- ## Installation
27
-
28
- ```bash
29
- pip install pytestomatio
30
- ```
31
-
32
- ## Usage
33
-
34
- Synchronize tests to testomat.io and get back test id.
35
- Tests will not be executed
36
-
37
- ```bash
38
- pytest --testomatio sync
39
- ```
40
-
41
- Remove all test ids from source code. Tests will not be executed
42
-
43
- ```bash
44
- pytest --testomatio remove
45
- ```
46
-
47
- Run pytest and send test results into testomat.io.
48
- Test can be sent to testomat.io without ids in your test code. If testomat.io failed to match tests by title, it will create
49
- new tests for the run
50
-
51
- ```bash
52
- pytest --testomatio report
53
- ```
54
-
55
- Run pytest with debug parameter to get test data collected in metadata.json file
56
-
57
- ```bash
58
- pytest --testomatio debug
59
- ```
60
-
61
- ## Configuration with environment variables
62
- You can use environment variable to control certain features of testomat.io
63
-
64
- | Env variable | What it does | Examples |
65
- |--------|--------|-------|
66
- | TESTOMATIO| Provides token for pytestomatio to access and push data to testomat.io | TESTOMATIO=tstmt_***** pytest --testomatio sync|
67
- |TESTOMATIO_RUN|Name of a test run to create on testomat.io|TESTOMATIO_RUN="Smoke test" pytest --testomatio report|
68
- |TESTOMATIO_RUN_ID| Id of existing test run to use for sending test results to | TESTOMATIO_RUN_ID=98dfas0 pytest --testomatio report |
69
- |TESTOMATIO_RUNGROUP_TITLE| Create a group (folder) for a test run | TESTOMATIO_RUNGROUP_TITLE="Release 2.0" pytest --testomatio report|
70
- |TESTOMATIO_ENV|Assign environment to a test run |TESTOMATIO_ENV="linux,chrome,1920x1080" pytest --testomatio report|
71
- |TESTOMATIO_LABELS|Assign labels to a test run |TESTOMATIO_ENV="smoke,regression" pytest --testomatio report|
72
- |TESTOMATIO_SYNC_LABELS|Assign labels to a test case when you synchronise test from code with testomat.io|TESTOMATIO_SYNC_LABELS="number:1,list:one,standalone" pytest --testomatio report|
73
- |TESTOMATIO_CODE_STYLE|If you are not sure, don't set this variable. Default value is 'default'|TESTOMATIO_CODE_STYLE=pep8 pytest --testomatio sync|
74
- |TESTOMATIO_CI_DOWNSTREAM|If set, pytestomatio will not set or update build url for a test run. This is useful in scenarios where build url is already set in the test run by Testomat.io for test runs that a created directly on Testomat.io.|TESTOMATIO_CI_DOWNSTREAM=true pytest --testomatio report|
75
-
76
-
77
- ### Run groups parameter
78
- There is environment variable `TESTOMATIO_RUNGROUP_TITLE` that can be used to specify run tests with specific group title.
79
-
80
- ### pytest.ini
81
- In case you are using private testomat.io service, create `pytest.ini` file in your project root directory. Specify
82
- testomat.io url in it
83
-
84
- ```ini
85
- [pytest]
86
- testomatio_url = https://app.testomat.io
87
-
88
- ```
89
-
90
- ### Submitting Test Run Environment
91
-
92
- to configure test environment, you can use additional option:
93
-
94
- ```bash
95
- pytest --testomatio report --testRunEnv "windows11,chrome,1920x1080"
96
- ```
97
-
98
- Environment values are comma separated, please use double quotation.
99
-
100
-
101
- ### Submitting Test Artifacts
102
-
103
- testomat.io does not store any screenshots, logs or other artifacts.
104
-
105
- In order to manage them it is advised to use S3 Buckets (GCP Storage).
106
- https://docs.testomat.io/usage/test-artifacts/
107
-
108
- Analyser needs to be aware of the cloud storage credentials.
109
- There are two options:
110
- 1. Enable **Share credentials with testomat.io Reporter** option in testomat.io Settings -> Artifacts.
111
- 2. Use environment variables `ACCESS_KEY_ID, SECRET_ACCESS_KEY, ENDPOINT, BUCKET, BUCKET_PATH`
112
-
113
- You would need to decide when you want to upload your test artifacts to cloud storage
114
-
115
- 1) Upload page screenshot when test fails, using fixtures [reference](https://docs.pytest.org/en/latest/example/simple.html#making-test-result-information-available-in-fixtures)
116
-
117
- ```python
118
- # content of conftest.py
119
- import pytest
120
- import random
121
- import os
122
- from typing import Dict
123
- from pytest import StashKey, CollectReport
124
- from playwright.sync_api import Page
125
-
126
- phase_report_key = StashKey[Dict[str, CollectReport]]()
127
-
128
- @pytest.hookimpl(wrapper=True, tryfirst=True)
129
- def pytest_runtest_makereport(item, call):
130
- rep = yield
131
- item.stash.setdefault(phase_report_key, {})[rep.when] = rep
132
- return rep
133
-
134
-
135
- @pytest.fixture(scope="function")
136
- def handle_artifacts(page: Page, request):
137
- yield
138
- report = request.node.stash[phase_report_key]
139
- if ("call" not in report) or report["setup"].failed or report["call"].failed:
140
- random_string = ''.join(random.choices(string.ascii_letters + string.digits, k=8))
141
-
142
- filename = f"{random_string}.png"
143
- screenshot_path = os.path.join(artifacts_dir, filename)
144
- page.screenshot(path=screenshot_path)
145
- # file_path - required, path to file to be uploaded
146
- # file_bytes - required, bytes of the file to be uploaded
147
- # key - required, file name in the s3 bucket
148
- # bucket_name - optional,name of the bucket to upload file to. Default value is taken from testomat.io
149
- artifact_url = pytest.testomatio.upload_file(screenshot_path, filename)
150
- # or
151
- # artifact_url = pytest.testomatio.upload_file_object(file_bytes, key, bucket_name)
152
- pytest.testomatio.add_artifacts(request.node, [artifact_url])
153
- page.close()
154
- ```
155
-
156
- ⚠️ Please take into account s3_connector available only after **pytest_collection_modifyitems()** hook is executed.
157
-
158
- 2) If you prefer to use pytest hooks - add `pytest_runtest_makereport` hook in your `conftest.py` file.
159
-
160
- ```python
161
- def pytest_runtest_makereport(item, call):
162
- artifact_url = pytest.testomatio.upload_file(screenshot_path, filename)
163
- pytest.testomatio.add_artifacts([artifact_url])
164
- ```
165
-
166
- ### Clarifications
167
-
168
- - tests can be synced even without `@patest.mark.testomatio('@T96c700e6')` decorator.
169
- - test title in testomat.io == test name in pytest
170
- - test suit title in testomat.io == test file name in pytest
171
-
172
- ## Example of test
173
-
174
- To make the experience more consistent, it uses standard pytest markers.
175
- testomat.io test id is a string value that starts with `@T` and has 8 symbols after.
176
-
177
- ```python
178
- from pytest import mark
179
-
180
-
181
- @mark.testomatio('@T96c700e6')
182
- def test_example():
183
- assert 2 + 2 == 4
184
- ```
185
-
186
- ### Compatibility table with [Testomatio check-tests](https://github.com/testomatio/check-tests)
187
-
188
- | Action | Compatibility | Method |
189
- |--------|--------|-------|
190
- | Report tests to testomat.io | complete | `pytest --testomatio report` |
191
- | Filter test by id to run | complete | `pytest --testomatio report --test-id="T00C73028\|T00C73029"` |
192
- | Importing test into testomat.io | complete | `pytest --testomatio sync` |
193
- | Exclude hook code of a test | N/A | N/A |
194
- | Include line number code of a test | N/A | N/A |
195
- | Import Parametrized Tests | complete | default behaviour |
196
- | Disable Detached Tests | complete | `pytest --testomatio sync --no-detach` |
197
- | Synchronous Import | complete | default behaviour |
198
- | Auto-assign Test IDs in Source Code | complete | default behaviour |
199
- | Keep Test IDs Between Projects | complete | `pytest --testomatio sync --create` |
200
- | Clean Test IDs | complete | `pytest --testomatio remove` |
201
- | Import Into a Branch | N/A | N/A |
202
- | Keep Structure of Source Code | complete | `pytest --testomatio sync --keep-structure` |
203
- | Delete Empty Suites | complete | `pytest --testomatio sync --no-empty` |
204
- | Import Into a Specific Folder | complete | `pytest --testomatio sync --directory "Windows\smoke"` |
205
- | Debugging | parity | `pytest --testomatio debug` |
206
-
207
-
208
- ## Test
209
- - import into empty project
210
- - updated test - (resync)
211
- - test run
212
- - test run into a folder
213
- - test run labels, tags
214
-
215
- ## TODO
216
- - retry test run update with less attributes, we get 500 from api
217
- - handler non configured s3 bucket error
218
- - Fix test duration
219
-
220
- ## Contribution
221
- Use python 3.12
222
-
223
- 1. `pip install ".[dev]"`
224
- 1. `python ./smoke.py`
225
- 1. Test things manually
226
- 1. Verify no regression bugs
227
- 1. `cz commit`
228
- 1. `cz bump`
229
- 1. `git push remoteName branchName --tags`