testeiya 0.2.0

package/skills/testeiya/testomatio-docs/docs/test-reporting/junit.md

@@ -0,0 +1,248 @@
+---
+title: JUnit Reporter
+description: Learn how to use the JUnit reporter with Testomat.io to upload XML test reports from various frameworks like JUnit, Pytest, NUnit, and more. Testomat.io automatically processes these reports to generate detailed test runs and create tests directly in the system. Supports Playwright, Ruby, Java, Python, and PHP.
+type: article
+url: https://docs.testomat.io/test-reporting/junit
+head:   
+  - tag: meta
+    attrs:
+      name: keywords
+      content: JUnit reporter, Testomat.io, test automation, XML report, Pytest, NUnit, test result import, Playwright, CI/CD integration, test management
+---
+## JUnit Reports
+
+JUnit XML format is standard among test runners on various platforms. Testomat.io can load XML reports from test runners and create tests in a project from it. If your framework is not supported yet, generate JUnit report and upload it into Testomat.io
+
+Testomat.io will not only create a run report but also collect all information about tests, including source code, and create tests in a system as regular importer do. The only difference that normal process of Testomat.io import does not require executing tests on import, while importing via reporter requires to have tests executed and XML report provided.
+
+Tested Frameworks:
+
+- JUnit (JUnit)
+- Python (Pytest)
+- Minitest (Ruby)
+- PHPUnit (PHP)
+- NUnit (C#)
+
+To enable Testomatio Reporter install `@testomatio/reporter` package
+
+Use one of your favorite package managers:
+
+```
+npm install @testomatio/reporter --save-dev
+```
+
+```
+pnpm install @testomatio/reporter --save-dev
+```
+
+```
+yarn add @testomatio/reporter --dev
+```
+
+Run your test framework and generate a JUnit report.
+
+Then import XML report into Testomat.io
+
+```
+npx @testomatio/reporter xml "{pattern}" --lang={lang}
+```
+
+- `pattern` - is a glob pattern to match all XML files from report. For instance, `"test/report/**.xml"` or just `report.xml`
+- `--lang` option can be specified to identify source code of the project. Example: `--lang=Ruby` or `--lang=Java` or `--lang=Python`. Possible values:
+  - `c#`
+  - `java`
+  - `ruby`
+  - `python`
+  - `php`
+- `--java-tests` option is avaiable for Java projects, and can be set if path to tests is different then `src/test/java`. When this option is enable, `lang` option is automatically set to `java`
+- `--env-file <envfile>` option to load environment variables from .env file. Inside env file TESTOMATIO credentials like `TESTOMATIO` api key or [bucket config for artifats](./artifacts.md).
+- `--timelimit <time>` set a timer to silently kill a long-running reporter process due to network or other issues. For instance, use `--set-timeout=3` to stop process after 3 secs.
+
+## Pytest
+
+Run pytest tests and generate a report to `report.xml`:
+
+```
+pytest --junit-xml report.xml
+```
+
+Import report with this command
+
+```
+TESTOMATIO={API_KEY} npx @testomatio/reporter xml report.xml --lang=python
+```
+
+## JUnit
+
+Run tests via Maven, make sure JUnit report was configured in `pom.xml`.
+
+```
+mvn clean test
+```
+
+Import report with this command:
+
+```
+TESTOMATIO={API_KEY} npx @testomatio/reporter xml "target/surefire-reports/*.xml" --java-tests
+```
+
+> [!NOTE]
+> If your tests are located in a folder other other than `src/test/java`, specify a path to test files using `--java-tests` option: `--java-tests="path/to/tests"`
+
+Screenshots or videos from tests are uploaded if test contains output with a path to file of following format:
+
+```
+file://path/to/screenshot.png
+```
+
+Use `System.out.println` to print an absulute path to file that should be uploaded as a screenshot.
+
+```java
+System.out.println("file://" + pathToScreenshot);
+```
+
+This will produce XML report which contains path to a file:
+
+```xml
+<testcase>
+  <system-out><![CDATA[
+    file://path/to/scrrenshot.png
+  ]]></system-out>
+</testcase>
+```
+
+When XML report is uploaded, all files from `file://` will be uploaded to corresponding tests.
+
+### Assign Test ID
+
+Test IDs can be taken applied to a test name. So to assign a test in a result to to test in Testomat.io with ID `@T8acca9eb` add this ID to a test name if your test framework allows setting test names as a string.
+
+```xml
+<testcase name="test name @T8acca9eb"> ....
+```
+
+To place a test inside a specific sute you can set a suite ID similarly to a test ID:
+
+```xml
+<testcase name="test name @S1428a8fa"> ....
+```
+
+If a file with a source code is available to a reporter, you can link test in the test body and specify Test ID as a comment:
+
+```java
+  @Test
+  public void testAddition() {
+    // @T8acca9eb
+    MyMath myMath = new MyMath();
+    int result = myMath.add(2, 3);
+    assertEquals(5, result);
+}
+```
+
+In this case `@TT8acca9eb` is ID of existing test inside Testomat.io project
+
+Alternatively, if the code can't be imported by a reported, use output inside a test to print its ID:
+
+```java
+public void testAddition() {
+  MyMath myMath = new MyMath();
+  int result = myMath.add(2, 3);
+  System.out.println("tid://@T8acca9eb"); // here we print test ID
+  assertEquals(5, result);
+}
+```
+
+## NUnit
+
+Generate NUnit XML report and run the following code:
+
+```
+TESTOMATIO={API_KEY} npx @testomatio/reporter xml "report.xml" --lang="c#"
+```
+
+If NUnit generates `<ResultFiles>` section in XML report, all items from it will be uploaded as artifacts. For instance, this is available for **Playwright DotNet** integration.
+
+```xml
+<Results>
+  <UnitTestResult>
+    <ResultFiles>
+      <ResultFile path="screenshot2.png" />
+      <ResultFile path="data.zip" />
+    </ResultFiles>
+  </UnitTestResult>
+```
+
+To upload an arbitrary file as artifact, print its absulute path to console:
+
+```c#
+Console.WriteLine("file://path/to/file.png");
+```
+
+### Assign Test ID
+
+To link test in source code with test in Testomat.io print a Test ID inside a test:
+
+```java
+Console.WriteLine("tid://@T8acca9eb"); // here we print test ID
+```
+
+Use `tid://` prefix with a existing Test ID to match test with ID.
+
+## Ruby Minitest
+
+```ruby
+
+
+reporters = [Minitest::Reporters::DefaultReporter.new(color: true)]
+
+reporters << Minitest::Reporters::JUnitReporter.new
+
+Minitest::Reporters.use! reporters
+```
+
+Launch tests:
+
+```
+rails test
+```
+
+Import reports from `test/reports` directory:
+
+```
+TESTOMATIO={API_KEY} npx @testomatio/reporter xml "test/reports/*.xml" --lang ruby
+```
+
+### Assign Test ID
+
+Add test ID to a test name prefixed with `@T`. For instance, to link test from source code to test with id `@T8acca9eb` add TID to a test title:
+
+```ruby
+  test 'export test as yaml @T8acca9eb' do
+    # ....
+  end
+```
+
+To place a test inside a specific sute you can set a suite ID similarly to a test ID:
+
+```ruby
+  test 'export test as yaml @S1428a8fa' do
+    # ....
+  end
+```
+
+To link test in source code with test in Testomat.io print a Test ID inside a test:
+
+```ruby
+puts "tid://@T8acca9eb" // here we print test ID
+```
+
+Use `tid://` prefix with a existing Test ID to match test with ID.
+
+## PHPUnit
+
+Generate PHPUnit XML and import it
+
+```
+TESTOMATIO={API_KEY} npx @testomatio/reporter xml "report.xml" --lang php
+```
+

package/skills/testeiya/testomatio-docs/docs/test-reporting/logger.md

@@ -0,0 +1,138 @@
+---
+title: Logger
+description: Learn how to use Testomat.io’s Logger to intercept and attach log messages from various logging frameworks (e.g., Winston, Pino) to your test reports. The Logger offers convenient syntax for logging, customizes output formats, and supports popular frameworks like Playwright, Jest, and Mocha.
+type: article
+url: https://docs.testomat.io/test-reporting/logger
+head:
+  - tag: meta
+    attrs:
+      name: keywords
+      content: Testomat.io, logger, test reporting, Playwright, Winston, Pino, Mocha, Jest, log interception, test automation, console logs
+---
+
+
+- intercepts `console` logs by default (requires `TESTOMATIO_INTERCEPT_CONSOLE_LOGS` to be set for some frameworks)
+- intercepts any logger messages (winston, pino, etc., specified by user)
+- attaches intercepted log messages to your report
+- provides own methods to add any info to your report (like `logger.info()`, `logger.debug()`, etc.)
+- has a varied and convenient syntax
+- automatically converts any data to human-readable format
+
+> Please check if your framework supported in the list below [Supported frameworks](#supported-frameworks)
+
+For proper logger work, your tests titles should contain testomatio test ids ([see details](https://docs.testomat.io/usage/continuous-integration/#assigning-ids)).
+
+The logger supports next methods: `assert`, `debug`, `error`, `info`, `log`, `trace` and `warn`.
+
+Logging functionality is represented by 2 entitites:
+
+Logger has ability to intercept other loggers. [usage](#usage)
+
+## Usage
+
+Log messages with different levels:
+
+```javascript
+const { logger } = require('@testomatio/reporter');
+// or
+import { logger } from '@testomatio/reporter';
+
+logger.info('message');
+logger.error('message');
+```
+
+Logger allows configuration (See [Configuration](#configuration) section for details).
+
+### Convenient syntax with "log" function
+
+```javascript
+const { log } = require('@testomatio/reporter');
+
+test('Your test @T12345678', async () => {
+  await page.login(user);
+  log`I was logged in with user ${user}`; // <<<<<
+  assert(loggedIn);
+});
+```
+
+More examples:
+
+```javascript
+const { log } = require('@testomatio/reporter');
+// or
+import { log } from '@testomatio/reporter';
+
+log`your message`;
+log`your message ${variable}`;
+```
+
+### Intercept logs from your logger (and attach them to testomatio report)
+
+```javascript
+const { logger } = require('@testomatio/reporter')
+
+logger.intercept(console); // intercept console
+
+describe('Your suite @S12345678', () => {
+  test('Your test @T12345678', async () => {
+    await page.login();
+    console.log('this message will be intercepted and added to your report'); // <<
+    assert(something);
+  });
+```
+
+Pass the logger object you want to intercept, not its name:\
+✅ `logger.intercept(pino)`\
+❌ `logger.intercept('pino')`
+
+NodeJS `console` is intercepted by default for some frameworks (CodeceptJS, Cucumber), to add console logs to your report just import logger in your test file and use console as usual. For other frameworks you need to set `TESTOMATIO_INTERCEPT_CONSOLE_LOGS` env variable to any truthy value. Or intercept console (or any other logger) manually (see examples above or below).
+
+### Examples for other loggers intercepting
+
+```javascript
+const logLevel = require('loglevel');
+const pino = require('pino')();
+const morgan = require('morgan');
+const tracer = require('tracer').console();
+const tracerColor = require('tracer').colorConsole();
+
+logger.intercept(logLevel);
+logger.intercept(pino);
+logger.intercept(morgan);
+logger.intercept(tracer);
+logger.intercept(tracerColor);
+```
+
+> Note: if you import testomatio logger as "logger", be sure not to import other loggers as "logger" too. Otherwise, you will get an error.
+
+## Configuration
+
+After you import logger, you can configure it:
+
+```javascript
+const { logger } = require('@testomatio/reporter');
+loggger.configure({
+  logLevel: 'WARN',
+  prettyObjects: true,
+});
+```
+
+List of available options:
+
+- `prettyObjects` [boolean] - if true, objects will be printed on multiple lines (easier to read). Default: `false` (object are printed on one line)
+- `logLevel` [error, warn, log, info, debug, trace, verbose, all] - ignores messages below the log level set. Also could be set by `LOG_LEVEL` env variable. Default: `all`
+
+### Supported frameworks
+
+This feature is under development right now. List of supported frameworks:
+
+- 🟢 CodeceptJS
+- 🔴 Cypress
+- 🟢 Cucumber
+- 🟢 Jest
+- 🟢 Mocha
+- 🟢 Newman (Postman) (console logs are added by testomatio reporter by default, no need to use logger)
+- 🟢 Playwright
+- 🟢 Puppeteer (using Jest)
+- 🔴 WebdriverIO
+

package/skills/testeiya/testomatio-docs/docs/test-reporting/php.md

@@ -0,0 +1,77 @@
+---
+title: PHP Test Frameworks
+description: Learn how to integrate PHP test frameworks like Codeception and PHPUnit with Testomat.io. The guide covers installation, setting API keys, and configuring test reporting. It supports seamless integration for automated test results with real-time reporting, enhancing test management workflows.
+type: article
+url: https://docs.testomat.io/test-reporting/php
+head:
+  - tag: meta
+    attrs:
+      name: keywords
+      content: Testomat.io, PHP, Codeception, PHPUnit, test reporting, automated testing, CI integration, API keys, real-time reports, test management
+---
+
+:::note
+ Taken from [PHP Reporter Readme](https://github.com/testomatio/php-reporter)
+:::
+ 
+
+## Installation
+
+```
+composer require testomatio/reporter --dev
+```
+
+### Codeception
+
+> [!NOTE]
+> For Codeception 4.x install `testomatio/php-reporter` version 0.4.0, for Codeception 5+ install `1.0.0` version of this reporter.
+
+Get API key from Testomatio application and set it as `TESTOMATIO` environment variable.
+Run your tests with`Testomatio\Reporter\Codeception` extension enabled: 
+
+On Linux/MacOS:
+
+```
+TESTOMATIO={apiKey} php vendor/bin/codecept run --ext "Testomatio\Reporter\Codeception"
+```
+
+On Windows
+
+```
+set TESTOMATIO={apiKey}&& php vendor/bin/codecept run  --ext "Testomatio\Reporter\Codeception"
+```
+
+Alternatively, you can add `Testomatio\Reporter\Codeception` extension to suite or global config. 
+Reporter will be started only when `TESTOMATIO` environment variable is set:
+
+```yml
+extensions:
+  enabled:
+    - Testomatio\Reporter\Codeception
+```
+
+### PHPUnit
+
+> PHPUnit v10+ is not implemented yet
+
+Add `Testomatio\Reporter\PHPUnit` listener to `phpunit.xml` or `phpunit.dist.xml`
+
+```xml
+<listeners>
+    <listener class="Testomatio\Reporter\PHPUnit" file="vendor/testomatio/php-reporter/src/PHPUnit.php" />
+</listeners>
+```
+Run tests and provide `TESTOMATIO` API key as environment variable:
+
+On Linux/MacOS:
+
+```
+TESTOMATIO={apiKey} php vendor/bin/phpunit
+```
+
+On Windows
+
+```
+set TESTOMATIO={apiKey}&& php vendor/bin/phpunit
+```
+

package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/bitbucket.md

@@ -0,0 +1,72 @@
+---
+title: Bitbucket
+description: Learn how to integrate Testomat.io with Bitbucket Pipes to streamline test reporting in pull requests. Automatically add test run summaries, including status, failed/passed/skipped test counts, stack traces, and screenshots. Configure Bitbucket with access tokens for seamless integration into your CI/CD pipeline using Playwright.
+type: article
+url: https://docs.testomat.io/test-reporting/pipes/bitbucket
+head:
+  - tag: meta
+    attrs:
+      name: keywords
+      content: Bitbucket integration, Testomat.io, test reporting, CI/CD, automated testing, Playwright, pull request, test summary, failed tests, access tokens
+---
+## Bitbucket Pipe
+
+Bitbucket Pipe adds a comment with a summary of a run to a Pull Request:
+
+![](./images/bitbucket.png)
+
+This summary will contain:
+
+- Status of a test run
+- Number of failed/passed/skipped tests
+- Stack traces of failing tests (first 10)
+- Screenshots of failed tests (if available)
+- List of 5 slowest tests
+
+**🔌 Тo enable Bitbucket pipe set BITBUCKET_ACCESS_TOKEN in Bitbucket Repository variables**
+
+To use the BITBUCKET_ACCESS_TOKEN from Repository Access Tokens in Bitbucket Pipelines, follow these steps:
+
+1. In Bitbucket, go to your repository settings.
+   ![Step 1](./images/bbk-1.png)
+2. Select "Repository Access Tokens" under the "Access management" section.
+   ![Step 2](./images/bbk-2.png)
+3. Create a new Access Token, granting it the necessary permissions (e.g., read and write for the repository).
+   ![Step 3](./images/bbk-3.png)
+
+Now, you need to add this token as an environment variable in Bitbucket Pipelines:
+
+1. Go to "Pipelines" in your repository settings.
+   ![Step 4](./images/bbk-4.png)
+2. Select "Repository variables" under the "Settings" section.
+   ![Step 5](./images/bbk-5.png)
+3. Add a new variable with the name BITBUCKET_ACCESS_TOKEN and paste the token
+   ![Step 6](./images/bbk-6.png)
+
+Once you've done that, your pipelines.yml configuration file will automatically use this token. Here's how it should look:
+
+```yaml
+image: atools/chrome-headless:java17-nodelts-latest
+
+pipelines:
+  pull-requests:
+    '**':
+      - step:
+          name: Run Playwright tests
+          script:
+            - npm install
+            - npx playwright install --with-deps chromium
+            - BITBUCKET_ACCESS_TOKEN=$BITBUCKET_ACCESS_TOKEN TESTOMATIO=$TESTOMATIO npx playwright test
+```
+
+### Keep Outdated Reports
+
+If a pipeline is executed multiple times, comment with previous reports will be deleted. To keep them pass `BITBUCKET_KEEP_OUTDATED_REPORTS` variable:
+
+```yaml
+script:
+  - npm install
+  - npx playwright install --with-deps chromium
+  - BITBUCKET_KEEP_OUTDATED_REPORTS=1 BITBUCKET_ACCESS_TOKEN=$BITBUCKET_ACCESS_TOKEN TESTOMATIO=$TESTOMATIO npx playwright test
+```
+

package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/csv.md

@@ -0,0 +1,23 @@
+---
+title: CSV
+description: The CSV Pipe in Testomat.io allows you to export test results as a CSV file by setting the TESTOMATIO_CSV_FILENAME environment variable. This functionality integrates with CI workflows to generate reports in a simple, readable format, enabling easy sharing and analysis of test results across teams.
+type: article
+url: https://docs.testomat.io/test-reporting/pipes/csv
+head: 
+  - tag: meta
+    attrs:
+      name: keywords
+      content: CSV Pipe, Testomat.io, test reporting, CI integration, CSV export, test results, automated testing, TESTOMATIO_CSV_FILENAME
+---
+## CSV Pipe
+
+CSV Pipe saves test result as CSV file
+
+**🔌 To enable CSV pipe set `TESTOMATIO_CSV_FILENAME` environment variable with the name of CSV file to save into**
+
+Add an env to run by specifying the `TESTOMATIO_CSV_FILENAME` variable.
+
+```bash
+TESTOMATIO_CSV_FILENAME="report.csv" <actual run command>
+```
+

package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/debug.md

@@ -0,0 +1,18 @@
+---
+title: Debug
+description: The Debug Pipe is used to store data for debugging purposes in a temporary json file.
+type: article
+url: https://docs.testomat.io/test-reporting/pipes/debug
+---
+
+## Debug Pipe
+
+Debug Pipe stores data for debugging purposes in a temporary json file.
+
+**🔌 To enable Debug pipe set `TESTOMATIO_DEBUG` environment variable with value `true` or `1` **
+
+Add an env to run by specifying the `TESTOMATIO_DEBUG` variable.
+
+```bash
+TESTOMATIO_DEBUG=1 <actual run command>
+```

package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/github.md

@@ -0,0 +1,63 @@
+---
+title: Github
+description: The GitHub Pipe feature in Testomat.io enables automated comments on pull requests, summarizing test results with key metrics like passed/failed/skipped tests, stack traces, and slowest tests. It integrates through the GH_PAT environment variable, with options to keep previous reports using GH_KEEP_OUTDATED_REPORTS. This enhances CI workflows by providing detailed feedback directly in GitHub.
+type: article
+url: https://docs.testomat.io/test-reporting/pipes/github
+head:  
+  - tag: meta
+    attrs:
+      name: keywords
+      content: GitHub Pipe, Testomat.io, pull request comments, test reporting, CI integration, GitHub Actions, GH_PAT, automated testing, test management
+---
+## GitHub Pipe
+
+GitHub Pipe adds a comment with a summary of a run to a Pull Request:
+
+![](./images/github.png)
+
+This summary will contain:
+
+- Status of a test run
+- Number of failed/passed/skipped tests
+- Stack traces of failing tests (first 20)
+- Screenshots of failed tests (if available)
+- List of 5 slowest tests
+
+**🔌 To enable GitHub pipe set `GH_PAT` environment with GitHub Private Access Token**
+
+When using GitHub Actions inside Pull Request you can set GH_PAT from `github.token` system variable. Here is a sample workflow:
+
+```yaml
+name: CI
+
+on:
+  push:
+    branches:
+      - main
+  pull_request:
+
+jobs:
+  test:
+    permissions:
+      # These permissions are required for testomat to create comments on pull requests
+      issues: write
+      pull-requests: write
+    # pre-execution steps
+    - name: Run Tests
+      run: <actual test command with @testomatio/reporter enabled>
+      env:
+        GH_PAT: ${{ github.token }}
+```
+
+### Keep Outdated Reports
+
+If a pipeline is executed multiple times, comment with previous reports will be deleted. To keep them pass `GH_KEEP_OUTDATED_REPORTS` variable:
+
+```yaml
+- name: Run Tests
+  run: <actual test command with @testomatio/reporter enabled>
+  env:
+    GH_PAT: ${{ github.token }}
+    GH_KEEP_OUTDATED_REPORTS: 1
+```
+

package/skills/testeiya/testomatio-docs/docs/test-reporting/pipes/gitlab.md

@@ -0,0 +1,29 @@
+---
+title: GitLab
+description: The GitLab Pipe in Testomat.io enables automatic test result reporting directly into GitLab merge requests. It provides detailed test metrics, including passed, failed, and skipped tests, along with stack traces. The integration is achieved through environment variables like GITLAB_TOKEN, improving visibility into test results within the GitLab CI/CD pipeline.
+type: article
+url: https://docs.testomat.io/test-reporting/pipes/gitlab
+head: 
+  - tag: meta
+    attrs:
+      name: keywords
+      content: GitLab Pipe, Testomat.io, merge request comments, test reporting, CI integration, GitLab CI/CD, GITLAB_TOKEN, automated testing, test management
+---
+## GitLab Pipe
+
+Similarly to [GitHub Pipe](./github.md#github-pipe), GitLab Pipe adds a comment with a summary of a run to a Merge Request:
+
+This summary will contain:
+
+- Status of a test run
+- Number of failed/passed/skipped tests
+- Stack traces of failing tests (first 20)
+- Screenshots of failed tests (if available)
+- List of 5 slowest tests
+
+**🔌 To enable GitLab pipe set `GITLAB_PAT` environment with GitLab Private Access Token**
+
+### Keep Outdated Reports
+
+If a pipeline is executed multiple times, comment with previous reports will be deleted. To keep them set `GITLAB_KEEP_OUTDATED_REPORTS=1`.
+