@testomatio/reporter 1.0.0-beta.4 → 1.0.0

package/Changelog.md

@@ -2,7 +2,14 @@
 
 <!-- pending release updates -->
 
-* Added `TESTOMATIO_SHARED_RUN` option to use a shared run for parallel executions
+* Added [`TESTOMATIO_SHARED_RUN` option](https://github.com/testomatio/reporter/blob/master/docs/pipes.md#reporting-parallel-execution-to-to-same-run) to use a shared run for parallel executions
+* Reworked [documentation](https://github.com/testomatio/reporter/tree/master#readme).
+* Added an option to obtain [S3 configuration](https://github.com/testomatio/reporter/blob/master/docs/artifacts.md#configuration) from Testomat.io
+* Introduced [pipes](https://github.com/testomatio/reporter/blob/master/docs/pipes.md):
+  * GitHub
+  * GitLab
+  * CSV Pipe
+
 
 # 0.7.6
 

package/README.md

@@ -1,603 +1,145 @@
-# @testomatio/reporter
+# Testomatio Reporter
 
-Library for sending test run reports to your [testomat.io](https://testomat.io) project.
 
-## Installation
-
-Get the `{API_KEY}` from testomat.
-
-You can refer sample tests from example folder of this repo. This is a basic example. If you need something full fledged you can refer this [example repo](https://github.com/testomatio/examples).
-
-Add `@testomatio/reporter` package to your project:
-
-```bash
-npm i @testomatio/reporter --save
-```
-
-For testcafe use testcafe reporter:
-
-```bash
-npm i testcafe-reporter-testomatio
-```
-
-For newman use:
-
-```bash
-npm i newman-reporter-testomatio --save-dev
-```
-
->`newman` and `newman-reporter-testomatio` should be installed in the same directory.
-\
-If you run your tests using globally installed newman (`newman run ...`), intall `newman-reporter-testomatio` globally too (`npm i newman-reporter-testomatio -g`).
-\
-If you use locally installed newman (within the project) (`npx newman run ...`), install `newman-reporter-testomatio` locally (`npm i newman-reporter-testomatio`).
-You can verify installed packages via `npm list` or `npm list -g`.
-
-## Usage
-
-### CodeceptJS
-
-Make sure you load all your tests using [check-tests](https://github.com/testomatio/check-tests#cli).
-
-Add plugin to [codecept conf](https://github.com/testomatio/reporter/blob/master/example/codecept/codecept.conf.js#L23):
-
-```javascript
-plugins: {
-  testomatio: {
-    enabled: true,
-    require: '@testomatio/reporter/lib/adapter/codecept',
-    apiKey: process.env.TESTOMATIO || 'API_KEY', // pass in api key via config or env variable
-  }
-}
-```
-
-Run the following command from you project folder:
-
-```bash
-TESTOMATIO={API_KEY} npx codeceptjs run
-```
-
-#### CodeceptJS Parallel Run
-
-If tests run parallel, like workers in CodeceptJS use `start-test-run` command to get proper reports:
-
-```bash
-TESTOMATIO={API_KEY} npx start-test-run -c 'npx codeceptjs run-workers 2'
-```
-
-> Specify a command to run with `-c` option in `start-test-run`
-
-Use `--env-file <envfile>` option to load environment variables from .env file. Inside env file TESTOMATIO credentials like `TESTOMATIO` api key or [S3 config](#attaching-test-artifacts) can be stored.
-
-
-### Playwright
-
-Add a reporter to Playwright config:
-
-```javascript
-reporter: [
-  ['list'],
-  [
-    '@testomatio/reporter/lib/adapter/playwright.js',
-    {
-      apiKey: process.env.TESTOMATIO,
-    },
-  ],
-];
-```
-
-Run the following command from you project folder:
-
-```bash
-TESTOMATIO={API_KEY} npx playwright test
-```
-
-### Mocha
-
-Load the test using using `check-tests` if not done already. Get the test id from testomat account and add it to your mocha test like in this [example](https://github.com/testomatio/reporter/blob/master/example/mocha/test/index.test.js#L4).
-
-Run the following command from you project folder:
-
-```bash
-mocha --reporter ./node_modules/testomat-reporter/lib/adapter/mocha.js --reporter-options apiKey={API_KEY}
-```
-
-### Jest
-
-Load the test using using `check-tests`. Add the test id to your tests like in this [example](https://github.com/testomatio/reporter/blob/master/example/jest/index.test.js#L1).
-
-Add the following line to [jest.config.js](https://github.com/testomatio/reporter/blob/master/example/jest/jest.config.js#L100):
-
-```javascript
-reporters: ['default', ['@testomatio/reporter/lib/adapter/jest.js', { apiKey: process.env.TESTOMATIO }]],
-```
-
-Run your tests.
-
-### Cucumber
-
-Load you test using [`check-cucumber`](https://github.com/testomatio/check-cucumber).
-
-Run the following command from you project folder:
-
-```bash
-TESTOMATIO={API_KEY} ./node_modules/.bin/cucumber-js --format ./node_modules/@testomatio/reporter/lib/adapter/cucumber.js
-```
-
-### TestCafe
-
-Load the test using using `check-tests`.
-
-Run the following command from you project folder:
-
-```bash
-TESTOMATIO={API_KEY} npx testcafe chrome -r testomatio
-```
-
-### Newman (Postman)
-
-Run collection and specify `testomatio` as reporter:
-
-```bash
-TESTOMATIO={API_KEY} npx newman run {collection_name.json} -r testomatio
-```
-
-> _`check-tests` not supported for newman for now, tests will be created on testomatio by default_
-
-
-### Cypress
-
-Load the test using using `check-tests`.
-
-Register our `cypress-plugin` in `cypress/plugins/index.js`:
-
-```javascript
-const testomatioReporter = require('@testomatio/reporter/lib/adapter/cypress-plugin');
-
-/**
- * @type {Cypress.PluginConfig}
- */
-module.exports = (on, config) => {
-  // `on` is used to hook into various events Cypress emits
-  // `config` is the resolved Cypress config
-
-  testomatioReporter(on, config);
-
-  return config;
-};
-```
-
-Run the following command from you project folder:
-
-```bash
-TESTOMATIO={API_KEY} npx cypress run
-```
-
-### Protractor
-
-Load the test using using `check-tests`.
-
-Add the following lines to [conf.js](https://github.com/angular/protractor/blob/5.4.1/example/conf.js):
-
-```javascript
-const JasmineReporter = require('@testomatio/reporter/lib/adapter/jasmine');
-
-exports.config = {
-  onPrepare: () => {
-    jasmine.getEnv().addReporter(new JasmineReporter({ apiKey: process.env.TESTOMATIO }));
-  },
-};
-```
-
-Run the following command from you project folder:
-
-```bash
-TESTOMATIO={API_KEY} npx start-test-run -c 'npx protractor conf.js'
-```
-
-### WebdriverIO
-
-Load the test using using `check-tests`.
-
-Add the following lines to [wdio.conf.js](https://webdriver.io/docs/configurationfile/):
-
-```javascript
-const testomatio = require('@testomatio/reporter/lib/adapter/webdriver');
-
-exports.config = {
-  // ...
-  reporters: [
-    [testomatio, {
-      apiKey: $ {
-        process.env.TESTOMATIO
-      }
-    }]
-  ]
-}
-```
-
-For making screenshots on failed tests add the following hook to `wdio.conf.js`:
-
-```js
-    afterTest: function (test, context, { error, result, duration, passed, retries }) {
-        if (error) {
-            browser.takeScreenshot()
-        }
-    },
-```
-
-Run the following command from you project folder:
-
-```bash
-TESTOMATIO={API_KEY} npx start-test-run -c 'npx wdio wdio.conf.js'
-```
-
-## Pipes
-Pipes allow you to get report inside different systems (e.g. Pull request comment, database etc)
-For now next pipes available:
-
-### GitHub
-This pipe adds comment with run report to GitHub Pull Request.
-
-To use it:
-1. run your tests using github actions in Pull Request
-2. pass `GH_PAT` (GitHub Personal Access Token) as environment variable.
-
-> Last report (comment) will be replaced with the new one.
-To leave previous report pass `GITHUB_KEEP_OUTDATED_REPORTS=1` env variable.
-
-### GitLab
-This pipe adds comment with run report to GitLab Merge Request.
-
-To use it:
-1. run your tests in Merge Request (pipeline trigger should be `merge_request`)
-2. pass `GITLAB_PAT` (GitLab Personal Access Token) as environment variable.
-
-> Last report (comment) will be replaced with the new one.
-To leave previous report pass `GITLAB_KEEP_OUTDATED_REPORTS=1` env variable.
+👋 Hey, do you need some test reporting?
 
-## JUnit Reports
+Testomat.io Reporter is a library that integrates with popular **JavaScript and TypeScript** test frameworks to provide a common interface for test reporting. By default, Testomat.io Reporter works with our reporting cloud service [Testomat.io App](https://testomat.io), however it is not locked to it. Reporter can be used as a standalone tool.
 
-> **Notice** JUnit reports are supported since 0.6.0
+## Features
 
-Other frameworks and languages are supported via JUnit reports.
+Testomat.io Reporter (this npm package) supports:
 
-JUnit XML format is standard among test runners on various platforms. Testomat.io can load XML reports from test runners and create tests in a project from it. If your framework is not supported yet, generate JUnit report and upload it into Testomat.io
+* 🏄 Integarion with all popular [JavaScript/TypeScript frameworks](./docs/frameworks.md)
+* 🗄️ Screenshots, videos, traces [uploaded into S3 bucket](./docs/artifacts.md)
+* 🔎 Stack traces and error messages
+* 🐙 [GitHub](./docs/pipes.md#github-pipe) & [GitLab](./docs/pipes.md#gitlab-pipe) integration
+* 🚅 Realtime reports
+* 🗃️ Other test frameworks supported via [JUNit XML](./docs/junit.md)
+* 🚶‍♀️ Steps *(work in progress)*
+* ☁️  Custom properties and metadata *(work in progress)*
+* 💯 Free & open-source.
+* 📊 Public and private Run reports on cloud via [Testomat.io App](https://testomat.io) 👇
 
-Testomat.io will not only create a run report but also collect all information about tests, including source code, and create tests in a system as regular importer do. The only difference that normal process of Testomat.io import does not require executing tests on import, while importing via repoter requires to have tests executed and XML report provided.
 
-Tested Frameworks:
+![](./docs/images/app.png)
 
-* JUnit (JUnit)
-* Python (Pytest)
-* Minitest (Ruby)
-* PHPUnit (PHP)
-* NUnit (C#)
+## How It Works
 
+Testomat.io Reporter provides common API to store and organize test reports.
+It can receive test result data from any [test framework](./docs/frameworks.md) and send it to different services via [pipes](./docs/pipes.md).
 
-To import JUnit reports into Testomat.io **NodeJS >=14 is required**.
+| 🌊 Input         | 📊 Output                                 |
+|---------------|----------------------------------------|
+| Playwright    | Report to GitHub                       |
+| Cypress       | Report to GitLab                       |
+| Jest          | Report to [Testomat.io](https://testomat.io) |
+| ...   | ... your custom report   |
 
-Package `@testomatio/reporter` should be installed:
+If you use multiple test frameworks and you need to use one customizable reporter, check Testomat.io Reporter, as you can adjust it once and attach it to all your projects.
 
-```
-npm i @testomatio/reporter@latest --save-dev
-```
-
-
-Run your test framework and generate a JUnit report.
-
-Then import XML report into Testomat.io
-
-```
-TESTOMATIO={API_KEY} npx report-xml "{pattern}" --lang={lang}
-```
-
-* `pattern` - is a glob pattern to match all XML files from report. For instance, `"test/report/**.xml"` or just `report.xml`
-* `--lang` option can be specified to identify source code of the project. Example: `--lang=Ruby` or `--lang=Java` or `--lang=Python`. Possible values:
-  * `c#`
-  * `java`
-  * `ruby`
-  * `python`
-  * `php`
-* `--java-tests` option is avaiable for Java projects, and can be set if path to tests is different then `src/test/java`. When this option is enable, `lang` option is automatically set to `java`
-* `--env-file <envfile>` option to load environment variables from .env file. Inside env file TESTOMATIO credentials like `TESTOMATIO` api key or [S3 config](#attaching-test-artifacts) can be stored.
-* `--timelimit <time>` set a timer to silently kill a long-running reporter process due to network or other issues. For instance, use `--set-timeout=3` to stop process after 3 secs.
-
-
-> *Notice:* All options from [Advanced Usage](#advanced-usage) are also available for JUnit reporter
-
-Check the list of examples:
-
-#### Example: Pytest
-
-Run pytest tests and generate a report to `report.xml`:
-
-```
-pytest --junit-xml report.xml
-```
-
-Import report with this command
-
-```
-TESTOMATIO={API_KEY} npx report-xml report.xml --lang=python
-```
-
-#### Example: JUnit with Maven
-
-Run tests via Maven, make sure JUnit report was configured in `pom.xml`.
-
-```
-mvn clean test
-```
-
-Import report with this command:
-
-```
-TESTOMATIO={API_KEY} npx report-xml "target/surefire-reports/*.xml" --java-tests
-```
-
-> You can specify `--java-test` option to set a path to tests if they are located in path other than `src/test/java`
-
-#### Example: NUnit
-
-Generate NUnit XML report and run the following code:
-
-```
-TESTOMATIO={API_KEY} npx report-xml "report.xml" --lang="c#"
-```
-
-#### Example: Ruby on Rails with Minitest
-
-```ruby
-# test_helper.rb:
-
-reporters = [Minitest::Reporters::DefaultReporter.new(color: true)]
-# enable JUnit reporter
-reporters << Minitest::Reporters::JUnitReporter.new
-
-Minitest::Reporters.use! reporters
-```
-
-Launch tests:
-
-```
-rails test
-```
-
-Import reports from `test/reports` directory:
-
-```
-TESTOMATIO={API_KEY} npx report-xml "test/reports/*.xml" --lang ruby
-```
-
-### Artifacts
-
-Screenshots or videos from tests are uploaded if a test contains output with a path to file of following format:
+![](./docs/images/reporter-architecture.png)
 
-```
-file://path/to/screenshot.png
-```
-
-For instance, inside Java test you can use `System.out.println` to print a path to file that should be uploaded as a screenshot.
-
-```java
-System.out.println("file://" + pathToScreenshot);
-```
-
-This will produce XML report which contains path to a file:
-
-```xml
-<testcase>
-  <system-out><![CDATA[
-    file://path/to/scrrenshot.png
-  ]]></system-out>
-</testcase>
-```
-
-When uploaded XML report, all files from `file://` will be uploaded to corresponding tests.
+Artifacts like screenshots, videos, traces, are **uploaded to your own cloud storage** via S3 protocol. Artifacts can be uplaoded privately or publicly, and used in reports.
 
+## Installation
 
-## Advanced Usage
+To enable Testomat.io Reporter install `@testomatio/reporter` package
 
-### Create Unmatched Tests
 
-Testomat.io will not create tests from the report if they have not been previously imported. To create tests during the report `TESTOMATIO_CREATE` option can be used:
+Use one of your favorite package managers:
 
-```bash
-TESTOMATIO={API_KEY} TESTOMATIO_CREATE=1 <actual run command>
 ```
-
-### Add Report to Run by ID
-
-This feature is widely used when a run is executed on CI.
-A run is created before the test is started and it is marked as `scheduled`. Then
-a report is assigned to that run using `TESTOMATIO_RUN` environment variable and `{RUN_ID}` of a run:
-
-```bash
-TESTOMATIO={API_KEY} TESTOMATIO_RUN={RUN_ID} <actual run command>
+npm install @testomatio/reporter --save-dev
 ```
 
-### Do Not Finalize Run
-
-If multiple reports are added to the same run, each of them should not finalize the run. 
-In this case use `TESTOMATIO_PROCEED=1` environment variable, so the Run will be shown as `Running`
-
 ```
-TESTOMATIO={API_KEY} TESTOMATIO_PROCEED=1 TESTOMATIO_RUN={RUN_ID} <actual run command>
+pnpm install @testomatio/reporter --save-dev
 ```
 
-After all reports were attached and run can be execute the following command:
-
 ```
-TESTOMATIO={API_KEY} TESTOMATIO_RUN={RUN_ID} npx start-test-run --finish
+yarn add @testomatio/reporter --dev
 ```
 
-### Setting Report Title
+## Getting Started
 
-Give a title to your reports by passing it as environment variable to `TESTOMATIO_TITLE`.
+### 1️⃣ Attach Reporter to the Test Runner
 
-```bash
-TESTOMATIO={API_KEY} TESTOMATIO_TITLE="title for the report" <actual run command>
-```
+* ####  [Playwright](./docs/frameworks.md#playwright)
+* #### [CodeceptJS](./docs/frameworks.md#CodeceptJS)
+* #### [Cypress](./docs/frameworks.md#Cypress)
+* #### [Jest](./docs/frameworks.md#Jest)
+* #### [Mocha](./docs/frameworks.md#Mocha)
+* #### [WebDriverIO](./docs/frameworks.md#WebDriverIO)
+* #### [TestCafe](./docs/frameworks.md#TestCafe)
+* #### [Detox](./docs/frameworks.md#Detox)
+* #### [Codeception](https://github.com/testomatio/php-reporter)
+* #### [Newman (Postman)](./docs/frameworks.md#Newman)
+* #### [JUnit](./docs/junit.md#junit)
+* #### [NUnit](./docs/junit.md#nunit)
+* #### [PyTest](./docs/junit.md#pytest)
+* #### [PHPUnit](./docs/junit.md#phpunit)
+* #### [Protractor](./docs/frameworks.md#protractor)
 
-### Reporting Parallel Executionto To Same Run
+or any [other via JUnit](./docs/junit.md) report....
 
-Provide a shared unique title to all runs that will be running in parallel, and add `TESTOMATIO_SHARED_RUN` environment var. So all reports will be sent to this run.
+### 2️⃣ Configure Reports
 
-```bash
-TESTOMATIO={API_KEY} TESTOMATIO_TITLE="report for commit ${GIT_COMMIT}" TESTOMATIO_SHARED_RUN=1 <actual run command>
-```
+* [Create report on Testomat.io](./docs/pipes.md#testomatio-pipe).
+* [Create brief summary report for GitHub Pull Request](./docs/pipes.md#github-pipe) 👇
+* [Create brief summary report for GitLab Merge Request](./docs/pipes.md#gitlab-pipe).
+* [Configure other pipes](./docs/pipes.md) for other ways to process test results output.
 
-We recommend using a commit hash as a title to create a new Run. In this case we ensure that run title is unique and will be the same for all parallel jobs running exactly for this commit.
 
+![GitHub report](./docs/images/github.png)
 
-### Adding Report to RunGroup
+GitHub report published as a comment to Pull Request:
 
-Create/Add run to group by providing `TESTOMATIO_RUNGROUP_TITLE`:
+### 3️⃣ Enable Artifacts Storage
 
-```sh
-TESTOMATIO={API_KEY} TESTOMATIO_RUNGROUP_TITLE="Build ${BUILD_ID}" <actual run command>
-```
+1. Create bucket on AWS, Google Cloud, or any other cloud storage provider supporting S3 protocol. 
+2. [Pass S3 credentials](./docs/artifacts.md) to reporter to enable artifacts uploading.
 
-### Adding Environments to Run
+### 4️⃣ Add to CI Pipeline
 
-Add environments to run by providing `TESTOMATIO_ENV` as comma seperated values:
+After you tested reporter locally add it to your CI pipeline.
 
-```bash
-TESTOMATIO={API_KEY} TESTOMATIO_ENV="Windows, Chrome" <actual run command>
-```
+> We prepared some [example workflows](./docs/workflows.md) that might help you to get it running.
 
-### Save test results to .csv file
-Add an env to run by specifying the `TESTOMATIO_CSV_FILENAME` variable.
+---
 
-1) using default report name:
+🎉 **You are all set!**
 
-```bash
-TESTOMATIO={API_KEY} TESTOMATIO_CSV_FILENAME="report.csv" <actual run command>
-```
+Bring this reporter on CI and never lose test results again!
 
-2) using unique report name:
 
-```bash
-TESTOMATIO={API_KEY} TESTOMATIO_CSV_FILENAME="test.csv" <actual run command>
-```
-_It's create a new /export folder with csv files_
+## Documentation
 
+* 🛠️ [Frameworks](./docs/frameworks.md)
+* ⛲ [Pipes](./docs/pipes.md)
+* 📓 [JUnit](./docs/junit.md)
+* 🗄️ [Artifacts](./docs/artifacts.md)
+* 🔂 [Workflows](./docs/workflows.md)
 
-### Attaching Test Artifacts
+## Development
 
-To save a test artifacts (screenshots and videos) of a failed test use S3 storage.
-Please note, that the **storage is not connected to Testomatio**.
-This allows you to store your artifacts on your own account and not expose S3 credentials.
 
-To save screenshots provide a configuration for S3 bucket via environment variables:
+### REST API 
 
-- **S3_REGION** - Region your bucket lies.
-- **S3_BUCKET** - Bucket name.
-- **S3_ACCESS_KEY_ID** - Access key.
-- **S3_SECRET_ACCESS_KEY** - Secret.
-- **S3_ENDPOINT** - for providers other than AWS
+Testomat.io App uses REST API to collect data from the reporter.
 
-By default tests artifacts are uploaded to bucket with `public-read` permission.
-In this case uploaded files will be publicly accessible in Internet.
-These public links will be used by [testomat.io](https://testomat.io) to display images and videos.
+[👉 API Reference](https://testomatio.github.io/reporter/)
 
-To upload files with `private` access bucket add `TESTOMATIO_PRIVATE_ARTIFACTS=1` environment value.
-Then update provide the same S3 credentials in "Settings > Artifacts" section of a [testomat.io](https://testomat.io) project,
-so [testomat.io](https://testomat.io) could connect to the same bucket and fetch uploaded artifacts.
-Links to files will be pre-signed and expires automatically in 10 minutes.
+### Debug Logs
 
-Example upload configuration in environment variables:
+To enable verbose logging run tests with `DEBUG` environment variable:
 
-##### AWS
+To print all reporter logs:
 
-```bash
-TESTOMATIO_PRIVATE_ARTIFACTS=1
-S3_ACCESS_KEY_ID=11111111111111111111
-S3_SECRET_ACCESS_KEY=2222222222222222222222222222222222222222222
-S3_BUCKET=artifacts
-S3_REGION=us-west-1
 ```
-
-##### DigitalOcean
-
-```bash
-TESTOMATIO_PRIVATE_ARTIFACTS=1
-S3_ENDPOINT=https://ams3.digitaloceanspaces.com
-S3_ACCESS_KEY_ID=11111111111111111111
-S3_SECRET_ACCESS_KEY=2222222222222222222222222222222222222222222
-S3_BUCKET=artifacts
-S3_REGION=ams3
+DEBUG=@testomatio/reporter:*
 ```
+To print all reporter logs of a specific pipe:
 
-##### Minio
-
-```bash
-S3_ENDPOINT=http://company.storage.com
-S3_ACCESS_KEY_ID=minio
-S3_SECRET_ACCESS_KEY=minio123
-S3_BUCKET=testomatio
-S3_FORCE_PATH_STYLE=true
 ```
-
-> It is important to add S3_FORCE_PATH_STYLE var for minio setup
-
-For local testing, it is recommended to store configuration in `.env` file. If you load configuration from a test runner, use [dotenv](https://www.npmjs.com/package/dotenv) library to load it.
-
-If you run a reporter via `start-test-run` or `report-xml` command use `--env-file` option to load variables from .env file:
-
-```
-npx start-test-run --env-file .env
-npx report-xml --env-file .env
+DEBUG=@testomatio/reporter:pipe:github
 ```
-
-On CI set environment variables in CI config.
-
-Test artifacts are automatically uploaded for these test runners:
-
-- CodeceptJS
-- Playwright
-- Cypress
-- WebdriverIO
-
-To manually attach an artifact and upload it for a test use `global.testomatioArtifacts` array:
-
-```javascript
-// attach a picture inside a test
-global.testomatioArtifacts.push('img/file.png');
-// attach a picture and add a name to it
-global.testomatioArtifacts.push({ name: 'Screenshot', path: 'img/file.png' });
-```
-
-Artifacts will be uploaded for the current test when it is finished.
-
-To disable uploading artifacts add `TESTOMATIO_DISABLE_ARTIFACTS` environment variable:
-
-```bash
-TESTOMATIO_DISABLE_ARTIFACTS=1
-```
-
-### Starting an Empty Run
-
-If you want to create a run and obtain its `{RUN_ID}` from [testomat.io](https://testomat.io) you can use `--launch` option:
-
-```bash
-TESTOMATIO={API_KEY} npx start-test-run --launch
-```
-
-This command will return `{RUN_ID}` which you can pass to other testrunner processes.
-
-> When executed with `--launch` a command provided by `-c` flag is ignored
-
-### Manually Finishing Run
-
-If you want to finish a run started by `--launch` use `--finish` option. `TESTOMATIO_RUN` environment variable is required:
-
-```bash
-TESTOMATIO={API_KEY} TESTOMATIO_RUN={RUN_ID} npx start-test-run --finish
-```
-
-### Debug logs
-Pass `DEBUG` variable with module name e.g. `DEBUG=@testomatio/reporter:pipe:github`.
-(Module name could be taken directly from the required module code).
-To log all debug info pass `DEBUG=*`.

package/lib/adapter/codecept.js

@@ -33,6 +33,7 @@ if (MAJOR_VERSION < 3) {
 function CodeceptReporter(config) {
   let failedTests = [];
   let videos = [];
+  let traces = [];
   const reportTestPromises = [];
 
   const testTimeMap = {};
@@ -54,6 +55,7 @@ function CodeceptReporter(config) {
   event.dispatcher.on(event.all.before, () => {
     recorder.add('Creating new run', () => client.createRun());
     videos = [];
+    traces = [];
   });
 
   event.dispatcher.on(event.test.before, () => {
@@ -70,25 +72,13 @@ function CodeceptReporter(config) {
   });
 
   event.dispatcher.on(event.all.result, async () => {
+    debug('waiting for all tests to be reported');
     // all tests were reported and we can upload videos
     await Promise.all(reportTestPromises);
 
-    if (videos.length && upload.isArtifactsEnabled()) {
-      console.log(APP_PREFIX, `🎞️  Uploading ${videos.length} videos...`);
-
-      const promises = [];
-      for (const video of videos) {
-        const { testId, title, path, type } = video;
-        const file = { path, type, title };
-        promises.push(
-          client.addTestRun(undefined, {
-            ...stripExampleFromTitle(title),
-            test_id: testId,
-            files: [file],
-          }),
-        );
-      }
-      await Promise.all(promises);
+    if (upload.isArtifactsEnabled()) {
+      uploadAttachments(client, videos, '🎞️  Uploading', 'video');
+      uploadAttachments(client, traces, '📁 Uploading', 'trace');
     }
 
     const status = failedTests.length === 0 ? STATUS.PASSED : STATUS.FAILED;
@@ -166,9 +156,11 @@ function CodeceptReporter(config) {
     }).then(pipes => {
       testId = pipes.filter(p => p.pipe.includes('Testomatio'))[0]?.result?.data?.test_id;
 
+      debug("artifacts", artifacts);
+      
       for (const aid in artifacts) {
-        if (aid.startsWith('video')) videos.push({ testId, title, path: artifacts[aid], type: 'video/webm' }); 
-        if (aid.startsWith('trace')) files.push({ testId, title, path: artifacts[aid], type: 'application/zip' }); 
+        if (aid.startsWith('video')) videos.push({ testId, title, path: artifacts[aid], type: 'video/webm' });
+        if (aid.startsWith('trace')) traces.push({ testId, title, path: artifacts[aid], type: 'application/zip' });
       }
     });
 
@@ -242,6 +234,24 @@ function CodeceptReporter(config) {
   });
 }
 
+async function uploadAttachments(client, attachments, messagePrefix, attachmentType) {
+  if (attachments.length > 0) {
+    console.log(APP_PREFIX, `Attachments: ${messagePrefix} ${attachments.length} ${attachmentType}/-s ...`);
+
+    const promises = attachments.map(async (attachment) => {
+      const { testId, title, path, type } = attachment;
+      const file = { path, type, title };
+      return client.addTestRun(undefined, {
+        ...stripExampleFromTitle(title),
+        test_id: testId,
+        files: [file],
+      });
+    });
+
+    await Promise.all(promises);
+  }
+}
+
 function parseTest(tags) {
   if (tags) {
     for (const tag of tags) {

package/lib/client.js

@@ -11,6 +11,7 @@ const pipesFactory = require('./pipe');
 /**
  * @typedef {import('../types').TestData} TestData
  * @typedef {import('../types').RunStatus} RunStatus
+ * @typedef {import('../types').PipeResult} PipeResult  
  */
 
 class Client {
@@ -33,7 +34,7 @@ class Client {
   /**
    * Used to create a new Test run
    *
-   * @returns {Promise<void>} - resolves to Run id which should be used to update / add test
+   * @returns {Promise<any>} - resolves to Run id which should be used to update / add test
    */
   createRun() {
     // all pipes disabled, skipping
@@ -61,7 +62,7 @@ class Client {
    * @param {string|undefined} status
    * @param {TestData} [testData]
    * @param {string[]} [storeArtifacts]
-   * @returns {Promise<void>}
+   * @returns {Promise<PipeResult[]>}
    */
   async addTestRun(status, testData, storeArtifacts = []) {
     // all pipes disabled, skipping
@@ -147,7 +148,7 @@ class Client {
           const result = await p.addTest(data);
           return { pipe: p.toString(), result };
         } catch (err) {
-          console.log(APP_PREFIX, pipe.toString(), err);
+          console.log(APP_PREFIX, p.toString(), err);
         }
       }
     )));
@@ -160,7 +161,7 @@ class Client {
    * Updates the status of the current test run and finishes the run.
    * @param {RunStatus} status - The status of the current test run. Must be one of "passed", "failed", or "finished"
    * @param {boolean} [isParallel] - Whether the current test run was executed in parallel with other tests.
-   * @returns {Promise<void>} - A Promise that resolves when finishes the run.
+   * @returns {Promise<any>} - A Promise that resolves when finishes the run.
    */
   updateRunStatus(status, isParallel = false) {
     // all pipes disabled, skipping

package/lib/pipe/github.js

@@ -39,8 +39,8 @@ class GitHubPipe {
   async createRun() {}
 
   addTest(test) {
-    debug('Adding test:', test);
     if (!this.isEnabled) return;
+    debug('Adding test:', test);
 
     const index = this.tests.findIndex(t => isSameTest(t, test));
     // update if they were already added
@@ -200,7 +200,7 @@ function fullName(t) {
 }
 
 async function deletePreviousReport(octokit, owner, repo, issue, hiddenCommentData) {
-  if (process.env.GITHUB_KEEP_OUTDATED_REPORTS) return;
+  if (process.env.GH_KEEP_OUTDATED_REPORTS) return;
 
   // get comments
   let comments = [];

package/lib/pipe/testomatio.js

@@ -169,6 +169,7 @@ class TestomatioPipe {
 
 module.exports = TestomatioPipe;
 
+
 function setS3Credentials(artifacts) {  
   if (!Object.keys(artifacts).length) return;
 
@@ -181,4 +182,4 @@ function setS3Credentials(artifacts) {
   if (artifacts.ENDPOINT) process.env.S3_ENDPOINT = artifacts.ENDPOINT; 
   if (artifacts.presign) process.env.TESTOMATIO_PRIVATE_ARTIFACTS = '1'; 
   resetConfig();
-}
+}

package/lib/reporter.js

@@ -5,5 +5,6 @@ const TRArtifacts = require('./ArtifactStorage');
 module.exports = {
   TestomatClient,
   TRConstants,
-  TRArtifacts
+  TRArtifacts,
+  addArtifact: TRArtifacts.artifact,
 };

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@testomatio/reporter",
-  "version": "1.0.0-beta.4",
+  "version": "1.0.0",
   "description": "Testomatio Reporter Client",
   "main": "./lib/reporter.js",
   "typings": "typings/index.d.ts",