@testomatio/reporter 0.6.0-beta.1 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.eslintrc.js +2 -1
- package/.github/workflows/node.js.yml +1 -1
- package/README.md +88 -3
- package/lib/adapter/cypress-plugin/index.js +2 -2
- package/lib/bin/reportXml.js +1 -0
- package/lib/junit-adapter/java.js +4 -4
- package/lib/junit-adapter/python.js +4 -4
- package/lib/junit-adapter/ruby.js +1 -3
- package/lib/util.js +5 -7
- package/lib/xmlReader.js +15 -13
- package/package.json +1 -1
- package/tests/data/codecept.xml +87 -87
package/.eslintrc.js
CHANGED
|
@@ -11,7 +11,7 @@ module.exports = {
|
|
|
11
11
|
SharedArrayBuffer: 'readonly',
|
|
12
12
|
},
|
|
13
13
|
parserOptions: {
|
|
14
|
-
ecmaVersion:
|
|
14
|
+
ecmaVersion: 2020,
|
|
15
15
|
},
|
|
16
16
|
rules: {
|
|
17
17
|
'max-len': ['error', { code: 120 }],
|
|
@@ -25,6 +25,7 @@ module.exports = {
|
|
|
25
25
|
'no-use-before-define': 0,
|
|
26
26
|
'object-curly-newline': 0,
|
|
27
27
|
'consistent-return': 0,
|
|
28
|
+
'no-await-in-loop': 0,
|
|
28
29
|
'no-param-reassign': 0,
|
|
29
30
|
'operator-linebreak': 0,
|
|
30
31
|
'prefer-destructuring': 0,
|
package/README.md
CHANGED
|
@@ -207,13 +207,13 @@ TESTOMATIO={API_KEY} npx @testomatio/reporter -c 'npx wdio wdio.conf.js'
|
|
|
207
207
|
|
|
208
208
|
## JUnit Reports
|
|
209
209
|
|
|
210
|
+
> **Notice** JUnit reports are supported since 0.6.0
|
|
211
|
+
|
|
210
212
|
Other frameworks and languages are supported via JUnit reports.
|
|
211
213
|
|
|
212
214
|
JUnit XML format is standard among test runners on various platforms. Testomat.io can load XML reports from test runners and create tests in a project from it. If your framework is not supported yet, generate JUnit report and upload it into Testomat.io
|
|
213
215
|
|
|
214
|
-
|
|
215
|
-
TESTOMATIO={API_KEY} npx @testomatio/reporter -c 'npx wdio wdio.conf.js'
|
|
216
|
-
```
|
|
216
|
+
Testomat.io will not only create a run report but also collect all information about tests, including source code, and create tests in a system as regular importer do. The only difference that normal process of Testomat.io import does not require executing tests on import, while importing via repoter requires to have tests executed and XML report provided.
|
|
217
217
|
|
|
218
218
|
Tested Frameworks:
|
|
219
219
|
|
|
@@ -223,6 +223,91 @@ Tested Frameworks:
|
|
|
223
223
|
* PHPUnit (PHP)
|
|
224
224
|
|
|
225
225
|
|
|
226
|
+
To import JUnit reports into Testomat.io **NodeJS >=14 is required**.
|
|
227
|
+
|
|
228
|
+
Package `@testomatio/reporter` should be installed:
|
|
229
|
+
|
|
230
|
+
```
|
|
231
|
+
npm i @testomatio/reporter@latest
|
|
232
|
+
```
|
|
233
|
+
|
|
234
|
+
For local development it is recommended to install @testomatio/reporter globally:
|
|
235
|
+
|
|
236
|
+
```
|
|
237
|
+
npm i -g @testomatio/reporter@latest
|
|
238
|
+
```
|
|
239
|
+
|
|
240
|
+
Run your test framework and generate a JUnit report.
|
|
241
|
+
|
|
242
|
+
Then import XML report into Testomat.io
|
|
243
|
+
|
|
244
|
+
```
|
|
245
|
+
TESTOMATIO={API_KEY} npx report-xml "{pattern}" --lang={lang}
|
|
246
|
+
```
|
|
247
|
+
|
|
248
|
+
* `pattern` - is a glob pattern to match all XML files from report. For instance, `"test/report/**.xml"` or just `report.xml`
|
|
249
|
+
* `--lang` option should be specified to identify source code of the project. Example: `--lang=Ruby` or `--lang=Java` or `--lang=Python`.
|
|
250
|
+
* `--java-tests` option is avaiable for Java projects, and can be set if path to tests is different then `src/test/java`. When this option is enable, `lang` option is automatically set to `java`
|
|
251
|
+
|
|
252
|
+
|
|
253
|
+
> *Notice:* All options from [Advanced Usage](#advanced-usage) are also available for JUnit reporter
|
|
254
|
+
|
|
255
|
+
Check the list of examples:
|
|
256
|
+
|
|
257
|
+
#### Example: Pytest
|
|
258
|
+
|
|
259
|
+
Run pytest tests and generate a report to `report.xml`:
|
|
260
|
+
|
|
261
|
+
```
|
|
262
|
+
pytest --junit-xml report.xml
|
|
263
|
+
```
|
|
264
|
+
|
|
265
|
+
Import report with this command
|
|
266
|
+
|
|
267
|
+
```
|
|
268
|
+
TESTOMATIO={API_KEY} npx report-xml report.xml --lang=python
|
|
269
|
+
```
|
|
270
|
+
|
|
271
|
+
#### Example: JUnit with Maven
|
|
272
|
+
|
|
273
|
+
Run tests via Maven, make sure JUnit report was configured in `pom.xml`.
|
|
274
|
+
|
|
275
|
+
```
|
|
276
|
+
maven clean test
|
|
277
|
+
```
|
|
278
|
+
|
|
279
|
+
Import report with this command:
|
|
280
|
+
|
|
281
|
+
```
|
|
282
|
+
TESTOMATIO={API_KEY} npx report-xml "target/surefire-reports/*.xml" --java-tests
|
|
283
|
+
```
|
|
284
|
+
|
|
285
|
+
> You can specify `--java-test` option to set a path to tests if they are located in path other than `src/test/java`
|
|
286
|
+
|
|
287
|
+
#### Example: Ruby on Rails with Minitest
|
|
288
|
+
|
|
289
|
+
```ruby
|
|
290
|
+
# test_helper.rb:
|
|
291
|
+
|
|
292
|
+
reporters = [Minitest::Reporters::DefaultReporter.new(color: true)]
|
|
293
|
+
# enable JUnit reporter
|
|
294
|
+
reporters << Minitest::Reporters::JUnitReporter.new
|
|
295
|
+
|
|
296
|
+
Minitest::Reporters.use! reporters
|
|
297
|
+
```
|
|
298
|
+
|
|
299
|
+
Launch tests:
|
|
300
|
+
|
|
301
|
+
```
|
|
302
|
+
rails test
|
|
303
|
+
```
|
|
304
|
+
|
|
305
|
+
Import reports from `test/reports` directory:
|
|
306
|
+
|
|
307
|
+
```
|
|
308
|
+
TESTOMATIO={API_KEY} npx report-xml "test/reports/*.xml" --lang ruby
|
|
309
|
+
```
|
|
310
|
+
|
|
226
311
|
## Advanced Usage
|
|
227
312
|
|
|
228
313
|
### Create Unmatched Tests
|
|
@@ -29,7 +29,7 @@ const testomatioReporter = on => {
|
|
|
29
29
|
const error = latestAttempt.error;
|
|
30
30
|
|
|
31
31
|
let title = test.title.pop();
|
|
32
|
-
|
|
32
|
+
const examples = title.match(/\(example (#\d+)\)/);
|
|
33
33
|
let example = null;
|
|
34
34
|
if (examples && examples[1]) example = { example: examples[1] };
|
|
35
35
|
title = title.replace(/\(example #\d+\)/, '').trim();
|
|
@@ -40,7 +40,7 @@ const testomatioReporter = on => {
|
|
|
40
40
|
const suiteId = parseSuite(suiteTitle);
|
|
41
41
|
|
|
42
42
|
if (error) {
|
|
43
|
-
error.inspect = function() {
|
|
43
|
+
error.inspect = function() { // eslint-disable-line func-names
|
|
44
44
|
if (this && this.codeFrame) {
|
|
45
45
|
return this.codeFrame.frame;
|
|
46
46
|
}
|
package/lib/bin/reportXml.js
CHANGED
|
@@ -22,6 +22,7 @@ program
|
|
|
22
22
|
}
|
|
23
23
|
let { javaTests, lang } = opts;
|
|
24
24
|
if (javaTests === true) javaTests = 'src/test/java';
|
|
25
|
+
lang = lang?.toLowerCase();
|
|
25
26
|
const runReader = new XmlReader({ javaTests, lang });
|
|
26
27
|
const files = glob.sync(pattern, { cwd: opts.dir || process.cwd() });
|
|
27
28
|
if (!files.length) {
|
|
@@ -20,20 +20,20 @@ class JavaAdapter extends Adapter {
|
|
|
20
20
|
}
|
|
21
21
|
|
|
22
22
|
formatStack(t) {
|
|
23
|
-
|
|
23
|
+
const stack = super.formatStack(t);
|
|
24
24
|
|
|
25
25
|
const file = t.suite_title.split('.');
|
|
26
26
|
|
|
27
|
-
const fileLine = `at .*${file[file.length - 1]}\.java
|
|
27
|
+
const fileLine = `at .*${file[file.length - 1]}\.java:(\\d+)` // eslint-disable-line no-useless-escape
|
|
28
28
|
const regexp = new RegExp(fileLine,"g")
|
|
29
29
|
return stack.replace(regexp, `${this.opts.javaTests}${path.sep}${namespaceToFileName(t.suite_title)}:$1:`);
|
|
30
30
|
}
|
|
31
31
|
}
|
|
32
32
|
|
|
33
33
|
function namespaceToFileName(fileName) {
|
|
34
|
-
|
|
34
|
+
const fileParts = fileName.split('.')
|
|
35
35
|
fileParts[fileParts.length - 1] = fileParts[fileParts.length - 1]?.replace(/\$.*/, '')
|
|
36
|
-
return fileParts.join(path.sep)
|
|
36
|
+
return `${fileParts.join(path.sep) }.java`;
|
|
37
37
|
|
|
38
38
|
}
|
|
39
39
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
const path = require('path');
|
|
2
|
-
const Adapter = require('./adapter');
|
|
3
2
|
const fs = require('fs');
|
|
3
|
+
const Adapter = require('./adapter');
|
|
4
4
|
|
|
5
5
|
class PythonAdapter extends Adapter {
|
|
6
6
|
|
|
@@ -27,11 +27,11 @@ class PythonAdapter extends Adapter {
|
|
|
27
27
|
}
|
|
28
28
|
|
|
29
29
|
function namespaceToFileName(fileName) {
|
|
30
|
-
|
|
30
|
+
const fileParts = fileName.split('.')
|
|
31
31
|
|
|
32
32
|
while (fileParts.length > 0) {
|
|
33
|
-
const file = fileParts.join(path.sep)
|
|
34
|
-
if (fs.existsSync(fileParts.join(path.sep)
|
|
33
|
+
const file = `${fileParts.join(path.sep) }.py`;
|
|
34
|
+
if (fs.existsSync(`${fileParts.join(path.sep) }.py`)) {
|
|
35
35
|
return file;
|
|
36
36
|
}
|
|
37
37
|
fileParts.pop();
|
|
@@ -1,12 +1,10 @@
|
|
|
1
|
-
const path = require('path');
|
|
2
1
|
const Adapter = require('./adapter');
|
|
3
|
-
const fs = require('fs');
|
|
4
2
|
|
|
5
3
|
class RubyAdapter extends Adapter {
|
|
6
4
|
|
|
7
5
|
formatStack(t) {
|
|
8
6
|
const stack = super.formatStack(t);
|
|
9
|
-
return stack.replace(/\[(
|
|
7
|
+
return stack.replace(/\[(.*?:.\d*)\]/g, '\n$1')
|
|
10
8
|
}
|
|
11
9
|
}
|
|
12
10
|
|
package/lib/util.js
CHANGED
|
@@ -1,5 +1,3 @@
|
|
|
1
|
-
const { fstat } = require('fs');
|
|
2
|
-
const { includes, split } = require('lodash');
|
|
3
1
|
const { URL } = require('url');
|
|
4
2
|
const { sep } = require('path');
|
|
5
3
|
const chalk = require('chalk');
|
|
@@ -69,8 +67,8 @@ const fetchSourceCodeFromStackTrace = (stack = '') => {
|
|
|
69
67
|
.filter(l => isValid(l.split(':')[0]))
|
|
70
68
|
|
|
71
69
|
// // filter out 3rd party libs
|
|
72
|
-
.filter(l => !l.includes(
|
|
73
|
-
.filter(l => !l.includes(
|
|
70
|
+
.filter(l => !l.includes(`vendor${ sep}`))
|
|
71
|
+
.filter(l => !l.includes(`node_modules${ sep}`))
|
|
74
72
|
.filter(l => fs.existsSync(l.split(':')[0]))
|
|
75
73
|
.filter(l => fs.lstatSync(l.split(':')[0]).isFile())
|
|
76
74
|
|
|
@@ -99,7 +97,7 @@ const fetchSourceCode = (contents, opts = {}) => {
|
|
|
99
97
|
|
|
100
98
|
// remove special chars from title
|
|
101
99
|
if (!lineIndex && opts.title) {
|
|
102
|
-
const title = opts.title.replace(/[
|
|
100
|
+
const title = opts.title.replace(/[([@].*/g, '')
|
|
103
101
|
lineIndex = lines.findIndex(l => l.includes(title))
|
|
104
102
|
}
|
|
105
103
|
|
|
@@ -118,7 +116,7 @@ const fetchSourceCode = (contents, opts = {}) => {
|
|
|
118
116
|
if (opts.lang === 'php' && lines[i].includes(' private function ')) break;
|
|
119
117
|
if (opts.lang === 'php' && lines[i].includes(' protected function ')) break;
|
|
120
118
|
if (opts.lang === 'php' && lines[i].includes(' public function ')) break;
|
|
121
|
-
if (opts.lang === 'python' && lines[i].trim().match(
|
|
119
|
+
if (opts.lang === 'python' && lines[i].trim().match(/^@\w+/)) break;
|
|
122
120
|
if (opts.lang === 'python' && lines[i].includes(' def ')) break;
|
|
123
121
|
if (opts.lang === 'ruby' && lines[i].includes(' def ')) break;
|
|
124
122
|
if (opts.lang === 'ruby' && lines[i].includes(' test ')) break;
|
|
@@ -129,7 +127,7 @@ const fetchSourceCode = (contents, opts = {}) => {
|
|
|
129
127
|
if (opts.lang === 'ts' && lines[i].includes(' test(')) break;
|
|
130
128
|
if (opts.lang === 'js' && lines[i].includes(' it(')) break;
|
|
131
129
|
if (opts.lang === 'js' && lines[i].includes(' test(')) break;
|
|
132
|
-
if (opts.lang === 'java' && lines[i].trim().match(
|
|
130
|
+
if (opts.lang === 'java' && lines[i].trim().match(/^@\w+/)) break;
|
|
133
131
|
if (opts.lang === 'java' && lines[i].includes(' public void ')) break;
|
|
134
132
|
if (opts.lang === 'java' && lines[i].includes(' class ')) break;
|
|
135
133
|
}
|
package/lib/xmlReader.js
CHANGED
|
@@ -52,7 +52,7 @@ class XmlReader {
|
|
|
52
52
|
if (this.opts.javaTests || this.stats.language === 'java') {
|
|
53
53
|
this.adapter = new JavaAdapter(this.opts);
|
|
54
54
|
}
|
|
55
|
-
if (this.stats.language
|
|
55
|
+
if (this.stats.language === 'js') {
|
|
56
56
|
this.adapter = new JavaScriptAdapter(this.opts);
|
|
57
57
|
}
|
|
58
58
|
if (this.stats.language === 'python') {
|
|
@@ -76,7 +76,7 @@ class XmlReader {
|
|
|
76
76
|
console.log(jsonResult)
|
|
77
77
|
throw new Error("Format can't be parsed")
|
|
78
78
|
}
|
|
79
|
-
const { testsuite, name, tests, failures, errors
|
|
79
|
+
const { testsuite, name, tests, failures, errors } = jsonSuite;
|
|
80
80
|
|
|
81
81
|
const resultTests = processTestSuite(testsuite);
|
|
82
82
|
|
|
@@ -120,7 +120,7 @@ class XmlReader {
|
|
|
120
120
|
fetchSourceCode() {
|
|
121
121
|
this.tests.forEach(t => {
|
|
122
122
|
try {
|
|
123
|
-
|
|
123
|
+
const file = this.adapter.getFilePath(t)
|
|
124
124
|
if (!file) return;
|
|
125
125
|
|
|
126
126
|
if (!this.stats.language) {
|
|
@@ -135,7 +135,7 @@ class XmlReader {
|
|
|
135
135
|
const contents = fs.readFileSync(file).toString();
|
|
136
136
|
t.code = fetchSourceCode(contents, { ...t, lang: this.stats.language })
|
|
137
137
|
} catch (err) {
|
|
138
|
-
if (process.env
|
|
138
|
+
if (process.env.DEBUG) {
|
|
139
139
|
console.log(err)
|
|
140
140
|
}
|
|
141
141
|
}
|
|
@@ -173,20 +173,22 @@ class XmlReader {
|
|
|
173
173
|
}
|
|
174
174
|
|
|
175
175
|
formatStack(t) {
|
|
176
|
-
|
|
176
|
+
const stack = this.adapter.formatStack(t);
|
|
177
177
|
|
|
178
178
|
const sourcePart = fetchSourceCodeFromStackTrace(stack);
|
|
179
179
|
|
|
180
180
|
if (!sourcePart) return stack;
|
|
181
181
|
|
|
182
|
-
|
|
182
|
+
const separator = chalk.bold.red('################[ Failure ]################');
|
|
183
|
+
|
|
184
|
+
return `${stack}\n\n${separator}\n${fetchSourceCodeFromStackTrace(stack)}`;
|
|
183
185
|
}
|
|
184
186
|
|
|
185
187
|
async uploadArtifacts() {
|
|
186
188
|
if (!this.runId) return;
|
|
187
|
-
for (const
|
|
188
|
-
const files = fetchFilesFromStackTrace(
|
|
189
|
-
|
|
189
|
+
for (const test of this.tests.filter(t => !!t.stack)) {
|
|
190
|
+
const files = fetchFilesFromStackTrace(test.stack);
|
|
191
|
+
test.artifacts = await Promise.all(files.map(f => upload.uploadFileByPath(f, this.runId)));
|
|
190
192
|
}
|
|
191
193
|
}
|
|
192
194
|
|
|
@@ -234,7 +236,7 @@ class XmlReader {
|
|
|
234
236
|
this.formatErrors();
|
|
235
237
|
this.formatTests();
|
|
236
238
|
|
|
237
|
-
if (process.env
|
|
239
|
+
if (process.env.DEBUG) {
|
|
238
240
|
console.log({
|
|
239
241
|
...this.stats,
|
|
240
242
|
tests: this.tests,
|
|
@@ -284,7 +286,7 @@ class XmlReader {
|
|
|
284
286
|
if (!isValidUrl(this.requestParams.url)) {
|
|
285
287
|
console.log(
|
|
286
288
|
APP_PREFIX,
|
|
287
|
-
chalk.red(`Error creating report on Testomat.io, report url '${
|
|
289
|
+
chalk.red(`Error creating report on Testomat.io, report url '${this.requestParams.url}' is invalid`),
|
|
288
290
|
);
|
|
289
291
|
return;
|
|
290
292
|
}
|
|
@@ -302,7 +304,7 @@ function reduceTestCases(prev, item) {
|
|
|
302
304
|
testCases = [testCases]
|
|
303
305
|
}
|
|
304
306
|
testCases.filter(t => !!t).forEach(testCaseItem => {
|
|
305
|
-
|
|
307
|
+
const file = testCaseItem.file || item.filepath || '';
|
|
306
308
|
|
|
307
309
|
let stack = '';
|
|
308
310
|
let message = '';
|
|
@@ -316,7 +318,7 @@ function reduceTestCases(prev, item) {
|
|
|
316
318
|
if (!message) message = stack.trim().split('\n')[0];
|
|
317
319
|
|
|
318
320
|
// prepend system output
|
|
319
|
-
stack = `${testCaseItem['system-out'] || testCaseItem
|
|
321
|
+
stack = `${testCaseItem['system-out'] || testCaseItem.log || ''}\n\n${stack}`.trim()
|
|
320
322
|
|
|
321
323
|
prev.push({
|
|
322
324
|
create: true,
|
package/package.json
CHANGED
package/tests/data/codecept.xml
CHANGED
|
@@ -2,96 +2,96 @@
|
|
|
2
2
|
<testsuites>
|
|
3
3
|
<testsuite name="cli" tests="89" assertions="881" errors="0" failures="10" skipped="2" useless="0" time="20.333714">
|
|
4
4
|
|
|
5
|
-
<testcase name="runCestWithTwoFailedTest" class="RunCest" file="/
|
|
6
|
-
<testcase name="runHtml" class="RunCest" file="/
|
|
7
|
-
<testcase name="runTestWithArrayExamples" class="RunCest" file="/
|
|
8
|
-
<testcase name="throwErrorIfBootstrapNotFound" class="RunCest" file="/
|
|
9
|
-
<testcase name="runOneFile" class="RunCest" file="/
|
|
10
|
-
<testcase name="runOneGroupByAttr" class="RunCest" file="/
|
|
11
|
-
<testcase name="runOneGroupWithDataProviders" class="RunCest" file="/
|
|
12
|
-
<testcase name="filterCestsByDataProviderNumber" class="RunCest" file="/
|
|
13
|
-
<testcase name="runGherkinTest" class="RunCest" file="/
|
|
14
|
-
<testcase name="runTestWithSubSteps" class="RunCest" file="/
|
|
5
|
+
<testcase name="runCestWithTwoFailedTest" class="RunCest" file="tests/data/cli/RunCest.php" feature="run cest with two failed test" time="0.119537" assertions="4"/>
|
|
6
|
+
<testcase name="runHtml" class="RunCest" file="tests/data/cli/RunCest.php" feature="execute tests with html output" time="0.116880" assertions="1"/>
|
|
7
|
+
<testcase name="runTestWithArrayExamples" class="RunCest" file="tests/data/cli/RunCest.php" feature="run test with array examples" time="0.096469" assertions="4"/>
|
|
8
|
+
<testcase name="throwErrorIfBootstrapNotFound" class="RunCest" file="tests/data/cli/RunCest.php" feature="execute one test" time="0.078476" assertions="3"/>
|
|
9
|
+
<testcase name="runOneFile" class="RunCest" file="tests/data/cli/RunCest.php" feature="execute one test" time="0.082889" assertions="1"/>
|
|
10
|
+
<testcase name="runOneGroupByAttr" class="RunCest" file="tests/data/cli/RunCest.php" feature="run one group by attr" time="0.105880" assertions="2"/>
|
|
11
|
+
<testcase name="runOneGroupWithDataProviders" class="RunCest" file="tests/data/cli/RunCest.php" feature="run one group with data providers" time="0.107412" assertions="5"/>
|
|
12
|
+
<testcase name="filterCestsByDataProviderNumber" class="RunCest" file="tests/data/cli/RunCest.php" feature="filter cests by data provider number" time="0.087259" assertions="4"/>
|
|
13
|
+
<testcase name="runGherkinTest" class="RunCest" file="tests/data/cli/RunCest.php" feature="run gherkin test" time="0.107440" assertions="7"/>
|
|
14
|
+
<testcase name="runTestWithSubSteps" class="RunCest" file="tests/data/cli/RunCest.php" feature="run test with sub steps" time="0.000815" assertions="0">
|
|
15
15
|
<skipped/>
|
|
16
16
|
</testcase>
|
|
17
|
-
<testcase name="runWithDepends" class="RunCest" file="/
|
|
18
|
-
<testcase name="runTestsByShards" class="RunCest" file="/
|
|
19
|
-
<testcase name="runErrorTest" class="RunCest" file="/
|
|
20
|
-
<testcase name="runOneFileWithColors" class="RunCest" file="/
|
|
21
|
-
<testcase name="runTestsDoesntFail" class="RunCest" file="/
|
|
22
|
-
<testcase name="filterCestsByExampleNumber" class="RunCest" file="/
|
|
23
|
-
<testcase name="runTestWithAnnotationExamples" class="RunCest" file="/
|
|
24
|
-
<testcase name="runDependentCest" class="RunCest" file="/
|
|
25
|
-
<testcase name="runTestWithAnnotationDataprovider" class="RunCest" file="/
|
|
26
|
-
<testcase name="runGherkinScenarioWithMultipleStepDefinitions" class="RunCest" file="/
|
|
27
|
-
<testcase name="runOneGroup" class="RunCest" file="/
|
|
28
|
-
<testcase name="runTestsWithFilterDoesntFail" class="RunCest" file="/
|
|
29
|
-
<testcase name="runTestWithJsonExamples" class="RunCest" file="/
|
|
30
|
-
<testcase name="runWithBeforeAfter" class="RunCest" file="/
|
|
31
|
-
<testcase name="skipGroupOfCest" class="RunCest" file="/
|
|
32
|
-
<testcase name="testsWithConditionalFails" class="RunCest" file="/
|
|
33
|
-
<testcase name="runWithUnitSkipped" class="RunCest" file="/
|
|
34
|
-
<testcase name="runHtmlCheckReport" class="RunCest" file="/
|
|
35
|
-
<testcase name="runHtmlCheckReport" class="RunCest" file="/
|
|
36
|
-
<testcase name="runHtmlCheckReport" class="RunCest" file="/
|
|
37
|
-
<testcase name="runHtmlCheckReport" class="RunCest" file="/
|
|
38
|
-
<testcase name="runHtmlCheckReport" class="RunCest" file="/
|
|
39
|
-
<testcase name="runHtmlCheckReport" class="RunCest" file="/
|
|
40
|
-
<testcase name="runHtmlCheckReport" class="RunCest" file="/
|
|
41
|
-
<testcase name="runHtmlCheckReport" class="RunCest" file="/
|
|
42
|
-
<testcase name="runHtmlCheckReport" class="RunCest" file="/
|
|
43
|
-
<testcase name="runHtmlCheckReport" class="RunCest" file="/
|
|
44
|
-
<testcase name="runHtmlCheckReport" class="RunCest" file="/
|
|
45
|
-
<testcase name="runHtmlCheckReport" class="RunCest" file="/
|
|
46
|
-
<testcase name="runHtmlCheckReport" class="RunCest" file="/
|
|
47
|
-
<testcase name="runTestsWithSteps" class="RunCest" file="/
|
|
48
|
-
<testcase name="runWithDataprovider" class="RunCest" file="/
|
|
49
|
-
<testcase name="runXmlReport" class="RunCest" file="/
|
|
50
|
-
<testcase name="runTestsWithGrep" class="RunCest" file="/
|
|
51
|
-
<testcase name="runHtmlWithPhpBrowserCheckReport" class="RunCest" file="/
|
|
52
|
-
<testcase name="skipRunOneGroup" class="RunCest" file="/
|
|
53
|
-
<testcase name="runTestsWithFilter" class="RunCest" file="/
|
|
54
|
-
<testcase name="runGherkinScenarioOutline" class="RunCest" file="/
|
|
55
|
-
<testcase name="runTestsWithDependencyInjections" class="RunCest" file="/
|
|
56
|
-
<testcase name="runWithExamples" class="RunCest" file="/
|
|
57
|
-
<testcase name="runPhpUnitXmlReport" class="RunCest" file="/
|
|
58
|
-
<testcase name="runInvalidDataProvider" class="RunCest" file="/
|
|
59
|
-
<testcase name="runTestWithDataProviders" class="RunCest" file="/
|
|
60
|
-
<testcase name="runFailingGherkinTest" class="RunCest" file="/
|
|
61
|
-
<testcase name="showSameOrderOfFilesOnSeed" class="RunCest" file="/
|
|
62
|
-
<testcase name="runWithUnitIncomplete" class="RunCest" file="/
|
|
63
|
-
<testcase name="showSeedNumberOnShuffle" class="RunCest" file="/
|
|
64
|
-
<testcase name="filterTestsByDataProviderCaseNumber" class="RunCest" file="/
|
|
65
|
-
<testcase name="runTwoSuites" class="RunCest" file="/
|
|
66
|
-
<testcase name="runCustomBootstrap" class="RunCest" file="/
|
|
67
|
-
<testcase name="runOneTestFromCest" class="RunCest" file="/
|
|
68
|
-
<testcase name="runTestWithFailedScenario" class="RunCest" file="/
|
|
17
|
+
<testcase name="runWithDepends" class="RunCest" file="tests/data/cli/RunCest.php" feature="run with depends" time="0.083950" assertions="1"/>
|
|
18
|
+
<testcase name="runTestsByShards" class="RunCest" file="tests/data/cli/RunCest.php" feature="run tests by shards" time="0.273580" assertions="9"/>
|
|
19
|
+
<testcase name="runErrorTest" class="RunCest" file="tests/data/cli/RunCest.php" feature="run error test" time="0.098815" assertions="3"/>
|
|
20
|
+
<testcase name="runOneFileWithColors" class="RunCest" file="tests/data/cli/RunCest.php" feature="execute one test" time="0.084870" assertions="2"/>
|
|
21
|
+
<testcase name="runTestsDoesntFail" class="RunCest" file="tests/data/cli/RunCest.php" feature="run tests doesnt fail" time="0.089660" assertions="1"/>
|
|
22
|
+
<testcase name="filterCestsByExampleNumber" class="RunCest" file="tests/data/cli/RunCest.php" feature="filter cests by example number" time="0.099362" assertions="5"/>
|
|
23
|
+
<testcase name="runTestWithAnnotationExamples" class="RunCest" file="tests/data/cli/RunCest.php" feature="run test with annotation examples" time="0.082711" assertions="4"/>
|
|
24
|
+
<testcase name="runDependentCest" class="RunCest" file="tests/data/cli/RunCest.php" feature="run dependent cest" time="0.081131" assertions="1"/>
|
|
25
|
+
<testcase name="runTestWithAnnotationDataprovider" class="RunCest" file="tests/data/cli/RunCest.php" feature="run test with annotation dataprovider" time="0.160254" assertions="1"/>
|
|
26
|
+
<testcase name="runGherkinScenarioWithMultipleStepDefinitions" class="RunCest" file="tests/data/cli/RunCest.php" feature="run gherkin scenario with multiple step definitions" time="0.088234" assertions="4"/>
|
|
27
|
+
<testcase name="runOneGroup" class="RunCest" file="tests/data/cli/RunCest.php" feature="run one group" time="0.076610" assertions="3"/>
|
|
28
|
+
<testcase name="runTestsWithFilterDoesntFail" class="RunCest" file="tests/data/cli/RunCest.php" feature="run tests with filter doesnt fail" time="0.159520" assertions="2"/>
|
|
29
|
+
<testcase name="runTestWithJsonExamples" class="RunCest" file="tests/data/cli/RunCest.php" feature="run test with json examples" time="0.082773" assertions="4"/>
|
|
30
|
+
<testcase name="runWithBeforeAfter" class="RunCest" file="tests/data/cli/RunCest.php" feature="run with before after" time="0.078491" assertions="4"/>
|
|
31
|
+
<testcase name="skipGroupOfCest" class="RunCest" file="tests/data/cli/RunCest.php" feature="skip group of cest" time="0.170683" assertions="5"/>
|
|
32
|
+
<testcase name="testsWithConditionalFails" class="RunCest" file="tests/data/cli/RunCest.php" feature="tests with conditional fails" time="0.097338" assertions="4"/>
|
|
33
|
+
<testcase name="runWithUnitSkipped" class="RunCest" file="tests/data/cli/RunCest.php" feature="run with unit skipped" time="0.097461" assertions="1"/>
|
|
34
|
+
<testcase name="runHtmlCheckReport" class="RunCest" file="tests/data/cli/RunCest.php" feature="verify that all steps are rendered correctly in HTML report (' . $test . ') | "CodeceptionIssue4413Cest:twoCommentStepsInARow"" time="0.073369" assertions="2"/>
|
|
35
|
+
<testcase name="runHtmlCheckReport" class="RunCest" file="tests/data/cli/RunCest.php" feature="verify that all steps are rendered correctly in HTML report (' . $test . ') | "CodeceptionIssue4413Cest:twoCommentStepsInARowViaPageObjectActor"" time="0.072299" assertions="2"/>
|
|
36
|
+
<testcase name="runHtmlCheckReport" class="RunCest" file="tests/data/cli/RunCest.php" feature="verify that all steps are rendered correctly in HTML report (' . $test . ') | "CodeceptionIssue4413Cest:twoCommentStepsWithOneSubStepInBetween"" time="0.096977" assertions="2"/>
|
|
37
|
+
<testcase name="runHtmlCheckReport" class="RunCest" file="tests/data/cli/RunCest.php" feature="verify that all steps are rendered correctly in HTML report (' . $test . ') | "CodeceptionIssue4413Cest:commentStepsWithDifferentSubStepsInBetweenAndAfter"" time="0.076719" assertions="2"/>
|
|
38
|
+
<testcase name="runHtmlCheckReport" class="RunCest" file="tests/data/cli/RunCest.php" feature="verify that all steps are rendered correctly in HTML report (' . $test . ') | "CodeceptionIssue4413Cest:differentSubSteps"" time="0.075144" assertions="2"/>
|
|
39
|
+
<testcase name="runHtmlCheckReport" class="RunCest" file="tests/data/cli/RunCest.php" feature="verify that all steps are rendered correctly in HTML report (' . $test . ') | "CodeceptionIssue4413Cest:commentStepsWithDifferentSubStepsOnceNestedInBetweenAndAfter"" time="0.092805" assertions="2"/>
|
|
40
|
+
<testcase name="runHtmlCheckReport" class="RunCest" file="tests/data/cli/RunCest.php" feature="verify that all steps are rendered correctly in HTML report (' . $test . ') | "CodeceptionIssue4413Cest:commentStepsWithDifferentSubStepsOnceNestedInBetweenAndAfter2"" time="0.098153" assertions="2"/>
|
|
41
|
+
<testcase name="runHtmlCheckReport" class="RunCest" file="tests/data/cli/RunCest.php" feature="verify that all steps are rendered correctly in HTML report (' . $test . ') | "CodeceptionIssue4413Cest:nestedSubStepFollowedByOtherSubStep"" time="0.078239" assertions="2"/>
|
|
42
|
+
<testcase name="runHtmlCheckReport" class="RunCest" file="tests/data/cli/RunCest.php" feature="verify that all steps are rendered correctly in HTML report (' . $test . ') | "CodeceptionIssue4413Cest:nestedSubStepFollowedByOtherSubStep2"" time="0.095552" assertions="2"/>
|
|
43
|
+
<testcase name="runHtmlCheckReport" class="RunCest" file="tests/data/cli/RunCest.php" feature="verify that all steps are rendered correctly in HTML report (' . $test . ') | "CodeceptionIssue4413Cest:twoIdentialSubStepsInARow"" time="0.077251" assertions="2"/>
|
|
44
|
+
<testcase name="runHtmlCheckReport" class="RunCest" file="tests/data/cli/RunCest.php" feature="verify that all steps are rendered correctly in HTML report (' . $test . ') | "CodeceptionIssue4413Cest:twoIdentialSubStepsInARowFollowedByAnotherSubStep"" time="0.078100" assertions="2"/>
|
|
45
|
+
<testcase name="runHtmlCheckReport" class="RunCest" file="tests/data/cli/RunCest.php" feature="verify that all steps are rendered correctly in HTML report (' . $test . ') | "CodeceptionIssue4413Cest:twoIdentialSubStepsWithAnotherSubStepInBetween"" time="0.096649" assertions="2"/>
|
|
46
|
+
<testcase name="runHtmlCheckReport" class="RunCest" file="tests/data/cli/RunCest.php" feature="verify that all steps are rendered correctly in HTML report (' . $test . ') | "CodeceptionIssue4413Cest:subStepFollowedByTwoIdentialSubSteps"" time="0.097845" assertions="2"/>
|
|
47
|
+
<testcase name="runTestsWithSteps" class="RunCest" file="tests/data/cli/RunCest.php" feature="run tests with steps" time="0.097513" assertions="1"/>
|
|
48
|
+
<testcase name="runWithDataprovider" class="RunCest" file="tests/data/cli/RunCest.php" feature="run with dataprovider" time="0.101674" assertions="1"/>
|
|
49
|
+
<testcase name="runXmlReport" class="RunCest" file="tests/data/cli/RunCest.php" feature="check xml reports" time="0.100334" assertions="5"/>
|
|
50
|
+
<testcase name="runTestsWithGrep" class="RunCest" file="tests/data/cli/RunCest.php" feature="run tests with grep" time="0.177239" assertions="3"/>
|
|
51
|
+
<testcase name="runHtmlWithPhpBrowserCheckReport" class="RunCest" file="tests/data/cli/RunCest.php" feature="execute tests with PhpBrowser with html output and check html" time="0.282488" assertions="5"/>
|
|
52
|
+
<testcase name="skipRunOneGroup" class="RunCest" file="tests/data/cli/RunCest.php" feature="skip run one group" time="0.098319" assertions="3"/>
|
|
53
|
+
<testcase name="runTestsWithFilter" class="RunCest" file="tests/data/cli/RunCest.php" feature="run tests with filter" time="0.086682" assertions="2"/>
|
|
54
|
+
<testcase name="runGherkinScenarioOutline" class="RunCest" file="tests/data/cli/RunCest.php" feature="run gherkin scenario outline" time="0.103988" assertions="1"/>
|
|
55
|
+
<testcase name="runTestsWithDependencyInjections" class="RunCest" file="tests/data/cli/RunCest.php" feature="run tests with dependency injections" time="0.080304" assertions="7"/>
|
|
56
|
+
<testcase name="runWithExamples" class="RunCest" file="tests/data/cli/RunCest.php" feature="run with examples" time="0.100186" assertions="1"/>
|
|
57
|
+
<testcase name="runPhpUnitXmlReport" class="RunCest" file="tests/data/cli/RunCest.php" feature="check phpunit xml reports" time="0.101651" assertions="10"/>
|
|
58
|
+
<testcase name="runInvalidDataProvider" class="RunCest" file="tests/data/cli/RunCest.php" feature="run invalid data provider" time="0.111115" assertions="4"/>
|
|
59
|
+
<testcase name="runTestWithDataProviders" class="RunCest" file="tests/data/cli/RunCest.php" feature="run test with data providers" time="0.095058" assertions="5"/>
|
|
60
|
+
<testcase name="runFailingGherkinTest" class="RunCest" file="tests/data/cli/RunCest.php" feature="run failing gherkin test" time="0.076423" assertions="2"/>
|
|
61
|
+
<testcase name="showSameOrderOfFilesOnSeed" class="RunCest" file="tests/data/cli/RunCest.php" feature="show same order of files on seed" time="0.306368" assertions="3"/>
|
|
62
|
+
<testcase name="runWithUnitIncomplete" class="RunCest" file="tests/data/cli/RunCest.php" feature="run with unit incomplete" time="0.097535" assertions="1"/>
|
|
63
|
+
<testcase name="showSeedNumberOnShuffle" class="RunCest" file="tests/data/cli/RunCest.php" feature="show seed number on shuffle" time="0.223366" assertions="2"/>
|
|
64
|
+
<testcase name="filterTestsByDataProviderCaseNumber" class="RunCest" file="tests/data/cli/RunCest.php" feature="filter tests by data provider case number" time="0.100420" assertions="5"/>
|
|
65
|
+
<testcase name="runTwoSuites" class="RunCest" file="tests/data/cli/RunCest.php" feature="run two suites" time="0.098478" assertions="3"/>
|
|
66
|
+
<testcase name="runCustomBootstrap" class="RunCest" file="tests/data/cli/RunCest.php" feature="execute one test" time="0.095968" assertions="3"/>
|
|
67
|
+
<testcase name="runOneTestFromCest" class="RunCest" file="tests/data/cli/RunCest.php" feature="run one test from cest" time="0.073993" assertions="2"/>
|
|
68
|
+
<testcase name="runTestWithFailedScenario" class="RunCest" file="tests/data/cli/RunCest.php" feature="run test with failed scenario" time="0.000200" assertions="0">
|
|
69
69
|
<skipped/>
|
|
70
70
|
</testcase>
|
|
71
|
-
<testcase name="runPhpUnitXmlReportsInStrictMode" class="RunCest" file="/
|
|
72
|
-
<testcase name="runBootstrapInGlobalConfig" class="RunCest" file="/
|
|
73
|
-
<testcase name="reportersConfigurationSectionIsNotSupported" class="RunCest" file="/
|
|
74
|
-
<testcase name="runFailedTestAndCheckOutput" class="RunCest" file="/
|
|
75
|
-
<testcase name="runTestWithFailFastDefault" class="RunCest" file="/
|
|
76
|
-
<testcase name="runTestWithComplexExample" class="RunCest" file="/
|
|
77
|
-
<testcase name="runTestWithAnnotationExamplesFromGroupFileTest" class="RunCest" file="/
|
|
78
|
-
<testcase name="filterTestsByDataProviderCaseNumberRange" class="RunCest" file="/
|
|
79
|
-
<testcase name="runSuiteWhenNameMatchesExistingDirectory" class="RunCest" file="/
|
|
80
|
-
<testcase name="runBootstrapInSuiteConfig" class="RunCest" file="/
|
|
81
|
-
<testcase name="runOneTestFromUnit" class="RunCest" file="/
|
|
82
|
-
<testcase name="reportsCorrectFailedStep" class="RunCest" file="/
|
|
83
|
-
<testcase name="overrideModuleOptions" class="RunCest" file="/
|
|
84
|
-
<testcase name="runTestWithException" class="RunCest" file="/
|
|
85
|
-
<testcase name="runWithCustomOutputPath" class="RunCest" file="/
|
|
86
|
-
<testcase name="filterTestsByDataProviderCaseName" class="RunCest" file="/
|
|
87
|
-
<testcase name="filterTestsWithoutSpecifyingSuite" class="RunCest" file="/
|
|
88
|
-
<testcase name="runXmlReportsInStrictMode" class="RunCest" file="/
|
|
89
|
-
<testcase name="runCustomReport" class="RunCest" file="/
|
|
90
|
-
<testcase name="runDependentTest" class="RunCest" file="/
|
|
91
|
-
<testcase name="runTestWithFailFastCustom" class="RunCest" file="/
|
|
92
|
-
<testcase name="runTestWithCustomSetupMethod" class="RunCest" file="/
|
|
93
|
-
<testcase name="runCompactReport" class="RunCest" file="/
|
|
94
|
-
<testcase name="skipSuites" class="RunCest" file="/
|
|
71
|
+
<testcase name="runPhpUnitXmlReportsInStrictMode" class="RunCest" file="tests/data/cli/RunCest.php" feature="check phpunit xml in strict mode" time="0.080945" assertions="10"/>
|
|
72
|
+
<testcase name="runBootstrapInGlobalConfig" class="RunCest" file="tests/data/cli/RunCest.php" feature="execute one test" time="0.099138" assertions="3"/>
|
|
73
|
+
<testcase name="reportersConfigurationSectionIsNotSupported" class="RunCest" file="tests/data/cli/RunCest.php" feature="reporters configuration section is not supported" time="0.200293" assertions="3"/>
|
|
74
|
+
<testcase name="runFailedTestAndCheckOutput" class="RunCest" file="tests/data/cli/RunCest.php" feature="run failed test and check output" time="0.092119" assertions="4"/>
|
|
75
|
+
<testcase name="runTestWithFailFastDefault" class="RunCest" file="tests/data/cli/RunCest.php" feature="run test with fail fast default" time="0.193308" assertions="4"/>
|
|
76
|
+
<testcase name="runTestWithComplexExample" class="RunCest" file="tests/data/cli/RunCest.php" feature="run test with complex example" time="0.084370" assertions="5"/>
|
|
77
|
+
<testcase name="runTestWithAnnotationExamplesFromGroupFileTest" class="RunCest" file="tests/data/cli/RunCest.php" feature="run test with annotation examples from group file test" time="0.091098" assertions="1"/>
|
|
78
|
+
<testcase name="filterTestsByDataProviderCaseNumberRange" class="RunCest" file="tests/data/cli/RunCest.php" feature="filter tests by data provider case number range" time="0.074344" assertions="5"/>
|
|
79
|
+
<testcase name="runSuiteWhenNameMatchesExistingDirectory" class="RunCest" file="tests/data/cli/RunCest.php" feature="run suite when name matches existing directory" time="0.069087" assertions="1"/>
|
|
80
|
+
<testcase name="runBootstrapInSuiteConfig" class="RunCest" file="tests/data/cli/RunCest.php" feature="execute one test" time="0.080465" assertions="3"/>
|
|
81
|
+
<testcase name="runOneTestFromUnit" class="RunCest" file="tests/data/cli/RunCest.php" feature="run one test from unit" time="0.083712" assertions="3"/>
|
|
82
|
+
<testcase name="reportsCorrectFailedStep" class="RunCest" file="tests/data/cli/RunCest.php" feature="reports correct failed step" time="0.083000" assertions="2"/>
|
|
83
|
+
<testcase name="overrideModuleOptions" class="RunCest" file="tests/data/cli/RunCest.php" feature="override module options" time="0.166803" assertions="2"/>
|
|
84
|
+
<testcase name="runTestWithException" class="RunCest" file="tests/data/cli/RunCest.php" feature="run test with exception" time="0.087845" assertions="4"/>
|
|
85
|
+
<testcase name="runWithCustomOutputPath" class="RunCest" file="tests/data/cli/RunCest.php" feature="run with custom output path" time="0.085757" assertions="7"/>
|
|
86
|
+
<testcase name="filterTestsByDataProviderCaseName" class="RunCest" file="tests/data/cli/RunCest.php" feature="filter tests by data provider case name" time="0.092092" assertions="5"/>
|
|
87
|
+
<testcase name="filterTestsWithoutSpecifyingSuite" class="RunCest" file="tests/data/cli/RunCest.php" feature="filter tests without specifying suite" time="0.075297" assertions="1"/>
|
|
88
|
+
<testcase name="runXmlReportsInStrictMode" class="RunCest" file="tests/data/cli/RunCest.php" feature="check xml in strict mode" time="0.084044" assertions="5"/>
|
|
89
|
+
<testcase name="runCustomReport" class="RunCest" file="tests/data/cli/RunCest.php" feature="run custom report" time="0.080532" assertions="2"/>
|
|
90
|
+
<testcase name="runDependentTest" class="RunCest" file="tests/data/cli/RunCest.php" feature="run dependent test" time="0.178287" assertions="2"/>
|
|
91
|
+
<testcase name="runTestWithFailFastCustom" class="RunCest" file="tests/data/cli/RunCest.php" feature="run test with fail fast custom" time="0.168278" assertions="2"/>
|
|
92
|
+
<testcase name="runTestWithCustomSetupMethod" class="RunCest" file="tests/data/cli/RunCest.php" feature="run test with custom setup method" time="0.083309" assertions="1"/>
|
|
93
|
+
<testcase name="runCompactReport" class="RunCest" file="tests/data/cli/RunCest.php" feature="run compact report" time="0.074003" assertions="1"/>
|
|
94
|
+
<testcase name="skipSuites" class="RunCest" file="tests/data/cli/RunCest.php" feature="skip suites" time="0.064021" assertions="4"/>
|
|
95
95
|
<testcase name="runAllSnapshotTests" class="SnapshotCest" file="tests/data/cli/SnapshotCest.php" feature="run all snapshot tests" time="0.058300" assertions="1">
|
|
96
96
|
<failure type="PHPUnit\Framework\AssertionFailedError">SnapshotCest: Run all snapshot tests
|
|
97
97
|
Result code was 1.
|
|
@@ -147,4 +147,4 @@ tests/data/cli/SnapshotCest.php:92
|
|
|
147
147
|
</failure>
|
|
148
148
|
</testcase>
|
|
149
149
|
</testsuite>
|
|
150
|
-
</testsuites>
|
|
150
|
+
</testsuites>
|