@micro-os-plus/micro-test-plus 2.1.1 → 3.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +42 -0
- package/CMakeLists.txt +15 -26
- package/LICENSE-Boost +23 -0
- package/README.md +1007 -255
- package/docs/NOTES.md +7 -0
- package/include/micro-os-plus/detail.h +733 -0
- package/include/micro-os-plus/inlines.h +182 -0
- package/include/micro-os-plus/literals.h +285 -0
- package/include/micro-os-plus/math.h +205 -0
- package/include/micro-os-plus/micro-test-plus.h +488 -102
- package/include/micro-os-plus/reflection.h +134 -0
- package/include/micro-os-plus/test-reporter-inlines.h +231 -0
- package/include/micro-os-plus/test-reporter.h +347 -0
- package/include/micro-os-plus/test-runner.h +132 -0
- package/include/micro-os-plus/test-suite.h +242 -0
- package/include/micro-os-plus/type-traits.h +346 -0
- package/meson.build +35 -11
- package/package.json +243 -147
- package/src/micro-test-plus.cpp +105 -188
- package/src/test-reporter.cpp +423 -0
- package/src/test-runner.cpp +183 -0
- package/src/test-suite.cpp +117 -0
package/README.md
CHANGED
|
@@ -1,9 +1,9 @@
|
|
|
1
1
|
[](https://github.com/micro-os-plus/micro-test-plus-xpack/blob/xpack/LICENSE)
|
|
2
2
|
[](https://github.com/micro-os-plus/micro-test-plus-xpack/actions?query=workflow%3A%22CI+on+Push%22)
|
|
3
3
|
|
|
4
|
-
# A source xPack with µTest++, a
|
|
4
|
+
# A source library xPack with µTest++, a testing framework for embedded platforms
|
|
5
5
|
|
|
6
|
-
The **µTest++** project (_micro test plus_) provides a
|
|
6
|
+
The **µTest++** project (_micro test plus_) provides a relatively simple
|
|
7
7
|
testing framework, intended for running unit tests on embedded
|
|
8
8
|
platforms.
|
|
9
9
|
|
|
@@ -12,16 +12,16 @@ The project is hosted on GitHub as
|
|
|
12
12
|
|
|
13
13
|
## Maintainer info
|
|
14
14
|
|
|
15
|
-
This page is addressed to developers who plan to include this
|
|
16
|
-
into their own projects.
|
|
15
|
+
This page is addressed to developers who plan to include this source
|
|
16
|
+
library into their own projects.
|
|
17
17
|
|
|
18
|
-
For maintainer
|
|
18
|
+
For maintainer info, please see the
|
|
19
19
|
[README-MAINTAINER](README-MAINTAINER.md) file.
|
|
20
20
|
|
|
21
21
|
## Install
|
|
22
22
|
|
|
23
|
-
As a source xPacks, the easiest way to add it to a project is via
|
|
24
|
-
but it can also be used as any Git project, for example as a submodule.
|
|
23
|
+
As a source library xPacks, the easiest way to add it to a project is via
|
|
24
|
+
**xpm**, but it can also be used as any Git project, for example as a submodule.
|
|
25
25
|
|
|
26
26
|
### Prerequisites
|
|
27
27
|
|
|
@@ -29,7 +29,7 @@ A recent [xpm](https://xpack.github.io/xpm/),
|
|
|
29
29
|
which is a portable [Node.js](https://nodejs.org/) command line application.
|
|
30
30
|
|
|
31
31
|
For details please follow the instructions in the
|
|
32
|
-
[install](https://xpack.github.io/install/) page.
|
|
32
|
+
[xPack install](https://xpack.github.io/install/) page.
|
|
33
33
|
|
|
34
34
|
### xpm
|
|
35
35
|
|
|
@@ -42,6 +42,8 @@ cd my-project
|
|
|
42
42
|
xpm init # Unless a package.json is already present
|
|
43
43
|
|
|
44
44
|
xpm install @micro-os-plus/micro-test-plus@latest
|
|
45
|
+
|
|
46
|
+
ls -l xpacks/micro-os-plus-micro-test-plus
|
|
45
47
|
```
|
|
46
48
|
|
|
47
49
|
### Git submodule
|
|
@@ -71,309 +73,1034 @@ Pull Requests should be directed to this branch.
|
|
|
71
73
|
When new releases are published, the `xpack-develop` branch is merged
|
|
72
74
|
into `xpack`.
|
|
73
75
|
|
|
74
|
-
##
|
|
76
|
+
## Developer info
|
|
75
77
|
|
|
76
|
-
The
|
|
77
|
-
|
|
78
|
+
The xPack Build framework already includes several ready to use
|
|
79
|
+
testing frameworks:
|
|
78
80
|
|
|
79
|
-
-
|
|
80
|
-
|
|
81
|
-
-
|
|
82
|
-
- each test case may perform any number of tests checks
|
|
83
|
-
- each test check either succeeds or fails
|
|
84
|
-
- the test progress is shown on STDOUT, with each test check on a separate
|
|
85
|
-
line, prefixed with either a check sign (✓) or a cross sign (✗)
|
|
86
|
-
- the main result of the test is passed back as the process exit code
|
|
81
|
+
- [Google Test](https://github.com/xpack-3rd-party/googletest-xpack)
|
|
82
|
+
- [Catch2](https://github.com/xpack-3rd-party/catch2-xpack)
|
|
83
|
+
- [Boost UT](https://github.com/xpack-3rd-party/boost-ut-xpack))
|
|
87
84
|
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
0 as exit value.
|
|
85
|
+
However, they all are quite heavy in terms of memory resources, and the
|
|
86
|
+
learning curve for mastering them is quite steep.
|
|
91
87
|
|
|
92
|
-
|
|
88
|
+
Thus, for embedded projects, a simpler solution, with a smaller
|
|
89
|
+
memory footprint, was considered a useful addition.
|
|
93
90
|
|
|
94
|
-
|
|
95
|
-
written in C++ too (although there are no
|
|
96
|
-
major reasons to prevent adding C wrappers).
|
|
91
|
+
### Overview
|
|
97
92
|
|
|
98
|
-
|
|
99
|
-
|
|
93
|
+
The initial version of the **µTest++** framework was inspired mainly by
|
|
94
|
+
[Node tap](https://node-tap.org) and aimed for simplicity.
|
|
95
|
+
The later v3.x was a full rework inspired by
|
|
96
|
+
[Boost UT](https://boost-ext.github.io/ut/).
|
|
100
97
|
|
|
101
|
-
|
|
102
|
-
are already made CMake and meson configuration files (see below).
|
|
98
|
+
The main characteristics of µTest++, basically inherited from Boost UT, are:
|
|
103
99
|
|
|
104
|
-
|
|
100
|
+
- intended to test both C and C++ projects
|
|
101
|
+
- modern C++ 20 code (this was also the reason
|
|
102
|
+
to raise the bar to C++ 20 for the entire µOS++ project)
|
|
103
|
+
- macro free (while preserving the nice feature of being able to report
|
|
104
|
+
the file name and line number for failed tests)
|
|
105
|
+
- expectations, assumptions, exceptions
|
|
106
|
+
- test cases, test suites
|
|
107
|
+
- automatic test suites registration
|
|
105
108
|
|
|
106
|
-
|
|
109
|
+
As major differentiator from Boost UT:
|
|
107
110
|
|
|
108
|
-
-
|
|
111
|
+
- reduced memory footprint, since there are no dependencies on
|
|
112
|
+
the standard C++ stream library
|
|
113
|
+
- a slightly simplified API
|
|
109
114
|
|
|
110
|
-
|
|
115
|
+
### Concepts and features
|
|
111
116
|
|
|
112
|
-
|
|
117
|
+
- for complex applications, test cases can be grouped in test suites
|
|
118
|
+
- test suites can be located in separate compilation units, and automatically
|
|
119
|
+
register themselves to the runner;
|
|
120
|
+
- a test suite is a named sequence of test cases;
|
|
121
|
+
- a test case is a sequence of test conditions (or simply tests),
|
|
122
|
+
which are expected to be true;
|
|
123
|
+
- tests are based on logical expressions, which usually
|
|
124
|
+
compute a result and compare it to an expected value
|
|
125
|
+
- for C++ projects: it is also possible to check if, while evaluating
|
|
126
|
+
an expression, exceptions are thrown or not;
|
|
127
|
+
- each test either succeeds or fails; the runner keeps counts of them;
|
|
128
|
+
- assumptions are hard conditions expected to be true in order for the test
|
|
129
|
+
to be able to run;
|
|
130
|
+
- failed assumptions abort the test;
|
|
131
|
+
- the test progress is shown on STDOUT, with each tests on a
|
|
132
|
+
separate line, prefixed with either a check sign (✓) or a cross sign (✗);
|
|
133
|
+
- failed tests display the location in the file and, if possible,
|
|
134
|
+
the actual values used in the expression evaluation;
|
|
135
|
+
- the main result of the test is passed back to the system as the process
|
|
136
|
+
exit code.
|
|
113
137
|
|
|
114
|
-
|
|
138
|
+
A test suite is considered successful
|
|
139
|
+
if there is at least one successful test (expectation or assumption)
|
|
140
|
+
and there are no failed tests.
|
|
141
|
+
|
|
142
|
+
If all tests suites are successful, the process returns 0 as exit value.
|
|
143
|
+
|
|
144
|
+
### ISTQB Glossary
|
|
145
|
+
|
|
146
|
+
The **International Software Testing Qualification Board** defines some terms
|
|
147
|
+
used in testing frameworks:
|
|
148
|
+
|
|
149
|
+
- test condition: a testable aspect of a component or system identified
|
|
150
|
+
as a basis for testing (implemented in µTest++ as calls to `expect()` or
|
|
151
|
+
`assume()` functions);
|
|
152
|
+
- test case: a set of preconditions, inputs, actions (where applicable),
|
|
153
|
+
expected results and postconditions, developed based on test conditions
|
|
154
|
+
(implemented in µTest++ as calls to the `test_case()` function)
|
|
155
|
+
- test suite: a set of test scripts or test procedures to be executed in
|
|
156
|
+
a specific test run (implemented in µTest++ as instances of the
|
|
157
|
+
`test_suite` class).
|
|
115
158
|
|
|
116
|
-
|
|
159
|
+
For more details see: <http://glossary.istqb.org/en/search/test%20case>.
|
|
160
|
+
|
|
161
|
+
### Getting started
|
|
162
|
+
|
|
163
|
+
The absolute minimal test has a single test case, with a single expectation;
|
|
164
|
+
for example:
|
|
117
165
|
|
|
118
166
|
```c++
|
|
119
167
|
#include <micro-os-plus/micro-test-plus.h>
|
|
168
|
+
|
|
169
|
+
int
|
|
170
|
+
main(int argc, char* argv[])
|
|
171
|
+
{
|
|
172
|
+
using namespace micro_os_plus::micro_test_plus;
|
|
173
|
+
|
|
174
|
+
initialize(argc, argv, "Minimal");
|
|
175
|
+
|
|
176
|
+
test_case ("Check truth", [] {
|
|
177
|
+
expect (true);
|
|
178
|
+
})
|
|
179
|
+
|
|
180
|
+
return exit_code ();
|
|
181
|
+
}
|
|
120
182
|
```
|
|
121
183
|
|
|
122
|
-
|
|
184
|
+
When running this test, the output looks like:
|
|
123
185
|
|
|
124
|
-
|
|
186
|
+
```console
|
|
187
|
+
Minimal - test suite started
|
|
125
188
|
|
|
126
|
-
|
|
189
|
+
✓ Check truth - test case passed (1 check)
|
|
190
|
+
|
|
191
|
+
Minimal - test suite passed (1 check in 1 test case)
|
|
192
|
+
```
|
|
127
193
|
|
|
128
|
-
|
|
129
|
-
|
|
194
|
+
A slightly more useful example would check the result of a computed value;
|
|
195
|
+
for example:
|
|
130
196
|
|
|
131
|
-
|
|
197
|
+
```c++
|
|
198
|
+
#include <micro-os-plus/micro-test-plus.h>
|
|
132
199
|
|
|
133
|
-
|
|
200
|
+
static int
|
|
201
|
+
compute_answer()
|
|
202
|
+
{
|
|
203
|
+
return 42;
|
|
204
|
+
}
|
|
134
205
|
|
|
135
|
-
|
|
206
|
+
int
|
|
207
|
+
main(int argc, char* argv[])
|
|
208
|
+
{
|
|
209
|
+
using namespace micro_os_plus::micro_test_plus;
|
|
136
210
|
|
|
137
|
-
|
|
211
|
+
initialize(argc, argv, "The Answer");
|
|
138
212
|
|
|
139
|
-
|
|
213
|
+
test_case ("Check answer", [] {
|
|
214
|
+
expect (compute_answer() == 42, "answer is 42");
|
|
215
|
+
});
|
|
140
216
|
|
|
141
|
-
|
|
142
|
-
|
|
217
|
+
return exit_code ();
|
|
218
|
+
}
|
|
219
|
+
```
|
|
143
220
|
|
|
144
|
-
|
|
221
|
+
```console
|
|
222
|
+
The Answer - test suite started
|
|
145
223
|
|
|
146
|
-
|
|
147
|
-
folder to the build:
|
|
224
|
+
✓ Check answer - test case passed (1 check)
|
|
148
225
|
|
|
149
|
-
|
|
150
|
-
add_subdirectory("xpacks/micro-os-plus-micro-test-plus")`
|
|
226
|
+
The Answer - test suite passed (1 check passed, 0 checks failed, in 1 test case)
|
|
151
227
|
```
|
|
152
228
|
|
|
153
|
-
|
|
154
|
-
|
|
229
|
+
In case that the function returns the wrong answer, the test will also fail;
|
|
230
|
+
for example:
|
|
155
231
|
|
|
156
|
-
```
|
|
157
|
-
|
|
158
|
-
|
|
232
|
+
```c++
|
|
233
|
+
static int
|
|
234
|
+
compute_answer()
|
|
235
|
+
{
|
|
236
|
+
return 42 + 1;
|
|
237
|
+
}
|
|
238
|
+
```
|
|
159
239
|
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
|
|
240
|
+
In this case the test will fail with:
|
|
241
|
+
|
|
242
|
+
```console
|
|
243
|
+
The Answer - test suite started
|
|
244
|
+
|
|
245
|
+
• Check answer - test case started
|
|
246
|
+
✗ answer is 42 FAILED (answer.cpp:17)
|
|
247
|
+
✗ Check answer - test case FAILED (0 checks passed, 1 check failed)
|
|
248
|
+
|
|
249
|
+
The Answer - test suite FAILED (0 checks passed, 1 check failed, in 1 test case)
|
|
164
250
|
```
|
|
165
251
|
|
|
166
|
-
|
|
252
|
+
The output identifies the failed test as located at line 17, but does not
|
|
253
|
+
provide more details, for example it does not tell what was the actual
|
|
254
|
+
wrong answer.
|
|
167
255
|
|
|
168
|
-
To
|
|
169
|
-
|
|
256
|
+
To get such useful information, the test should be slightly more elaborate,
|
|
257
|
+
and must use some custom comparators or operators; for example:
|
|
170
258
|
|
|
171
|
-
```
|
|
172
|
-
|
|
259
|
+
```c++
|
|
260
|
+
// ...
|
|
261
|
+
|
|
262
|
+
int
|
|
263
|
+
main(int argc, char* argv[])
|
|
264
|
+
{
|
|
265
|
+
using namespace micro_os_plus::micro_test_plus;
|
|
266
|
+
|
|
267
|
+
initialize(argc, argv, "The Answer");
|
|
268
|
+
|
|
269
|
+
test_case ("Check answer with comparator", [] {
|
|
270
|
+
expect (eq (compute_answer (), 42)) << "answer is 42";
|
|
271
|
+
});
|
|
272
|
+
|
|
273
|
+
test_case ("Check answer with operator", [] {
|
|
274
|
+
using namespace micro_os_plus::micro_test_plus::operators;
|
|
275
|
+
using namespace micro_os_plus::micro_test_plus::literals;
|
|
276
|
+
|
|
277
|
+
expect (compute_answer () == 42_i) << "answer is 42";
|
|
278
|
+
expect (_i {compute_answer ()} == 42) << "answer is 42";
|
|
279
|
+
});
|
|
280
|
+
|
|
281
|
+
return exit_code ();
|
|
282
|
+
}
|
|
173
283
|
```
|
|
174
284
|
|
|
175
|
-
The result
|
|
176
|
-
dependency with:
|
|
285
|
+
The result would look like:
|
|
177
286
|
|
|
178
|
-
```
|
|
179
|
-
|
|
180
|
-
your-target,
|
|
287
|
+
```console
|
|
288
|
+
The Answer - test suite started
|
|
181
289
|
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
290
|
+
• Check answer with comparator - test case started
|
|
291
|
+
✗ answer is 42 FAILED (answer.cpp:17, 43 == 42)
|
|
292
|
+
✗ Check answer with comparator - test case FAILED (0 checks passed, 1 check failed)
|
|
293
|
+
|
|
294
|
+
• Check answer with operator - test case started
|
|
295
|
+
✗ answer is 42 FAILED (answer.cpp:24, 43 == 42)
|
|
296
|
+
✗ answer is 42 FAILED (answer.cpp:25, 43 == 42)
|
|
297
|
+
✗ Check answer with operator - test case FAILED (0 checks passed, 1 check failed)
|
|
298
|
+
|
|
299
|
+
The Answer - test suite FAILED (0 checks passed, 3 checks failed, in 2 test cases)
|
|
187
300
|
```
|
|
188
301
|
|
|
189
|
-
|
|
302
|
+
In the first case, `eq()` is a function that basically compares almost
|
|
303
|
+
everything and is able to keep track of its operands. There are similar
|
|
304
|
+
functions for all comparisons.
|
|
305
|
+
|
|
306
|
+
In the second case, a custom operator is used. To avoid interferences
|
|
307
|
+
with other operators, it is defined in a separate namespace (which must
|
|
308
|
+
be explicitly refered, as shown) and matches only some specific types.
|
|
309
|
+
|
|
310
|
+
To cast the integer constant `42` to this specific type, a custom literal
|
|
311
|
+
is available (`_i`), which is also defined in a separate namespace.
|
|
312
|
+
|
|
313
|
+
In addition to literals used to define constants, there are also definitions
|
|
314
|
+
which can be used to cast expressions.
|
|
190
315
|
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
316
|
+
For the operator to match, it is necessary for at least one of the operands
|
|
317
|
+
to be of the specific type, usually the constant using a literal, but if both
|
|
318
|
+
are expression, at least one of them must be casted.
|
|
319
|
+
|
|
320
|
+
### C++ API
|
|
321
|
+
|
|
322
|
+
Aiming simplicity, µTest++ provides only a very limited number of primitives
|
|
323
|
+
used to check expectations and assumptions.
|
|
324
|
+
|
|
325
|
+
#### Expectations
|
|
326
|
+
|
|
327
|
+
Expectations are checks whose results are counted and do not
|
|
328
|
+
break the test (as opposed to assumptions, which abort the test).
|
|
329
|
+
|
|
330
|
+
```C++
|
|
331
|
+
template <class Expr_T, type_traits::requires_t<....>>
|
|
332
|
+
bool expect(const Expr_T& expr);
|
|
333
|
+
```
|
|
334
|
+
|
|
335
|
+
The template matches only expressions that evaluate to
|
|
336
|
+
a boolean or use custom comparators/operators derived from an
|
|
337
|
+
internal `detail::op` type.
|
|
338
|
+
|
|
339
|
+
For generic checks performed outside the testing framework, the results can
|
|
340
|
+
be reported with `expect(true)` or `expect(false)`.
|
|
341
|
+
|
|
342
|
+
#### Assumptions
|
|
343
|
+
|
|
344
|
+
Assumptions are checks that abort the test if the results are false.
|
|
345
|
+
|
|
346
|
+
```C++
|
|
347
|
+
template <class Expr_T, type_traits::requires_t<....>>
|
|
348
|
+
bool assume(const Expr_T& expr);
|
|
349
|
+
```
|
|
350
|
+
|
|
351
|
+
Similarly, the template matches only expressions that evaluate to
|
|
352
|
+
a boolean or use custom comparators/operators derived from a
|
|
353
|
+
internal `detail::op` type.
|
|
354
|
+
|
|
355
|
+
#### Function comparators
|
|
356
|
+
|
|
357
|
+
Expectations and assumptions may use any expression evaluating to a
|
|
358
|
+
boolean value, but in order to nicely report the difference between expected
|
|
359
|
+
and actual values in failed
|
|
360
|
+
conditions, the following generic comparators are available:
|
|
194
361
|
|
|
195
362
|
```c++
|
|
196
|
-
|
|
363
|
+
template <class Lhs_T, class Rhs_T>
|
|
364
|
+
auto eq(const Lhs_T& lhs, const Rhs_T& rhs);
|
|
197
365
|
|
|
198
|
-
|
|
366
|
+
template <class Lhs_T, class Rhs_T>
|
|
367
|
+
auto ne(const Lhs_T& lhs, const Rhs_T& rhs);
|
|
199
368
|
|
|
200
|
-
|
|
369
|
+
template <class Lhs_T, class Rhs_T>
|
|
370
|
+
auto lt(const Lhs_T& lhs, const Rhs_T& rhs);
|
|
201
371
|
|
|
202
|
-
|
|
203
|
-
|
|
204
|
-
test_case_something (micro_test_plus::session& t);
|
|
372
|
+
template <class Lhs_T, class Rhs_T>
|
|
373
|
+
auto le(const Lhs_T& lhs, const Rhs_T& rhs);
|
|
205
374
|
|
|
206
|
-
|
|
207
|
-
|
|
375
|
+
template <class Lhs_T, class Rhs_T>
|
|
376
|
+
auto gt(const Lhs_T& lhs, const Rhs_T& rhs);
|
|
208
377
|
|
|
209
|
-
|
|
378
|
+
template <class Lhs_T, class Rhs_T>
|
|
379
|
+
auto ge(const Lhs_T& lhs, const Rhs_T& rhs);
|
|
380
|
+
```
|
|
210
381
|
|
|
211
|
-
|
|
212
|
-
test_case_exception_thrown (micro_test_plus::session& t);
|
|
382
|
+
Similar templates are defined for pointer comparators.
|
|
213
383
|
|
|
214
|
-
|
|
215
|
-
test_case_exception_not_thrown (micro_test_plus::session& t);
|
|
384
|
+
Examples:
|
|
216
385
|
|
|
217
|
-
|
|
386
|
+
```c++
|
|
387
|
+
expect (eq (compute_answer (), 42)) << "answer is 42";
|
|
388
|
+
expect (ne (compute_answer (), 43)) << "answer is not 43";
|
|
389
|
+
expect (lt (compute_answer (), 43)) << "answer is < 43";
|
|
390
|
+
expect (le (compute_answer (), 43)) << "answer is <= 42";
|
|
391
|
+
expect (gt (compute_answer (), 41)) << "answer is > 43";
|
|
392
|
+
expect (ge (compute_answer (), 42)) << "answer is >= 42";
|
|
393
|
+
|
|
394
|
+
expect (compute_condition ()) << "condition is true";
|
|
395
|
+
```
|
|
218
396
|
|
|
219
|
-
|
|
220
|
-
|
|
397
|
+
When such comparator functions are used, failed checks also display the
|
|
398
|
+
actual values compared during the test; for example:
|
|
221
399
|
|
|
222
|
-
|
|
223
|
-
|
|
400
|
+
```console
|
|
401
|
+
Check failed comparisons
|
|
402
|
+
✗ actual != 42 FAILED (unit-test.cpp:286, 42 != 42)
|
|
403
|
+
✗ FAILED (unit-test.cpp:307, 42 != 42)
|
|
404
|
+
✗ 42 != 42_i FAILED (unit-test.cpp:310, 42 != 42)
|
|
405
|
+
✗ (actual == 42) and (actual != 42.0) FAILED (unit-test.cpp:781, (42 == 42 and 42.000000 != 42.000000))
|
|
406
|
+
```
|
|
224
407
|
|
|
225
|
-
|
|
226
|
-
compute_condition (void);
|
|
408
|
+
### Logical function operators
|
|
227
409
|
|
|
228
|
-
|
|
410
|
+
Complex expressions can be checked in a single line, using the logical
|
|
411
|
+
`_and()`, `_or()` and `_not()` functions. The names are prefixed since
|
|
412
|
+
`and`, `or` and `not` are reserved words in C/C++.
|
|
229
413
|
|
|
230
|
-
|
|
231
|
-
|
|
414
|
+
```c++
|
|
415
|
+
template <class Lhs_T, class Rhs_T>
|
|
416
|
+
auto _and (const Lhs_T& lhs, const Rhs_T& rhs);
|
|
232
417
|
|
|
233
|
-
|
|
418
|
+
template <class Lhs_T, class Rhs_T>
|
|
419
|
+
auto _or (const Lhs_T& lhs, const Rhs_T& rhs);
|
|
234
420
|
|
|
235
|
-
|
|
421
|
+
template <class Expr_T>
|
|
422
|
+
auto _not (const Expr_T& expr);
|
|
423
|
+
```
|
|
236
424
|
|
|
237
|
-
|
|
238
|
-
static char** g_argv;
|
|
425
|
+
Examples:
|
|
239
426
|
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
|
|
243
|
-
{
|
|
244
|
-
micro_test_plus::session t (argc, argv);
|
|
427
|
+
```c++
|
|
428
|
+
expect(_and (eq (compute_answer (), 42), eq (compute_float (), 42.0)));
|
|
429
|
+
```
|
|
245
430
|
|
|
246
|
-
|
|
247
|
-
|
|
431
|
+
A slightly more readable syntax is available with custom operators,
|
|
432
|
+
as shown below.
|
|
248
433
|
|
|
249
|
-
|
|
434
|
+
#### Comparing strings
|
|
250
435
|
|
|
251
|
-
|
|
436
|
+
In C/C++, plain strings are actually pointers to characters, and simply
|
|
437
|
+
comparing them does not compare the content but the memory addresses.
|
|
252
438
|
|
|
253
|
-
|
|
439
|
+
For string comparisons to compare the content, use `string_view`:
|
|
254
440
|
|
|
255
|
-
|
|
441
|
+
```c++
|
|
442
|
+
#include <string_view>
|
|
443
|
+
using namespace std::literals; // For the "sv" literal.
|
|
444
|
+
// ...
|
|
256
445
|
|
|
257
|
-
|
|
258
|
-
|
|
446
|
+
expect (eq (std::string_view{ compute_ultimate_answer () }, "fortytwo"sv))
|
|
447
|
+
<< "ultimate_answer is 'fortytwo'";
|
|
448
|
+
```
|
|
259
449
|
|
|
260
|
-
|
|
261
|
-
"Check if exceptions are not thrown");
|
|
450
|
+
#### Comparing containers
|
|
262
451
|
|
|
263
|
-
|
|
452
|
+
Containers can be compared for equality. The comparison
|
|
453
|
+
is done by iterating and comparing each member.
|
|
264
454
|
|
|
265
|
-
|
|
266
|
-
}
|
|
455
|
+
```c++
|
|
456
|
+
expect (eq (std::vector<int>{ 1, 2 }, std::vector<int>{ 1, 2 }))
|
|
457
|
+
<< "vector{ 1, 2 } eq vector{ 1, 2 }";
|
|
267
458
|
|
|
268
|
-
|
|
459
|
+
expect (ne (std::vector<int>{ 1, 2, 3 }, std::vector<int>{ 1, 2, 4 })
|
|
460
|
+
<< "vector{ 1, 2, 3 } ne vector{ 1, 2, 4 }";
|
|
461
|
+
```
|
|
269
462
|
|
|
270
|
-
|
|
271
|
-
int
|
|
272
|
-
compute_one (void)
|
|
273
|
-
{
|
|
274
|
-
return 1;
|
|
275
|
-
}
|
|
463
|
+
#### Operators
|
|
276
464
|
|
|
277
|
-
|
|
278
|
-
|
|
279
|
-
{
|
|
280
|
-
return "aaa";
|
|
281
|
-
}
|
|
465
|
+
As in most other C++ test frameworks, it is
|
|
466
|
+
also possible to overload the `==`, `!=`, `<`, `>`, `<=`, `>=` operators.
|
|
282
467
|
|
|
283
|
-
|
|
284
|
-
|
|
285
|
-
|
|
286
|
-
|
|
468
|
+
To avoid possible interferences with other operators
|
|
469
|
+
defined by the application, these operators match only for operands of
|
|
470
|
+
specific types and are located in a separate namespace
|
|
471
|
+
(`micro_test_plus::operators`); when applied to regular values, the
|
|
472
|
+
standard operands are used; the comparisons are performed properly,
|
|
473
|
+
but in case of failures the actual values are not shown.
|
|
474
|
+
|
|
475
|
+
The following operators match only operands derived from the local
|
|
476
|
+
`detail::op` type, which can be enforced for constant values by using the
|
|
477
|
+
provided literals (like `1_i`) or, for dynamic values, by using the
|
|
478
|
+
provided casts (like `_i {expression}`, which are actually the
|
|
479
|
+
constructors of the internal classes):
|
|
480
|
+
|
|
481
|
+
```c++
|
|
482
|
+
template <class Lhs_T, class Rhs_T, type_traits::requires_t<....>>
|
|
483
|
+
bool operator== (const Lhs_T& lhs, const Rhs_T& rhs);
|
|
484
|
+
|
|
485
|
+
template <class Lhs_T, class Rhs_T, type_traits::requires_t<....>>
|
|
486
|
+
bool operator!= (const Lhs_T& lhs, const Rhs_T& rhs);
|
|
487
|
+
|
|
488
|
+
template <class Lhs_T, class Rhs_T, type_traits::requires_t<....>>
|
|
489
|
+
bool operator< (const Lhs_T& lhs, const Rhs_T& rhs);
|
|
490
|
+
|
|
491
|
+
template <class Lhs_T, class Rhs_T, type_traits::requires_t<....>>
|
|
492
|
+
bool operator<= (const Lhs_T& lhs, const Rhs_T& rhs);
|
|
493
|
+
|
|
494
|
+
template <class Lhs_T, class Rhs_T, type_traits::requires_t<....>>
|
|
495
|
+
bool operator> (const Lhs_T& lhs, const Rhs_T& rhs);
|
|
496
|
+
|
|
497
|
+
template <class Lhs_T, class Rhs_T, type_traits::requires_t<....>>
|
|
498
|
+
bool operator>= (const Lhs_T& lhs, const Rhs_T& rhs);
|
|
499
|
+
```
|
|
500
|
+
|
|
501
|
+
Examples:
|
|
502
|
+
|
|
503
|
+
```c++
|
|
504
|
+
test_case ("Operators", [] {
|
|
505
|
+
using namespace micro_test_plus::operators;
|
|
506
|
+
using namespace micro_test_plus::literals;
|
|
507
|
+
|
|
508
|
+
expect (compute_answer () == 42_i) << "answer is 42 (with literal)";
|
|
509
|
+
expect (_i {compute_answer ()} == 42) << "answer is 42 (with cast)";
|
|
510
|
+
expect (compute_answer () != 43_i) << "answer is not 43";
|
|
511
|
+
expect (compute_answer () < 43_i) << "answer is < 43";
|
|
512
|
+
expect (compute_answer () <= 43_i) << "answer is <= 42";
|
|
513
|
+
expect (compute_answer () > 41_i) << "answer is > 43";
|
|
514
|
+
expect (compute_answer () >= 42_i) << "answer is >= 42";
|
|
515
|
+
});
|
|
516
|
+
```
|
|
517
|
+
|
|
518
|
+
In addition, equality operators are also provided for `string_view`
|
|
519
|
+
objects and for iterable containers:
|
|
520
|
+
|
|
521
|
+
```c++
|
|
522
|
+
bool operator== (std::string_view lhs, std::string_view rhs);
|
|
523
|
+
bool operator!= (std::string_view lhs, std::string_view rhs);
|
|
524
|
+
|
|
525
|
+
template <class T, type_traits::requires_t<type_traits::is_container_v<T>>>
|
|
526
|
+
bool operator== (T&& lhs, T&& rhs);
|
|
527
|
+
|
|
528
|
+
template <class T, type_traits::requires_t<type_traits::is_container_v<T>>>
|
|
529
|
+
bool operator!= (T&& lhs, T&& rhs);
|
|
530
|
+
```
|
|
531
|
+
|
|
532
|
+
Examples:
|
|
533
|
+
|
|
534
|
+
```c++
|
|
535
|
+
#include <string_view>
|
|
536
|
+
using namespace std::literals; // For the "sv" literal.
|
|
537
|
+
// ...
|
|
538
|
+
|
|
539
|
+
test_case ("Operators", [] {
|
|
540
|
+
using namespace micro_test_plus::operators;
|
|
541
|
+
|
|
542
|
+
expect (std::string_view{ compute_ultimate_answer () } == "fortytwo"sv)
|
|
543
|
+
<< "ultimate answer == 'fortytwo'";
|
|
544
|
+
|
|
545
|
+
expect (std::vector<int>{ 1, 2 } == std::vector<int>{ 1, 2 })
|
|
546
|
+
<< "vector{ 1, 2 } == vector{ 1, 2 }";
|
|
547
|
+
|
|
548
|
+
expect (std::vector<int>{ 1, 2, 3 } != std::vector<int>{ 1, 2, 4 })
|
|
549
|
+
<< "vector{ 1, 2, 3 } != vector{ 1, 2, 4 }";
|
|
550
|
+
});
|
|
551
|
+
```
|
|
552
|
+
|
|
553
|
+
### Logical operators
|
|
554
|
+
|
|
555
|
+
Similarly, logical operators are defined:
|
|
556
|
+
|
|
557
|
+
```c++
|
|
558
|
+
template <class Lhs_T, class Rhs_T, type_traits::requires_t<....>>
|
|
559
|
+
bool operator and (const Lhs_T& lhs, const Rhs_T& rhs);
|
|
560
|
+
|
|
561
|
+
template <class Lhs_T, class Rhs_T, type_traits::requires_t<....>>
|
|
562
|
+
bool operator or (const Lhs_T& lhs, const Rhs_T& rhs);
|
|
563
|
+
|
|
564
|
+
template <class T, type_traits::requires_t<....>>
|
|
565
|
+
bool operator not (const T& t);
|
|
566
|
+
```
|
|
567
|
+
|
|
568
|
+
They can be used in exactly the same way as standard operators, but the
|
|
569
|
+
additional functionality is enabled only when matching the typed operands.
|
|
570
|
+
|
|
571
|
+
Examples:
|
|
572
|
+
|
|
573
|
+
```c++
|
|
574
|
+
expect (compute_answer () == 42_i && compute_float () == 42.0_f);
|
|
575
|
+
```
|
|
576
|
+
|
|
577
|
+
#### Literals and wrappers
|
|
578
|
+
|
|
579
|
+
For converting constants to recognised typed operands, the following
|
|
580
|
+
literal operators are available in the separate namespace `literals`:
|
|
581
|
+
|
|
582
|
+
```c++
|
|
583
|
+
namespace literals {
|
|
584
|
+
auto operator""_i (); // int
|
|
585
|
+
auto operator""_s (); // short
|
|
586
|
+
auto operator""_c (); // char
|
|
587
|
+
auto operator""_sc () // signed char
|
|
588
|
+
auto operator""_l (); // long
|
|
589
|
+
auto operator""_ll (); // long long
|
|
590
|
+
auto operator""_u (); // unsigned
|
|
591
|
+
auto operator""_uc (); // unsigned char
|
|
592
|
+
auto operator""_us (); // unsigned short
|
|
593
|
+
auto operator""_ul (); // unsigned long
|
|
594
|
+
auto operator""_ull (); // unsigned long long
|
|
595
|
+
auto operator""_i8 (); // int8_t
|
|
596
|
+
auto operator""_i16 (); // int16_t
|
|
597
|
+
auto operator""_i32 (); // int32_t
|
|
598
|
+
auto operator""_i64 (); // int64_t
|
|
599
|
+
auto operator""_u8 (); // uint8_t
|
|
600
|
+
auto operator""_u16 (); // uint16_t
|
|
601
|
+
auto operator""_u32 (); // uint32_t
|
|
602
|
+
auto operator""_u64 (); // uint64_t
|
|
603
|
+
auto operator""_f (); // float
|
|
604
|
+
auto operator""_d (); // double
|
|
605
|
+
auto operator""_ld (); // long double
|
|
606
|
+
auto operator""_b (); // bool
|
|
287
607
|
}
|
|
608
|
+
```
|
|
288
609
|
|
|
289
|
-
|
|
610
|
+
Similarly, for dynamic values, there are wrappers that convert them to
|
|
611
|
+
recognised types:
|
|
290
612
|
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
|
|
294
|
-
|
|
613
|
+
```c++
|
|
614
|
+
using _b = type_traits::value<bool>;
|
|
615
|
+
using _c = type_traits::value<char>;
|
|
616
|
+
using _sc = type_traits::value<signed char>;
|
|
617
|
+
using _s = type_traits::value<short>;
|
|
618
|
+
using _i = type_traits::value<int>;
|
|
619
|
+
using _l = type_traits::value<long>;
|
|
620
|
+
using _ll = type_traits::value<long long>;
|
|
621
|
+
using _u = type_traits::value<unsigned>;
|
|
622
|
+
using _uc = type_traits::value<unsigned char>;
|
|
623
|
+
using _us = type_traits::value<unsigned short>;
|
|
624
|
+
using _ul = type_traits::value<unsigned long>;
|
|
625
|
+
using _ull = type_traits::value<unsigned long long>;
|
|
626
|
+
using _i8 = type_traits::value<std::int8_t>;
|
|
627
|
+
using _i16 = type_traits::value<std::int16_t>;
|
|
628
|
+
using _i32 = type_traits::value<std::int32_t>;
|
|
629
|
+
using _i64 = type_traits::value<std::int64_t>;
|
|
630
|
+
using _u8 = type_traits::value<std::uint8_t>;
|
|
631
|
+
using _u16 = type_traits::value<std::uint16_t>;
|
|
632
|
+
using _u32 = type_traits::value<std::uint32_t>;
|
|
633
|
+
using _u64 = type_traits::value<std::uint64_t>;
|
|
634
|
+
using _f = type_traits::value<float>;
|
|
635
|
+
using _d = type_traits::value<double>;
|
|
636
|
+
using _ld = type_traits::value<long double>;
|
|
637
|
+
|
|
638
|
+
// Template for wrapping any other type.
|
|
639
|
+
template <class T>
|
|
640
|
+
struct _t : type_traits::value<T>
|
|
641
|
+
{
|
|
642
|
+
constexpr explicit _t (const T& t) : type_traits::value<T>{ t }
|
|
295
643
|
{
|
|
296
|
-
throw "kaboom";
|
|
297
644
|
}
|
|
298
|
-
}
|
|
645
|
+
};
|
|
646
|
+
```
|
|
299
647
|
|
|
300
|
-
|
|
648
|
+
Examples:
|
|
301
649
|
|
|
302
|
-
|
|
650
|
+
```c++
|
|
651
|
+
expect (_i {answer} == 42_i);
|
|
652
|
+
expect (_f {expression} == 42_f);
|
|
653
|
+
```
|
|
654
|
+
|
|
655
|
+
#### Function comparators vs. operators & literals
|
|
656
|
+
|
|
657
|
+
A very good question is which to use, functions like `eq()` or the
|
|
658
|
+
overloaded operators.
|
|
659
|
+
|
|
660
|
+
Functions guarantee that the nice feature of showing the actual values
|
|
661
|
+
when expectations fail is always available. Also the syntax is more on the
|
|
662
|
+
traditional side, and for some it may look simpler and easier to read.
|
|
663
|
+
|
|
664
|
+
Operators are generally easier to recognise than function calls,
|
|
665
|
+
but require the hack with the type wrappers and literals to enforce the
|
|
666
|
+
types, otherwise the actual values will not be displayed when the
|
|
667
|
+
expectations fail.
|
|
668
|
+
|
|
669
|
+
Both syntaxes are functional, and, once understood the differences,
|
|
670
|
+
the issue is a matter of personal preferences.
|
|
671
|
+
|
|
672
|
+
#### Explicit namespace
|
|
303
673
|
|
|
304
|
-
|
|
305
|
-
|
|
306
|
-
|
|
674
|
+
If for any reasons, the definitions in the `micro_test_plus` namespace
|
|
675
|
+
interfere with application definitions, it is recommended to
|
|
676
|
+
use the comparator functions, which can be more easily invoked
|
|
677
|
+
with explicit namespaces, possibly aliased to shorter names.
|
|
678
|
+
|
|
679
|
+
Example:
|
|
680
|
+
|
|
681
|
+
```c++
|
|
307
682
|
{
|
|
308
|
-
|
|
309
|
-
|
|
310
|
-
|
|
683
|
+
namespace mt = micro_os_plus::micro_test_plus;
|
|
684
|
+
|
|
685
|
+
mt::test_case ("Check answer", [] {
|
|
686
|
+
mt::expect (mt::eq (compute_answer (), 42)) << "answer is 42";
|
|
687
|
+
});
|
|
688
|
+
}
|
|
689
|
+
```
|
|
690
|
+
|
|
691
|
+
#### Exceptions
|
|
692
|
+
|
|
693
|
+
Specific to C++, a testing framework must be able check if an expression
|
|
694
|
+
(usually a function call), throws or not an exception.
|
|
695
|
+
|
|
696
|
+
The following function templates allow to check for any exception,
|
|
697
|
+
for a specific exception, or for no exception at all:
|
|
698
|
+
|
|
699
|
+
```C++
|
|
700
|
+
// Check for any exception.
|
|
701
|
+
template <class Callable_T>
|
|
702
|
+
auto throws (const Callable_T& expr);
|
|
703
|
+
|
|
704
|
+
// Check for a specific exception.
|
|
705
|
+
template <class Exception_T, class Callable_T>
|
|
706
|
+
auto throws (const Callable_T& expr);
|
|
707
|
+
|
|
708
|
+
// Check for no exception at all.
|
|
709
|
+
template <class Callable_T>
|
|
710
|
+
auto nothrow (const Callable_T& expr);
|
|
711
|
+
```
|
|
712
|
+
|
|
713
|
+
Examples:
|
|
714
|
+
|
|
715
|
+
```c++
|
|
716
|
+
expect (throws ([] { exercise_throw (true); })) << "exception thrown";
|
|
717
|
+
|
|
718
|
+
expect (throws<std::runtime_error> ([] { throw std::runtime_error{ "" }; }))
|
|
719
|
+
<< "std::runtime_error thrown";
|
|
720
|
+
|
|
721
|
+
expect (nothrow ([] { exercise_throw (false); })) << "exception not thrown";
|
|
722
|
+
```
|
|
723
|
+
|
|
724
|
+
If a more elaborate logic is required, for example for expecting multiple
|
|
725
|
+
exceptions, use an explicit `try` with multiple `catch` statements and
|
|
726
|
+
report the results with `expect(true)` or `expect(false)`.
|
|
727
|
+
|
|
728
|
+
```c++
|
|
729
|
+
try
|
|
730
|
+
{
|
|
731
|
+
compute_answer ();
|
|
732
|
+
}
|
|
733
|
+
catch (const std::overflow_error& e)
|
|
734
|
+
{
|
|
735
|
+
expect (true) << "std::overflow_error thrown";
|
|
736
|
+
}
|
|
737
|
+
catch (const std::runtime_error& e)
|
|
738
|
+
{
|
|
739
|
+
expect (true) << "std::runtime_error thrown";
|
|
740
|
+
}
|
|
741
|
+
catch (...)
|
|
742
|
+
{
|
|
743
|
+
expect (false) << "known exception thrown";
|
|
744
|
+
}
|
|
745
|
+
```
|
|
746
|
+
|
|
747
|
+
#### Test cases
|
|
748
|
+
|
|
749
|
+
Test cases group several checks done in the same environment.
|
|
750
|
+
|
|
751
|
+
There can be any number of test cases, and each test case is performed
|
|
752
|
+
by invoking
|
|
753
|
+
a function, parametrised with a name, a callable (usually a lambda),
|
|
754
|
+
and optional arguments:
|
|
755
|
+
|
|
756
|
+
```C++
|
|
757
|
+
template <typename Callable_T, typename... Args_T>
|
|
758
|
+
void test_case (const char* name, Callable_T&& func, Args_T&&... arguments);
|
|
759
|
+
```
|
|
760
|
+
|
|
761
|
+
Examples:
|
|
762
|
+
|
|
763
|
+
```c++
|
|
764
|
+
using namespace micro_os_plus::micro_test_plus;
|
|
765
|
+
|
|
766
|
+
test_case ("Check various conditions", [] {
|
|
767
|
+
expect (eq (compute_answer (), 42)) << "answer eq 42";
|
|
768
|
+
expect (ne (compute_answer (), 43)) << "answer ne 43";
|
|
769
|
+
});
|
|
770
|
+
|
|
771
|
+
test_case ("Check various conditions with operators", [] {
|
|
772
|
+
using namespace micro_os_plus::micro_test_plus::operators;
|
|
773
|
+
using namespace micro_os_plus::micro_test_plus::literals;
|
|
774
|
+
|
|
775
|
+
expect (compute_answer () == 42_i) << "answer == 42";
|
|
776
|
+
expect (compute_answer () != 43_i) << "answer != 43";
|
|
777
|
+
});
|
|
778
|
+
```
|
|
779
|
+
|
|
780
|
+
#### Test runner initialization
|
|
781
|
+
|
|
782
|
+
The test runner is initialised with the process arguments and a
|
|
783
|
+
name, which is used for the default test suite:
|
|
784
|
+
|
|
785
|
+
```C++
|
|
786
|
+
void initialize (int argc, char* argv[], const char* name = "Main");
|
|
787
|
+
```
|
|
788
|
+
|
|
789
|
+
The arguments are used for controlling the verbosity level.
|
|
790
|
+
|
|
791
|
+
#### Return the test result
|
|
792
|
+
|
|
793
|
+
The final test result that must be returned to the system
|
|
794
|
+
(0 for pass, 1 for fail), can be obtained with:
|
|
795
|
+
|
|
796
|
+
```C++
|
|
797
|
+
int exit_code (void);
|
|
798
|
+
```
|
|
799
|
+
|
|
800
|
+
This call also triggers the execution of all global test suites.
|
|
801
|
+
|
|
802
|
+
For examples, see before.
|
|
803
|
+
|
|
804
|
+
#### Test suites
|
|
805
|
+
|
|
806
|
+
Test suites are named sequences of test cases.
|
|
807
|
+
|
|
808
|
+
The test cases defined in `main()` are considered to be part of
|
|
809
|
+
the default (or main) test suite, and are executed immediately
|
|
810
|
+
when invoked.
|
|
811
|
+
|
|
812
|
+
For complex applications there can be multiple test
|
|
813
|
+
suites, usually in separate source files.
|
|
814
|
+
|
|
815
|
+
In order to make self-registration possible, test suites are classes,
|
|
816
|
+
constructed with a name and a callable (usually a lambda),
|
|
817
|
+
which chains the execution of the test cases:
|
|
311
818
|
|
|
312
|
-
|
|
313
|
-
|
|
819
|
+
```C++
|
|
820
|
+
template <typename Callable_T, typename... Args_T>
|
|
821
|
+
class test_suite : public test_suite_base
|
|
822
|
+
{
|
|
823
|
+
public:
|
|
824
|
+
test_suite (const char* name, Callable_T&& callable,
|
|
825
|
+
Args_T&&... arguments);
|
|
826
|
+
// ...
|
|
827
|
+
}
|
|
828
|
+
```
|
|
829
|
+
|
|
830
|
+
It is recommended to instantiate the test suites as static objects.
|
|
831
|
+
The self-registration is done in the constructor.
|
|
832
|
+
Test suites defined in different compilation units can be executed in any
|
|
833
|
+
order (since the order in which the
|
|
834
|
+
static constructors are invoked is not specified);
|
|
835
|
+
thus there should be no dependencies between test suites.
|
|
836
|
+
|
|
837
|
+
Test suites are executed when the function `exit_code()` is invoked.
|
|
838
|
+
|
|
839
|
+
Examples:
|
|
840
|
+
|
|
841
|
+
```c++
|
|
842
|
+
using namespace micro_os_plus::micro_test_plus;
|
|
843
|
+
|
|
844
|
+
static test_suite ts_1
|
|
845
|
+
= { "Separate", [] {
|
|
314
846
|
|
|
315
|
-
|
|
316
|
-
|
|
847
|
+
test_case ("Check one", [] {
|
|
848
|
+
expect (true) << "Passed";
|
|
849
|
+
});
|
|
850
|
+
|
|
851
|
+
test_case ("Check two", [] {
|
|
852
|
+
expect (true) << "Passed";
|
|
853
|
+
});
|
|
854
|
+
}};
|
|
855
|
+
```
|
|
856
|
+
|
|
857
|
+
#### Utility functions
|
|
858
|
+
|
|
859
|
+
For tests comparing strings, in addition to exact matches, it is also possible
|
|
860
|
+
to check matches with patterns like `*` (for any characters) and `?` (for a
|
|
861
|
+
single character):
|
|
862
|
+
|
|
863
|
+
```c++
|
|
864
|
+
namespace utility {
|
|
865
|
+
bool is_match (std::string_view input, std::string_view pattern);
|
|
317
866
|
}
|
|
867
|
+
```
|
|
318
868
|
|
|
319
|
-
|
|
320
|
-
test_case_args (micro_test_plus::session& t)
|
|
321
|
-
{
|
|
322
|
-
MTP_EXPECT_EQ (t, g_argc, 3, "argc == 3");
|
|
869
|
+
Examples:
|
|
323
870
|
|
|
324
|
-
|
|
325
|
-
|
|
326
|
-
|
|
327
|
-
|
|
871
|
+
```c++
|
|
872
|
+
expect (utility::is_match ("abc", "a?c")) << "abc matches a?c";
|
|
873
|
+
expect (utility::is_match ("abc", "a*c")) << "abc matches a*c";
|
|
874
|
+
```
|
|
328
875
|
|
|
329
|
-
|
|
330
|
-
|
|
331
|
-
|
|
332
|
-
|
|
876
|
+
Also for tests handling strings, the following function template allows to
|
|
877
|
+
split a string into a vector of substrings, using a delimiter:
|
|
878
|
+
|
|
879
|
+
```c++
|
|
880
|
+
namespace utility {
|
|
881
|
+
template <class T, class Delim_T>
|
|
882
|
+
auto split (T input, Delim_T delim) -> std::vector<T>;
|
|
333
883
|
}
|
|
884
|
+
```
|
|
885
|
+
|
|
886
|
+
Example:
|
|
887
|
+
|
|
888
|
+
```c++
|
|
889
|
+
expect (std::vector<std::string_view>{ "a", "b" }
|
|
890
|
+
== utility::split<std::string_view> ("a.b", "."))
|
|
891
|
+
<< "a.b splits into [a,b]";
|
|
892
|
+
```
|
|
893
|
+
|
|
894
|
+
#### Custom types
|
|
895
|
+
|
|
896
|
+
It is possible to extend the comparators with templates matching custom
|
|
897
|
+
types, but this is a non-trivial task and requires a good knowledge of
|
|
898
|
+
C++.
|
|
899
|
+
|
|
900
|
+
TODO: add a test to show how to do this.
|
|
901
|
+
|
|
902
|
+
### Command line options
|
|
903
|
+
|
|
904
|
+
#### Verbosity
|
|
905
|
+
|
|
906
|
+
By default, the test reporter shows detailed results only for test cases
|
|
907
|
+
that failed; successful test cases are shown as a single line with
|
|
908
|
+
the total counts of passed/failed checks.
|
|
909
|
+
|
|
910
|
+
To control the verbosity use one of the following command line options:
|
|
911
|
+
|
|
912
|
+
- `--verbose`: show all expectations, regardless of the result
|
|
913
|
+
- `--quiet`: show only the test suite totals
|
|
914
|
+
- `--silent`: suppress all output and only return the exit code
|
|
915
|
+
|
|
916
|
+
### Memory footprint
|
|
917
|
+
|
|
918
|
+
The memory footprint of unit tests based on µTest++ is definitely smaller than
|
|
919
|
+
that of traditional C++ testing framework, since the `iostream` library is not
|
|
920
|
+
used.
|
|
921
|
+
|
|
922
|
+
However, the use of templates for implementing the comparators and
|
|
923
|
+
operators should be carefully observed for platforms with really
|
|
924
|
+
limited amounts of memory, since each pair of different operands
|
|
925
|
+
contributes to the program size.
|
|
926
|
+
|
|
927
|
+
At the limit, µTest++ can be used only with regular boolean expressions,
|
|
928
|
+
without custom comparators and operators, and still be able to provide
|
|
929
|
+
the basic functionality of testing various conditions, but without
|
|
930
|
+
the optional features of displaying the actual values compared.
|
|
931
|
+
|
|
932
|
+
Also, please note that the memory footprint on `debug`, built with `-O0`,
|
|
933
|
+
is significantly larger than on `release`. If necessary, the optimization
|
|
934
|
+
for the `debug` build can be increased to `-Og`, and save some memory.
|
|
935
|
+
|
|
936
|
+
### Build & integration info
|
|
937
|
+
|
|
938
|
+
The project is written in C++, and the tests are expected to be
|
|
939
|
+
written in C++ too, but the tested code can also be written in plain C.
|
|
940
|
+
The framework source code was compiled with GCC 11, clang 12
|
|
941
|
+
and arm-none-eabi-gcc 10, and should be warning free.
|
|
942
|
+
|
|
943
|
+
To run on embedded platforms, the test framework requires a minimum
|
|
944
|
+
of support from the system, like writing to the
|
|
945
|
+
output stream. Any such environments are acceptable, but for standalone
|
|
946
|
+
tests the most common solution is to use **Arm semihosting**.
|
|
947
|
+
|
|
948
|
+
To ease the integration of this package into user projects, there
|
|
949
|
+
are already made CMake and meson configuration files (see below).
|
|
950
|
+
|
|
951
|
+
For other build systems, consider the following details:
|
|
952
|
+
|
|
953
|
+
#### Include folders
|
|
954
|
+
|
|
955
|
+
The following folder should be used during the build:
|
|
956
|
+
|
|
957
|
+
- `include`
|
|
958
|
+
|
|
959
|
+
The header file to be included is:
|
|
960
|
+
|
|
961
|
+
```c++
|
|
962
|
+
#include <micro-os-plus/micro-test-plus.h>
|
|
963
|
+
```
|
|
964
|
+
|
|
965
|
+
#### Source folders
|
|
966
|
+
|
|
967
|
+
The source files to be added are:
|
|
968
|
+
|
|
969
|
+
- `src/micro-test-plus.cpp`
|
|
970
|
+
- `src/test-reporter.cpp`
|
|
971
|
+
- `src/test-runner.cpp`
|
|
972
|
+
- `src/test-suite.cpp`
|
|
973
|
+
|
|
974
|
+
#### Preprocessor definitions
|
|
975
|
+
|
|
976
|
+
- `MICRO_OS_PLUS_TRACE` - to include the trace
|
|
977
|
+
- `MICRO_TEST_PLUS_TRACE` to enable some tracing messages
|
|
978
|
+
|
|
979
|
+
#### Compiler options
|
|
980
|
+
|
|
981
|
+
- `-std=c++20` or higher for C++ sources
|
|
982
|
+
|
|
983
|
+
#### C++ Namespaces
|
|
984
|
+
|
|
985
|
+
- `micro_os_plus::micro_test_plus`
|
|
986
|
+
- `micro_os_plus::micro_test_plus::operators`
|
|
987
|
+
- `micro_os_plus::micro_test_plus::literals`
|
|
988
|
+
- `micro_os_plus::micro_test_plus::utility`
|
|
989
|
+
|
|
990
|
+
`micro_os_plus` is the top µOS++ namespace, and `micro_test_plus` is the
|
|
991
|
+
µTest++ namespace.
|
|
992
|
+
|
|
993
|
+
The `operators` namespace defines the custom operators, and the `literals`
|
|
994
|
+
namespace defines the literals (like `1_i`);
|
|
995
|
+
|
|
996
|
+
#### C++ Classes
|
|
997
|
+
|
|
998
|
+
- `micro_os_plus::micro_test_plus::test_suite`
|
|
999
|
+
|
|
1000
|
+
#### Dependencies
|
|
1001
|
+
|
|
1002
|
+
- none
|
|
1003
|
+
|
|
1004
|
+
#### CMake
|
|
1005
|
+
|
|
1006
|
+
To integrate the µTest++ source library into a CMake application, add this
|
|
1007
|
+
folder to the build:
|
|
1008
|
+
|
|
1009
|
+
```cmake
|
|
1010
|
+
add_subdirectory("xpacks/micro-os-plus-micro-test-plus")`
|
|
1011
|
+
```
|
|
1012
|
+
|
|
1013
|
+
The result is an interface library that can be added as an application
|
|
1014
|
+
dependency with:
|
|
1015
|
+
|
|
1016
|
+
```cmake
|
|
1017
|
+
target_link_libraries(your-target PRIVATE
|
|
1018
|
+
|
|
1019
|
+
micro-os-plus::micro-test-plus
|
|
1020
|
+
)
|
|
1021
|
+
```
|
|
1022
|
+
|
|
1023
|
+
#### meson
|
|
1024
|
+
|
|
1025
|
+
To integrate the µTest++ source library into a meson application, add this
|
|
1026
|
+
folder to the build:
|
|
1027
|
+
|
|
1028
|
+
```meson
|
|
1029
|
+
subdir('xpacks/micro-os-plus-micro-test-plus')
|
|
1030
|
+
```
|
|
1031
|
+
|
|
1032
|
+
The result is a dependency object that can be added
|
|
1033
|
+
to an application with:
|
|
1034
|
+
|
|
1035
|
+
```meson
|
|
1036
|
+
exe = executable(
|
|
1037
|
+
your-target,
|
|
1038
|
+
link_with: [
|
|
1039
|
+
# Nothing, not static.
|
|
1040
|
+
],
|
|
1041
|
+
dependencies: [
|
|
1042
|
+
micro_os_plus_micro_test_plus_dependency,
|
|
1043
|
+
]
|
|
1044
|
+
)
|
|
1045
|
+
```
|
|
1046
|
+
|
|
1047
|
+
### Examples
|
|
1048
|
+
|
|
1049
|
+
An example showing how to use the µTest++ framework is
|
|
1050
|
+
available in
|
|
1051
|
+
[tests/src/sample-test.cpp](tests/src/sample-test.cpp).
|
|
1052
|
+
|
|
1053
|
+
Here are some excerpts:
|
|
1054
|
+
|
|
1055
|
+
```c++
|
|
1056
|
+
#include <micro-os-plus/micro-test-plus.h>
|
|
334
1057
|
|
|
335
1058
|
// ----------------------------------------------------------------------------
|
|
336
1059
|
|
|
337
|
-
|
|
1060
|
+
// ...
|
|
338
1061
|
|
|
339
|
-
//
|
|
340
|
-
|
|
341
|
-
|
|
1062
|
+
// The test suite.
|
|
1063
|
+
int
|
|
1064
|
+
main (int argc, char* argv[])
|
|
342
1065
|
{
|
|
343
|
-
|
|
344
|
-
{
|
|
345
|
-
// Do something that throws.
|
|
346
|
-
exercise_throw (true);
|
|
1066
|
+
using namespace micro_os_plus::micro_test_plus;
|
|
347
1067
|
|
|
348
|
-
|
|
349
|
-
MTP_FAIL (t, "exception not thrown");
|
|
350
|
-
}
|
|
351
|
-
catch (...)
|
|
352
|
-
{
|
|
353
|
-
// Got it.
|
|
354
|
-
MTP_PASS (t, "exception thrown");
|
|
355
|
-
}
|
|
356
|
-
}
|
|
1068
|
+
initialize (argc, argv, "Sample");
|
|
357
1069
|
|
|
358
|
-
|
|
359
|
-
|
|
360
|
-
|
|
361
|
-
|
|
362
|
-
|
|
363
|
-
// Do something that may throw, but it doesn't.
|
|
364
|
-
exercise_throw (false);
|
|
1070
|
+
test_case ("Check various conditions", [] {
|
|
1071
|
+
expect (eq (compute_answer (), 42)) << "answer is 42";
|
|
1072
|
+
expect (ne (compute_answer (), 43)) << "answer is not 43";
|
|
1073
|
+
expect (compute_condition ()) << "condition() is true";
|
|
1074
|
+
});
|
|
365
1075
|
|
|
366
|
-
|
|
367
|
-
|
|
368
|
-
|
|
369
|
-
|
|
370
|
-
|
|
371
|
-
|
|
372
|
-
|
|
373
|
-
}
|
|
1076
|
+
test_case ("Check various conditions with operators", [] {
|
|
1077
|
+
using namespace micro_test_plus::operators;
|
|
1078
|
+
using namespace micro_test_plus::literals;
|
|
1079
|
+
|
|
1080
|
+
expect (compute_answer () == 42_i) << "answer == 42 (with literal)";
|
|
1081
|
+
expect (_i {compute_answer ()} == 42) << "answer == 42 (with cast)";
|
|
1082
|
+
expect (compute_answer () != 43_i) << "answer != 43";
|
|
1083
|
+
});
|
|
1084
|
+
|
|
1085
|
+
test_case ("Check parameterised", [] {
|
|
1086
|
+
auto f = [] (int i) { return i + 42; };
|
|
1087
|
+
expect (eq (f (1), 43)) << "lambda == 43";
|
|
1088
|
+
});
|
|
1089
|
+
|
|
1090
|
+
#if defined(__EXCEPTIONS)
|
|
1091
|
+
|
|
1092
|
+
test_case ("Check exceptions", [] {
|
|
1093
|
+
auto exercise_throw = [] { throw std::runtime_error{ "" }; }
|
|
1094
|
+
expect (throws<std::runtime_error> (exercise_throw))
|
|
1095
|
+
<< "std::runtime_error thrown";
|
|
1096
|
+
});
|
|
374
1097
|
|
|
375
1098
|
#endif // defined(__EXCEPTIONS)
|
|
376
1099
|
|
|
1100
|
+
return exit_code ();
|
|
1101
|
+
}
|
|
1102
|
+
|
|
1103
|
+
// ----------------------------------------------------------------------------
|
|
377
1104
|
```
|
|
378
1105
|
|
|
379
1106
|
The output of running such a test looks like:
|
|
@@ -386,67 +1113,89 @@ $ xpm run test-native
|
|
|
386
1113
|
...
|
|
387
1114
|
> Executing task: xpm run test --config native-cmake-debug <
|
|
388
1115
|
|
|
389
|
-
> cd build/native-cmake-
|
|
390
|
-
|
|
391
|
-
|
|
392
|
-
|
|
393
|
-
|
|
394
|
-
|
|
395
|
-
|
|
396
|
-
|
|
397
|
-
|
|
398
|
-
|
|
399
|
-
test
|
|
400
|
-
|
|
401
|
-
|
|
402
|
-
1:
|
|
1116
|
+
> cd build/native-cmake-release && ctest -V
|
|
1117
|
+
...
|
|
1118
|
+
Start 1: sample-test
|
|
1119
|
+
|
|
1120
|
+
1: Test command: /Users/ilg/My\ Files/WKS\ Projects/micro-os-plus.github/xPacks/micro-test-plus-xpack.git/build/native-cmake-release/platform-bin/sample-test "one" "two"
|
|
1121
|
+
1: Test timeout computed to be: 10000000
|
|
1122
|
+
1: Built with clang Apple LLVM 13.0.0 (clang-1300.0.29.30), no FP, with exceptions.
|
|
1123
|
+
1:
|
|
1124
|
+
1: Sample - test suite started
|
|
1125
|
+
1:
|
|
1126
|
+
1: • Check various conditions - test case started
|
|
1127
|
+
1: ✓ compute_one() == 1
|
|
1128
|
+
1: ✓ compute_aaa() == 'aaa'
|
|
1129
|
+
1: ✓ condition() is true
|
|
1130
|
+
1: ✓ Check various conditions - test case passed (3 checks)
|
|
1131
|
+
1:
|
|
1132
|
+
1: • Check parameterised - test case started
|
|
1133
|
+
1: ✓ lambda == 43
|
|
1134
|
+
1: ✓ Check parameterised - test case passed (1 check)
|
|
1135
|
+
1:
|
|
1136
|
+
1: • Check exceptions - test case started
|
|
1137
|
+
1: ✓ std::runtime_error thrown
|
|
1138
|
+
1: ✓ Check exceptions - test case passed (1 check)
|
|
1139
|
+
1:
|
|
1140
|
+
1: Sample - test suite passed (5 tests in 3 test cases)
|
|
1141
|
+
1/2 Test #1: sample-test ...................... Passed 0.00 sec
|
|
403
1142
|
...
|
|
404
|
-
1/2 Test #1: unit-test ........................ Passed 0.00 sec
|
|
405
|
-
test 2
|
|
406
|
-
Start 2: sample-test
|
|
407
|
-
|
|
408
|
-
2: Test command: /Users/ilg/My\ Files/WKS\ Projects/micro-os-plus.github/xPacks/micro-test-plus-xpack.git/build/native-cmake-debug/platform/sample-test "one" "two"
|
|
409
|
-
2: Test timeout computed to be: 10000000
|
|
410
|
-
2: Built with clang Apple LLVM 13.0.0 (clang-1300.0.29.30), no FP, with exceptions, with DEBUG.
|
|
411
|
-
2: argv[] = '/Users/ilg/My Files/WKS Projects/micro-os-plus.github/xPacks/micro-test-plus-xpack.git/build/native-cmake-debug/platform/sample-test' 'one' 'two'
|
|
412
|
-
2:
|
|
413
|
-
2: Sample test started
|
|
414
|
-
2:
|
|
415
|
-
2: Check various conditions
|
|
416
|
-
2: ✓ compute_one() == 1
|
|
417
|
-
2: ✓ compute_aaa() == 'aaa'
|
|
418
|
-
2: ✓ condition() is true
|
|
419
|
-
2:
|
|
420
|
-
2: Check args
|
|
421
|
-
2: ✓ argc == 3
|
|
422
|
-
2: ✓ argv[1] == 'one'
|
|
423
|
-
2: ✓ argv[2] == 'two'
|
|
424
|
-
2:
|
|
425
|
-
2: Check if exceptions are thrown
|
|
426
|
-
2: ✓ exception thrown
|
|
427
|
-
2:
|
|
428
|
-
2: Check if exceptions are not thrown
|
|
429
|
-
2: ✓ exception not thrown
|
|
430
|
-
2:
|
|
431
|
-
2: Sample test passed (8 tests in 4 test cases)
|
|
432
|
-
2/2 Test #2: sample-test ...................... Passed 0.00 sec
|
|
433
|
-
|
|
434
|
-
100% tests passed, 0 tests failed out of 2
|
|
435
|
-
|
|
436
|
-
Total Test time (real) = 0.01 sec
|
|
437
1143
|
```
|
|
438
1144
|
|
|
439
1145
|
### Known problems
|
|
440
1146
|
|
|
441
1147
|
- none
|
|
442
1148
|
|
|
1149
|
+
### TODO
|
|
1150
|
+
|
|
1151
|
+
- add code to show how to define custom comparators
|
|
1152
|
+
- move documentation to future µOS++ web site
|
|
1153
|
+
|
|
443
1154
|
### Tests
|
|
444
1155
|
|
|
445
1156
|
The project is fully tested via GitHub
|
|
446
1157
|
[Actions](https://github.com/micro-os-plus/micro-test-plus-xpack/actions/)
|
|
447
1158
|
on each push.
|
|
448
|
-
|
|
449
|
-
|
|
1159
|
+
|
|
1160
|
+
The test platforms are GNU/Linux, macOS and Windows; native tests are
|
|
1161
|
+
compiled with GCC and clang; tests for embedded platforms are compiled
|
|
1162
|
+
with arm-none-eabi-gcc and run via QEMU.
|
|
1163
|
+
|
|
1164
|
+
There are two set of tests, one that runs on every push, with a
|
|
1165
|
+
limited number of tests, and a set that is triggered manually,
|
|
1166
|
+
usually before releases, and runs all tests on all supported
|
|
1167
|
+
platforms.
|
|
1168
|
+
|
|
1169
|
+
The full set can be run manually with the following commands:
|
|
1170
|
+
|
|
1171
|
+
```sh
|
|
1172
|
+
cd ~Work/micro-test-plus-xpack.git
|
|
1173
|
+
|
|
1174
|
+
xpm run install-all
|
|
1175
|
+
xpm run test-all
|
|
1176
|
+
```
|
|
1177
|
+
|
|
1178
|
+
## Change log - incompatible changes
|
|
1179
|
+
|
|
1180
|
+
According to [semver](https://semver.org) rules:
|
|
1181
|
+
|
|
1182
|
+
> Major version X (X.y.z | X > 0) MUST be incremented if any
|
|
1183
|
+
backwards incompatible changes are introduced to the public API.
|
|
1184
|
+
|
|
1185
|
+
The incompatible changes, in reverse chronological order,
|
|
1186
|
+
are:
|
|
1187
|
+
|
|
1188
|
+
- v3.x: major rework, with full set of comparators, exceptions,
|
|
1189
|
+
function templates for test cases and class templates for test suites;
|
|
1190
|
+
- v2.3.x: deprecate `run_test_case(func, name)` in favour o
|
|
1191
|
+
`run_test_case(name, func)`, to prepare for variadic templates
|
|
1192
|
+
- v2.x: the C++ namespace was renamed from `os` to `micro_os_plus`;
|
|
1193
|
+
- v1.x: the code was extracted from the mono-repo µOS++ project.
|
|
1194
|
+
|
|
1195
|
+
## Credits
|
|
1196
|
+
|
|
1197
|
+
Many thanks to the [Boost UT](https://github.com/boost-ext/ut) project
|
|
1198
|
+
for the inspiration and for major parts of the code.
|
|
450
1199
|
|
|
451
1200
|
## License
|
|
452
1201
|
|
|
@@ -454,3 +1203,6 @@ The original content is released under the
|
|
|
454
1203
|
[MIT License](https://opensource.org/licenses/MIT/),
|
|
455
1204
|
with all rights reserved to
|
|
456
1205
|
[Liviu Ionescu](https://github.com/ilg-ul/).
|
|
1206
|
+
|
|
1207
|
+
The code from Boost UT is released under the terms of the
|
|
1208
|
+
[Boost Version 1.0 Software License](https://www.boost.org/LICENSE_1_0.txt).
|