ceedling 1.0.1 → 1.1.1

data/docs/mkdocs/plugins/gcov/troubleshooting.md

@@ -0,0 +1,78 @@
+# Advanced & Troubleshooting
+
+## Advanced usage
+
+Details of interest for this plugin to be modified or made use of using 
+Ceedling’s advanced features are primarily contained in 
+[defaults_gcov.rb](../../snapshot/plugins/gcov/config/defaults_gcov.rb) and [defaults.yml](../../snapshot/plugins/gcov/config/defaults.yml).
+
+## "gcovr not found"
+
+`gcovr` is a Python-based application. Depending on the particulars of its 
+installation and your platform, you may encounter a "gcovr not found" error. 
+This is usually related to complications of running a Python script as an 
+executable.
+
+### Check your `PATH`
+
+The problem may be as simple to solve as ensuring your user or system path 
+include the path to `python` and/or the `gcovr` script. `gcovr` may be 
+successfully installed and findable by Python; this does not necessarily 
+mean that shell commands Ceedling spawns can find these tools.
+
+Options:
+
+1. Modify your user or system path to include your Python installation, `gcovr`
+   location, or both.
+1. Use Ceedling’s `:environment` project configuration with its special 
+   handling of `PATH` to modify the search path Ceedling accesses when it 
+   executes shell commands. xample below.
+
+```yaml
+:environment:
+  - :path:               # Concatenates the following with OS-specific path separator             
+     - <path to add>     # Add Python and/or `gcovr` path
+     - "#{ENV['PATH']}"  # Fetch existing path entries
+```
+
+### Redefine `gcovr` to call Python directly
+
+Another solution is simple in concept. Instead of calling `gcovr` directly, call 
+`python` with the `gcovr` script as a command line argument (followed by all of 
+the configured `gcovr` arguments).
+
+To implement the solution, we make use of two features:
+
+* `gcovr`’s tool `:executable` definition that looks up an environment variable.
+* Ceedling’s `:environment` settings to redefine `gcovr`.
+
+Gcovr’s tool defintion, like many of Ceedling’s tool defintions, defaults to an
+environment variable (`GCOVR`) if it is defined. If we set that environment
+variable to call Python with the path to the `gcovr` script, Ceedling will call
+that instead of only `gcovr`. Ceedling enables you to set environment variables
+that only exist while it runs.
+
+In your project file:
+
+```yaml
+:environment:
+  # Fill in / omit paths on your system as appropritate to your circumstances
+  - :gcovr: <path>/python <path>/gcovr
+```
+
+Alternatively, a slightly more elegant approach may work in some cases:
+
+```yaml
+:environment:
+  - ":gcovr: python #{`which gcovr`}" # Shell out to look up the path to gcovr
+```
+
+A variation of this concept relies on Python’s knowledge of its runtime
+environment and packages:
+
+```yaml
+:environment:
+  - :gcovr: python -m gcovr # Call the gcovr module
+```
+
+<br/><br/>

data/docs/mkdocs/plugins/index.md

@@ -0,0 +1,261 @@
+# Ceedling Plugins
+
+Ceedling includes a number of built-in plugins. Each plugin linked below
+includes full documentation of its capabilities and configuration options.
+
+!!! tip "Need a CI test build report?"
+    [Test Suite Report Log Factory](report-tests-log-factory.md) provides
+    a handful of popular reports and the ability to create your own.
+
+!!! tip "Have custom needs?"
+    Many find the handy-dandy [Command Hooks plugin](command-hooks.md)
+    is often enough as it allows you to connect your own scripts and tools 
+    to Ceedling build steps. Alternatively, you can 
+    [create your own plugins](../development/plugins/index.md).
+
+See the [`:plugins` configuration reference](../configuration/reference/plugins.md) 
+for how to enable built-in plugins and load custom plugins in your project.
+
+---
+
+## Understanding plugin build duplication
+
+!!! warning
+    Build-related plugins intentionally execute duplicated builds.
+
+Plugins including [Gcov](gcov/index.md), [Bullseye](bullseye.md), and [Valgrind](valgrind.md)
+recreate test build tasks. That is, the command line build task `test:foo` also exists 
+as `gcov:foo`, `bullseye:foo`, and/or `valgrind:foo` if these plugins are enabled.
+Each of these triggers a full rebuild of the named test with instrumentation 
+activated. **This duplication is intentional and very necessary.**
+
+These plugins modify executable code or the environment in which a test suite 
+executable runs. Duplicating and segregating the build tasks addresses several 
+important issues:
+
+1. **Conflicting interactions**. Enabling multiple plugins without 
+   duplicated tasks (i.e. integrated build tasks) can bring about conflicts and
+   undefined behavior. For instance, Gcov’s instrumentation does not
+   necessarily play nice with Valgrind’s memory-instrumented emulation
+   runtime, and the two should be kept entirely segregated.
+2. **Unforeseen effects**. These plugins can directly alter the functioning of
+   the code under test. For instance, coverage instrumentation inserted into
+   your compiled code can alter its function or timing. As such, it is vital
+   to maintain distinct builds to compare them. Your vanilla test suite build
+   may function slightly differently than the coverage-instrumented build.
+3. **Optional instrumentation**. The `<plugin>:` command line task convention
+   also provides a very simple opt-in scheme. Plugins automatically enabled 
+   for every build drives the desire to optional disable them. Valgrind testing
+   can require 25-50x the execution time of a test suite, and you likely will
+   want the option to easily disable it. While Ceedling could certainly provide
+   that option at the command line for otherwise always-on plugins, this
+   plugin task namespacing scheme naturally accomplished this along with the
+   needs of (1) and (2).
+
+---
+
+## Directory
+
+### Console Test Results
+
+<div class="grid cards" markdown>
+
+-   :material-monitor-screenshot: **[`report_tests_pretty_stdout`](report-tests-pretty-stdout.md)**
+
+    ---
+
+    The default plugin for printing test results to the console. Produces a
+    well-formatted list of summary statistics, ignored and failed tests, and
+    any extraneous output collected from test fixtures.
+
+-   :material-monitor-screenshot: **[`report_tests_ide_stdout`](report-tests-ide-stdout.md)**
+
+    ---
+
+    Formats test results for the console such that an IDE executing Ceedling
+    tasks can recognize file paths and line numbers in test failures and link
+    them for direct file navigation.
+
+-   :material-monitor-screenshot: **[`report_tests_gtestlike_stdout`](report-tests-gtestlike-stdout.md)**
+
+    ---
+
+    Prints test results to the console in a format that mimics
+    [Google Test's output][gtest-sample-output] — both human readable and
+    recognized by a variety of reporting tools, IDEs, and CI servers.
+
+-   :material-monitor-screenshot: **[`report_tests_teamcity_stdout`](report-tests-teamcity-stdout.md)**
+
+    ---
+
+    Processes test results into [TeamCity] service messages printed to the
+    console, allowing the CI server to extract build steps and test results
+    from software builds.
+
+</div>
+
+### Test Results Reports
+
+<div class="grid cards" markdown>
+
+-   :material-file-chart: **[`report_tests_log_factory`](report-tests-log-factory.md)**
+
+    ---
+
+    Produces any or all of four useful test suite reports in JSON, JUnit XML,
+    CppUnit XML, or HTML format. Also provides a mechanism for custom reports 
+    with a small amount of Ruby rather than a full plugin.
+
+-   :material-file-chart: **[`report_tests_raw_output_log`](report-tests-raw-output-log.md)**
+
+    ---
+
+    Captures extraneous console output generated by test executables —
+    typically for debugging — to log files named after the test executables.
+
+-   :material-file-chart: **[`report_build_warnings_log`](report-build-warnings-log.md)**
+
+    ---
+
+    Scans the output of build tools for console warning notices and produces
+    a simple text file that collects all such warning messages.
+
+</div>
+
+### Static & Dynamic Analysis
+
+!!! tip "Ceedling includes integrated `gdb` support"
+    Tracking down weird test failures can require multiple approaches and tools. 
+    Diagnosing a bug causing a test executable crash can benefit from either or 
+    both of these plugins plus Ceedling’s 
+    [backtrace feature](../configuration/reference/project.md#use_backtrace).
+
+<div class="grid cards" markdown>
+
+-   :material-file-check: **[`cppcheck`](cppcheck.md)**
+
+    ---
+
+    Adds Ceedling tasks to execute static analysis with [Cppcheck]. Cppcheck helps
+    find bugs related to undefined behavior, dangerous code patterns, and
+    coding style. Cppcheck is even able to analyze non-standard syntax C/C++ code, 
+    as is common in embedded projects. The plugin can produce various reports.
+
+-   :material-memory: **[`valgrind`](valgrind.md)**
+
+    ---
+
+    Adds Ceedling tasks to run test executables under [Valgrind] memory error
+    detection. Valgrind catches memory leaks, invalid accesses, and use-after-free
+    bugs. The plugin produces per-test log files in the build artifacts directory.
+
+</div>
+
+### Code Coverage
+
+<div class="grid cards" markdown>
+
+-   :material-percent: **[`gcov`](gcov/index.md)**
+
+    ---
+
+    Adds Ceedling tasks to execute tests with GNU code coverage
+    instrumentation. Manages use of [gcov], optional Python-based [GCovr],
+    and .Net-based [ReportGenerator] to produce coverage reports in a
+    variety of formats.
+
+-   :material-percent: **[`bullseye`](bullseye.md)**
+
+    ---
+
+    Adds Ceedling tasks to execute tests with code coverage instrumentation
+    provided by the commercial [Bullseye] tool, which provides visualization
+    and report generation from the coverage results.
+
+</div>
+
+### Mocking & Test Doubles
+
+<div class="grid cards" markdown>
+
+-   :material-ghost: **[`fff`](fff.md)**
+
+    ---
+
+    Replaces Ceedling’s CMock-based mock generation with the
+    [Fake Function Framework][FFF] — an alternative approach to
+    [test doubles][test-doubles] using simple fake functions.
+
+<!--Spacer so the single fff card does not stretch across multiple grid columns-->
+<div style="visibility:hidden"></div>
+
+</div>
+
+### Build Integration
+
+<div class="grid cards" markdown>
+
+-   :material-script-text-play: **[`command_hooks`](command-hooks.md)**
+
+    ---
+
+    Connects Ceedling's build events to tool entries you define in your
+    project configuration. Easily attach your own scripts or command line
+    utilities to build steps without writing a full custom plugin.
+
+-   :material-database: **[`compile_commands_json_db`](compile-commands-json-db.md)**
+
+    ---
+
+    Creates a [JSON Compilation Database][json-compilation-database] — a
+    `compile_commands.json` file useful to any editor or IDE that implements
+    [`clangd`][clangd] Language Server Protocol support.
+
+-   :material-bell-ring: **[`beep`](beep.md)**
+
+    ---
+
+    Provides a simple audio notification when a test build completes suite
+    execution or fails due to a build error, intended to support developers
+    running time-consuming test suites locally in the background.
+
+</div>
+
+### Project & Release Builds
+
+<div class="grid cards" markdown>
+
+-   :material-package-variant-closed: **[`module_generator`](module-generator.md)**
+
+    ---
+
+    Saves time in Test-Driven Development by generating a templated triplet
+    of source file, header file, and test file — scaffolded such that they
+    refer to one another — with convenient command line tasks.
+
+-   :material-package-variant-closed: **[`dependencies`](dependencies.md)**
+
+    ---
+
+    Manages release build dependencies (e.g. static libraries) including 
+    fetching those dependencies and calling a given dependency‘s build 
+    process to generate the components needed by your Ceedling release 
+    build target.
+
+</div>
+
+[gtest-sample-output]: https://subscription.packtpub.com/book/programming/9781800208988/11/ch11lvl1sec31/controlling-output-with-google-test
+[TeamCity]: https://jetbrains.com/teamcity
+[Cppcheck]: https://cppcheck.sourceforge.io
+[Valgrind]: https://valgrind.org
+[gcov]: http://gcc.gnu.org/onlinedocs/gcc/Gcov.html
+[GCovr]: https://www.gcovr.com/
+[ReportGenerator]: https://reportgenerator.io
+[Bullseye]: http://www.bullseye.com
+[FFF]: https://github.com/meekrosoft/fff
+[test-doubles]: https://blog.pragmatists.com/test-doubles-fakes-mocks-and-stubs-1a7491dfa3da
+[json-compilation-database]: https://clang.llvm.org/docs/JSONCompilationDatabase.html
+[clangd]: https://clangd.llvm.org
+[IDEs]: https://www.throwtheswitch.org/ide
+
+<br/><br/>

data/{plugins/module_generator/README.md → docs/mkdocs/plugins/module-generator.md}

@@ -1,18 +1,17 @@
-ceedling-module-generator
-=========================
+# Module Generator
 
-## Plugin Overview
+This plugin adds a pair of new commands to Ceedling allowing you to generate or remove 
+module skeletons according to predefined templates.
 
-The module_generator plugin adds a pair of new commands to Ceedling, allowing
-you to make or remove modules according to predefined templates. With a single call,
-Ceedling can generate a source, header, and test file for a new module. If given a
-pattern, it can even create a series of submodules to support specific design patterns.
-Finally, it can just as easily remove related modules, avoiding the need to delete
-each individually.
+## Overview
+With a single call, Ceedling can generate a source, header, and test file for a new 
+module. If given a pattern, it can even create a series of submodules to support 
+specific design patterns. Finally, it can just as easily remove related modules, 
+avoiding the need to delete each individually.
 
 Let's say, for example, that you want to create a single module named `MadScience`.
 
-```
+```sh
 ceedling module:create[MadScience]
 ```
 
@@ -22,12 +21,12 @@ specified a different default (see configuration). It will create three files:
 `MadScience.c`, `MadScience.h`, and `TestMadScience.c`. *NOTE* that it is important that
 there are no spaces between the brackets. We know, it's annoying... but it's the rules.
 
-### Patterns
+## Patterns
 
 You can also create an entire pattern of files. To do that, just add a second argument
 to the pattern ID. Something like this:
 
-```
+```sh
 ceedling module:create[SecretLair,mch]
 ```
 
@@ -37,17 +36,18 @@ that nice?
 
 The module generator understands the following patterns:
 
- - `src` -- generate only a source file
- - `test` -- generate only a test file
- - `dh` -- generate 6 files for the driver-hardware pattern
- - `dih` -- generate 9 files for the driver-interrupt-hardware pattern
- - `mch` -- generate 9 files for the model-conductor-hardware pattern
- - `mvp` -- generate 9 files for the model-view-presenter pattern
+- `src` -- generate only a source file
+- `test` -- generate only a test file
+- `dh` -- generate 6 files for the driver-hardware pattern
+- `dih` -- generate 9 files for the driver-interrupt-hardware pattern
+- `mch` -- generate 9 files for the model-conductor-hardware pattern
+- `mvp` -- generate 9 files for the model-view-presenter pattern
 
-### Paths
+## Paths
 
 The directories found in the project `:paths:` are reused. You can also specify an alternative default generation path using: 
-```
+
+```yaml
 :module_generator:
   :path_src: src/
   :path_inc: src/
@@ -60,13 +60,13 @@ It can do that too! You can give it a hint as to where to find your files. The p
 here is fairly basic, but it is usually sufficient. It works perfectly if your directory structure
 matches a common pattern. For example, let's say you issue this command:
 
-```
+```sh
 ceedling module:create[lab:SecretLair,mch]
 ```
 
 Say your directory structure looks like this:
 
-```
+```yaml
 :paths:
   :source:
     - lab/src
@@ -80,13 +80,13 @@ Say your directory structure looks like this:
 
 In this case, the `lab:` hint would make the module generator guess you want your files here:
 
- - source files: `lab/src` (because it's a close match)
- - include files: `lab/src` (because no include paths were listed)
- - test files: `lab/test` (because it's a close match)
+- source files: `lab/src` (because it's a close match)
+- include files: `lab/src` (because no include paths were listed)
+- test files: `lab/test` (because it's a close match)
 
 Instead, if your directory structure looks like this:
 
-```
+```yaml
 :paths:
   :source:
     - src/**   #this might contain subfolders lab, lair, and other
@@ -98,9 +98,9 @@ Instead, if your directory structure looks like this:
 
 In this case, the `lab:` hint would make the module generator guess you want your files here:
 
- - source files: `src/lab` (because it's a close match)
- - include files: `inc/lab` (because it's a close match)
- - test files: `test` (because there wasn't a close match, and this was the first entry on our list)
+- source files: `src/lab` (because it's a close match)
+- include files: `inc/lab` (because it's a close match)
+- test files: `test` (because there wasn't a close match, and this was the first entry on our list)
 
 You can see that more complicated structures will have files placed in the wrong place from time to
 time... no worries... you can move the file after it's created... but if your project has any kind of 
@@ -129,7 +129,7 @@ subdirectory is supposed to be there. If the subdir is NOT there, it will automa
 
 Let's try an example with our previous path specification:
 
-```
+```yaml
 :paths:
   :source:
     - src/**   #this might contain subfolders lab, lair, and other
@@ -141,15 +141,15 @@ Let's try an example with our previous path specification:
 
 Then, use the following command:
 
-```
+```sh
 ceedling module:create[newlab/SecretLair]
 ```
 
 This will create the following 3 files:
 
- - `src/newlab/SecretLair.c`
- - `inc/newlab/SecretLair.h`
- - `test/newlab/TestSecretLair.c`
+- `src/newlab/SecretLair.c`
+- `inc/newlab/SecretLair.h`
+- `test/newlab/TestSecretLair.c`
 
 Technically, you can start with a colon and specify part of a path, then use a slash
 to specify the remainder of the path to use. In most circumstances, this should work... however
@@ -166,7 +166,7 @@ a file that Ceedling can see, when it can't.
 Similarly, you can create stubs for all functions in a header file just by making a single call
 to your handy `stub` feature, like this:
 
-```
+```sh
 ceedling module:stub[SecretLair]
 ```
 
@@ -184,7 +184,7 @@ defaults for your own needs. For example, new source and header files will be au
 placed in the `src/` folder while tests will go in the `test/` folder. That's great if your project
 follows the default ceedling structure... but what if you have a different structure?
 
-```
+```yaml
 :module_generator:
   :naming: :bumpy
   :includes: 
@@ -204,7 +204,7 @@ Now I've redirected the location where modules are going to be generated.
 You can make it so that all of your files are generated with a standard include list. This is done
 by adding to the `:includes` array. For example:
 
-```
+```yaml
 :module_generator:
   :includes:
     :tst:
@@ -222,7 +222,7 @@ put that corporate copyright notice (or maybe a copyleft notice, if that's your
 Notice there is a separate template for source files, include files, and test files. Also, 
 your boilerplates can optionally contain `%1$s` which will inject the filename into that spot.
 
-```
+```yaml
 :module_generator:
   :boilerplates: 
     :src: |
@@ -246,7 +246,7 @@ your boilerplates can optionally contain `%1$s` which will inject the filename i
 You can specify the "#ifdef TEST" at the top of the test files with a custom define.
 This example will put a "#ifdef CEEDLING_TEST" at the top of the test files.  
 
-```
+```yaml
 :module_generator:
   :test_define: CEEDLING_TEST
 ```
@@ -259,9 +259,9 @@ generate files like `MY_NEW_MODULE.c`. This keeps everyone on your team behaving
 
 Your options for `:naming:` are as follows:
 
-  - `:bumpy` - BumpyFilesLooksLikeSo
-  - `:camel` - camelFilesAreSimilarButStartLow
-  - `:snake` - snake_case_is_all_lower_and_uses_underscores
-  - `:caps`  - CAPS_FEELS_LIKE_YOU_ARE_SCREAMING
-
+    - `:bumpy` - BumpyFilesLooksLikeSo
+    - `:camel` - camelFilesAreSimilarButStartLow
+    - `:snake` - snake_case_is_all_lower_and_uses_underscores
+    - `:caps`  - CAPS_FEELS_LIKE_YOU_ARE_SCREAMING
 
+<br/><br/>

data/{plugins/report_build_warnings_log/README.md → docs/mkdocs/plugins/report-build-warnings-log.md}

@@ -1,8 +1,8 @@
-# Ceedling Plugin: Build Warnings Log
+# Build Warnings Log
 
 Capture build process warnings from command tools to a plain text log file.
 
-# Plugin Overview
+## Overview
 
 This plugin captures warning messages output by command line tools throughout a
 build. At the end of a build, any collected warning messages are written to one 
@@ -19,7 +19,7 @@ linking.
 
 Log files are written to `<build root>/artifacts/<context>/`.
 
-# Setup
+## Setup
 
 Enable the plugin in your Ceedling project file:
 
@@ -29,7 +29,7 @@ Enable the plugin in your Ceedling project file:
     - report_build_warnings_log
 ```
 
-# Configuration
+## Configuration
 
 To change the default filename of `warning.log`, add your desired filename to
 your configuration file using `:report_build_warnings_log:` ↳ `:filename`.
@@ -38,3 +38,5 @@ your configuration file using `:report_build_warnings_log:` ↳ `:filename`.
 :report_build_warnings_log:
   :filename: more_better_filename.ext
 ```
+
+<br/><br/>

data/{plugins/report_tests_gtestlike_stdout/README.md → docs/mkdocs/plugins/report-tests-gtestlike-stdout.md}

@@ -1,8 +1,8 @@
-# Ceedling Plugin: GTest-like Test Suite Console Report
+# GTest-like Console Report
 
 Prints to the console ($stdout) test suite results in a GTest-like format.
 
-# Plugin Overview
+## Plugin Overview
 
 This plugin is intended to be used in place of the more commonly used "pretty" 
 test report plugin. Like its sibling, this plugin ollects raw test results from
@@ -12,7 +12,7 @@ readable summary form — specifically the GoogleTest format.
 This plugin is most useful when using an IDE or working with a CI system that
 understands the GTest console logging format.
 
-# Setup
+## Setup
 
 Enable the plugin in your Ceedling project by adding 
 `report_tests_gtestlike_stdout` to the list of enabled plugins instead of any 
@@ -24,13 +24,13 @@ other `report_tests_*_stdout` plugin.
     - report_tests_gtestlike_stdout
 ```
 
-# Configuration
+## Configuration
 
 No additional configuration is needed once the plugin is enabled.
 
-# Plugin Output
+## Plugin Output
 
-## Ceedling mapped to GoogleTest reporting elements
+### Ceedling mapped to GoogleTest reporting elements
 
 Ceedling's conventions and output map to GTest format as the following:
 
@@ -47,7 +47,7 @@ collect and format statistics until the end of a build. This plugin duplicates
 the tense of the wording in a GTest report, but it is unintentionally somewhat 
 misleading.
 
-## Example output (snippet)
+### Example output (snippet)
 
 The GTest format is verbose. It lists all tests with success and failure results.
 
@@ -94,3 +94,5 @@ test/TestModel.c(21): error: Function TaskScheduler_Init() called more times tha
 
  1 FAILED TESTS
 ```
+
+<br/><br/>

data/{plugins/report_tests_ide_stdout/README.md → docs/mkdocs/plugins/report-tests-ide-stdout.md}

@@ -1,8 +1,8 @@
-# Ceedling Plugin: IDE Test Suite Console Report
+# IDE Test Suite Console Report
 
 Prints to the console ($stdout) test suite results with a test failure filepath and line number format understood by nearly any IDE.
 
-# Plugin Overview
+## Plugin Overview
 
 This plugin is intended to be used in place of the more commonly used "pretty" 
 test report plugin. Like its sibling, this plugin ollects raw test results from
@@ -18,23 +18,23 @@ standard of a sort recognized almost universally.
 The end result is that test failures in your IDE's build window can become 
 links that jump directly to failing test cases.
 
-# Setup
+## Setup
 
 Enable the plugin in your project.yml by adding `report_tests_ide_stdout` to 
 the list of enabled plugins instead of any other `report_tests_*_stdout` 
 plugin.
 
-``` YAML
+```YAML
 :plugins:
   :enabled:
     - report_tests_ide_stdout
 ```
 
-# Configuration
+## Configuration
 
 No additional configuration is needed once the plugin is enabled.
 
-# Example Output
+## Example Output
 
 ```sh
  > ceedling test:Model
@@ -57,6 +57,7 @@ IGNORED: 0
 ---------------------
 BUILD FAILURE SUMMARY
 ---------------------
-Unit test failures.
+ • Unit test failures.
 ```
 
+<br/><br/>