branchproof 0.5.0 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +10 -0
- data/README.md +53 -6
- data/doc/Branchproof/Report.md +10 -0
- data/doc/CHANGELOG.md +10 -0
- data/doc/README.md +53 -6
- data/lib/branchproof/analyzer.rb +131 -3
- data/lib/branchproof/cli.rb +2 -2
- data/lib/branchproof/focused_report.rb +17 -3
- data/lib/branchproof/report.rb +155 -9
- data/lib/branchproof/version.rb +1 -1
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: b5d48afceb9c6608abf5ebdb2aa6f402dd11b75f43421b49686cc49fd95641c3
|
|
4
|
+
data.tar.gz: 647789d869c3a01f1809eacc871df99e4e0f81c7d8e70ffdb27228f4fd5c797e
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: b5f87cd4aee8588f7a86ba6a91730b7b4a787c256a0485f6dc278acbd948a5a1399669d0e2903371b28f1b7058e5339efb85fdb13d31110944e6462bdfe55b6a
|
|
7
|
+
data.tar.gz: fb81f01ba2e42014364959afcdce9c8cd218611299d8bbd27a0a9fd5ed1ce1f2d311993f834b9f960934deadab659848257010717c2bdcc6814ea01870f985e2
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,15 @@
|
|
|
1
1
|
## [Unreleased]
|
|
2
2
|
|
|
3
|
+
## [0.6.0] - 2026-09-10
|
|
4
|
+
|
|
5
|
+
- Report Decision, Condition, and Condition/Decision Coverage alongside
|
|
6
|
+
existing MC/DC evidence from one test execution.
|
|
7
|
+
- Show explicit coverage counts, supporting tests, and missing Boolean
|
|
8
|
+
observations in decision, condition, and test views and JSON reports.
|
|
9
|
+
- Calculate the same coverage criteria at every reporting level and retain
|
|
10
|
+
analysis in level-1 snapshots for offline inspection at higher detail.
|
|
11
|
+
Continue to support legacy snapshots without analysis at level 1.
|
|
12
|
+
|
|
3
13
|
## [0.5.0] - 2026-09-10
|
|
4
14
|
|
|
5
15
|
- Show source filenames beside diagnostics in decision, condition, and test
|
data/README.md
CHANGED
|
@@ -126,6 +126,50 @@ Additional tests outside this MC/DC evidence set may improve coverage.
|
|
|
126
126
|
The test execution still runs once for the selected level. Use `--format
|
|
127
127
|
json` when a consumer needs the complete identifiers and versioned schema.
|
|
128
128
|
|
|
129
|
+
### Coverage ladder
|
|
130
|
+
|
|
131
|
+
Each successful `analyze` run calculates four criteria from the same completed
|
|
132
|
+
observations. The report shows their status for each supported decision and
|
|
133
|
+
aggregate counts with explicit denominators:
|
|
134
|
+
|
|
135
|
+
| Criterion | Requirement | Aggregate denominator |
|
|
136
|
+
| --- | --- | --- |
|
|
137
|
+
| Decision | The decision produced both true and false | Supported decisions |
|
|
138
|
+
| Condition | Every atomic condition evaluated both true and false | Two required truth values per supported condition |
|
|
139
|
+
| Condition/Decision | Both Decision and Condition Coverage hold for a decision | Supported decisions |
|
|
140
|
+
| MC/DC | Every condition has an independence witness pair | Supported conditions |
|
|
141
|
+
|
|
142
|
+
For `logged_in? && admin?`, observations `[F-] => F` and `[TT] => T` give:
|
|
143
|
+
|
|
144
|
+
```text
|
|
145
|
+
Decision PASS
|
|
146
|
+
Condition FAIL (3/4 values observed)
|
|
147
|
+
Condition/Decision FAIL
|
|
148
|
+
MC/DC FAIL (1/2 conditions proven)
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
The skipped `admin?` in `[F-]` counts as neither true nor false. Conditions
|
|
152
|
+
that did evaluate count toward Condition Coverage even when their value was
|
|
153
|
+
masked by another condition. Here, the two observations prove independence
|
|
154
|
+
for `logged_in?`; `admin?` still needs `[TF] => F`.
|
|
155
|
+
|
|
156
|
+
Decision and Condition Coverage are calculated independently from the captured
|
|
157
|
+
evidence. Condition/Decision requires both; MC/DC adds independence evidence.
|
|
158
|
+
Unsupported decisions are excluded from every denominator, while
|
|
159
|
+
unexecuted supported decisions remain in scope. Empty denominators are N/A.
|
|
160
|
+
|
|
161
|
+
Reports retain the observations and owning tests for each decision outcome
|
|
162
|
+
and condition value. Missing values describe required runtime observations,
|
|
163
|
+
not application inputs or a guarantee that the path is reachable. Unattributed
|
|
164
|
+
observations can provide truth-value evidence without identifying a test;
|
|
165
|
+
the report's completeness and diagnostics still apply.
|
|
166
|
+
|
|
167
|
+
JSON stores aggregate counts under `analysis.coverage`, per-decision results
|
|
168
|
+
under `analysis.decisions[].coverage`, and condition value evidence under
|
|
169
|
+
`analysis.decisions[].condition_results[].coverage`. Existing MC/DC witness
|
|
170
|
+
pairs, counterpart constraints, and raw vectors remain available. Statuses
|
|
171
|
+
distinguish `covered`, `partial`, `unexecuted`, and `unsupported` results.
|
|
172
|
+
|
|
129
173
|
### Find missing cases
|
|
130
174
|
|
|
131
175
|
Use `--missing-only` to focus the terminal report on conditions that still
|
|
@@ -175,15 +219,17 @@ test inputs; they do not by themselves prove a bug in the application.
|
|
|
175
219
|
|
|
176
220
|
The levels select how much of the one-run result is displayed:
|
|
177
221
|
|
|
178
|
-
* Level 1 reports observed vectors grouped by their
|
|
222
|
+
* Level 1 reports the coverage ladder and observed vectors grouped by their
|
|
223
|
+
raw decision outcomes.
|
|
179
224
|
* Level 2 includes the smallest supporting sets found for vectors and tests.
|
|
180
225
|
* Level 3 (the default) includes condition independence witnesses and
|
|
181
226
|
counterpart constraints, as well as the Level 1 and 2 evidence.
|
|
182
227
|
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
228
|
+
All levels calculate the same criteria from one captured run. The level
|
|
229
|
+
controls displayed evidence, not the coverage criterion; JSON retains the
|
|
230
|
+
analysis even at Level 1. A failed, unsupported, or incomplete run is reported
|
|
231
|
+
with its status and diagnostics and cannot become a successful coverage
|
|
232
|
+
result by changing the display level.
|
|
187
233
|
|
|
188
234
|
### Focused condition and test views
|
|
189
235
|
|
|
@@ -232,7 +278,8 @@ artifacts when you need to retain multiple runs.
|
|
|
232
278
|
The `report` command reads the saved document without loading the application
|
|
233
279
|
or running tests. It uses locations and metadata captured in the report, so
|
|
234
280
|
rendering remains useful after the original checkout has moved or been
|
|
235
|
-
removed.
|
|
281
|
+
removed. New snapshots retain the ladder at every level. Legacy snapshots
|
|
282
|
+
without analysis can still be rendered at Level 1; levels 2 and 3 require
|
|
236
283
|
analysis in the saved report. The repository ignores `.branchproof/`; choose a
|
|
237
284
|
different path and CI artifact policy when a project needs to retain reports.
|
|
238
285
|
Saved JSON includes existing raw metadata such as test names and expressions;
|
data/doc/Branchproof/Report.md
CHANGED
|
@@ -19,9 +19,19 @@ Not documented.
|
|
|
19
19
|
Not documented.
|
|
20
20
|
|
|
21
21
|
## Public Instance Methods
|
|
22
|
+
### `condition_coverage_evidence(decision_id:, condition_id:)` <a id="method-i-condition_coverage_evidence"></a> <a id="condition_coverage_evidence-instance_method"></a>
|
|
23
|
+
Returns condition-value evidence for focused renderers without exposing the
|
|
24
|
+
report's internal document traversal or mutating saved records.
|
|
25
|
+
|
|
22
26
|
### `condition_explanation(decision_id:, condition_id:)` <a id="method-i-condition_explanation"></a> <a id="condition_explanation-instance_method"></a>
|
|
23
27
|
Shares the existing missing-case wording with focused terminal views.
|
|
24
28
|
|
|
29
|
+
### `coverage_ladder_lines()` <a id="method-i-coverage_ladder_lines"></a> <a id="coverage_ladder_lines-instance_method"></a>
|
|
30
|
+
Render the shared ladder in every terminal view.
|
|
31
|
+
|
|
32
|
+
### `coverage_status_label(status)` <a id="method-i-coverage_status_label"></a> <a id="coverage_status_label-instance_method"></a>
|
|
33
|
+
Not documented.
|
|
34
|
+
|
|
25
35
|
### `diagnostic_message(diagnostic)` <a id="method-i-diagnostic_message"></a> <a id="diagnostic_message-instance_method"></a>
|
|
26
36
|
Formats source context consistently in live and saved terminal views.
|
|
27
37
|
|
data/doc/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,15 @@
|
|
|
1
1
|
## [Unreleased]
|
|
2
2
|
|
|
3
|
+
## [0.6.0] - 2026-09-10
|
|
4
|
+
|
|
5
|
+
- Report Decision, Condition, and Condition/Decision Coverage alongside
|
|
6
|
+
existing MC/DC evidence from one test execution.
|
|
7
|
+
- Show explicit coverage counts, supporting tests, and missing Boolean
|
|
8
|
+
observations in decision, condition, and test views and JSON reports.
|
|
9
|
+
- Calculate the same coverage criteria at every reporting level and retain
|
|
10
|
+
analysis in level-1 snapshots for offline inspection at higher detail.
|
|
11
|
+
Continue to support legacy snapshots without analysis at level 1.
|
|
12
|
+
|
|
3
13
|
## [0.5.0] - 2026-09-10
|
|
4
14
|
|
|
5
15
|
- Show source filenames beside diagnostics in decision, condition, and test
|
data/doc/README.md
CHANGED
|
@@ -126,6 +126,50 @@ Additional tests outside this MC/DC evidence set may improve coverage.
|
|
|
126
126
|
The test execution still runs once for the selected level. Use `--format
|
|
127
127
|
json` when a consumer needs the complete identifiers and versioned schema.
|
|
128
128
|
|
|
129
|
+
### Coverage ladder
|
|
130
|
+
|
|
131
|
+
Each successful `analyze` run calculates four criteria from the same completed
|
|
132
|
+
observations. The report shows their status for each supported decision and
|
|
133
|
+
aggregate counts with explicit denominators:
|
|
134
|
+
|
|
135
|
+
| Criterion | Requirement | Aggregate denominator |
|
|
136
|
+
| --- | --- | --- |
|
|
137
|
+
| Decision | The decision produced both true and false | Supported decisions |
|
|
138
|
+
| Condition | Every atomic condition evaluated both true and false | Two required truth values per supported condition |
|
|
139
|
+
| Condition/Decision | Both Decision and Condition Coverage hold for a decision | Supported decisions |
|
|
140
|
+
| MC/DC | Every condition has an independence witness pair | Supported conditions |
|
|
141
|
+
|
|
142
|
+
For `logged_in? && admin?`, observations `[F-] => F` and `[TT] => T` give:
|
|
143
|
+
|
|
144
|
+
```text
|
|
145
|
+
Decision PASS
|
|
146
|
+
Condition FAIL (3/4 values observed)
|
|
147
|
+
Condition/Decision FAIL
|
|
148
|
+
MC/DC FAIL (1/2 conditions proven)
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
The skipped `admin?` in `[F-]` counts as neither true nor false. Conditions
|
|
152
|
+
that did evaluate count toward Condition Coverage even when their value was
|
|
153
|
+
masked by another condition. Here, the two observations prove independence
|
|
154
|
+
for `logged_in?`; `admin?` still needs `[TF] => F`.
|
|
155
|
+
|
|
156
|
+
Decision and Condition Coverage are calculated independently from the captured
|
|
157
|
+
evidence. Condition/Decision requires both; MC/DC adds independence evidence.
|
|
158
|
+
Unsupported decisions are excluded from every denominator, while
|
|
159
|
+
unexecuted supported decisions remain in scope. Empty denominators are N/A.
|
|
160
|
+
|
|
161
|
+
Reports retain the observations and owning tests for each decision outcome
|
|
162
|
+
and condition value. Missing values describe required runtime observations,
|
|
163
|
+
not application inputs or a guarantee that the path is reachable. Unattributed
|
|
164
|
+
observations can provide truth-value evidence without identifying a test;
|
|
165
|
+
the report's completeness and diagnostics still apply.
|
|
166
|
+
|
|
167
|
+
JSON stores aggregate counts under `analysis.coverage`, per-decision results
|
|
168
|
+
under `analysis.decisions[].coverage`, and condition value evidence under
|
|
169
|
+
`analysis.decisions[].condition_results[].coverage`. Existing MC/DC witness
|
|
170
|
+
pairs, counterpart constraints, and raw vectors remain available. Statuses
|
|
171
|
+
distinguish `covered`, `partial`, `unexecuted`, and `unsupported` results.
|
|
172
|
+
|
|
129
173
|
### Find missing cases
|
|
130
174
|
|
|
131
175
|
Use `--missing-only` to focus the terminal report on conditions that still
|
|
@@ -175,15 +219,17 @@ test inputs; they do not by themselves prove a bug in the application.
|
|
|
175
219
|
|
|
176
220
|
The levels select how much of the one-run result is displayed:
|
|
177
221
|
|
|
178
|
-
* Level 1 reports observed vectors grouped by their
|
|
222
|
+
* Level 1 reports the coverage ladder and observed vectors grouped by their
|
|
223
|
+
raw decision outcomes.
|
|
179
224
|
* Level 2 includes the smallest supporting sets found for vectors and tests.
|
|
180
225
|
* Level 3 (the default) includes condition independence witnesses and
|
|
181
226
|
counterpart constraints, as well as the Level 1 and 2 evidence.
|
|
182
227
|
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
228
|
+
All levels calculate the same criteria from one captured run. The level
|
|
229
|
+
controls displayed evidence, not the coverage criterion; JSON retains the
|
|
230
|
+
analysis even at Level 1. A failed, unsupported, or incomplete run is reported
|
|
231
|
+
with its status and diagnostics and cannot become a successful coverage
|
|
232
|
+
result by changing the display level.
|
|
187
233
|
|
|
188
234
|
### Focused condition and test views
|
|
189
235
|
|
|
@@ -232,7 +278,8 @@ artifacts when you need to retain multiple runs.
|
|
|
232
278
|
The `report` command reads the saved document without loading the application
|
|
233
279
|
or running tests. It uses locations and metadata captured in the report, so
|
|
234
280
|
rendering remains useful after the original checkout has moved or been
|
|
235
|
-
removed.
|
|
281
|
+
removed. New snapshots retain the ladder at every level. Legacy snapshots
|
|
282
|
+
without analysis can still be rendered at Level 1; levels 2 and 3 require
|
|
236
283
|
analysis in the saved report. The repository ignores `.branchproof/`; choose a
|
|
237
284
|
different path and CI artifact policy when a project needs to retain reports.
|
|
238
285
|
Saved JSON includes existing raw metadata such as test names and expressions;
|
data/lib/branchproof/analyzer.rb
CHANGED
|
@@ -30,7 +30,8 @@ module Branchproof
|
|
|
30
30
|
proven_count: proven,
|
|
31
31
|
eligible_count: eligible,
|
|
32
32
|
completeness: completeness(decisions),
|
|
33
|
-
diagnostics: decisions.flat_map { |decision| decision[:diagnostics] } + @invalid_diagnostics
|
|
33
|
+
diagnostics: decisions.flat_map { |decision| decision[:diagnostics] } + @invalid_diagnostics,
|
|
34
|
+
coverage: aggregate_coverage(decisions)
|
|
34
35
|
}.freeze
|
|
35
36
|
end
|
|
36
37
|
|
|
@@ -97,7 +98,7 @@ module Branchproof
|
|
|
97
98
|
unless id(decision, :support_status).to_s.empty? || id(decision, :support_status).to_s.upcase == "SUPPORTED"
|
|
98
99
|
return {
|
|
99
100
|
decision_id: decision_id, effective_masks_by_vector: {}, condition_results: [], witness_buckets: {},
|
|
100
|
-
conditions: [], unsupported: true,
|
|
101
|
+
conditions: [], unsupported: true, coverage: unsupported_coverage,
|
|
101
102
|
completeness: { observation: true, attribution: true, analysis: true },
|
|
102
103
|
diagnostics: [diagnostic("unsupported_decision", "warning", decision_id, nil)]
|
|
103
104
|
}
|
|
@@ -136,9 +137,13 @@ module Branchproof
|
|
|
136
137
|
nil
|
|
137
138
|
else
|
|
138
139
|
(true_ids.empty? || false_ids.empty? ? "missing_effective_sign" : "no_independent_pair")
|
|
139
|
-
end
|
|
140
|
+
end,
|
|
141
|
+
coverage: condition_coverage(index, vectors)
|
|
140
142
|
}
|
|
141
143
|
end
|
|
144
|
+
decision_coverage = decision_coverage(vectors)
|
|
145
|
+
condition_coverage_summary = condition_summary(results)
|
|
146
|
+
mcdc = mcdc_coverage(results)
|
|
142
147
|
{
|
|
143
148
|
decision_id: decision_id,
|
|
144
149
|
effective_masks_by_vector: masks,
|
|
@@ -146,11 +151,134 @@ module Branchproof
|
|
|
146
151
|
witness_buckets: buckets,
|
|
147
152
|
edge_table: edge_table(decision[:tree]),
|
|
148
153
|
conditions: conditions(decision),
|
|
154
|
+
coverage: { decision: decision_coverage, condition: condition_coverage_summary,
|
|
155
|
+
condition_decision: conjunction_coverage(decision_coverage, condition_coverage_summary),
|
|
156
|
+
mcdc: mcdc },
|
|
149
157
|
completeness: { observation: true, attribution: true, analysis: !@analysis_invalid },
|
|
150
158
|
diagnostics: []
|
|
151
159
|
}
|
|
152
160
|
end
|
|
153
161
|
|
|
162
|
+
def unsupported_coverage
|
|
163
|
+
{ decision: { status: "unsupported", true_observed: false, false_observed: false,
|
|
164
|
+
covered_outcomes: 0, required_outcomes: 2, outcomes: [], missing_outcomes: [] },
|
|
165
|
+
condition: { status: "unsupported", covered_values: 0, required_values: 0,
|
|
166
|
+
covered_conditions: 0, condition_count: 0 },
|
|
167
|
+
condition_decision: { status: "unsupported" },
|
|
168
|
+
mcdc: { status: "unsupported", proven_conditions: 0, condition_count: 0 } }
|
|
169
|
+
end
|
|
170
|
+
|
|
171
|
+
def provenance(vector)
|
|
172
|
+
{ vector_id: id(vector, :id).to_s,
|
|
173
|
+
test_ids: Array(id(vector, :test_ids)).map(&:to_s).uniq.sort,
|
|
174
|
+
unattributed_count: id(vector, :unattributed_count).to_i }
|
|
175
|
+
end
|
|
176
|
+
|
|
177
|
+
def evidence_bucket(value, vectors)
|
|
178
|
+
matching = vectors.select do |vector|
|
|
179
|
+
observed?(vector, value[:index]) && self.value(vector, value[:index]) == value[:value]
|
|
180
|
+
end
|
|
181
|
+
evidence = matching.map { |vector| provenance(vector) }
|
|
182
|
+
{ value: value[:value], observed: !matching.empty?,
|
|
183
|
+
vector_ids: evidence.flat_map { |item| item[:vector_id] }.uniq.sort,
|
|
184
|
+
test_ids: evidence.flat_map { |item| item[:test_ids] }.uniq.sort,
|
|
185
|
+
unattributed_count: evidence.sum { |item| item[:unattributed_count] } }
|
|
186
|
+
end
|
|
187
|
+
|
|
188
|
+
def condition_coverage(index, vectors)
|
|
189
|
+
observed_vectors = vectors.select { |vector| observed?(vector, index) }
|
|
190
|
+
values = [true, false].map do |item|
|
|
191
|
+
evidence_bucket({ index: index, value: item }, observed_vectors)
|
|
192
|
+
end
|
|
193
|
+
covered = values.count { |entry| entry[:observed] }
|
|
194
|
+
{ status: coverage_status(covered, 2), true_observed: values[0][:observed],
|
|
195
|
+
false_observed: values[1][:observed], covered_values: covered, required_values: 2,
|
|
196
|
+
values: values, missing_values: values.reject { |entry| entry[:observed] }.map { |entry| entry[:value] } }
|
|
197
|
+
end
|
|
198
|
+
|
|
199
|
+
def decision_coverage(vectors)
|
|
200
|
+
outcomes = [false, true].map do |value|
|
|
201
|
+
matching = vectors.select { |vector| outcome(vector) == value }
|
|
202
|
+
evidence = matching.map { |vector| provenance(vector) }
|
|
203
|
+
{ value: value, observed: !matching.empty?,
|
|
204
|
+
vector_ids: evidence.flat_map { |item| item[:vector_id] }.uniq.sort,
|
|
205
|
+
test_ids: evidence.flat_map { |item| item[:test_ids] }.uniq.sort,
|
|
206
|
+
unattributed_count: evidence.sum { |item| item[:unattributed_count] } }
|
|
207
|
+
end
|
|
208
|
+
covered = outcomes.count { |entry| entry[:observed] }
|
|
209
|
+
{ status: coverage_status(covered, 2), true_observed: outcomes[1][:observed],
|
|
210
|
+
false_observed: outcomes[0][:observed], covered_outcomes: covered, required_outcomes: 2,
|
|
211
|
+
outcomes: outcomes,
|
|
212
|
+
missing_outcomes: outcomes.reject { |entry| entry[:observed] }.map { |entry| entry[:value] } }
|
|
213
|
+
end
|
|
214
|
+
|
|
215
|
+
def condition_summary(results)
|
|
216
|
+
values = results.sum { |result| result[:coverage][:covered_values] }
|
|
217
|
+
count = results.length
|
|
218
|
+
{ status: coverage_status(values, count * 2), covered_values: values,
|
|
219
|
+
required_values: count * 2,
|
|
220
|
+
covered_conditions: results.count { |result| result[:coverage][:status] == "covered" },
|
|
221
|
+
condition_count: count }
|
|
222
|
+
end
|
|
223
|
+
|
|
224
|
+
def mcdc_coverage(results)
|
|
225
|
+
proven = results.count { |result| result[:status] == "PROVEN" }
|
|
226
|
+
observed = results.any? { |result| result[:coverage][:covered_values].positive? }
|
|
227
|
+
status = if !observed
|
|
228
|
+
"unexecuted"
|
|
229
|
+
elsif proven == results.length
|
|
230
|
+
"covered"
|
|
231
|
+
else
|
|
232
|
+
"partial"
|
|
233
|
+
end
|
|
234
|
+
{ status: status, proven_conditions: proven, condition_count: results.length }
|
|
235
|
+
end
|
|
236
|
+
|
|
237
|
+
def conjunction_coverage(decision, condition)
|
|
238
|
+
status = if decision[:status] == "covered" && condition[:status] == "covered"
|
|
239
|
+
"covered"
|
|
240
|
+
elsif decision[:status] == "unexecuted" && condition[:status] == "unexecuted"
|
|
241
|
+
"unexecuted"
|
|
242
|
+
else
|
|
243
|
+
"partial"
|
|
244
|
+
end
|
|
245
|
+
{ status: status }
|
|
246
|
+
end
|
|
247
|
+
|
|
248
|
+
def coverage_status(covered, required)
|
|
249
|
+
return "unexecuted" if covered.zero?
|
|
250
|
+
return "covered" if covered == required
|
|
251
|
+
|
|
252
|
+
"partial"
|
|
253
|
+
end
|
|
254
|
+
|
|
255
|
+
def aggregate_coverage(decisions)
|
|
256
|
+
supported = decisions.reject { |decision| decision[:unsupported] }
|
|
257
|
+
decision_covered = supported.count do |decision|
|
|
258
|
+
decision[:coverage][:decision][:status] == "covered"
|
|
259
|
+
end
|
|
260
|
+
condition_decision_covered = supported.count do |decision|
|
|
261
|
+
decision[:coverage][:condition_decision][:status] == "covered"
|
|
262
|
+
end
|
|
263
|
+
condition_count = supported.sum { |decision| decision[:coverage][:condition][:condition_count] }
|
|
264
|
+
condition_values = supported.sum { |decision| decision[:coverage][:condition][:covered_values] }
|
|
265
|
+
covered_conditions = supported.sum do |decision|
|
|
266
|
+
decision[:coverage][:condition][:covered_conditions]
|
|
267
|
+
end
|
|
268
|
+
proven = supported.sum { |decision| decision[:coverage][:mcdc][:proven_conditions] }
|
|
269
|
+
percentage = ->(covered, required) { required.zero? ? nil : (covered.to_f / required * 100).round(2) }
|
|
270
|
+
{ decision: { covered_decisions: decision_covered, supported_decisions: supported.length,
|
|
271
|
+
percentage: percentage.call(decision_covered, supported.length) },
|
|
272
|
+
condition: { covered_values: condition_values, required_values: condition_count * 2,
|
|
273
|
+
covered_conditions: covered_conditions, condition_count: condition_count,
|
|
274
|
+
percentage: percentage.call(condition_values, condition_count * 2) },
|
|
275
|
+
condition_decision: { covered_decisions: condition_decision_covered,
|
|
276
|
+
supported_decisions: supported.length,
|
|
277
|
+
percentage: percentage.call(condition_decision_covered, supported.length) },
|
|
278
|
+
mcdc: { proven_conditions: proven, supported_conditions: condition_count,
|
|
279
|
+
percentage: percentage.call(proven, condition_count) } }
|
|
280
|
+
end
|
|
281
|
+
|
|
154
282
|
def valid_vector(vector, decision)
|
|
155
283
|
return nil unless id(vector, :decision_id) == id(decision, :id)
|
|
156
284
|
|
data/lib/branchproof/cli.rb
CHANGED
|
@@ -49,7 +49,7 @@ module Branchproof
|
|
|
49
49
|
message: merge_status[:reason].to_s }]
|
|
50
50
|
end
|
|
51
51
|
end
|
|
52
|
-
if
|
|
52
|
+
if value(baseline, :status).to_s == "PASSED"
|
|
53
53
|
analysis = Analyzer.new(inventory: inventory, evidence: evidence.snapshot, limits: options[:limits]).call
|
|
54
54
|
baseline[:analysis] = analysis
|
|
55
55
|
if options[:level] >= 2
|
|
@@ -70,7 +70,7 @@ module Branchproof
|
|
|
70
70
|
diagnostics = Array(value(inventory, :diagnostics)) + Array(value(baseline, :diagnostics)) +
|
|
71
71
|
Array(value(evidence.snapshot,
|
|
72
72
|
:diagnostics)) + Array(value(value(baseline, :analysis), :diagnostics))
|
|
73
|
-
analysis =
|
|
73
|
+
analysis = value(baseline, :analysis)
|
|
74
74
|
minima = options[:level] == 1 ? [] : Array(value(baseline, :minima))
|
|
75
75
|
report = Report.new(inventory: inventory, evidence: value(baseline, :evidence) || evidence.snapshot,
|
|
76
76
|
analysis: analysis, minima: minima, baseline: baseline, diagnostics: diagnostics,
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
-
# rubocop:disable Metrics/ClassLength, Metrics/AbcSize, Metrics/CyclomaticComplexity, Metrics/MethodLength, Metrics/PerceivedComplexity
|
|
3
|
+
# rubocop:disable Metrics/ClassLength, Metrics/AbcSize, Metrics/BlockLength, Metrics/CyclomaticComplexity, Metrics/MethodLength, Metrics/PerceivedComplexity
|
|
4
4
|
require "pathname"
|
|
5
5
|
|
|
6
6
|
module Branchproof
|
|
@@ -34,6 +34,7 @@ module Branchproof
|
|
|
34
34
|
end
|
|
35
35
|
lines << "Empty groups mean no recorded completed observation."
|
|
36
36
|
lines << ""
|
|
37
|
+
lines.concat(@coordinator.coverage_ladder_lines)
|
|
37
38
|
@view == :conditions ? render_conditions(lines) : render_tests(lines)
|
|
38
39
|
render_unowned(lines)
|
|
39
40
|
render_unsupported(lines)
|
|
@@ -52,7 +53,15 @@ module Branchproof
|
|
|
52
53
|
lines << "Condition: #{row[:expression]}"
|
|
53
54
|
lines << "Location: #{location(row[:relative_path], row[:line], unavailable: "condition line unavailable")}"
|
|
54
55
|
lines << "Decision: #{row[:decision_expression]} (condition #{row[:index]})"
|
|
55
|
-
|
|
56
|
+
status = if @level == 1 && !coverage_available?
|
|
57
|
+
"NOT CALCULATED"
|
|
58
|
+
else
|
|
59
|
+
row[:status] || "NOT_PROVEN"
|
|
60
|
+
end
|
|
61
|
+
lines << "MC/DC: #{status}"
|
|
62
|
+
if @level >= 2
|
|
63
|
+
lines.concat(@coordinator.condition_coverage_evidence(decision_id: row[:decision_id], condition_id: row[:id]))
|
|
64
|
+
end
|
|
56
65
|
render_group(lines, "Evaluated true by", row[:observed_true], row)
|
|
57
66
|
render_group(lines, "Evaluated false by", row[:observed_false], row)
|
|
58
67
|
render_group(lines, "Short-circuited in", row[:short_circuited], row)
|
|
@@ -79,6 +88,11 @@ module Branchproof
|
|
|
79
88
|
lines << "No missing conditions" if @missing_only && rows.empty?
|
|
80
89
|
end
|
|
81
90
|
|
|
91
|
+
def coverage_available?
|
|
92
|
+
analysis = fetch(@document, :analysis)
|
|
93
|
+
analysis && fetch(analysis, :coverage)
|
|
94
|
+
end
|
|
95
|
+
|
|
82
96
|
def render_constraint(lines, row)
|
|
83
97
|
lines << @coordinator.condition_explanation(decision_id: row[:decision_id], condition_id: row[:id])
|
|
84
98
|
end
|
|
@@ -202,4 +216,4 @@ module Branchproof
|
|
|
202
216
|
end
|
|
203
217
|
end
|
|
204
218
|
|
|
205
|
-
# rubocop:enable Metrics/ClassLength, Metrics/AbcSize, Metrics/CyclomaticComplexity, Metrics/MethodLength, Metrics/PerceivedComplexity
|
|
219
|
+
# rubocop:enable Metrics/ClassLength, Metrics/AbcSize, Metrics/BlockLength, Metrics/CyclomaticComplexity, Metrics/MethodLength, Metrics/PerceivedComplexity
|
data/lib/branchproof/report.rb
CHANGED
|
@@ -70,6 +70,31 @@ module Branchproof
|
|
|
70
70
|
"#{prefix}#{path}: #{message}"
|
|
71
71
|
end
|
|
72
72
|
|
|
73
|
+
# Returns condition-value evidence for focused renderers without exposing
|
|
74
|
+
# the report's internal document traversal or mutating saved records.
|
|
75
|
+
def condition_coverage_evidence(decision_id:, condition_id:)
|
|
76
|
+
decision = inventory_decisions.find { |item| value(item, :id).to_s == decision_id.to_s }
|
|
77
|
+
condition = Array(value(decision, :conditions)).find do |item|
|
|
78
|
+
value(item, :id).to_s == condition_id.to_s
|
|
79
|
+
end
|
|
80
|
+
coverage = value(condition_result(decision, condition), :coverage)
|
|
81
|
+
return [] unless coverage
|
|
82
|
+
|
|
83
|
+
@terminal_ids ||= terminal_ids
|
|
84
|
+
lines = Array(value(coverage, :values)).map do |entry|
|
|
85
|
+
value_label = value(entry, :value) ? "true" : "false"
|
|
86
|
+
owners = Array(value(entry, :test_ids)).map { |id| test_label(id) }
|
|
87
|
+
owners << "unattributed" if value(entry, :unattributed_count).to_i.positive?
|
|
88
|
+
evidence = owners.empty? ? "none recorded" : owners.uniq.join(", ")
|
|
89
|
+
" Value #{value_label}: #{value(entry, :observed) ? "observed" : "missing"}; tests: #{evidence}"
|
|
90
|
+
end
|
|
91
|
+
missing = Array(value(coverage, :missing_values))
|
|
92
|
+
unless missing.empty?
|
|
93
|
+
lines << " Missing values: #{missing.map { |item| item ? "true" : "false" }.join(", ")}"
|
|
94
|
+
end
|
|
95
|
+
lines
|
|
96
|
+
end
|
|
97
|
+
|
|
73
98
|
def exit_code
|
|
74
99
|
return 2 unless usage_valid?
|
|
75
100
|
|
|
@@ -94,7 +119,7 @@ module Branchproof
|
|
|
94
119
|
criterion_version: CRITERION_VERSION, runtime: RUBY_DESCRIPTION,
|
|
95
120
|
run_ids: Array(value(@evidence, :run_ids)),
|
|
96
121
|
source_inventory: @inventory, baseline: @baseline, observations: @evidence,
|
|
97
|
-
analysis: @
|
|
122
|
+
analysis: @analysis, minima: @minima, metrics: metrics,
|
|
98
123
|
diagnostics: @diagnostics, completeness: completeness,
|
|
99
124
|
run_metadata: @run_metadata)
|
|
100
125
|
end
|
|
@@ -115,6 +140,7 @@ module Branchproof
|
|
|
115
140
|
"Observations: #{metrics[:completed]} completed, #{metrics[:aborted]} aborted, " \
|
|
116
141
|
"#{metrics[:unattributed]} unattributed",
|
|
117
142
|
"Values: T=true, F=false, -=short-circuited"]
|
|
143
|
+
lines.concat(coverage_ladder_lines)
|
|
118
144
|
lines << missing_summary_line if @missing_only
|
|
119
145
|
lines << "Scope: supported decisions and conditions"
|
|
120
146
|
lines << ""
|
|
@@ -137,14 +163,16 @@ module Branchproof
|
|
|
137
163
|
conditions_to_render(decision).each do |condition|
|
|
138
164
|
result = condition_result(decision, condition)
|
|
139
165
|
detail = condition_detail(decision, condition, result)
|
|
140
|
-
status = if @analysis.nil? || @level == 1
|
|
166
|
+
status = if @analysis.nil? || (@level == 1 && !coverage_available?)
|
|
141
167
|
"NOT CALCULATED"
|
|
142
168
|
else
|
|
143
169
|
value(result, :status) || "NOT_PROVEN"
|
|
144
170
|
end
|
|
145
171
|
lines << " Condition #{value(condition, :index)}: #{value(condition, :expression)}"
|
|
146
172
|
lines << " #{status}#{detail}"
|
|
173
|
+
render_condition_coverage(lines, decision, condition, result) if @level >= 2 && coverage_available?
|
|
147
174
|
end
|
|
175
|
+
render_decision_coverage(lines, decision) if coverage_available?
|
|
148
176
|
vectors_to_render(decision).each do |vector|
|
|
149
177
|
values = Array(value(vector, :values)).map do |item|
|
|
150
178
|
if item.nil?
|
|
@@ -162,6 +190,64 @@ module Branchproof
|
|
|
162
190
|
lines << ""
|
|
163
191
|
end
|
|
164
192
|
|
|
193
|
+
def coverage_status_label(status)
|
|
194
|
+
{ "covered" => "PASS", "partial" => "FAIL", "unexecuted" => "UNEXECUTED", "unsupported" => "EXCLUDED" }.fetch(
|
|
195
|
+
status.to_s.downcase, status.to_s.upcase
|
|
196
|
+
)
|
|
197
|
+
end
|
|
198
|
+
|
|
199
|
+
def render_decision_coverage(lines, decision)
|
|
200
|
+
decision_coverage = value(value(analysis_for(decision), :coverage), :decision)
|
|
201
|
+
all_coverage = value(analysis_for(decision), :coverage) || {}
|
|
202
|
+
statuses = [["D", :decision], ["C", :condition], ["C/D", :condition_decision],
|
|
203
|
+
["MC/DC", :mcdc]].filter_map do |label, key|
|
|
204
|
+
row = value(all_coverage, key)
|
|
205
|
+
status = value(row, :status)
|
|
206
|
+
status ? criterion_status_text(label, row) : nil
|
|
207
|
+
end
|
|
208
|
+
lines << " Coverage: #{statuses.join(", ")}" unless statuses.empty?
|
|
209
|
+
coverage = decision_coverage
|
|
210
|
+
return unless coverage
|
|
211
|
+
|
|
212
|
+
return unless @level >= 2
|
|
213
|
+
|
|
214
|
+
Array(value(coverage, :outcomes)).each do |outcome|
|
|
215
|
+
value_label = value(outcome, :value) ? "true" : "false"
|
|
216
|
+
owners = Array(value(outcome, :test_ids)).map { |id| test_label(id) }
|
|
217
|
+
owners << "unattributed" if value(outcome, :unattributed_count).to_i.positive?
|
|
218
|
+
evidence = owners.empty? ? "none recorded" : owners.uniq.join(", ")
|
|
219
|
+
lines << " Outcome #{value_label}: #{value(outcome, :observed) ? "observed" : "missing"}; tests: #{evidence}"
|
|
220
|
+
end
|
|
221
|
+
missing = Array(value(coverage, :missing_outcomes))
|
|
222
|
+
return if missing.empty?
|
|
223
|
+
|
|
224
|
+
lines << " Missing decision outcomes: #{missing.map do |item|
|
|
225
|
+
item ? "true" : "false"
|
|
226
|
+
end.join(", ")}"
|
|
227
|
+
end
|
|
228
|
+
|
|
229
|
+
def render_condition_coverage(lines, _decision, condition, result)
|
|
230
|
+
coverage = value(result, :coverage)
|
|
231
|
+
return unless coverage
|
|
232
|
+
|
|
233
|
+
values = Array(value(coverage, :values))
|
|
234
|
+
return if values.empty?
|
|
235
|
+
|
|
236
|
+
values.each do |entry|
|
|
237
|
+
value_label = value(entry, :value) ? "true" : "false"
|
|
238
|
+
owners = Array(value(entry, :test_ids)).map { |id| test_label(id) }
|
|
239
|
+
owners << "unattributed" if value(entry, :unattributed_count).to_i.positive?
|
|
240
|
+
evidence = owners.empty? ? "none recorded" : owners.uniq.join(", ")
|
|
241
|
+
lines << " Value #{value_label}: #{value(entry, :observed) ? "observed" : "missing"}; tests: #{evidence}"
|
|
242
|
+
end
|
|
243
|
+
missing = Array(value(coverage, :missing_values))
|
|
244
|
+
return if missing.empty?
|
|
245
|
+
|
|
246
|
+
lines << " Missing values for #{value(condition, :expression)}: #{missing.map do |item|
|
|
247
|
+
item ? "true" : "false"
|
|
248
|
+
end.join(", ")}"
|
|
249
|
+
end
|
|
250
|
+
|
|
165
251
|
def render_minima(lines)
|
|
166
252
|
rows = Array(@minima).map do |minimum|
|
|
167
253
|
objective = value(minimum, :objective)
|
|
@@ -189,19 +275,19 @@ module Branchproof
|
|
|
189
275
|
end
|
|
190
276
|
|
|
191
277
|
def terminal_coverage_label
|
|
192
|
-
return "not calculated" if @analysis.nil?
|
|
278
|
+
return "not calculated" if @analysis.nil?
|
|
193
279
|
|
|
194
280
|
coverage_label
|
|
195
281
|
end
|
|
196
282
|
|
|
197
283
|
def terminal_coverage_line
|
|
198
|
-
return "MC/DC: not calculated" if @analysis.nil?
|
|
284
|
+
return "MC/DC: not calculated" if @analysis.nil?
|
|
199
285
|
|
|
200
286
|
"MC/DC: #{terminal_coverage_label} (#{metrics[:proven]}/#{metrics[:eligible_conditions]} conditions proven)"
|
|
201
287
|
end
|
|
202
288
|
|
|
203
289
|
def terminal_analysis_status
|
|
204
|
-
return "NOT CALCULATED" if @analysis.nil?
|
|
290
|
+
return "NOT CALCULATED" if @analysis.nil?
|
|
205
291
|
|
|
206
292
|
analysis_status
|
|
207
293
|
end
|
|
@@ -473,7 +559,7 @@ module Branchproof
|
|
|
473
559
|
end
|
|
474
560
|
|
|
475
561
|
def valid_for_requested_level?
|
|
476
|
-
return completeness[:observation] && completeness[:attribution] if @level == 1
|
|
562
|
+
return completeness[:observation] && completeness[:attribution] if @level == 1 && @analysis.nil?
|
|
477
563
|
|
|
478
564
|
completeness.values.all? { |item| item == true }
|
|
479
565
|
end
|
|
@@ -486,13 +572,14 @@ module Branchproof
|
|
|
486
572
|
end
|
|
487
573
|
|
|
488
574
|
def analysis_status
|
|
489
|
-
return "NOT_REQUESTED" if @analysis.nil?
|
|
575
|
+
return "NOT_REQUESTED" if @analysis.nil?
|
|
490
576
|
|
|
491
577
|
valid_for_requested_level? ? "COMPLETE" : "PARTIAL"
|
|
492
578
|
end
|
|
493
579
|
|
|
494
580
|
def incomplete?
|
|
495
|
-
!completeness[:observation] || !completeness[:attribution] ||
|
|
581
|
+
!completeness[:observation] || !completeness[:attribution] ||
|
|
582
|
+
(analysis_available? && !completeness[:analysis])
|
|
496
583
|
end
|
|
497
584
|
|
|
498
585
|
def vectors_for(decision)
|
|
@@ -524,7 +611,65 @@ module Branchproof
|
|
|
524
611
|
end
|
|
525
612
|
|
|
526
613
|
def analysis_available?
|
|
527
|
-
!@analysis.nil?
|
|
614
|
+
!@analysis.nil?
|
|
615
|
+
end
|
|
616
|
+
|
|
617
|
+
def coverage_available?
|
|
618
|
+
analysis_available? && !value(@analysis, :coverage).nil?
|
|
619
|
+
end
|
|
620
|
+
|
|
621
|
+
# Render the shared ladder in every terminal view.
|
|
622
|
+
def coverage_ladder_lines
|
|
623
|
+
return [] unless coverage_available?
|
|
624
|
+
|
|
625
|
+
aggregate = value(@analysis, :coverage) || {}
|
|
626
|
+
rows = [["D", :decision, :covered_decisions, :supported_decisions],
|
|
627
|
+
["C", :condition, :covered_values, :required_values],
|
|
628
|
+
["C/D", :condition_decision, :covered_decisions, :supported_decisions],
|
|
629
|
+
["MC/DC", :mcdc, :proven_conditions, :supported_conditions]]
|
|
630
|
+
lines = ["Coverage ladder:"]
|
|
631
|
+
rows.each do |label, key, numerator_key, denominator_key|
|
|
632
|
+
row = value(aggregate, key)
|
|
633
|
+
next unless row
|
|
634
|
+
|
|
635
|
+
percentage = value(row, :percentage)
|
|
636
|
+
numerator = value(row, numerator_key) || 0
|
|
637
|
+
denominator = value(row, denominator_key) || 0
|
|
638
|
+
shown = percentage.nil? ? "N/A" : "#{percentage}%"
|
|
639
|
+
denominator_label = if ["D", "C/D"].include?(label)
|
|
640
|
+
"decisions"
|
|
641
|
+
else
|
|
642
|
+
label == "C" ? "truth values" : "conditions"
|
|
643
|
+
end
|
|
644
|
+
qualifier = incomplete? ? " (lower bound)" : ""
|
|
645
|
+
description = case label
|
|
646
|
+
when "D" then "Decision coverage"
|
|
647
|
+
when "C" then "Condition coverage"
|
|
648
|
+
when "C/D" then "Condition/decision coverage"
|
|
649
|
+
else "MC/DC coverage"
|
|
650
|
+
end
|
|
651
|
+
lines << " #{label} (#{description}): #{shown} (#{numerator}/#{denominator} #{denominator_label})#{qualifier}"
|
|
652
|
+
end
|
|
653
|
+
lines << ""
|
|
654
|
+
lines
|
|
655
|
+
end
|
|
656
|
+
|
|
657
|
+
def criterion_status_text(label, row)
|
|
658
|
+
raw_status = value(row, :status).to_s.downcase
|
|
659
|
+
status = coverage_status_label(raw_status)
|
|
660
|
+
return "#{label}=#{status}" if raw_status == "unsupported"
|
|
661
|
+
|
|
662
|
+
case label
|
|
663
|
+
when "D"
|
|
664
|
+
"#{label}=#{status} (#{value(row, :covered_outcomes) || 0}/#{value(row, :required_outcomes) || 0} outcomes)"
|
|
665
|
+
when "C"
|
|
666
|
+
"#{label}=#{status} (#{value(row, :covered_values) || 0}/#{value(row, :required_values) || 0} values; " \
|
|
667
|
+
"#{value(row, :covered_conditions) || 0}/#{value(row, :condition_count) || 0} conditions fully covered)"
|
|
668
|
+
when "MC/DC"
|
|
669
|
+
"#{label}=#{status} (#{value(row, :proven_conditions) || 0}/#{value(row, :condition_count) || 0} conditions)"
|
|
670
|
+
else
|
|
671
|
+
"#{label}=#{status}"
|
|
672
|
+
end
|
|
528
673
|
end
|
|
529
674
|
|
|
530
675
|
def analysis_complete?
|
|
@@ -604,5 +749,6 @@ module Branchproof
|
|
|
604
749
|
def normalize_unknown(object)
|
|
605
750
|
object.to_s
|
|
606
751
|
end
|
|
752
|
+
public :condition_coverage_evidence, :coverage_ladder_lines, :coverage_status_label
|
|
607
753
|
end
|
|
608
754
|
end
|
data/lib/branchproof/version.rb
CHANGED