branchproof 0.10.0 → 0.11.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +30 -0
- data/README.md +153 -10
- data/doc/Branchproof/Configuration.md +28 -0
- data/doc/Branchproof/CoveragePolicy.md +27 -0
- data/doc/Branchproof/FocusedReport.md +1 -1
- data/doc/Branchproof/Report.md +8 -2
- data/doc/Branchproof/ReportSelection.md +42 -0
- data/doc/Branchproof/Source.md +5 -0
- data/doc/Branchproof.md +4 -1
- data/doc/CHANGELOG.md +30 -0
- data/doc/README.md +153 -10
- data/lib/branchproof/cli.rb +120 -13
- data/lib/branchproof/comparison.rb +6 -1
- data/lib/branchproof/configuration.rb +111 -0
- data/lib/branchproof/coverage_policy.rb +169 -0
- data/lib/branchproof/evidence.rb +37 -2
- data/lib/branchproof/focused_report.rb +144 -27
- data/lib/branchproof/report.rb +97 -9
- data/lib/branchproof/report_selection.rb +158 -0
- data/lib/branchproof/saved_report.rb +29 -3
- data/lib/branchproof/source.rb +10 -1
- data/lib/branchproof/version.rb +1 -1
- data/lib/branchproof.rb +3 -0
- data/llms.txt +4 -1
- data/sig/branchproof.rbs +35 -3
- metadata +8 -2
data/doc/README.md
CHANGED
|
@@ -107,6 +107,72 @@ files are selected explicitly. The worker prepends the project's `lib` and
|
|
|
107
107
|
`test` directories (`lib` and `spec` for RSpec) to its child load path, so application `require` calls
|
|
108
108
|
resolve without changing the parent process.
|
|
109
109
|
|
|
110
|
+
Project defaults can be checked into `.branchproof.json` at the project root:
|
|
111
|
+
|
|
112
|
+
```json
|
|
113
|
+
{
|
|
114
|
+
"schema_version": 1,
|
|
115
|
+
"project": "rails",
|
|
116
|
+
"framework": "minitest",
|
|
117
|
+
"sources": ["app/**/*.rb"],
|
|
118
|
+
"tests": ["test/**/*_test.rb"],
|
|
119
|
+
"exclude": ["app/generated/**/*.rb"],
|
|
120
|
+
"minimum": { "mcdc": 80 }
|
|
121
|
+
}
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
`project` accepts `auto`, `ruby`, or `rails`; `framework` accepts `auto`,
|
|
125
|
+
`minitest`, or `rspec`. `sources` and `tests` replace their corresponding
|
|
126
|
+
defaults, while `exclude` removes matching source files before inventory. A
|
|
127
|
+
configuration file may live elsewhere when passed with `--config PATH`; its
|
|
128
|
+
patterns are still resolved from the project root. Use `--no-config` to disable
|
|
129
|
+
the default file. Command-line project, framework, source, and test selections
|
|
130
|
+
take precedence over the file. Configured `exclude` patterns are applied
|
|
131
|
+
before source inventory and are recorded in run metadata for comparison.
|
|
132
|
+
`--config` and `--no-config` cannot be used together. RSpec configuration
|
|
133
|
+
selectors continue to conflict with an explicit test selection, including one
|
|
134
|
+
supplied by this file.
|
|
135
|
+
|
|
136
|
+
The `minimum` values are validated percentages from 0 through 100 for the five
|
|
137
|
+
supported criteria: `decision`, `condition`, `condition_decision`, `mcdc`, and
|
|
138
|
+
`decision_table`. A policy is evaluated against the exact numerator and
|
|
139
|
+
denominator counts in the report, so a value such as `66.67` is not rounded
|
|
140
|
+
before it is compared. The policy is recorded in JSON under
|
|
141
|
+
`coverage_policy`.
|
|
142
|
+
|
|
143
|
+
The command line can add or replace individual policy entries with repeatable
|
|
144
|
+
`--minimum` options. Use `criterion=threshold`, for example:
|
|
145
|
+
|
|
146
|
+
```sh
|
|
147
|
+
bundle exec branchproof analyze 'lib/**/*.rb' --minimum mcdc=80 \
|
|
148
|
+
--minimum decision_table=75
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
CLI values take precedence over matching `minimum` keys from `.branchproof.json`.
|
|
152
|
+
Specifying the same criterion more than once on the command line is an error.
|
|
153
|
+
The five criteria are evaluated independently. A threshold met exactly passes;
|
|
154
|
+
a result below it fails. A requested gate with a zero denominator, unavailable
|
|
155
|
+
coverage, or incomplete observations or analysis is `unavailable`, and exits
|
|
156
|
+
with status 2. A below-threshold gate exits 1. A failed test run makes
|
|
157
|
+
requested policy gates unavailable and exits 2; without a policy, a failed test
|
|
158
|
+
run retains the existing exit status 1. With no policy, the existing report
|
|
159
|
+
exit behavior remains unchanged.
|
|
160
|
+
|
|
161
|
+
For a Minitest project, a focused configuration can select the library and
|
|
162
|
+
test trees directly:
|
|
163
|
+
|
|
164
|
+
```json
|
|
165
|
+
{ "schema_version": 1, "framework": "minitest",
|
|
166
|
+
"sources": ["lib/**/*.rb"], "tests": ["test/**/*_test.rb"] }
|
|
167
|
+
```
|
|
168
|
+
|
|
169
|
+
For an RSpec project, use its spec patterns instead:
|
|
170
|
+
|
|
171
|
+
```json
|
|
172
|
+
{ "schema_version": 1, "framework": "rspec",
|
|
173
|
+
"sources": ["app/**/*.rb"], "tests": ["spec/**/*_spec.rb"] }
|
|
174
|
+
```
|
|
175
|
+
|
|
110
176
|
Rails analysis boots the application inside the isolated worker after
|
|
111
177
|
Branchproof's loader and the selected framework hooks are installed. The
|
|
112
178
|
application's `rails_helper` owns requiring and configuring `rspec/rails` after
|
|
@@ -208,6 +274,15 @@ that did evaluate count toward Condition Coverage even when their value was
|
|
|
208
274
|
masked by another condition. Here, the two observations prove independence
|
|
209
275
|
for `logged_in?`; `admin?` still needs `[TF] => F`.
|
|
210
276
|
|
|
277
|
+
This is also a compact test-design walkthrough. A test that records
|
|
278
|
+
`[F-] => F` proves the false branch of `logged_in?`; a test that records
|
|
279
|
+
`[TT] => T` proves the true branch and establishes the current positive path.
|
|
280
|
+
Adding a test that records `[TF] => F` evaluates `admin?` false while
|
|
281
|
+
`logged_in?` stays true, so it supplies the missing independence witness and
|
|
282
|
+
improves the condition, Condition/Decision, MC/DC, and applicable decision-table
|
|
283
|
+
results. The report names the missing vector and its owning test when it is
|
|
284
|
+
observed.
|
|
285
|
+
|
|
211
286
|
Decision and Condition Coverage are calculated independently from the captured
|
|
212
287
|
evidence. Condition/Decision requires both; MC/DC adds independence evidence.
|
|
213
288
|
Decision Table Coverage is calculated independently of all of them: MC/DC asks
|
|
@@ -501,6 +576,34 @@ execution. JSON output always contains the complete evidence document, so an
|
|
|
501
576
|
explicit view cannot be combined with `--format json`. The `mcdc` executable
|
|
502
577
|
accepts the same arguments for existing scripts.
|
|
503
578
|
|
|
579
|
+
Terminal reports can be narrowed with `--focus PATH[:LINE]` and bounded with
|
|
580
|
+
`--top N`, where `N` is a positive integer. These options are accepted by
|
|
581
|
+
`analyze` and `report`, and are terminal-only. Focus matches the source spans
|
|
582
|
+
captured in the report; rendering does not reopen or need the original source
|
|
583
|
+
file. `--focus` and `--top` affect displayed detail only: the coverage summary,
|
|
584
|
+
policy gates, diagnostics, exit status, and global counts remain unchanged.
|
|
585
|
+
|
|
586
|
+
The default view limits decisions. In `--view decision-tables`, the units are
|
|
587
|
+
decision tables; in `--view conditions`, condition and alternative rows; and
|
|
588
|
+
in `--view tests`, test rows. Rows are ordered deterministically by project-
|
|
589
|
+
relative path and source line, with deterministic tie-breakers for each view:
|
|
590
|
+
decisions use column and stable ID; decision tables use decision ID; conditions
|
|
591
|
+
and alternatives use column, decision ID, and condition or alternative index;
|
|
592
|
+
tests use test name and ID. `--top` follows that source order; it does not rank
|
|
593
|
+
rows by risk or coverage. For example:
|
|
594
|
+
|
|
595
|
+
```sh
|
|
596
|
+
bundle exec branchproof analyze 'lib/**/*.rb' --focus lib/access.rb:12 --top 5
|
|
597
|
+
bundle exec branchproof report .branchproof/current.json --view conditions \
|
|
598
|
+
--focus lib/access.rb --top 10
|
|
599
|
+
```
|
|
600
|
+
|
|
601
|
+
Focus and top selections never create a new test run or alter policy scope.
|
|
602
|
+
They also do not generate tests. RSpec rerun labels remain the existing
|
|
603
|
+
recorded selectors and commands. The main finding rows respect the selection;
|
|
604
|
+
supplemental unexecuted, unattributed, and unsupported sections retain their
|
|
605
|
+
run-wide counts rather than expanding into all detail rows.
|
|
606
|
+
|
|
504
607
|
### Saved reports and offline comparison
|
|
505
608
|
|
|
506
609
|
Reports are saved only when requested. Create a local artifact directory and
|
|
@@ -518,13 +621,26 @@ The output's parent directory must already exist. Replacing a baseline is an
|
|
|
518
621
|
explicit `analyze --format json --output` operation; keep CI snapshots as
|
|
519
622
|
artifacts when you need to retain multiple runs.
|
|
520
623
|
|
|
521
|
-
The `report` command reads the saved document without loading the application
|
|
522
|
-
or running tests. It uses locations and metadata
|
|
523
|
-
rendering remains useful after the original
|
|
524
|
-
|
|
525
|
-
|
|
526
|
-
|
|
527
|
-
|
|
624
|
+
The `report` command reads the saved document without loading the application,
|
|
625
|
+
project configuration, or running tests. It uses locations and metadata
|
|
626
|
+
captured in the report, so rendering remains useful after the original
|
|
627
|
+
checkout has moved or been removed. A saved report's policy is inherited by
|
|
628
|
+
default. Repeatable `--minimum criterion=threshold` options override matching
|
|
629
|
+
saved policy entries for that rendering without changing the snapshot file;
|
|
630
|
+
other saved thresholds remain inherited. Offline rendering never consults the
|
|
631
|
+
current `.branchproof.json`.
|
|
632
|
+
|
|
633
|
+
New live JSON reports use schema `1.4` and include the required
|
|
634
|
+
`coverage_policy` object with the normalized requested minima and recomputed
|
|
635
|
+
gate results. The report validates those results from its exact coverage
|
|
636
|
+
counts rather than trusting a persisted percentage. Readers continue to accept
|
|
637
|
+
schemas `1.0` through `1.3`; an offline policy overlay is optional and
|
|
638
|
+
preserves the input snapshot's schema version, including for legacy reports.
|
|
639
|
+
New snapshots retain the ladder at every level. Legacy snapshots without
|
|
640
|
+
analysis can still be rendered at Level 1; levels 2 and 3 require analysis in
|
|
641
|
+
the saved report.
|
|
642
|
+
The repository ignores `.branchproof/`; choose a different path and CI artifact
|
|
643
|
+
policy when a project needs to retain reports.
|
|
528
644
|
Saved JSON includes existing raw metadata such as test names and expressions;
|
|
529
645
|
relative terminal labels do not mean every legacy JSON field is sanitized.
|
|
530
646
|
|
|
@@ -560,6 +676,29 @@ Invalid input or an incomplete comparison exits 2, which takes
|
|
|
560
676
|
precedence. Reports are explicit snapshots: comparison never creates history,
|
|
561
677
|
promotes a baseline, or overwrites either input.
|
|
562
678
|
|
|
679
|
+
### GitHub Actions artifacts
|
|
680
|
+
|
|
681
|
+
A coverage policy can fail a job while still preserving the report for review.
|
|
682
|
+
Create the output directory before invoking Branchproof and upload the report
|
|
683
|
+
with `if: always()` so failed gates and failed test runs leave an artifact:
|
|
684
|
+
|
|
685
|
+
```yaml
|
|
686
|
+
- name: Branchproof coverage
|
|
687
|
+
run: |
|
|
688
|
+
mkdir -p .branchproof
|
|
689
|
+
bundle exec branchproof analyze 'lib/**/*.rb' --test 'test/**/*_test.rb' \
|
|
690
|
+
--format json --output .branchproof/coverage.json \
|
|
691
|
+
--minimum mcdc=80 --minimum decision_table=75
|
|
692
|
+
|
|
693
|
+
- name: Upload Branchproof report
|
|
694
|
+
if: always()
|
|
695
|
+
uses: actions/upload-artifact@v4
|
|
696
|
+
with:
|
|
697
|
+
name: branchproof-coverage
|
|
698
|
+
path: .branchproof/coverage.json
|
|
699
|
+
if-no-files-found: warn
|
|
700
|
+
```
|
|
701
|
+
|
|
563
702
|
MC/DC has two related questions. Evaluation asks whether a condition was
|
|
564
703
|
observed with a value, including short-circuiting. Independent proof asks
|
|
565
704
|
whether the analyzer found a pair of observations where that condition changes
|
|
@@ -670,8 +809,8 @@ Ruby-defined custom `!` methods keep their runtime behavior;
|
|
|
670
809
|
evidence that contradicts Boolean negation is rejected instead of proving
|
|
671
810
|
coverage with an invalid logical model.
|
|
672
811
|
|
|
673
|
-
New reports use schema `1.
|
|
674
|
-
remain readable. Comparison distinguishes decision-table coverage movement
|
|
812
|
+
New reports use schema `1.4`; saved schema `1.0`, `1.1`, `1.2`, and `1.3`
|
|
813
|
+
reports remain readable. Comparison distinguishes decision-table coverage movement
|
|
675
814
|
(`rule coverage gained`, `rule coverage lost`) from analysis movement
|
|
676
815
|
(`rule reachability changed`), and treats a structurally changed decision as a
|
|
677
816
|
changed decision-table context instead of guessing which old rule a new rule
|
|
@@ -730,7 +869,11 @@ namespace. The CLI is the supported way to run a complete analysis;
|
|
|
730
869
|
child-environment policy,
|
|
731
870
|
and `Branchproof::RailsSupport` is the optional Rails boot boundary. The library
|
|
732
871
|
classes expose the source, runtime, evidence, analysis, and report contracts
|
|
733
|
-
for adapters and integrations.
|
|
872
|
+
for adapters and integrations. `Branchproof::CoveragePolicy#call` evaluates a
|
|
873
|
+
normalized `minimum` hash against a report document. `Report#coverage_policy`
|
|
874
|
+
returns the recomputed policy record, and `Report#coverage_policy_lines`
|
|
875
|
+
returns its terminal lines; both are available to integrations that need the
|
|
876
|
+
same gate result as the CLI.
|
|
734
877
|
|
|
735
878
|
## Development
|
|
736
879
|
|
data/lib/branchproof/cli.rb
CHANGED
|
@@ -77,7 +77,8 @@ module Branchproof
|
|
|
77
77
|
report = Report.new(inventory: inventory, evidence: value(baseline, :evidence) || snapshot,
|
|
78
78
|
analysis: analysis, minima: minima, baseline: baseline, diagnostics: diagnostics,
|
|
79
79
|
level: options[:level], missing_only: options[:missing_only], view: options[:view],
|
|
80
|
-
run_metadata: run_metadata(options, baseline)
|
|
80
|
+
run_metadata: run_metadata(options, baseline), minimum: options[:minimum],
|
|
81
|
+
focus: options[:focus], top: options[:top])
|
|
81
82
|
output_report(report, options)
|
|
82
83
|
report.exit_code
|
|
83
84
|
rescue ArgumentError => e
|
|
@@ -94,10 +95,13 @@ module Branchproof
|
|
|
94
95
|
@stdout.write(<<~HELP)
|
|
95
96
|
Usage:
|
|
96
97
|
branchproof analyze [SOURCE_GLOB ...] [--test TEST_GLOB] [--project auto|ruby|rails] [--framework auto|minitest|rspec]
|
|
97
|
-
[--view decisions|conditions|tests|decision-tables] [--level 1|2|3] [--missing-only]
|
|
98
|
-
[--
|
|
98
|
+
[--view decisions|conditions|tests|decision-tables] [--level 1|2|3] [--missing-only] [--minimum CRITERION=THRESHOLD]
|
|
99
|
+
[--focus PATH[:LINE]] [--top N]
|
|
100
|
+
[--format terminal|json] [--output PATH] [--limits PATH] [--config PATH|--no-config]
|
|
101
|
+
[--no-reachability] [-- RUNNER_ARGS]
|
|
99
102
|
branchproof report SNAPSHOT [--view decisions|conditions|tests|decision-tables]
|
|
100
|
-
[--level 1|2|3] [--missing-only] [--
|
|
103
|
+
[--level 1|2|3] [--missing-only] [--minimum CRITERION=THRESHOLD] [--focus PATH[:LINE]] [--top N]
|
|
104
|
+
[--format terminal|json] [--output PATH]
|
|
101
105
|
branchproof compare BEFORE AFTER [--format terminal|json] [--output PATH] [--fail-on-regression]
|
|
102
106
|
mcdc accepts the same commands as a compatibility alias.
|
|
103
107
|
JSON always contains full evidence; --view requires terminal output.
|
|
@@ -142,14 +146,15 @@ module Branchproof
|
|
|
142
146
|
end
|
|
143
147
|
|
|
144
148
|
report = Report.from_document(document: document, level: options[:level], view: options[:view],
|
|
145
|
-
missing_only: options[:missing_only]
|
|
149
|
+
missing_only: options[:missing_only], minimum: options[:minimum_overrides],
|
|
150
|
+
focus: options[:focus], top: options[:top])
|
|
146
151
|
output_report(report, options)
|
|
147
152
|
report.exit_code
|
|
148
153
|
end
|
|
149
154
|
end
|
|
150
155
|
|
|
151
156
|
def parse_offline(command, args)
|
|
152
|
-
options = { format: :terminal, view: :decisions, missing_only: false }
|
|
157
|
+
options = { format: :terminal, view: :decisions, missing_only: false, minimum_overrides: {} }
|
|
153
158
|
paths = []
|
|
154
159
|
until args.empty?
|
|
155
160
|
token = args.shift
|
|
@@ -166,7 +171,7 @@ module Branchproof
|
|
|
166
171
|
raise ArgumentError, "--fail-on-regression requires compare" unless command == "compare"
|
|
167
172
|
|
|
168
173
|
options[:fail_on_regression] = true
|
|
169
|
-
when "--view", "--level", "--missing-only"
|
|
174
|
+
when "--view", "--level", "--missing-only", "--minimum", "--focus", "--top"
|
|
170
175
|
raise ArgumentError, "#{token} requires report" unless command == "report"
|
|
171
176
|
|
|
172
177
|
case token
|
|
@@ -176,6 +181,14 @@ module Branchproof
|
|
|
176
181
|
when "--level"
|
|
177
182
|
options[:level] = Integer(args.shift.to_s, 10)
|
|
178
183
|
raise ArgumentError, "level must be 1, 2, or 3" unless (1..3).cover?(options[:level])
|
|
184
|
+
when "--minimum"
|
|
185
|
+
add_minimum_override!(options, args.shift)
|
|
186
|
+
when "--focus"
|
|
187
|
+
options[:focus] = args.shift
|
|
188
|
+
raise ArgumentError, "--focus requires PATH or PATH:LINE" if options[:focus].nil? || options[:focus].start_with?("-")
|
|
189
|
+
when "--top"
|
|
190
|
+
options[:top] = args.shift
|
|
191
|
+
raise ArgumentError, "--top requires a positive integer" if options[:top].nil?
|
|
179
192
|
else options[:missing_only] = true
|
|
180
193
|
end
|
|
181
194
|
else
|
|
@@ -188,6 +201,7 @@ module Branchproof
|
|
|
188
201
|
raise ArgumentError, "#{command} requires #{expected} saved report #{expected == 1 ? "path" : "paths"}" unless paths.length == expected
|
|
189
202
|
|
|
190
203
|
validate_view!(options)
|
|
204
|
+
validate_selection!(options)
|
|
191
205
|
[options, paths]
|
|
192
206
|
end
|
|
193
207
|
|
|
@@ -211,11 +225,18 @@ module Branchproof
|
|
|
211
225
|
metadata = {
|
|
212
226
|
captured_at: Time.now.utc.iso8601, requested_level: options[:level], project_kind: options[:project][:kind],
|
|
213
227
|
project_root: root, source_patterns: options[:source_patterns].map { |path| relative_path(path, root) },
|
|
228
|
+
exclude_patterns: Array(options[:exclude]).map { |path| relative_pattern(path, root) },
|
|
214
229
|
test_patterns: options[:test_patterns].map { |path| relative_path(path, root) },
|
|
215
230
|
test_files: Array(test_files).map { |path| relative_path(path, root) }, runner_args: options[:runner_args],
|
|
216
231
|
seed: value(baseline, :seed), limits: options[:limits], test_locations: locations,
|
|
217
232
|
reachability: options[:reachability]
|
|
218
233
|
}
|
|
234
|
+
if options[:configuration]
|
|
235
|
+
metadata[:excluded_files] = Array(options[:excluded_files]).map { |path| relative_path(path, root) }
|
|
236
|
+
metadata[:selected_source_files] = Array(options[:selected_source_files]).map do |path|
|
|
237
|
+
relative_path(path, root)
|
|
238
|
+
end
|
|
239
|
+
end
|
|
219
240
|
%i[selected_test_files selected_example_ids].each do |key|
|
|
220
241
|
metadata[key] = value(baseline, key) if value(baseline, key)
|
|
221
242
|
end
|
|
@@ -231,6 +252,10 @@ module Branchproof
|
|
|
231
252
|
Pathname.new(File.expand_path(path, root)).relative_path_from(Pathname.new(root)).to_s
|
|
232
253
|
end
|
|
233
254
|
|
|
255
|
+
def relative_pattern(path, root)
|
|
256
|
+
path.to_s.empty? ? path.to_s : relative_path(path, root)
|
|
257
|
+
end
|
|
258
|
+
|
|
234
259
|
def parse(argv)
|
|
235
260
|
return nil if argv.empty? || argv.first != "analyze"
|
|
236
261
|
|
|
@@ -240,7 +265,9 @@ module Branchproof
|
|
|
240
265
|
args = args[0...delimiter] if delimiter
|
|
241
266
|
options = { level: 3, format: :terminal, output: nil, tests: [], source_paths: [], limits: Limits.default,
|
|
242
267
|
runner_args: runner_args, project: nil, missing_only: false, view: :decisions,
|
|
243
|
-
reachability: true, project_mode: "auto", framework: "auto", explicit_tests: false
|
|
268
|
+
reachability: true, project_mode: "auto", framework: "auto", explicit_tests: false,
|
|
269
|
+
explicit_project: false, explicit_framework: false, explicit_sources: false,
|
|
270
|
+
config_path: nil, config_disabled: false, minimum_overrides: {} }
|
|
244
271
|
until args.empty?
|
|
245
272
|
token = args.shift
|
|
246
273
|
case token
|
|
@@ -249,6 +276,14 @@ module Branchproof
|
|
|
249
276
|
options[:explicit_view] = true
|
|
250
277
|
when "--missing-only"
|
|
251
278
|
options[:missing_only] = true
|
|
279
|
+
when "--minimum"
|
|
280
|
+
add_minimum_override!(options, args.shift)
|
|
281
|
+
when "--focus"
|
|
282
|
+
options[:focus] = args.shift
|
|
283
|
+
raise ArgumentError, "--focus requires PATH or PATH:LINE" if options[:focus].nil? || options[:focus].start_with?("-")
|
|
284
|
+
when "--top"
|
|
285
|
+
options[:top] = args.shift
|
|
286
|
+
raise ArgumentError, "--top requires a positive integer" if options[:top].nil?
|
|
252
287
|
when "--no-reachability"
|
|
253
288
|
options[:reachability] = false
|
|
254
289
|
when "--level"
|
|
@@ -272,9 +307,20 @@ module Branchproof
|
|
|
272
307
|
when "--framework"
|
|
273
308
|
options[:framework] = args.shift
|
|
274
309
|
raise ArgumentError, "--framework requires auto, minitest, or rspec" if options[:framework].nil? || options[:framework].empty?
|
|
310
|
+
|
|
311
|
+
options[:explicit_framework] = true
|
|
275
312
|
when "--project"
|
|
276
313
|
options[:project_mode] = args.shift
|
|
277
314
|
raise ArgumentError, "--project requires auto, ruby, or rails" if options[:project_mode].nil? || options[:project_mode].empty?
|
|
315
|
+
|
|
316
|
+
options[:explicit_project] = true
|
|
317
|
+
when "--config"
|
|
318
|
+
options[:config_path] = args.shift
|
|
319
|
+
raise ArgumentError, "--config requires a readable JSON path" if options[:config_path].nil? || options[:config_path].empty?
|
|
320
|
+
when "--no-config"
|
|
321
|
+
raise ArgumentError, "--config and --no-config are mutually exclusive" if options[:config_path]
|
|
322
|
+
|
|
323
|
+
options[:config_disabled] = true
|
|
278
324
|
when "--limits"
|
|
279
325
|
limits_path = args.shift
|
|
280
326
|
raise ArgumentError, "--limits requires a readable JSON path" unless limits_path && File.file?(limits_path)
|
|
@@ -286,10 +332,36 @@ module Branchproof
|
|
|
286
332
|
raise ArgumentError, "unknown option: #{token}" if token.start_with?("-")
|
|
287
333
|
|
|
288
334
|
options[:source_paths] << token
|
|
335
|
+
options[:explicit_sources] = true
|
|
336
|
+
end
|
|
337
|
+
end
|
|
338
|
+
root = Dir.pwd
|
|
339
|
+
if options[:config_path] && options[:config_disabled]
|
|
340
|
+
raise ArgumentError, "--config and --no-config are mutually exclusive"
|
|
341
|
+
end
|
|
342
|
+
|
|
343
|
+
configuration = Configuration.load(path: options[:config_path] || ".branchproof.json", root: root,
|
|
344
|
+
explicit: !options[:config_path].nil?, disabled: options[:config_disabled])
|
|
345
|
+
options[:configuration] = configuration
|
|
346
|
+
if configuration
|
|
347
|
+
options[:project_mode] = configuration[:project] if !options[:explicit_project] && configuration.key?(:project)
|
|
348
|
+
options[:framework] = configuration[:framework] if !options[:explicit_framework] && configuration.key?(:framework)
|
|
349
|
+
if !options[:explicit_sources] && configuration.key?(:sources)
|
|
350
|
+
options[:source_paths] = configuration[:sources].dup
|
|
351
|
+
end
|
|
352
|
+
if !options[:explicit_tests] && configuration.key?(:tests)
|
|
353
|
+
options[:tests] = configuration[:tests].dup
|
|
354
|
+
options[:explicit_tests] = true
|
|
289
355
|
end
|
|
356
|
+
options[:exclude] = Array(configuration[:exclude]).dup
|
|
357
|
+
options[:minimum] = configuration.fetch(:minimum, {}).dup.merge(options[:minimum_overrides])
|
|
358
|
+
else
|
|
359
|
+
options[:exclude] = []
|
|
360
|
+
options[:minimum] = options[:minimum_overrides].dup
|
|
290
361
|
end
|
|
291
|
-
options[:project] = Project.new(root:
|
|
362
|
+
options[:project] = Project.new(root: root, mode: options[:project_mode], framework: options[:framework]).to_h
|
|
292
363
|
validate_view!(options)
|
|
364
|
+
validate_selection!(options)
|
|
293
365
|
if options[:missing_only] && (options[:format] != :terminal || options[:level] == 1)
|
|
294
366
|
raise ArgumentError, "--missing-only requires terminal format and level 2 or 3"
|
|
295
367
|
end
|
|
@@ -302,7 +374,34 @@ module Branchproof
|
|
|
302
374
|
options
|
|
303
375
|
end
|
|
304
376
|
|
|
377
|
+
def add_minimum_override!(options, argument)
|
|
378
|
+
text = argument.to_s
|
|
379
|
+
match = text.match(/\A([a-z_]+)=([0-9]+(?:\.[0-9]+)?)\z/)
|
|
380
|
+
raise ArgumentError, "minimum must be CRITERION=THRESHOLD" unless match
|
|
381
|
+
|
|
382
|
+
criterion = match[1]
|
|
383
|
+
threshold_text = match[2]
|
|
384
|
+
threshold = threshold_text.include?(".") ? Float(threshold_text) : Integer(threshold_text, 10)
|
|
385
|
+
normalized = CoveragePolicy.normalize(criterion => threshold)
|
|
386
|
+
criterion = normalized.keys.first
|
|
387
|
+
raise ArgumentError, "duplicate coverage criterion: #{criterion}" if options[:minimum_overrides].key?(criterion)
|
|
388
|
+
|
|
389
|
+
options[:minimum_overrides][criterion] = normalized.fetch(criterion)
|
|
390
|
+
rescue ArgumentError
|
|
391
|
+
raise
|
|
392
|
+
rescue TypeError
|
|
393
|
+
raise ArgumentError, "minimum threshold must be a finite number from 0 to 100"
|
|
394
|
+
end
|
|
395
|
+
|
|
396
|
+
def validate_selection!(options)
|
|
397
|
+
return unless options[:focus] || options[:top]
|
|
398
|
+
raise ArgumentError, "focus and top filters are terminal-only" if options[:format] == :json
|
|
399
|
+
|
|
400
|
+
ReportSelection.new(focus: options[:focus], top: options[:top])
|
|
401
|
+
end
|
|
402
|
+
|
|
305
403
|
def build_inventory(options)
|
|
404
|
+
root = options[:project][:root]
|
|
306
405
|
test_paths = options[:tests].filter_map do |path|
|
|
307
406
|
File.realpath(path)
|
|
308
407
|
rescue StandardError
|
|
@@ -313,13 +412,21 @@ module Branchproof
|
|
|
313
412
|
rescue StandardError
|
|
314
413
|
nil
|
|
315
414
|
end
|
|
316
|
-
|
|
317
|
-
|
|
415
|
+
excluded = expand_paths(Array(options[:exclude]).reject(&:empty?), root: root)
|
|
416
|
+
excluded_paths = excluded.filter_map do |path|
|
|
417
|
+
File.realpath(path)
|
|
418
|
+
rescue StandardError
|
|
419
|
+
nil
|
|
420
|
+
end
|
|
421
|
+
selected = expand_paths(options[:source_paths], root: root).reject do |path|
|
|
422
|
+
relative = Pathname.new(path).relative_path_from(Pathname.new(root)).to_s
|
|
318
423
|
canonical = File.realpath(path)
|
|
319
424
|
relative.match?(%r{\A(?:test|spec|tool|vendor)(?:/|\z)}) ||
|
|
320
|
-
test_paths.include?(canonical) || loaded_paths.include?(canonical)
|
|
425
|
+
test_paths.include?(canonical) || loaded_paths.include?(canonical) || excluded_paths.include?(canonical)
|
|
321
426
|
end
|
|
322
|
-
|
|
427
|
+
options[:excluded_files] = excluded
|
|
428
|
+
options[:selected_source_files] = selected
|
|
429
|
+
Source.new(root: root, limits: options[:limits]).inventory(paths: selected)
|
|
323
430
|
end
|
|
324
431
|
|
|
325
432
|
def empty_evidence(inventory, options)
|
|
@@ -8,7 +8,7 @@ module Branchproof
|
|
|
8
8
|
# Compares two complete report documents without loading or executing the project.
|
|
9
9
|
class Comparison
|
|
10
10
|
SCHEMA_VERSION = "1.0"
|
|
11
|
-
SUPPORTED_REPORT_SCHEMAS = %w[1.0 1.1 1.2 1.3].freeze
|
|
11
|
+
SUPPORTED_REPORT_SCHEMAS = %w[1.0 1.1 1.2 1.3 1.4].freeze
|
|
12
12
|
CRITERION_VERSION = "masking_occurrence_v1"
|
|
13
13
|
|
|
14
14
|
def initialize(before:, after:)
|
|
@@ -481,6 +481,7 @@ module Branchproof
|
|
|
481
481
|
reasons << "#{key} differs" if !left.nil? && !right.nil? && left != right
|
|
482
482
|
end
|
|
483
483
|
reasons << "source selection differs" if source_selection(@before) != source_selection(@after)
|
|
484
|
+
reasons << "source exclusion scope differs" if exclusion_scope(@before) != exclusion_scope(@after)
|
|
484
485
|
reasons
|
|
485
486
|
end
|
|
486
487
|
|
|
@@ -517,6 +518,10 @@ module Branchproof
|
|
|
517
518
|
value(value(document, :run_metadata), :source_patterns)
|
|
518
519
|
end
|
|
519
520
|
|
|
521
|
+
def exclusion_scope(document)
|
|
522
|
+
Array(value(value(document, :run_metadata), :exclude_patterns))
|
|
523
|
+
end
|
|
524
|
+
|
|
520
525
|
def metadata_requirements
|
|
521
526
|
[@before, @after].each_with_index.filter_map do |document, index|
|
|
522
527
|
metadata = value(document, :run_metadata)
|
|
@@ -0,0 +1,111 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "json"
|
|
4
|
+
require_relative "coverage_policy"
|
|
5
|
+
|
|
6
|
+
module Branchproof
|
|
7
|
+
# Loads and validates the project-local .branchproof.json policy.
|
|
8
|
+
class Configuration
|
|
9
|
+
SCHEMA_VERSION = 1
|
|
10
|
+
PROJECTS = %w[auto ruby rails].freeze
|
|
11
|
+
FRAMEWORKS = %w[auto minitest rspec].freeze
|
|
12
|
+
MINIMUM_CRITERIA = CoveragePolicy::CRITERIA.keys.freeze
|
|
13
|
+
FIELDS = %w[schema_version project framework sources tests exclude minimum].freeze
|
|
14
|
+
|
|
15
|
+
class << self
|
|
16
|
+
def load(path:, root:, explicit: false, disabled: false)
|
|
17
|
+
return nil if disabled
|
|
18
|
+
|
|
19
|
+
root = File.expand_path(root)
|
|
20
|
+
path = File.expand_path(path, root)
|
|
21
|
+
return nil unless ensure_file!(path, explicit)
|
|
22
|
+
|
|
23
|
+
payload = read_payload(path)
|
|
24
|
+
validate(payload, path: path, root: root)
|
|
25
|
+
end
|
|
26
|
+
|
|
27
|
+
private
|
|
28
|
+
|
|
29
|
+
def ensure_file!(path, explicit)
|
|
30
|
+
unless File.exist?(path)
|
|
31
|
+
return nil unless explicit
|
|
32
|
+
|
|
33
|
+
raise ArgumentError, "configuration file does not exist: #{path}"
|
|
34
|
+
end
|
|
35
|
+
return true if File.file?(path)
|
|
36
|
+
|
|
37
|
+
raise ArgumentError, "configuration path is not a file: #{path}"
|
|
38
|
+
end
|
|
39
|
+
|
|
40
|
+
def read_payload(path)
|
|
41
|
+
JSON.parse(File.binread(path))
|
|
42
|
+
rescue JSON::ParserError => e
|
|
43
|
+
raise ArgumentError, "invalid JSON configuration: #{e.message}"
|
|
44
|
+
rescue SystemCallError => e
|
|
45
|
+
raise ArgumentError, "configuration could not be read: #{e.message}"
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
def validate(payload, path:, root:)
|
|
49
|
+
validate_shape!(payload)
|
|
50
|
+
validate_values!(payload)
|
|
51
|
+
|
|
52
|
+
payload.each_with_object({ schema_version: SCHEMA_VERSION, path: path, root: root }) do |(key, value), result|
|
|
53
|
+
next if key == "schema_version"
|
|
54
|
+
|
|
55
|
+
result[key.to_sym] = key == "minimum" ? value.transform_keys(&:to_s) : value
|
|
56
|
+
end
|
|
57
|
+
end
|
|
58
|
+
|
|
59
|
+
def validate_shape!(payload)
|
|
60
|
+
raise ArgumentError, "configuration must be a JSON object" unless payload.is_a?(Hash)
|
|
61
|
+
|
|
62
|
+
unknown = payload.keys - FIELDS
|
|
63
|
+
return if unknown.empty? && valid_schema_version?(payload["schema_version"])
|
|
64
|
+
|
|
65
|
+
raise ArgumentError, "configuration has unknown fields: #{unknown.join(", ")}" unless unknown.empty?
|
|
66
|
+
|
|
67
|
+
raise ArgumentError, "configuration schema_version must be #{SCHEMA_VERSION}"
|
|
68
|
+
end
|
|
69
|
+
|
|
70
|
+
def valid_schema_version?(value)
|
|
71
|
+
value.is_a?(Integer) && value == SCHEMA_VERSION
|
|
72
|
+
end
|
|
73
|
+
|
|
74
|
+
def validate_values!(payload)
|
|
75
|
+
validate_enum(payload, "project", PROJECTS)
|
|
76
|
+
validate_enum(payload, "framework", FRAMEWORKS)
|
|
77
|
+
%w[sources tests].each { |field| validate_nonempty_strings(payload, field) if payload.key?(field) }
|
|
78
|
+
validate_strings(payload, "exclude") if payload.key?("exclude")
|
|
79
|
+
validate_minimum(payload["minimum"]) if payload.key?("minimum")
|
|
80
|
+
end
|
|
81
|
+
|
|
82
|
+
def validate_enum(payload, field, values)
|
|
83
|
+
return unless payload.key?(field)
|
|
84
|
+
|
|
85
|
+
value = payload[field]
|
|
86
|
+
raise ArgumentError, "configuration #{field} must be a string" unless value.is_a?(String)
|
|
87
|
+
raise ArgumentError, "configuration #{field} must be #{values.join(", ")}" unless values.include?(value)
|
|
88
|
+
end
|
|
89
|
+
|
|
90
|
+
def validate_nonempty_strings(payload, field)
|
|
91
|
+
value = payload[field]
|
|
92
|
+
valid = value.is_a?(Array) && !value.empty? && value.all? { |item| item.is_a?(String) && !item.empty? }
|
|
93
|
+
raise ArgumentError, "configuration #{field} must be a nonempty array of strings" unless valid
|
|
94
|
+
end
|
|
95
|
+
|
|
96
|
+
def validate_strings(payload, field)
|
|
97
|
+
value = payload[field]
|
|
98
|
+
valid = value.is_a?(Array) && value.all? { |item| item.is_a?(String) && !item.empty? }
|
|
99
|
+
raise ArgumentError, "configuration #{field} must be an array of strings" unless valid
|
|
100
|
+
end
|
|
101
|
+
|
|
102
|
+
def validate_minimum(value)
|
|
103
|
+
raise ArgumentError, "configuration minimum must be an object" unless value.is_a?(Hash)
|
|
104
|
+
|
|
105
|
+
CoveragePolicy.normalize(value)
|
|
106
|
+
rescue ArgumentError => e
|
|
107
|
+
raise ArgumentError, "configuration minimum #{e.message.sub(/\Acoverage minimum for /, "")}"
|
|
108
|
+
end
|
|
109
|
+
end
|
|
110
|
+
end
|
|
111
|
+
end
|