quality_gate 0.2.0 → 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +4 -0
- data/README.md +37 -2
- data/lib/quality_gate/adapters/undercover.rb +30 -6
- data/lib/quality_gate/version.rb +1 -1
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 661dea9969d01c65c60ef01523cbdfc2ebae74be253bd3f78b5420b34aad934a
|
|
4
|
+
data.tar.gz: 6a34cb55d86a89885c635dd0c1b77d203c5432c4f85035b232510e23baa1aad2
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 0475a1a8e435e68effb9ea72bfbe3a82c69be9fb66e9c9e14c6cb983aec9354adb2754848a181dba35a128032e6131b1a3ed852defe5f838e1cb8d90589cc05a
|
|
7
|
+
data.tar.gz: 017605cf3e353c4e1861f03fd0e7fa2ac1ee1c6cedb6ba8fdfdbe99827bf1d186997b3ac794774f0259a23eb36b20a08f6a5957c8e67b7a43d65860755af4399
|
data/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,7 @@
|
|
|
1
|
+
## [0.2.1] - 2026-09-22
|
|
2
|
+
|
|
3
|
+
- Undercover reports stale coverage as an actionable `stale_coverage` finding with gate exit 1, telling operators to re-run the test suite before the coverage gate, instead of a parse failure with exit 2. Unknown validation reasons also fail closed and include their value; malformed output and CLI exit-status checks remain strict.
|
|
4
|
+
|
|
1
5
|
## [0.2.0] - 2026-09-21
|
|
2
6
|
|
|
3
7
|
- Gate commands accept `--format markdown` and `format: markdown`, printing a heading with the tally, a checks table, and a findings list for agents and pull request comments.
|
data/README.md
CHANGED
|
@@ -581,14 +581,49 @@ Clone the repository and install its dependencies:
|
|
|
581
581
|
bin/setup
|
|
582
582
|
```
|
|
583
583
|
|
|
584
|
-
Run the tests, lint checks, or complete default task:
|
|
584
|
+
Run the tests in two separate Ruby processes, lint checks, or complete serial default task:
|
|
585
585
|
|
|
586
586
|
```sh
|
|
587
|
-
bundle exec rake test
|
|
587
|
+
bundle exec rake test:parallel
|
|
588
588
|
bundle exec rubocop
|
|
589
589
|
bundle exec rake
|
|
590
590
|
```
|
|
591
591
|
|
|
592
|
+
`TEST_WORKERS=4 bundle exec rake test:parallel` changes the worker count.
|
|
593
|
+
`bundle exec rake test` preserves the serial runner and its Minitest filtering
|
|
594
|
+
options; `test:parallel` accepts the same `N`, `X`, and `A` filters. Named latency
|
|
595
|
+
checks (`QUALITY_GATE_ACCEPTANCE_TIMING=1`) always use one worker so concurrent
|
|
596
|
+
tests do not distort their measurements.
|
|
597
|
+
|
|
598
|
+
This repository's `quality_gate verify` uses the parallel task. Each worker writes
|
|
599
|
+
isolated coverage results; the parent merges them and generates reports only after
|
|
600
|
+
every worker succeeds and supplies coverage. `TEST_WORKERS=1 bundle exec quality_gate verify`
|
|
601
|
+
runs the same coverage pipeline with one worker. CI runs RuboCop separately and
|
|
602
|
+
lets verify run the full suite once per Ruby version.
|
|
603
|
+
|
|
604
|
+
To investigate test performance with [TestProf](https://github.com/test-prof/test-prof),
|
|
605
|
+
run the serial suite without coverage instrumentation:
|
|
606
|
+
|
|
607
|
+
```sh
|
|
608
|
+
QUALITY_GATE_PROFILE=1 COVERAGE=0 bundle exec rake test
|
|
609
|
+
```
|
|
610
|
+
|
|
611
|
+
This opt-in run prints a TagProf breakdown by test directory and an EventProf
|
|
612
|
+
report for `subprocess.quality_gate`, including the ten slowest suites and tests.
|
|
613
|
+
The event measures synchronous `Open3.capture2`, `capture2e`, and `capture3`
|
|
614
|
+
calls, including time waiting for child processes. It does not measure work
|
|
615
|
+
inside those processes or the adapter's separate `popen`/wait implementation.
|
|
616
|
+
The profiling switch is consumed before tests run so nested test projects keep
|
|
617
|
+
their normal output. Ordinary runs do not enable the profilers, and TestProf is
|
|
618
|
+
only a development/test dependency.
|
|
619
|
+
|
|
620
|
+
Use the serial task for a single report; parallel profiling produces a separate
|
|
621
|
+
report per worker. Existing Minitest filters still apply, for example:
|
|
622
|
+
|
|
623
|
+
```sh
|
|
624
|
+
QUALITY_GATE_PROFILE=1 COVERAGE=0 bundle exec rake test N=/test_clean_fixture/
|
|
625
|
+
```
|
|
626
|
+
|
|
592
627
|
The gem is linted by the configuration it ships, so the Sandi Metz budgets apply to its own code. Violations that predate those budgets are frozen file by file in `.rubocop_todo.yml`, which keeps the gate green while forcing new code to meet the budgets. That file is a ratchet: it may only shrink. When you change a file listed there, bring it under the budget and delete its entry; never add one. Regenerate it only after such a burn-down:
|
|
593
628
|
|
|
594
629
|
```sh
|
|
@@ -39,8 +39,7 @@ module QualityGate
|
|
|
39
39
|
|
|
40
40
|
stdout, stderr, status = capture_with_remaining_timeout(command)
|
|
41
41
|
validate_cli_envelope!(stdout, stderr, status)
|
|
42
|
-
findings = parse(stdout)
|
|
43
|
-
validate_cli_result!(status.exitstatus, findings)
|
|
42
|
+
findings = parse(stdout, exitstatus: status.exitstatus)
|
|
44
43
|
validate_findings!(findings, expected_tool: tool)
|
|
45
44
|
findings
|
|
46
45
|
rescue StandardError => e
|
|
@@ -49,11 +48,13 @@ module QualityGate
|
|
|
49
48
|
clear_call_budget
|
|
50
49
|
end
|
|
51
50
|
|
|
52
|
-
def parse(stdout)
|
|
51
|
+
def parse(stdout, exitstatus: nil)
|
|
53
52
|
document = parsed_document(stdout)
|
|
54
53
|
warnings, summary = report_parts(document)
|
|
55
54
|
findings = warnings.map { finding_from(_1) }
|
|
56
55
|
validate_summary!(summary, warnings)
|
|
56
|
+
validate_cli_result!(exitstatus, warnings)
|
|
57
|
+
findings << validation_finding(document["validation"]) unless document["validation"].nil?
|
|
57
58
|
findings
|
|
58
59
|
rescue JSON::ParserError, KeyError, TypeError, ArgumentError => e
|
|
59
60
|
raise ParseError.new(tool: name, reason: e.message)
|
|
@@ -97,8 +98,10 @@ module QualityGate
|
|
|
97
98
|
cli_contract_error!("Undercover output is missing its terminal timing footer")
|
|
98
99
|
end
|
|
99
100
|
|
|
100
|
-
def validate_cli_result!(exitstatus,
|
|
101
|
-
|
|
101
|
+
def validate_cli_result!(exitstatus, warnings)
|
|
102
|
+
return if exitstatus.nil?
|
|
103
|
+
|
|
104
|
+
expected_exitstatus = warnings.empty? ? 0 : 1
|
|
102
105
|
return if exitstatus == expected_exitstatus
|
|
103
106
|
|
|
104
107
|
cli_contract_error!("Undercover exit status #{exitstatus} disagrees with its warning count")
|
|
@@ -120,7 +123,6 @@ module QualityGate
|
|
|
120
123
|
summary = document.fetch("summary")
|
|
121
124
|
raise TypeError, "warnings must be an Array" unless warnings.is_a?(Array)
|
|
122
125
|
raise TypeError, "summary must be a mapping" unless summary.is_a?(Hash)
|
|
123
|
-
raise TypeError, "validation must be nil" unless document["validation"].nil?
|
|
124
126
|
|
|
125
127
|
[warnings, summary]
|
|
126
128
|
end
|
|
@@ -145,6 +147,28 @@ module QualityGate
|
|
|
145
147
|
)
|
|
146
148
|
end
|
|
147
149
|
|
|
150
|
+
def validation_finding(validation)
|
|
151
|
+
raise TypeError, "validation must be a String or nil" unless validation.is_a?(String)
|
|
152
|
+
|
|
153
|
+
Finding.new(
|
|
154
|
+
tool: name,
|
|
155
|
+
file: "",
|
|
156
|
+
line: 0,
|
|
157
|
+
rule: validation == "stale_coverage" ? "stale_coverage" : "undercover_validation",
|
|
158
|
+
severity: :error,
|
|
159
|
+
message: validation_message(validation)
|
|
160
|
+
)
|
|
161
|
+
end
|
|
162
|
+
|
|
163
|
+
def validation_message(validation)
|
|
164
|
+
if validation == "stale_coverage"
|
|
165
|
+
"Undercover coverage data predates the changed files. Re-run the test suite before the coverage gate."
|
|
166
|
+
else
|
|
167
|
+
"Undercover validation failed: #{validation.inspect}. Coverage cannot be trusted; " \
|
|
168
|
+
"resolve the validation issue and re-run the test suite before the coverage gate."
|
|
169
|
+
end
|
|
170
|
+
end
|
|
171
|
+
|
|
148
172
|
def skip_reason
|
|
149
173
|
return "Undercover skipped: repository is shallow; set compare_point explicitly." if shallow_repository?
|
|
150
174
|
return detached_reason if detached_head? && default_branch_present?
|
data/lib/quality_gate/version.rb
CHANGED