rail_verdict 1.0.0 → 1.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +478 -118
- data/lib/rail_verdict/analyzers/bundler_audit.rb +65 -1
- data/lib/rail_verdict/check.rb +10 -3
- data/lib/rail_verdict/configuration.rb +13 -2
- data/lib/rail_verdict/doctor.rb +1 -1
- data/lib/rail_verdict/schema_validator.rb +4 -1
- data/lib/rail_verdict/version.rb +1 -1
- data/schemas/configuration-v1.5.schema.json +149 -0
- metadata +2 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 62a7032233c6c05db93caede4d2fad10181a04aff006780106e92b474fe18195
|
|
4
|
+
data.tar.gz: 5a87a7f930f2bf519561404be6fbcec92c41c2596216beaf06ab0aa7a84fd01c
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: cbac4c960c4c1392d957ba96dc63776b7406a6da4f7f596735ea3635294dab6f99e94a5a99f022437bff51396354f8ba59ff4cab9f9abc7a5ce52ca0e60a4cf2
|
|
7
|
+
data.tar.gz: f83c03803134e56beb991545bb2dd1ddc8f43bb5beb821c2540161af623c34005389ccd04d4589944b738996442f30ab82be237a1ba9d832318760ca99d8a139
|
data/README.md
CHANGED
|
@@ -1,192 +1,552 @@
|
|
|
1
1
|
# RailVerdict
|
|
2
2
|
|
|
3
|
-
**
|
|
3
|
+
**Deterministic, offline, fail-closed merge verification for Ruby on Rails.**
|
|
4
4
|
|
|
5
5
|
[](https://rubygems.org/gems/rail_verdict)
|
|
6
6
|
[](LICENSE)
|
|
7
7
|
[](https://www.ruby-lang.org)
|
|
8
8
|
|
|
9
|
-
RailVerdict collects evidence from
|
|
9
|
+
RailVerdict is a deterministic verification framework for Ruby on Rails applications. It collects evidence from existing quality tools, normalizes it into stable findings, applies project policy and historical baselines, and produces a machine-readable `PASS` / `WARN` / `FAIL` decision that humans, CI systems, and AI coding agents can trust.
|
|
10
10
|
|
|
11
|
-
|
|
11
|
+
```
|
|
12
|
+
Evidence discovers.
|
|
13
|
+
Verification decides.
|
|
14
|
+
Intelligence explains.
|
|
15
|
+
Agents act.
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
> **One command. One gate.** No SaaS, no accounts, no telemetry, no hosted service. Core verification is 100% local and offline.
|
|
12
19
|
|
|
13
20
|
---
|
|
14
21
|
|
|
15
|
-
## Why RailVerdict
|
|
22
|
+
## Why RailVerdict Exists
|
|
16
23
|
|
|
17
|
-
|
|
24
|
+
In modern Rails development, source code is written and modified across multiple surfaces:
|
|
18
25
|
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
+
- **Software engineers** crafting features, migrations, and bug fixes;
|
|
27
|
+
- **Automated refactoring tools** updating syntax and framework deprecations;
|
|
28
|
+
- **AI coding agents** generating code, tests, and pull requests;
|
|
29
|
+
- **CI automation** running checks and test suites.
|
|
30
|
+
|
|
31
|
+
Running test suites alone is not enough to answer a fundamental question:
|
|
32
|
+
|
|
33
|
+
> **"Is this change safe to accept?"**
|
|
34
|
+
|
|
35
|
+
Existing tools produce fragmented formats and disparate semantics:
|
|
36
|
+
|
|
37
|
+
- **RuboCop** reports style and lint offenses;
|
|
38
|
+
- **RSpec** and **Minitest** report test outcomes and failures;
|
|
39
|
+
- **SimpleCov** reports line coverage metrics;
|
|
40
|
+
- **bundler-audit** reports gem dependency advisories;
|
|
41
|
+
- **Git** tracks what actually changed.
|
|
42
|
+
|
|
43
|
+
When AI coding agents enter this workflow, a new challenge appears: an AI model can explain findings and propose code changes, but **an AI model should not be the authority deciding whether code is safe to merge**.
|
|
44
|
+
|
|
45
|
+
RailVerdict solves this by establishing a **deterministic verification layer** between raw tooling and human engineers or coding agents. Given identical repository state, configuration, analyzer state, and baseline, RailVerdict produces the exact same evidence-backed `PASS`, `WARN`, or `FAIL` gate result — regardless of whether AI is enabled.
|
|
26
46
|
|
|
27
47
|
---
|
|
28
48
|
|
|
29
|
-
##
|
|
49
|
+
## Born from a Real Rails Application
|
|
50
|
+
|
|
51
|
+
RailVerdict was not created as an isolated demo or synthetic experiment. It originated from real-world engineering needs encountered while maintaining **IntegrarPlus**.
|
|
52
|
+
|
|
53
|
+
IntegrarPlus is a private Ruby on Rails platform developed for the operational needs of a multidisciplinary healthcare organization. It has grown continuously over time and contains multiple interconnected business modules and workflows. Development involves both traditional engineering and AI-assisted coding.
|
|
54
|
+
|
|
55
|
+
As the application grew, maintaining confidence across tests, authorization, security, regressions, code quality, and automated changes became increasingly important. RailVerdict emerged from the need for a reusable verification layer capable of giving both humans and coding agents a deterministic answer about the state of a change.
|
|
56
|
+
|
|
57
|
+
> **Information Firewall:** No IntegrarPlus private source code, database schemas, business rules, clinical workflows, or private data are included in RailVerdict. All public examples, tests, and validation fixtures are strictly synthetic. RailVerdict is a completely standalone, open-source verification framework reusable by any Rails application.
|
|
58
|
+
|
|
59
|
+
---
|
|
60
|
+
|
|
61
|
+
## Tested Beyond Unit Tests: RailVerdict Lab
|
|
62
|
+
|
|
63
|
+
Validation repository: [pedro-dalben/railverdict-lab](https://github.com/pedro-dalben/railverdict-lab)
|
|
64
|
+
|
|
65
|
+
To ensure RailVerdict works reliably in real-world conditions, it is continuously tested against **RailVerdict Lab** — an independent, public Rails application designed to exercise RailVerdict as an external consumer rather than testing only internal classes.
|
|
66
|
+
|
|
67
|
+
### Validation Scope
|
|
68
|
+
|
|
69
|
+
The Lab exercises RailVerdict across realistic operational scenarios:
|
|
70
|
+
|
|
71
|
+
- Installing and running RailVerdict as an external gem dependency in a real Rails codebase;
|
|
72
|
+
- Introducing controlled regressions to verify expected `PASS`, `FAIL`, and `INCOMPLETE` gate behavior;
|
|
73
|
+
- Validating Git-aware changed-scope verification (`--changed --base`);
|
|
74
|
+
- Testing analyzer availability, timeout, and failure modes;
|
|
75
|
+
- Validating CLI commands, structured JSON, and SARIF output;
|
|
76
|
+
- Validating Model Context Protocol (MCP) tool execution;
|
|
77
|
+
- Validating repair packet generation and repair verification;
|
|
78
|
+
- Exercising fail-closed behavior on missing or malformed analyzer output;
|
|
79
|
+
- Testing release artifact installation from clean environments.
|
|
80
|
+
|
|
81
|
+
### 21 / 21 External Validation Campaign
|
|
82
|
+
|
|
83
|
+
The 1.0 release closeout validated **21 out of 21** external scenarios:
|
|
84
|
+
|
|
85
|
+
| Category | Scenarios | Result |
|
|
86
|
+
|---|---:|:---:|
|
|
87
|
+
| **Core Verification** | 12 / 12 | PASS |
|
|
88
|
+
| **Operational & CI** | 6 / 6 | PASS |
|
|
89
|
+
| **Release Closeout** | 3 / 3 | PASS |
|
|
90
|
+
| **Total** | **21 / 21** | **PASS** |
|
|
91
|
+
|
|
92
|
+
These controlled validation scenarios represent rigorous external regression verification rather than a claim of mathematically bug-free software. During development, the Lab uncovered real defects — such as test failure scoping edge cases — which were fixed before the 1.0 release.
|
|
93
|
+
|
|
94
|
+
---
|
|
95
|
+
|
|
96
|
+
## The Verification Pipeline
|
|
97
|
+
|
|
98
|
+
```
|
|
99
|
+
+-------------------------------------------------------------+
|
|
100
|
+
| Rails Project |
|
|
101
|
+
+-------------------------------------------------------------+
|
|
102
|
+
|
|
|
103
|
+
v
|
|
104
|
+
+-------------------------------------------------------------+
|
|
105
|
+
| 1. Evidence Layer (Analyzers) |
|
|
106
|
+
| RuboCop · Minitest · RSpec · SimpleCov · bundler-audit |
|
|
107
|
+
+-------------------------------------------------------------+
|
|
108
|
+
|
|
|
109
|
+
v
|
|
110
|
+
+-------------------------------------------------------------+
|
|
111
|
+
| Normalized Findings (v1) |
|
|
112
|
+
| Analyzer-independent · Fingerprinted · Stable |
|
|
113
|
+
+-------------------------------------------------------------+
|
|
114
|
+
|
|
|
115
|
+
v
|
|
116
|
+
+-------------------------------------------------------------+
|
|
117
|
+
| 2. Verification Core (Policy) |
|
|
118
|
+
| Baseline (no-new-debt) · Waivers · Policy Evaluation |
|
|
119
|
+
+-------------------------------------------------------------+
|
|
120
|
+
|
|
|
121
|
+
v
|
|
122
|
+
+-------------------------------------------------------------+
|
|
123
|
+
| Deterministic Gate: PASS / WARN / FAIL |
|
|
124
|
+
| (or INCOMPLETE / exit 2) |
|
|
125
|
+
+-------------------------------------------------------------+
|
|
126
|
+
| | |
|
|
127
|
+
v v v
|
|
128
|
+
+---------------+ +---------------+ +---------------+
|
|
129
|
+
| Outputs | | 3. Advisory AI| | 4. Agent Loop |
|
|
130
|
+
| Console, JSON,| | Explain & | | RepairPackets,|
|
|
131
|
+
| SARIF, CI exit| | Investigate | | MCP server, |
|
|
132
|
+
| (0, 1, 2) | | (opt-in only)| | coding agents |
|
|
133
|
+
+---------------+ +---------------+ +---------------+
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
- **Analyzers provide evidence.**
|
|
137
|
+
- **Policy owns the decision.**
|
|
138
|
+
- **AI never controls the deterministic GateResult.**
|
|
139
|
+
|
|
140
|
+
---
|
|
141
|
+
|
|
142
|
+
## Installation
|
|
143
|
+
|
|
144
|
+
Install the gem directly:
|
|
30
145
|
|
|
31
146
|
```bash
|
|
32
147
|
gem install rail_verdict
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
Or add it to your application's `Gemfile`:
|
|
33
151
|
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
railverdict check # full verification → console (or INCOMPLETE baseline_required)
|
|
39
|
-
railverdict baseline create # atomic baseline from a complete run (requires PASS/WARN/FAIL, not INCOMPLETE)
|
|
40
|
-
railverdict check --format json # versioned JSON on stdout, diagnostics on stderr
|
|
41
|
-
railverdict check --changed --base main # changed-scope gate from deterministic Git diff
|
|
42
|
-
railverdict findings --format json # projection of normalized findings
|
|
152
|
+
```ruby
|
|
153
|
+
group :development, :test do
|
|
154
|
+
gem "rail_verdict", require: false
|
|
155
|
+
end
|
|
43
156
|
```
|
|
44
157
|
|
|
45
|
-
|
|
158
|
+
Then install dependencies:
|
|
46
159
|
|
|
47
|
-
|
|
160
|
+
```bash
|
|
161
|
+
bundle install
|
|
162
|
+
```
|
|
48
163
|
|
|
49
164
|
---
|
|
50
165
|
|
|
51
|
-
##
|
|
166
|
+
## Quick Start
|
|
52
167
|
|
|
53
|
-
|
|
168
|
+
You can verify your Rails repository with RailVerdict in under five minutes:
|
|
54
169
|
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
| **Minitest** | RailVerdict-owned reporter → `minitest-reporter-v1` JSON | `>= 5, < 7` |
|
|
59
|
-
| **RSpec** | `--format json` | `>= 3.13, < 4` |
|
|
60
|
-
| **SimpleCov** | Public `coverage/coverage.json` v1 (never `.resultset.json`/HTML) | `>= 1, < 2` |
|
|
61
|
-
| **bundler-audit** | `check --format json` (never `update`) | `>= 0.9.3, < 1` |
|
|
170
|
+
```bash
|
|
171
|
+
# 1. Initialize default configuration (.railverdict.yml)
|
|
172
|
+
railverdict init
|
|
62
173
|
|
|
63
|
-
|
|
174
|
+
# 2. Check configuration and analyzer readiness
|
|
175
|
+
railverdict doctor
|
|
64
176
|
|
|
65
|
-
|
|
177
|
+
# 3. Create an initial baseline for existing technical debt
|
|
178
|
+
railverdict baseline create
|
|
179
|
+
|
|
180
|
+
# 4. Run verification
|
|
181
|
+
railverdict check
|
|
182
|
+
|
|
183
|
+
# 5. Inspect normalized findings
|
|
184
|
+
railverdict findings
|
|
185
|
+
```
|
|
186
|
+
|
|
187
|
+
### Example: A Passing Gate
|
|
188
|
+
|
|
189
|
+
When all required analyzers succeed and no new debt is introduced:
|
|
190
|
+
|
|
191
|
+
```
|
|
192
|
+
RailVerdict Verification: PASS
|
|
193
|
+
Policy: no_new_debt (complete)
|
|
194
|
+
Analyzers: 5 run (5 complete)
|
|
195
|
+
Findings: 0 introduced, 14 existing (baseline), 0 blocking
|
|
196
|
+
Exit: 0
|
|
197
|
+
```
|
|
198
|
+
|
|
199
|
+
### Example: A Policy Failure
|
|
66
200
|
|
|
67
|
-
|
|
201
|
+
When a change introduces a new offense or test failure:
|
|
68
202
|
|
|
69
203
|
```
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
Comparison (introduced / existing / resolved / changed / moved · rename-aware)
|
|
77
|
-
│
|
|
78
|
-
▼
|
|
79
|
-
Policy (advisory · no_new_debt · strict) ──► GateResult (immutable)
|
|
80
|
-
│ │
|
|
81
|
-
▼ ▼
|
|
82
|
-
Console / JSON / SARIF / GitHub Annotations exit code
|
|
204
|
+
RailVerdict Verification: FAIL
|
|
205
|
+
Policy: no_new_debt (failed)
|
|
206
|
+
Analyzers: 5 run (5 complete)
|
|
207
|
+
Findings: 1 introduced (blocking), 14 existing (baseline)
|
|
208
|
+
- [rubocop] Lint/UselessAssignment in app/models/user.rb:42 (introduced)
|
|
209
|
+
Exit: 1
|
|
83
210
|
```
|
|
84
211
|
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
-
|
|
89
|
-
|
|
212
|
+
### Analyzer timeouts
|
|
213
|
+
|
|
214
|
+
Analyzer processes have a 30-second timeout by default. For a legitimately
|
|
215
|
+
long-running analyzer, use configuration schema `version: 1.5` and set the
|
|
216
|
+
timeout on that analyzer only:
|
|
217
|
+
|
|
218
|
+
```yaml
|
|
219
|
+
version: 1.5
|
|
220
|
+
mode: strict
|
|
221
|
+
analyzers:
|
|
222
|
+
rubocop:
|
|
223
|
+
enabled: true
|
|
224
|
+
required: true
|
|
225
|
+
rspec:
|
|
226
|
+
enabled: true
|
|
227
|
+
required: true
|
|
228
|
+
timeout_seconds: 600
|
|
229
|
+
```
|
|
230
|
+
|
|
231
|
+
`timeout_seconds` must be an integer from 1 through 3600. An analyzer without
|
|
232
|
+
an explicit value, including every analyzer in older configuration versions,
|
|
233
|
+
continues to use 30 seconds. A timeout is incomplete evidence, never a normal
|
|
234
|
+
finding: a required timeout produces `INCOMPLETE` and exit code `2`.
|
|
235
|
+
|
|
236
|
+
RailVerdict 1.0.1 has no CLI timeout override; the versioned configuration is
|
|
237
|
+
the supported public surface. SimpleCov accepts the same setting for a uniform
|
|
238
|
+
configuration contract, but reads a local coverage artifact rather than
|
|
239
|
+
starting an analyzer process.
|
|
240
|
+
|
|
241
|
+
---
|
|
242
|
+
|
|
243
|
+
## The Default Policy Model: No New Debt
|
|
244
|
+
|
|
245
|
+
RailVerdict supports three policy modes: `no_new_debt` (default), `strict`, and `advisory`.
|
|
246
|
+
|
|
247
|
+
### Why "No New Debt" Matters
|
|
248
|
+
|
|
249
|
+
Large, mature Rails applications often contain existing technical debt: legacy style offenses, pending test skips, or partial test coverage. Requiring teams to fix all historical issues before adopting verification creates an impossible barrier.
|
|
250
|
+
|
|
251
|
+
RailVerdict's `no_new_debt` mode solves this by separating **historical debt** from **new changes**:
|
|
252
|
+
|
|
253
|
+
> **"Existing debt is known and recorded. New debt is blocked."**
|
|
254
|
+
|
|
255
|
+
### How Baselines Work
|
|
256
|
+
|
|
257
|
+
1. **Atomic Creation:** `railverdict baseline create` runs a complete verification and atomically records SHA-256 fingerprints of current findings into `.railverdict-baseline.json`.
|
|
258
|
+
2. **Comparison:** Subsequent runs classify findings into `introduced`, `existing`, `resolved`, `changed`, or `moved`.
|
|
259
|
+
3. **Selective Enforcement:** Only `introduced` (new) findings block the merge gate in `no_new_debt` mode.
|
|
260
|
+
4. **Read-Only Verification:** `railverdict check` is strictly read-only and never mutates baselines.
|
|
261
|
+
5. **Waivers:** Time-bounded, exact-fingerprint exemptions can be documented with owners and UTC expiration dates without silencing evidence.
|
|
90
262
|
|
|
91
263
|
---
|
|
92
264
|
|
|
93
|
-
## Changed
|
|
265
|
+
## Changed-Scope Verification
|
|
266
|
+
|
|
267
|
+
For pull requests, CI builds, and coding agents, RailVerdict supports incremental, Git-aware verification:
|
|
94
268
|
|
|
95
269
|
```bash
|
|
96
|
-
|
|
97
|
-
railverdict check --changed --base
|
|
270
|
+
# Verify only changes against the main branch
|
|
271
|
+
railverdict check --changed --base origin/main
|
|
272
|
+
|
|
273
|
+
# Verify against a local Git revision
|
|
274
|
+
railverdict check --changed --base HEAD~1
|
|
275
|
+
|
|
276
|
+
# Output versioned JSON for machine consumers
|
|
277
|
+
railverdict check --changed --base origin/main --format json
|
|
98
278
|
```
|
|
99
279
|
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
-
|
|
280
|
+
### Key Capabilities
|
|
281
|
+
|
|
282
|
+
- **Merge-Base Resolution:** Computes the exact `merge-base` between `HEAD` and the target branch;
|
|
283
|
+
- **NUL-Safe Diff Parsing:** Accurately handles renames, deletions, and binary files;
|
|
284
|
+
- **Changed-Line Coverage:** Evaluates whether newly added or modified executable lines are covered by tests;
|
|
285
|
+
- **Fail-Closed Git Boundary:** If the base revision is missing or repository history is shallow, RailVerdict returns `INCOMPLETE` (exit code 2) rather than guessing or silently passing.
|
|
286
|
+
|
|
287
|
+
---
|
|
288
|
+
|
|
289
|
+
## Supported Analyzers
|
|
290
|
+
|
|
291
|
+
All analyzers in RailVerdict are **external and owned by the target project**. RailVerdict invokes existing executables via argument arrays (`argv`) without shell interpolation and never silently installs or vendors third-party packages.
|
|
292
|
+
|
|
293
|
+
| Analyzer | Purpose | Supported Versions | How RailVerdict Consumes It |
|
|
294
|
+
|---|---|---|---|
|
|
295
|
+
| **RuboCop** (+ `rubocop-rails`) | Style, linting, Rails conventions | RuboCop `>= 1.72, < 2`<br>`rubocop-rails >= 2, < 3` | Runs `bundle exec rubocop --format json`; captures plugin versions and configuration digest. |
|
|
296
|
+
| **Minitest** | Unit and integration tests | `>= 5, < 7` | Consumes test results via RailVerdict's owned JSON reporter (`minitest-reporter-v1`). |
|
|
297
|
+
| **RSpec** | Unit and integration specs | `>= 3.13, < 4` | Consumes test results via standard `--format json`. |
|
|
298
|
+
| **SimpleCov** | Code and changed-line coverage | `>= 1, < 2` | Ingests versioned public `coverage/coverage.json` v1 (never parses internal `.resultset.json`). |
|
|
299
|
+
| **bundler-audit** | Gem dependency vulnerabilities | `>= 0.9.3, < 1` | Runs `bundle exec bundler-audit check --format json` (never runs automatic updates). Robustly extracts JSON when advisory-DB download notices precede the payload. |
|
|
103
300
|
|
|
104
|
-
|
|
301
|
+
> **Brakeman Status:** Brakeman support is **not included** in 1.0 (on HOLD pending legal and licensing review). Third-party analyzers retain their respective upstream licenses.
|
|
105
302
|
|
|
106
303
|
---
|
|
107
304
|
|
|
108
|
-
##
|
|
305
|
+
## Outputs, CI, and Exit Codes
|
|
306
|
+
|
|
307
|
+
RailVerdict produces structured output for both human developers and automated systems.
|
|
308
|
+
|
|
309
|
+
### Output Formats
|
|
109
310
|
|
|
110
|
-
|
|
311
|
+
- **Console (`--format console`):** Human-readable terminal output (default);
|
|
312
|
+
- **JSON (`--format json`):** Single-document, canonical JSON complying with [`result-v1.schema.json`](schemas/result-v1.schema.json) on `stdout`, diagnostics on `stderr`;
|
|
313
|
+
- **SARIF (`--format sarif`):** Standard SARIF 2.1.0 projection for GitHub Code Scanning and IDEs.
|
|
111
314
|
|
|
112
|
-
|
|
315
|
+
### Exit Code Contract
|
|
113
316
|
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
317
|
+
| Exit Code | Meaning | Gate Status |
|
|
318
|
+
|---:|---|---|
|
|
319
|
+
| **`0`** | Verification succeeded without blocking issues. | `PASS` or non-blocking `WARN` |
|
|
320
|
+
| **`1`** | Policy violation detected (new offenses, failing tests, etc.). | `FAIL` |
|
|
321
|
+
| **`2`** | Incomplete evidence, tool failure, missing base, or configuration error. | `INCOMPLETE` |
|
|
322
|
+
| **`130`** | Execution interrupted by user (`SIGINT`). | `INTERRUPTED` |
|
|
120
323
|
|
|
121
|
-
|
|
324
|
+
### Why Exit Code 2 is Vital
|
|
122
325
|
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
326
|
+
If a required analyzer is missing, times out, crashes, or produces malformed output, that run must **never** be interpreted as "zero offenses" or a `PASS`. RailVerdict fails closed: incomplete required evidence produces `INCOMPLETE` and exits with code `2`, preventing broken pipelines from silently passing.
|
|
327
|
+
|
|
328
|
+
### GitHub Actions Integration
|
|
329
|
+
|
|
330
|
+
```yaml
|
|
331
|
+
name: Verification
|
|
332
|
+
on:
|
|
333
|
+
pull_request:
|
|
334
|
+
types: [opened, synchronize, reopened]
|
|
335
|
+
|
|
336
|
+
permissions:
|
|
337
|
+
contents: read
|
|
338
|
+
|
|
339
|
+
jobs:
|
|
340
|
+
railverdict:
|
|
341
|
+
runs-on: ubuntu-latest
|
|
342
|
+
steps:
|
|
343
|
+
- uses: actions/checkout@v4
|
|
344
|
+
with:
|
|
345
|
+
fetch-depth: 0
|
|
346
|
+
|
|
347
|
+
- uses: ruby/setup-ruby@v1
|
|
348
|
+
with:
|
|
349
|
+
bundler-cache: true
|
|
350
|
+
|
|
351
|
+
- name: Run RailVerdict
|
|
352
|
+
run: bundle exec railverdict check --changed --base ${{ github.event.pull_request.base.sha }}
|
|
129
353
|
```
|
|
130
354
|
|
|
355
|
+
See [`docs/github-actions.md`](docs/github-actions.md) for full CI configuration examples.
|
|
356
|
+
|
|
131
357
|
---
|
|
132
358
|
|
|
133
|
-
##
|
|
359
|
+
## Optional AI Intelligence (Advisory Only)
|
|
134
360
|
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
base: main # fallback for --changed when --base is not passed
|
|
361
|
+
RailVerdict includes an optional intelligence layer designed to assist with finding explanations and triage:
|
|
362
|
+
|
|
363
|
+
```bash
|
|
364
|
+
# Explain a specific finding
|
|
365
|
+
railverdict explain <finding-id|fingerprint>
|
|
366
|
+
|
|
367
|
+
# Preview context that would be sent (no network)
|
|
368
|
+
railverdict explain <finding-id> --preview-context
|
|
369
|
+
|
|
370
|
+
# Investigate top blocking findings
|
|
371
|
+
railverdict investigate --limit 3
|
|
147
372
|
```
|
|
148
373
|
|
|
149
|
-
Strict
|
|
374
|
+
### Strict Advisory Boundaries
|
|
375
|
+
|
|
376
|
+
- **Opt-in Only:** AI is disabled by default. Enabling remote AI requires explicit configuration (`ai.enabled: true` and `ai.remote.enabled: true`).
|
|
377
|
+
- **Zero Gate Authority:** AI **never** controls `GateResult`. It cannot change a `FAIL` to a `PASS`, override project policy, or bypass checks.
|
|
378
|
+
- **Privacy & Redaction:** Sensitive tokens, credentials, and private patterns are automatically scrubbed (`trust: redacted` by default).
|
|
379
|
+
- **Offline Core:** All deterministic verification functions work 100% offline without network access or AI configuration.
|
|
380
|
+
|
|
381
|
+
See [`docs/ai.md`](docs/ai.md) and [`docs/privacy.md`](docs/privacy.md) for details.
|
|
150
382
|
|
|
151
383
|
---
|
|
152
384
|
|
|
153
|
-
##
|
|
385
|
+
## Coding Agents & The Repair Loop
|
|
154
386
|
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
387
|
+
RailVerdict provides a structured verification loop for AI coding agents (such as Claude, Codex, or custom agents):
|
|
388
|
+
|
|
389
|
+
```
|
|
390
|
+
Agent modifies code
|
|
391
|
+
│
|
|
392
|
+
▼
|
|
393
|
+
railverdict check (returns FAIL / exit 1)
|
|
394
|
+
│
|
|
395
|
+
▼
|
|
396
|
+
railverdict repair <finding-ref>
|
|
397
|
+
│
|
|
398
|
+
▼
|
|
399
|
+
RepairPacket (bounded, structured context)
|
|
400
|
+
│
|
|
401
|
+
▼
|
|
402
|
+
Agent applies fix
|
|
403
|
+
│
|
|
404
|
+
▼
|
|
405
|
+
railverdict check (Verifier classifies outcome)
|
|
406
|
+
```
|
|
407
|
+
|
|
408
|
+
### RepairPacket v1
|
|
409
|
+
|
|
410
|
+
When an agent needs to fix a finding, `railverdict repair <finding-ref>` generates a deterministic `RepairPacket v1`:
|
|
411
|
+
|
|
412
|
+
- **Bounded Context:** Capped at 256 KiB with strict snippet and diff limits;
|
|
413
|
+
- **Trust Boundary:** Delimits `TRUSTED_RAILVERDICT_INSTRUCTIONS` from untrusted repository text;
|
|
414
|
+
- **Secret Redacted:** Scans and removes sensitive values before emission;
|
|
415
|
+
- **Outcome Classification:** `RailVerdict::Repair::Verifier` classifies repair outcomes as `fixed`, `still_present`, `changed`, `moved`, `regressed`, or `incomplete`.
|
|
416
|
+
|
|
417
|
+
RailVerdict never directly mutates application code; it provides the verifiable contract that allows external agents to repair code safely. See [`docs/repair-workflow.md`](docs/repair-workflow.md).
|
|
170
418
|
|
|
171
419
|
---
|
|
172
420
|
|
|
173
|
-
##
|
|
421
|
+
## Model Context Protocol (MCP)
|
|
422
|
+
|
|
423
|
+
RailVerdict includes a native [Model Context Protocol (MCP)](https://modelcontextprotocol.io) stdio server, allowing coding agents to interact with verification as structured tools:
|
|
174
424
|
|
|
175
425
|
```bash
|
|
176
|
-
|
|
177
|
-
bundle exec rake test # synthetic fixtures only, deterministic
|
|
178
|
-
bundle exec rubocop
|
|
426
|
+
railverdict mcp serve
|
|
179
427
|
```
|
|
180
428
|
|
|
181
|
-
|
|
182
|
-
|
|
429
|
+
### Implemented MCP Tools
|
|
430
|
+
|
|
431
|
+
| Tool | Purpose |
|
|
432
|
+
|---|---|
|
|
433
|
+
| `verify` | Executes verification (`full` or `changed`) and returns canonical `GateResult`. |
|
|
434
|
+
| `list_findings` | Lists, filters, and paginates normalized findings. |
|
|
435
|
+
| `get_finding` | Retrieves detailed finding data, evidence references, and context. |
|
|
436
|
+
| `build_repair_packet` | Builds a bounded `RepairPacket v1` for a specific finding. |
|
|
437
|
+
| `verify_repair` | Reruns verification and classifies whether a repair succeeded or regressed. |
|
|
438
|
+
| `explain` | Generates an advisory AI explanation for a finding. |
|
|
439
|
+
| `investigate` | Investigates top blocking findings across the codebase. |
|
|
440
|
+
|
|
441
|
+
### MCP Security Properties
|
|
442
|
+
|
|
443
|
+
- **Stdio Transport:** Operates over standard I/O with no open ports or background daemons;
|
|
444
|
+
- **Repository Containment:** Paths are verified and strictly contained within the repository root;
|
|
445
|
+
- **Read-Only Tools:** All tools are marked `readOnlyHint: true`; RailVerdict verifies while external agents edit;
|
|
446
|
+
- **Mutex Serialization:** Verification runs are serialized to prevent concurrent execution conflicts.
|
|
447
|
+
|
|
448
|
+
See [`docs/mcp.md`](docs/mcp.md) for configuration examples.
|
|
183
449
|
|
|
184
450
|
---
|
|
185
451
|
|
|
186
|
-
##
|
|
452
|
+
## Rails-Aware Context (Without Booting Rails)
|
|
187
453
|
|
|
188
|
-
|
|
454
|
+
When verifying changes (`--changed`), RailVerdict enriches results with **bounded Rails-aware context** without booting the Rails application, loading ActiveRecord, or executing `routes.rb` and `schema.rb`:
|
|
455
|
+
|
|
456
|
+
- **Related Tests:** Identifies candidate test and spec files for modified models and controllers;
|
|
457
|
+
- **Policies:** Locates corresponding Pundit/ActionPolicy authorization policies;
|
|
458
|
+
- **Views:** Identifies associated view templates;
|
|
459
|
+
- **Routes & Schema:** Extracts literal route definitions and schema table fragments.
|
|
460
|
+
|
|
461
|
+
Each context element includes an explicit confidence rating (`exact`, `conventional`, `inferred`, `unresolved`) and provenance. Context extraction failures degrade gracefully without causing gate failures. See [`docs/rails-context.md`](docs/rails-context.md).
|
|
462
|
+
|
|
463
|
+
---
|
|
464
|
+
|
|
465
|
+
## Security & Fail-Closed Architecture
|
|
466
|
+
|
|
467
|
+
RailVerdict is engineered around a comprehensive threat model documented in [`SECURITY.md`](SECURITY.md):
|
|
468
|
+
|
|
469
|
+
- **Fail-Closed Gate:** Incomplete, missing, or malformed evidence cannot produce a `PASS`;
|
|
470
|
+
- **Safe Subprocess Execution:** Invokes executables exclusively via argument arrays (`argv`), eliminating shell-injection vulnerabilities;
|
|
471
|
+
- **Resource Bounds:** Monotonic timeouts, output limits, and process-group signal termination (`SIGTERM`/`SIGKILL`) prevent resource exhaustion;
|
|
472
|
+
- **Path Containment:** Normalizes paths and rejects directory traversal or symlink escapes;
|
|
473
|
+
- **Information Firewall:** Strict public provenance controls prevent internal data or private patterns from entering artifacts;
|
|
474
|
+
- **Subprocess Limitation:** Subprocess containment provides argument and resource isolation; it is not an operating system kernel sandbox.
|
|
475
|
+
|
|
476
|
+
---
|
|
477
|
+
|
|
478
|
+
## Architecture Overview
|
|
479
|
+
|
|
480
|
+
RailVerdict is structured into four distinct conceptual layers:
|
|
481
|
+
|
|
482
|
+
1. **Evidence Layer:** Executes external quality tools safely and captures facts;
|
|
483
|
+
2. **Verification Core:** Normalizes findings, manages fingerprints and baselines, and evaluates policy to produce an immutable `GateResult`;
|
|
484
|
+
3. **Intelligence Layer:** Provides optional, advisory explanations without gate authority;
|
|
485
|
+
4. **Agent Layer:** Exposes deterministic interfaces (CLI, JSON, SARIF, RepairPackets, MCP) for human engineers and coding agents.
|
|
486
|
+
|
|
487
|
+
```
|
|
488
|
+
Evidence discovers.
|
|
489
|
+
Verification decides.
|
|
490
|
+
Intelligence explains.
|
|
491
|
+
Agents act.
|
|
492
|
+
```
|
|
493
|
+
|
|
494
|
+
For detailed architectural principles and component designs, see [`ARCHITECTURE.md`](ARCHITECTURE.md), [`PROJECT.md`](PROJECT.md), and [`PHILOSOPHY.md`](PHILOSOPHY.md).
|
|
495
|
+
|
|
496
|
+
---
|
|
497
|
+
|
|
498
|
+
## What RailVerdict Is NOT
|
|
499
|
+
|
|
500
|
+
To maintain clear technical boundaries, RailVerdict is explicitly **NOT**:
|
|
501
|
+
|
|
502
|
+
- **NOT a replacement for test frameworks or linters:** It does not replace RSpec, Minitest, or RuboCop; it normalizes and verifies their evidence.
|
|
503
|
+
- **NOT an AI code reviewer that decides safety:** AI cannot override or determine the gate decision.
|
|
504
|
+
- **NOT a hosted SaaS or cloud dashboard:** There are no user accounts, billing, hosted control planes, or telemetry.
|
|
505
|
+
- **NOT an auto-fixing bot:** It does not autonomously commit or rewrite application source code.
|
|
506
|
+
- **NOT a full semantic code graph:** Rails relationships are bounded, heuristic, and labeled with confidence rather than deep AST graphs.
|
|
507
|
+
|
|
508
|
+
---
|
|
509
|
+
|
|
510
|
+
## Project Status
|
|
511
|
+
|
|
512
|
+
- **Release Version:** `1.0.1`
|
|
513
|
+
- **License:** [MIT](LICENSE) (see [NOTICE](NOTICE))
|
|
189
514
|
- **Trademarks:** [TRADEMARKS.md](TRADEMARKS.md)
|
|
190
|
-
- **Foundation:** [docs/foundation.md](docs/foundation.md) —
|
|
191
|
-
- **
|
|
192
|
-
- **
|
|
515
|
+
- **Foundation & Legal:** [docs/foundation.md](docs/foundation.md) — preliminary screen found no obvious software/tool conflict; NOT LEGAL CLEARANCE; qualified trademark review NOT PERFORMED — NON-BLOCKING BY MAINTAINER DECISION 2026-08-19 (Pedro Dalben).
|
|
516
|
+
- **Ruby Compatibility:** `>= 3.3` (tested on Ruby 3.3, 3.4, and 4.0)
|
|
517
|
+
- **Target Rails Context:** `>= 8.0`
|
|
518
|
+
- **Supply Chain:** Released via RubyGems Trusted Publishing and GitHub OIDC.
|
|
519
|
+
|
|
520
|
+
---
|
|
521
|
+
|
|
522
|
+
## Documentation Index
|
|
523
|
+
|
|
524
|
+
| Topic | Primary Documents |
|
|
525
|
+
|---|---|
|
|
526
|
+
| **Product & Philosophy** | [`PROJECT.md`](PROJECT.md) · [`PHILOSOPHY.md`](PHILOSOPHY.md) · [`ARCHITECTURE.md`](ARCHITECTURE.md) · [`ROADMAP.md`](ROADMAP.md) |
|
|
527
|
+
| **Contracts & Schemas** | [`docs/contracts.md`](docs/contracts.md) · [`schemas/finding-v1.schema.json`](schemas/finding-v1.schema.json) · [`schemas/configuration-v1.schema.json`](schemas/configuration-v1.schema.json) · [`schemas/result-v1.schema.json`](schemas/result-v1.schema.json) |
|
|
528
|
+
| **Examples** | [`examples/finding-v1.json`](examples/finding-v1.json) · [`examples/configuration-v1.yml`](examples/configuration-v1.yml) · [`examples/configuration-v1.5.yml`](examples/configuration-v1.5.yml) · [`examples/result-v1.json`](examples/result-v1.json) |
|
|
529
|
+
| **Analyzers & Baselines** | [`docs/analyzers.md`](docs/analyzers.md) · [`docs/baselines.md`](docs/baselines.md) |
|
|
530
|
+
| **CI, SARIF & Git Scope** | [`docs/github-actions.md`](docs/github-actions.md) |
|
|
531
|
+
| **Rails-Aware Context** | [`docs/rails-context.md`](docs/rails-context.md) |
|
|
532
|
+
| **Agent Protocols & Repair** | [`docs/mcp.md`](docs/mcp.md) · [`docs/repair-workflow.md`](docs/repair-workflow.md) |
|
|
533
|
+
| **AI & Privacy** | [`docs/ai.md`](docs/ai.md) · [`docs/privacy.md`](docs/privacy.md) |
|
|
534
|
+
| **Security & Firewall** | [`SECURITY.md`](SECURITY.md) · [`docs/foundation.md`](docs/foundation.md) |
|
|
535
|
+
| **Architecture Decisions** | [`ADRs`](docs/adr/) |
|
|
536
|
+
| **Legal & Trademarks** | [`LICENSE`](LICENSE) · [`NOTICE`](NOTICE) · [`TRADEMARKS.md`](TRADEMARKS.md) |
|
|
537
|
+
| **Changelog** | [`CHANGELOG.md`](CHANGELOG.md) |
|
|
538
|
+
| **Public Validation Lab** | [pedro-dalben/railverdict-lab](https://github.com/pedro-dalben/railverdict-lab) |
|
|
539
|
+
|
|
540
|
+
---
|
|
541
|
+
|
|
542
|
+
## Contributing & Issues
|
|
543
|
+
|
|
544
|
+
Contributions and issue reports are welcome. Please open an issue on GitHub for:
|
|
545
|
+
|
|
546
|
+
- Analyzer compatibility and version range feedback;
|
|
547
|
+
- False positives or false negatives in evidence normalization;
|
|
548
|
+
- Rails version or convention compatibility;
|
|
549
|
+
- CLI and MCP ergonomics and developer experience.
|
|
550
|
+
|
|
551
|
+
GitHub Repository: [https://github.com/pedro-dalben/RailVerdict](https://github.com/pedro-dalben/RailVerdict)
|
|
552
|
+
Issues: [https://github.com/pedro-dalben/RailVerdict/issues](https://github.com/pedro-dalben/RailVerdict/issues)
|
|
@@ -69,9 +69,12 @@ module RailVerdict
|
|
|
69
69
|
return [Shared.failure_result(analyzer_id: ANALYZER_ID, invocation: invocation, status: "truncated", message: Shared.detail_for(result), tool_version: tool_version), []] if Shared.truncated?(result)
|
|
70
70
|
return [Shared.failure_result(analyzer_id: ANALYZER_ID, invocation: invocation, status: "timed_out", message: Shared.detail_for(result), tool_version: tool_version), []] if result.status == :timed_out
|
|
71
71
|
return [Shared.failure_result(analyzer_id: ANALYZER_ID, invocation: invocation, status: "signaled", message: Shared.detail_for(result), tool_version: tool_version), []] if result.status == :signaled
|
|
72
|
+
if result.exit_code.nil? || result.exit_code > 1
|
|
73
|
+
return [Shared.failure_result(analyzer_id: ANALYZER_ID, invocation: invocation, status: "failed", message: Shared.execution_message(result), tool_version: tool_version), []]
|
|
74
|
+
end
|
|
72
75
|
|
|
73
76
|
begin
|
|
74
|
-
document =
|
|
77
|
+
document = parse_json_document(result.stdout)
|
|
75
78
|
rescue JSON::ParserError => error
|
|
76
79
|
return [Shared.failure_result(analyzer_id: ANALYZER_ID, invocation: invocation, status: "parse_failed", message: Shared.bounded_message(error.message), tool_version: tool_version), []]
|
|
77
80
|
end
|
|
@@ -171,6 +174,67 @@ module RailVerdict
|
|
|
171
174
|
raise MalformedOutput, "bundler-audit entry #{index} is malformed: #{error.message}"
|
|
172
175
|
end
|
|
173
176
|
|
|
177
|
+
def parse_json_document(stdout)
|
|
178
|
+
text = stdout.to_s
|
|
179
|
+
json_start = text.index(/[\[{]/)
|
|
180
|
+
last_error = nil
|
|
181
|
+
|
|
182
|
+
while json_start
|
|
183
|
+
begin
|
|
184
|
+
json_end = json_document_end(text, json_start)
|
|
185
|
+
parsed = JSON.parse(text[json_start..json_end])
|
|
186
|
+
trailing = text[(json_end + 1)..].to_s.strip
|
|
187
|
+
unless trailing.empty?
|
|
188
|
+
raise JSON::ParserError, "trailing content after JSON document: #{trailing[0, 80]}"
|
|
189
|
+
end
|
|
190
|
+
|
|
191
|
+
return parsed
|
|
192
|
+
rescue JSON::ParserError => error
|
|
193
|
+
raise error if error.message.start_with?("trailing content")
|
|
194
|
+
|
|
195
|
+
last_error = error
|
|
196
|
+
json_start = text.index(/[\[{]/, json_start + 1)
|
|
197
|
+
end
|
|
198
|
+
end
|
|
199
|
+
|
|
200
|
+
raise(last_error || JSON::ParserError.new("no JSON document found"))
|
|
201
|
+
end
|
|
202
|
+
|
|
203
|
+
def json_document_end(text, start)
|
|
204
|
+
stack = []
|
|
205
|
+
in_string = false
|
|
206
|
+
escaped = false
|
|
207
|
+
|
|
208
|
+
text[start..].each_char.with_index do |character, offset|
|
|
209
|
+
index = start + offset
|
|
210
|
+
if in_string
|
|
211
|
+
if escaped
|
|
212
|
+
escaped = false
|
|
213
|
+
elsif character == "\\"
|
|
214
|
+
escaped = true
|
|
215
|
+
elsif character == '"'
|
|
216
|
+
in_string = false
|
|
217
|
+
end
|
|
218
|
+
next
|
|
219
|
+
end
|
|
220
|
+
|
|
221
|
+
case character
|
|
222
|
+
when '"'
|
|
223
|
+
in_string = true
|
|
224
|
+
when "{", "["
|
|
225
|
+
stack << character
|
|
226
|
+
when "}", "]"
|
|
227
|
+
expected = character == "}" ? "{" : "["
|
|
228
|
+
raise JSON::ParserError, "mismatched JSON delimiter" unless stack.last == expected
|
|
229
|
+
|
|
230
|
+
stack.pop
|
|
231
|
+
return index if stack.empty?
|
|
232
|
+
end
|
|
233
|
+
end
|
|
234
|
+
|
|
235
|
+
raise JSON::ParserError, "incomplete JSON document"
|
|
236
|
+
end
|
|
237
|
+
|
|
174
238
|
def map_severity(raw)
|
|
175
239
|
case raw.to_s.downcase
|
|
176
240
|
when "critical" then "critical"
|
data/lib/rail_verdict/check.rb
CHANGED
|
@@ -26,7 +26,7 @@ module RailVerdict
|
|
|
26
26
|
configuration = Configuration.load(resolved_config)
|
|
27
27
|
return interrupted_outcome if interrupted&.call
|
|
28
28
|
|
|
29
|
-
probes = probe_enabled_analyzers(root, configuration, runner: runner, rubocop_command_resolver: rubocop_command_resolver,
|
|
29
|
+
probes = probe_enabled_analyzers(root, configuration, runner: runner, rubocop_command_resolver: rubocop_command_resolver, default_timeout_seconds: analyzer_timeout_seconds)
|
|
30
30
|
|
|
31
31
|
analyzer_versions = probes.transform_values(&:version)
|
|
32
32
|
|
|
@@ -94,11 +94,12 @@ module RailVerdict
|
|
|
94
94
|
|
|
95
95
|
adapter = build_adapter(name, rubocop_command_resolver)
|
|
96
96
|
probe = probes[name]
|
|
97
|
+
timeout_seconds = resolve_timeout_seconds(configuration, name, analyzer_timeout_seconds)
|
|
97
98
|
analyzer_result, analyzer_findings = adapter.run(
|
|
98
99
|
root,
|
|
99
100
|
runner: runner,
|
|
100
101
|
probe_result: probe,
|
|
101
|
-
timeout_seconds:
|
|
102
|
+
timeout_seconds: timeout_seconds
|
|
102
103
|
)
|
|
103
104
|
analyzer_results << analyzer_result
|
|
104
105
|
findings.concat(analyzer_findings)
|
|
@@ -395,7 +396,7 @@ module RailVerdict
|
|
|
395
396
|
end
|
|
396
397
|
private_class_method :summary_to_coverage_document
|
|
397
398
|
|
|
398
|
-
def probe_enabled_analyzers(root, configuration, runner:, rubocop_command_resolver:,
|
|
399
|
+
def probe_enabled_analyzers(root, configuration, runner:, rubocop_command_resolver:, default_timeout_seconds:)
|
|
399
400
|
probes = {}
|
|
400
401
|
configuration.analyzers.each do |name, selection|
|
|
401
402
|
next unless selection.fetch("enabled")
|
|
@@ -406,10 +407,16 @@ module RailVerdict
|
|
|
406
407
|
adapter = build_adapter(name, rubocop_command_resolver)
|
|
407
408
|
next unless adapter
|
|
408
409
|
|
|
410
|
+
timeout_seconds = resolve_timeout_seconds(configuration, name, default_timeout_seconds)
|
|
409
411
|
probes[name] = adapter.probe(root, runner: runner, timeout_seconds: timeout_seconds)
|
|
410
412
|
end
|
|
411
413
|
probes
|
|
412
414
|
end
|
|
413
415
|
private_class_method :probe_enabled_analyzers
|
|
416
|
+
|
|
417
|
+
def resolve_timeout_seconds(configuration, analyzer_name, default_timeout_seconds)
|
|
418
|
+
configuration.analyzer_timeout_seconds(analyzer_name.to_s) || default_timeout_seconds
|
|
419
|
+
end
|
|
420
|
+
private_class_method :resolve_timeout_seconds
|
|
414
421
|
end
|
|
415
422
|
end
|
|
@@ -5,8 +5,9 @@ require "digest"
|
|
|
5
5
|
module RailVerdict
|
|
6
6
|
class Configuration
|
|
7
7
|
DEFAULT_FILENAME = ".railverdict.yml"
|
|
8
|
-
CONFIGURATION_VERSION = 1.
|
|
9
|
-
SUPPORTED_VERSIONS = [1, 1.1, 1.2, 1.3, 1.4].freeze
|
|
8
|
+
CONFIGURATION_VERSION = 1.5
|
|
9
|
+
SUPPORTED_VERSIONS = [1, 1.1, 1.2, 1.3, 1.4, 1.5].freeze
|
|
10
|
+
DEFAULT_ANALYZER_TIMEOUT_SECONDS = 30
|
|
10
11
|
UTF8_BOM = "\xEF\xBB\xBF".b
|
|
11
12
|
|
|
12
13
|
attr_reader :version, :mode, :analyzers, :source_path, :digest
|
|
@@ -112,6 +113,16 @@ module RailVerdict
|
|
|
112
113
|
@raw_data&.dig("waivers", "path")
|
|
113
114
|
end
|
|
114
115
|
|
|
116
|
+
def analyzer_timeout_seconds(name)
|
|
117
|
+
selection = @analyzers[name.to_s]
|
|
118
|
+
return DEFAULT_ANALYZER_TIMEOUT_SECONDS unless selection
|
|
119
|
+
|
|
120
|
+
value = selection["timeout_seconds"]
|
|
121
|
+
return DEFAULT_ANALYZER_TIMEOUT_SECONDS if value.nil?
|
|
122
|
+
|
|
123
|
+
Integer(value)
|
|
124
|
+
end
|
|
125
|
+
|
|
115
126
|
def git_base
|
|
116
127
|
@raw_data&.dig("git", "base")
|
|
117
128
|
end
|
data/lib/rail_verdict/doctor.rb
CHANGED
|
@@ -19,7 +19,7 @@ module RailVerdict
|
|
|
19
19
|
adapter = build_adapter(name, rubocop_command_resolver)
|
|
20
20
|
next unless adapter
|
|
21
21
|
|
|
22
|
-
probes[name] = adapter.probe(root, runner: runner)
|
|
22
|
+
probes[name] = adapter.probe(root, runner: runner, timeout_seconds: configuration.analyzer_timeout_seconds(name))
|
|
23
23
|
end
|
|
24
24
|
analyzer_versions = probes.transform_values(&:version)
|
|
25
25
|
context = RunContext.build(
|
|
@@ -10,6 +10,7 @@ module RailVerdict
|
|
|
10
10
|
CONFIGURATION_V12_SCHEMA = "configuration-v1.2.schema.json"
|
|
11
11
|
CONFIGURATION_V13_SCHEMA = "configuration-v1.3.schema.json"
|
|
12
12
|
CONFIGURATION_V14_SCHEMA = "configuration-v1.4.schema.json"
|
|
13
|
+
CONFIGURATION_V15_SCHEMA = "configuration-v1.5.schema.json"
|
|
13
14
|
AI_ANALYSIS_SCHEMA = "ai-analysis-v1.schema.json"
|
|
14
15
|
FINDING_SCHEMA = "finding-v1.schema.json"
|
|
15
16
|
RESULT_SCHEMA = "result-v1.schema.json"
|
|
@@ -23,7 +24,9 @@ module RailVerdict
|
|
|
23
24
|
end
|
|
24
25
|
|
|
25
26
|
def self.validate_configuration(data)
|
|
26
|
-
schema_name = if data.is_a?(Hash) && data["version"] == 1.
|
|
27
|
+
schema_name = if data.is_a?(Hash) && data["version"] == 1.5
|
|
28
|
+
CONFIGURATION_V15_SCHEMA
|
|
29
|
+
elsif data.is_a?(Hash) && data["version"] == 1.4
|
|
27
30
|
CONFIGURATION_V14_SCHEMA
|
|
28
31
|
elsif data.is_a?(Hash) && data["version"] == 1.3
|
|
29
32
|
CONFIGURATION_V13_SCHEMA
|
data/lib/rail_verdict/version.rb
CHANGED
|
@@ -0,0 +1,149 @@
|
|
|
1
|
+
{
|
|
2
|
+
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
|
3
|
+
"$id": "https://railverdict.dev/schemas/configuration/v1.5.schema.json",
|
|
4
|
+
"title": "RailVerdict Parsed Configuration v1.5",
|
|
5
|
+
"description": "Versioned contract for .railverdict.yml with per-analyzer timeout_seconds.",
|
|
6
|
+
"type": "object",
|
|
7
|
+
"required": ["version", "mode", "analyzers"],
|
|
8
|
+
"properties": {
|
|
9
|
+
"version": { "const": 1.5 },
|
|
10
|
+
"mode": { "enum": ["advisory", "no_new_debt", "strict"] },
|
|
11
|
+
"analyzers": { "$ref": "#/$defs/analyzers" },
|
|
12
|
+
"baseline": { "$ref": "#/$defs/baselineConfig" },
|
|
13
|
+
"waivers": { "$ref": "#/$defs/waiversConfig" },
|
|
14
|
+
"git": { "$ref": "#/$defs/gitConfig" },
|
|
15
|
+
"ai": { "$ref": "#/$defs/aiConfig" }
|
|
16
|
+
},
|
|
17
|
+
"additionalProperties": false,
|
|
18
|
+
"unevaluatedProperties": false,
|
|
19
|
+
"$defs": {
|
|
20
|
+
"analyzers": {
|
|
21
|
+
"type": "object",
|
|
22
|
+
"required": ["rubocop"],
|
|
23
|
+
"properties": {
|
|
24
|
+
"rubocop": { "$ref": "#/$defs/analyzerSelection" },
|
|
25
|
+
"minitest": { "$ref": "#/$defs/analyzerSelection" },
|
|
26
|
+
"rspec": { "$ref": "#/$defs/analyzerSelection" },
|
|
27
|
+
"simplecov": { "$ref": "#/$defs/simplecovSelection" },
|
|
28
|
+
"bundler_audit": { "$ref": "#/$defs/bundlerAuditSelection" }
|
|
29
|
+
},
|
|
30
|
+
"additionalProperties": false,
|
|
31
|
+
"unevaluatedProperties": false
|
|
32
|
+
},
|
|
33
|
+
"analyzerSelection": {
|
|
34
|
+
"type": "object",
|
|
35
|
+
"required": ["enabled", "required"],
|
|
36
|
+
"properties": {
|
|
37
|
+
"enabled": { "type": "boolean" },
|
|
38
|
+
"required": { "type": "boolean" },
|
|
39
|
+
"timeout_seconds": { "type": "integer", "minimum": 1, "maximum": 3600 }
|
|
40
|
+
},
|
|
41
|
+
"additionalProperties": false,
|
|
42
|
+
"unevaluatedProperties": false,
|
|
43
|
+
"not": {
|
|
44
|
+
"required": ["enabled", "required"],
|
|
45
|
+
"properties": { "enabled": { "const": false }, "required": { "const": true } }
|
|
46
|
+
}
|
|
47
|
+
},
|
|
48
|
+
"simplecovSelection": {
|
|
49
|
+
"type": "object",
|
|
50
|
+
"required": ["enabled", "required"],
|
|
51
|
+
"properties": {
|
|
52
|
+
"enabled": { "type": "boolean" },
|
|
53
|
+
"required": { "type": "boolean" },
|
|
54
|
+
"coverage_path": { "type": "string", "minLength": 1, "maxLength": 1024 },
|
|
55
|
+
"freshness_window_seconds": { "type": "integer", "minimum": 1, "maximum": 2592000 },
|
|
56
|
+
"timeout_seconds": { "type": "integer", "minimum": 1, "maximum": 3600 }
|
|
57
|
+
},
|
|
58
|
+
"additionalProperties": false,
|
|
59
|
+
"unevaluatedProperties": false,
|
|
60
|
+
"not": {
|
|
61
|
+
"required": ["enabled", "required"],
|
|
62
|
+
"properties": { "enabled": { "const": false }, "required": { "const": true } }
|
|
63
|
+
}
|
|
64
|
+
},
|
|
65
|
+
"bundlerAuditSelection": {
|
|
66
|
+
"type": "object",
|
|
67
|
+
"required": ["enabled", "required"],
|
|
68
|
+
"properties": {
|
|
69
|
+
"enabled": { "type": "boolean" },
|
|
70
|
+
"required": { "type": "boolean" },
|
|
71
|
+
"timeout_seconds": { "type": "integer", "minimum": 1, "maximum": 3600 }
|
|
72
|
+
},
|
|
73
|
+
"additionalProperties": false,
|
|
74
|
+
"unevaluatedProperties": false,
|
|
75
|
+
"not": {
|
|
76
|
+
"required": ["enabled", "required"],
|
|
77
|
+
"properties": { "enabled": { "const": false }, "required": { "const": true } }
|
|
78
|
+
}
|
|
79
|
+
},
|
|
80
|
+
"baselineConfig": {
|
|
81
|
+
"type": "object",
|
|
82
|
+
"required": [],
|
|
83
|
+
"properties": { "path": { "type": "string", "minLength": 1, "maxLength": 1024 } },
|
|
84
|
+
"additionalProperties": false,
|
|
85
|
+
"unevaluatedProperties": false
|
|
86
|
+
},
|
|
87
|
+
"waiversConfig": {
|
|
88
|
+
"type": "object",
|
|
89
|
+
"required": [],
|
|
90
|
+
"properties": { "path": { "type": "string", "minLength": 1, "maxLength": 1024 } },
|
|
91
|
+
"additionalProperties": false,
|
|
92
|
+
"unevaluatedProperties": false
|
|
93
|
+
},
|
|
94
|
+
"gitConfig": {
|
|
95
|
+
"type": "object",
|
|
96
|
+
"required": [],
|
|
97
|
+
"properties": { "base": { "type": "string", "minLength": 1, "maxLength": 1024 } },
|
|
98
|
+
"additionalProperties": false,
|
|
99
|
+
"unevaluatedProperties": false
|
|
100
|
+
},
|
|
101
|
+
"aiConfig": {
|
|
102
|
+
"type": "object",
|
|
103
|
+
"required": [],
|
|
104
|
+
"properties": {
|
|
105
|
+
"enabled": { "type": "boolean" },
|
|
106
|
+
"mode": { "enum": ["off", "explain", "investigate"] },
|
|
107
|
+
"provider": { "type": "string", "minLength": 1, "maxLength": 64 },
|
|
108
|
+
"model": { "type": "string", "minLength": 1, "maxLength": 128 },
|
|
109
|
+
"remote": { "$ref": "#/$defs/aiRemoteConfig" },
|
|
110
|
+
"budgets": { "$ref": "#/$defs/aiBudgetsConfig" },
|
|
111
|
+
"cache": { "$ref": "#/$defs/aiCacheConfig" }
|
|
112
|
+
},
|
|
113
|
+
"additionalProperties": false,
|
|
114
|
+
"unevaluatedProperties": false
|
|
115
|
+
},
|
|
116
|
+
"aiRemoteConfig": {
|
|
117
|
+
"type": "object",
|
|
118
|
+
"required": [],
|
|
119
|
+
"properties": {
|
|
120
|
+
"enabled": { "type": "boolean" },
|
|
121
|
+
"trust": { "enum": ["redacted", "full"] },
|
|
122
|
+
"endpoint": { "type": "string", "minLength": 1, "maxLength": 1024 }
|
|
123
|
+
},
|
|
124
|
+
"additionalProperties": false,
|
|
125
|
+
"unevaluatedProperties": false
|
|
126
|
+
},
|
|
127
|
+
"aiBudgetsConfig": {
|
|
128
|
+
"type": "object",
|
|
129
|
+
"required": [],
|
|
130
|
+
"properties": {
|
|
131
|
+
"max_findings": { "type": "integer", "minimum": 1, "maximum": 10 },
|
|
132
|
+
"max_requests": { "type": "integer", "minimum": 1, "maximum": 10 },
|
|
133
|
+
"max_context_bytes": { "type": "integer", "minimum": 1024, "maximum": 262144 }
|
|
134
|
+
},
|
|
135
|
+
"additionalProperties": false,
|
|
136
|
+
"unevaluatedProperties": false
|
|
137
|
+
},
|
|
138
|
+
"aiCacheConfig": {
|
|
139
|
+
"type": "object",
|
|
140
|
+
"required": [],
|
|
141
|
+
"properties": {
|
|
142
|
+
"enabled": { "type": "boolean" },
|
|
143
|
+
"max_bytes": { "type": "integer", "minimum": 1024, "maximum": 104857600 }
|
|
144
|
+
},
|
|
145
|
+
"additionalProperties": false,
|
|
146
|
+
"unevaluatedProperties": false
|
|
147
|
+
}
|
|
148
|
+
}
|
|
149
|
+
}
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: rail_verdict
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 1.0.
|
|
4
|
+
version: 1.0.1
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Pedro Dalben
|
|
@@ -155,6 +155,7 @@ files:
|
|
|
155
155
|
- schemas/configuration-v1.2.schema.json
|
|
156
156
|
- schemas/configuration-v1.3.schema.json
|
|
157
157
|
- schemas/configuration-v1.4.schema.json
|
|
158
|
+
- schemas/configuration-v1.5.schema.json
|
|
158
159
|
- schemas/configuration-v1.schema.json
|
|
159
160
|
- schemas/coverage-v1.schema.json
|
|
160
161
|
- schemas/finding-v1.schema.json
|