rail_verdict 1.1.0 → 1.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +164 -15
- data/lib/rail_verdict/analyzers/_shared.rb +53 -3
- data/lib/rail_verdict/analyzers/bundler_audit.rb +3 -4
- data/lib/rail_verdict/analyzers/minitest.rb +3 -7
- data/lib/rail_verdict/analyzers/rspec.rb +22 -5
- data/lib/rail_verdict/analyzers/rubocop.rb +14 -6
- data/lib/rail_verdict/analyzers/simplecov.rb +197 -15
- data/lib/rail_verdict/baseline.rb +5 -1
- data/lib/rail_verdict/canonical_json.rb +31 -0
- data/lib/rail_verdict/check.rb +99 -7
- data/lib/rail_verdict/cli.rb +235 -1
- data/lib/rail_verdict/contracts/analyzer_result.rb +6 -2
- data/lib/rail_verdict/mcp/cache.rb +130 -163
- data/lib/rail_verdict/mcp/repository_root.rb +2 -8
- data/lib/rail_verdict/mcp/server.rb +5 -1
- data/lib/rail_verdict/mcp/tools/get_pr_intelligence.rb +101 -0
- data/lib/rail_verdict/mcp/tools/get_verification_receipt.rb +102 -0
- data/lib/rail_verdict/mcp/tools/verify.rb +31 -2
- data/lib/rail_verdict/path_safety.rb +39 -0
- data/lib/rail_verdict/pr_intelligence.rb +25 -0
- data/lib/rail_verdict/process_runner.rb +29 -12
- data/lib/rail_verdict/receipt.rb +482 -0
- data/lib/rail_verdict/repository_state.rb +305 -0
- data/lib/rail_verdict/schema_validator.rb +10 -0
- data/lib/rail_verdict/verification_environment.rb +220 -0
- data/lib/rail_verdict/verification_identity.rb +95 -0
- data/lib/rail_verdict/version.rb +1 -1
- data/lib/rail_verdict.rb +6 -0
- data/schemas/receipt-validation-v1.schema.json +47 -0
- data/schemas/verification-receipt-v1.schema.json +168 -0
- metadata +11 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 63845a29f84295397de20a23b72c7d6e0a9d6c969be35ac8081e2536b0a37cec
|
|
4
|
+
data.tar.gz: 6e778d530c6abe87db5997b9d5fe328ebd750ac1722ede75042195b5d98d92cf
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: beabe5b069a803549a6af55d03f16256e5ebe7e5ab6152672a7a1fa415a75827656108c01207860251fc8936fdc3bfb6397c1f53ca53f342da5ee913051d1c2a
|
|
7
|
+
data.tar.gz: 72e637013c3a4216a90fcb466c947b6b6e97bc9035166bb8b96b76b62579c76f1253eb765d1c56ac1e39dd82197489c3535a411fece8535c6cd340bd0e741ac9
|
data/README.md
CHANGED
|
@@ -78,18 +78,11 @@ The Lab exercises RailVerdict across realistic operational scenarios:
|
|
|
78
78
|
- Exercising fail-closed behavior on missing or malformed analyzer output;
|
|
79
79
|
- Testing release artifact installation from clean environments.
|
|
80
80
|
|
|
81
|
-
###
|
|
81
|
+
### External Validation Campaign
|
|
82
82
|
|
|
83
|
-
The 1.0
|
|
83
|
+
The Lab independently validates the published gem as an external consumer. 1.0 closeout validated **21 / 21** scenarios; **1.2.0** certification validates **81 scenarios** (14 categories, see Lab README) including Verification Receipts and PR Intelligence. See [railverdict-lab](https://github.com/pedro-dalben/railverdict-lab) for the canonical campaign and results.
|
|
84
84
|
|
|
85
|
-
|
|
86
|
-
|---|---:|:---:|
|
|
87
|
-
| **Core Verification** | 12 / 12 | PASS |
|
|
88
|
-
| **Operational & CI** | 6 / 6 | PASS |
|
|
89
|
-
| **Release Closeout** | 3 / 3 | PASS |
|
|
90
|
-
| **Total** | **21 / 21** | **PASS** |
|
|
91
|
-
|
|
92
|
-
These controlled validation scenarios represent rigorous external regression verification rather than a claim of mathematically bug-free software. During development, the Lab uncovered real defects — such as test failure scoping edge cases — which were fixed before the 1.0 release.
|
|
85
|
+
These controlled validation scenarios represent rigorous external regression verification rather than a claim of mathematically bug-free software. During development, the Lab uncovered real defects — such as test failure scoping edge cases and the 1.2 dogfooding hardening — which were fixed before release.
|
|
93
86
|
|
|
94
87
|
---
|
|
95
88
|
|
|
@@ -236,11 +229,73 @@ an explicit value, including every analyzer in older configuration versions,
|
|
|
236
229
|
continues to use 30 seconds. A timeout is incomplete evidence, never a normal
|
|
237
230
|
finding: a required timeout produces `INCOMPLETE` and exit code `2`.
|
|
238
231
|
|
|
239
|
-
|
|
232
|
+
There is no CLI timeout override; the versioned configuration (`version: 1.5`) is
|
|
240
233
|
the supported public surface. SimpleCov accepts the same setting for a uniform
|
|
241
234
|
configuration contract, but reads a local coverage artifact rather than
|
|
242
235
|
starting an analyzer process.
|
|
243
236
|
|
|
237
|
+
### 30-Second Demonstration (real output)
|
|
238
|
+
|
|
239
|
+
```console
|
|
240
|
+
$ railverdict check
|
|
241
|
+
|
|
242
|
+
RailVerdict Verification: PASS
|
|
243
|
+
Policy: no_new_debt (complete)
|
|
244
|
+
Analyzers: 5 run (5 complete)
|
|
245
|
+
Findings: 0 introduced, 0 existing, 0 blocking
|
|
246
|
+
Exit: 0
|
|
247
|
+
```
|
|
248
|
+
|
|
249
|
+
Introduce a controlled defect:
|
|
250
|
+
|
|
251
|
+
```ruby
|
|
252
|
+
# app/models/user.rb:42
|
|
253
|
+
unused = "oops" # Lint/UselessAssignment
|
|
254
|
+
```
|
|
255
|
+
|
|
256
|
+
```console
|
|
257
|
+
$ railverdict check
|
|
258
|
+
|
|
259
|
+
RailVerdict Verification: FAIL
|
|
260
|
+
Policy: no_new_debt (failed)
|
|
261
|
+
Findings: 1 introduced (blocking), 0 existing
|
|
262
|
+
- [rubocop] Lint/UselessAssignment in app/models/user.rb:42 (introduced)
|
|
263
|
+
Exit: 1
|
|
264
|
+
```
|
|
265
|
+
|
|
266
|
+
Fix it, rerun → `PASS`. See `docs/release/1.2-clean-room-rehearsal.md` for automated reproduction.
|
|
267
|
+
|
|
268
|
+
### Why not just CI?
|
|
269
|
+
|
|
270
|
+
CI answers **“did these jobs execute successfully?”** across fragmented outputs. RailVerdict answers **“given required evidence, repository state, baseline and policy, what is the deterministic verification decision for this change?”** — normalized findings, baseline-aware policy (`no_new_debt`), changed-scope (`--changed --base`), fail-closed `INCOMPLETE` on missing evidence, and machine contracts (JSON/SARIF/exits) plus Repository State Identity and Verification Receipts. It complements CI; the example in [GitHub Actions Integration](#github-actions-integration) runs RailVerdict inside CI.
|
|
271
|
+
|
|
272
|
+
### Why not just RSpec + RuboCop?
|
|
273
|
+
|
|
274
|
+
Those are the evidence. RailVerdict is the verifier:
|
|
275
|
+
|
|
276
|
+
```
|
|
277
|
+
RSpec says: 42 tests passed.
|
|
278
|
+
RuboCop says: 3 offenses.
|
|
279
|
+
SimpleCov says: 91.3% coverage.
|
|
280
|
+
Git says: these 7 lines changed.
|
|
281
|
+
RailVerdict says: FAIL — this change introduced a blocking finding.
|
|
282
|
+
```
|
|
283
|
+
|
|
284
|
+
They produce facts; policy owns the gate. `rubocop` + `rspec` + `minitest` + `simplecov` + `bundler-audit` + `Git` are inputs; `PASS/WARN/FAIL/INCOMPLETE` is the `GateResult` (see `docs/contracts.md`).
|
|
285
|
+
|
|
286
|
+
### Why this matters for AI agents
|
|
287
|
+
|
|
288
|
+
An AI agent can write code, run tests, and declare itself finished — but it should not be the authority deciding whether its own work is acceptable.
|
|
289
|
+
|
|
290
|
+
```
|
|
291
|
+
Agent --modifies--> Repository --verify--> RailVerdict
|
|
292
|
+
|--> deterministic evidence
|
|
293
|
+
|--> GateResult
|
|
294
|
+
`--> Verification Receipt
|
|
295
|
+
```
|
|
296
|
+
|
|
297
|
+
The agent proposes; the verification system decides. Receipts let agents (and humans) prove `fresh` vs `stale` after any edit — see below. AI inside RailVerdict (`explain`/`investigate`) is advisory only and never changes `GateResult`.
|
|
298
|
+
|
|
244
299
|
---
|
|
245
300
|
|
|
246
301
|
## The Default Policy Model: No New Debt
|
|
@@ -334,6 +389,38 @@ exit code `2`.
|
|
|
334
389
|
- **Changed-Line Coverage:** Evaluates whether newly added or modified executable lines are covered by tests;
|
|
335
390
|
- **Fail-Closed Git Boundary:** If the base revision is missing or repository history is shallow, RailVerdict returns `INCOMPLETE` (exit code 2) rather than guessing or silently passing.
|
|
336
391
|
|
|
392
|
+
### Verification Receipts (1.2)
|
|
393
|
+
|
|
394
|
+
Verification is only meaningful for the exact state that was verified. RailVerdict 1.2 binds every guarded verification to a deterministic **Repository State Identity** (HEAD + Git index snapshot + worktree delta with content hashes + configuration/baseline/waiver digests) and issues a machine-readable **Verification Receipt**:
|
|
395
|
+
|
|
396
|
+
```console
|
|
397
|
+
$ railverdict check
|
|
398
|
+
# => RailVerdict Verification: PASS (exit 0)
|
|
399
|
+
$ railverdict receipt create > receipt.json
|
|
400
|
+
# receipt_id = sha256:<64 hex> (no created_at, no timestamps)
|
|
401
|
+
|
|
402
|
+
$ railverdict receipt verify receipt.json --format json
|
|
403
|
+
{"schema_version":"1.0","status":"fresh","reasons":[],"gate":"PASS"}
|
|
404
|
+
```
|
|
405
|
+
|
|
406
|
+
Edit anything afterwards and the same receipt reports `stale`:
|
|
407
|
+
|
|
408
|
+
```console
|
|
409
|
+
$ echo "# changed after verification" >> app/models/user.rb
|
|
410
|
+
$ railverdict receipt verify receipt.json --format json
|
|
411
|
+
{"schema_version":"1.0","status":"stale","reasons":["worktree_changed"],"gate":"PASS"}
|
|
412
|
+
# exit 2 — current state is not what was verified
|
|
413
|
+
|
|
414
|
+
# Same for staged, config, baseline, waiver edits:
|
|
415
|
+
# head_changed | index_changed | worktree_changed | configuration_changed | baseline_changed | waivers_changed
|
|
416
|
+
```
|
|
417
|
+
|
|
418
|
+
If the repository mutates while analyzers run, receipt issuance fails closed with `repository_changed_during_verification`. Receipts exist for `PASS`, `FAIL`, and `INCOMPLETE`; they are deterministic integrity records — **not signed attestations** — a trusted CI remains the trust anchor when forgery is in scope. Full protocol in [docs/agent-verification.md](docs/agent-verification.md).
|
|
419
|
+
|
|
420
|
+
```
|
|
421
|
+
Deterministic Verification → PR Intelligence → Verification Receipt → Agent Verification Protocol
|
|
422
|
+
```
|
|
423
|
+
|
|
337
424
|
---
|
|
338
425
|
|
|
339
426
|
## Supported Analyzers
|
|
@@ -348,7 +435,7 @@ All analyzers in RailVerdict are **external and owned by the target project**. R
|
|
|
348
435
|
| **SimpleCov** | Code and changed-line coverage | `>= 1, < 2` | Ingests versioned public `coverage/coverage.json` v1 (never parses internal `.resultset.json`). |
|
|
349
436
|
| **bundler-audit** | Gem dependency vulnerabilities | `>= 0.9.3, < 1` | Runs `bundle exec bundler-audit check --format json` (never runs automatic updates). Robustly extracts JSON when advisory-DB download notices precede the payload. |
|
|
350
437
|
|
|
351
|
-
> **Brakeman Status:** Brakeman support is **not included** in 1.0 (on HOLD pending legal and licensing review). Third-party analyzers retain their respective upstream licenses.
|
|
438
|
+
> **Brakeman Status:** Brakeman support is **not included** in 1.2.0 (on HOLD pending legal and licensing review). Third-party analyzers retain their respective upstream licenses.
|
|
352
439
|
|
|
353
440
|
---
|
|
354
441
|
|
|
@@ -434,7 +521,44 @@ See [`docs/ai.md`](docs/ai.md) and [`docs/privacy.md`](docs/privacy.md) for deta
|
|
|
434
521
|
|
|
435
522
|
## Coding Agents & The Repair Loop
|
|
436
523
|
|
|
437
|
-
|
|
524
|
+
### Agent Completion Protocol (copyable for AGENTS.md / CLAUDE.md / Codex)
|
|
525
|
+
|
|
526
|
+
Before declaring any code-changing task complete:
|
|
527
|
+
|
|
528
|
+
1. Run the required project tests.
|
|
529
|
+
2. Run `bundle exec railverdict check` (or `check --changed --base <main>` in PRs).
|
|
530
|
+
3. If `FAIL`: remediate findings and rerun until `PASS` or explicitly report the blocker.
|
|
531
|
+
4. If `INCOMPLETE` (exit 2): do not claim verification succeeded — fix the evidence gap (missing analyzer, bad base, shallow history).
|
|
532
|
+
5. After final `PASS`, create `bundle exec railverdict receipt create > receipt.json`.
|
|
533
|
+
6. Do not modify repository state after receipt creation without reverifying — `receipt verify` must stay `fresh`.
|
|
534
|
+
|
|
535
|
+
`FAIL` means remediation required. `INCOMPLETE` must never be represented as `PASS`. See `docs/agent-verification.md` for the formal contract.
|
|
536
|
+
|
|
537
|
+
<details><summary>AGENTS.md snippet (paste into your repo)</summary>
|
|
538
|
+
|
|
539
|
+
```markdown
|
|
540
|
+
## Verification
|
|
541
|
+
|
|
542
|
+
Before completing any code-changing task:
|
|
543
|
+
|
|
544
|
+
bundle exec railverdict check
|
|
545
|
+
|
|
546
|
+
A task may only be reported as verified when RailVerdict returns PASS (exit 0).
|
|
547
|
+
|
|
548
|
+
FAIL (exit 1) means remediation is required.
|
|
549
|
+
|
|
550
|
+
INCOMPLETE (exit 2) means required evidence could not be obtained and must never be represented as PASS.
|
|
551
|
+
|
|
552
|
+
After the final PASS, create a verification receipt: bundle exec railverdict receipt create > receipt.json
|
|
553
|
+
|
|
554
|
+
Do not modify repository state after receipt creation without reverifying (receipt verify must be fresh).
|
|
555
|
+
```
|
|
556
|
+
|
|
557
|
+
Validated against 1.2.0: `init`, `doctor`, `check`, `pr`, `baseline create`, `findings`, `repair`, `receipt create|verify`, `mcp serve`.
|
|
558
|
+
|
|
559
|
+
</details>
|
|
560
|
+
|
|
561
|
+
RailVerdict also provides a structured verification loop for AI coding agents (such as Claude, Codex, or custom agents):
|
|
438
562
|
|
|
439
563
|
```
|
|
440
564
|
Agent modifies code
|
|
@@ -487,6 +611,8 @@ railverdict mcp serve
|
|
|
487
611
|
| `verify_repair` | Reruns verification and classifies whether a repair succeeded or regressed. |
|
|
488
612
|
| `explain` | Generates an advisory AI explanation for a finding. |
|
|
489
613
|
| `investigate` | Investigates top blocking findings across the codebase. |
|
|
614
|
+
| `get_verification_receipt` | Returns the Verification Receipt for the last `verify` without rerunning analyzers (refuses stale cache). |
|
|
615
|
+
| `get_pr_intelligence` | Returns PR Intelligence derived from the last `verify`'s GateResult (no rerun). |
|
|
490
616
|
|
|
491
617
|
### MCP Security Properties
|
|
492
618
|
|
|
@@ -559,7 +685,7 @@ To maintain clear technical boundaries, RailVerdict is explicitly **NOT**:
|
|
|
559
685
|
|
|
560
686
|
## Project Status
|
|
561
687
|
|
|
562
|
-
- **Release Version:** `1.0
|
|
688
|
+
- **Release Version:** `1.2.0`
|
|
563
689
|
- **License:** [MIT](LICENSE) (see [NOTICE](NOTICE))
|
|
564
690
|
- **Trademarks:** [TRADEMARKS.md](TRADEMARKS.md)
|
|
565
691
|
- **Foundation & Legal:** [docs/foundation.md](docs/foundation.md) — preliminary screen found no obvious software/tool conflict; NOT LEGAL CLEARANCE; qualified trademark review NOT PERFORMED — NON-BLOCKING BY MAINTAINER DECISION 2026-08-19 (Pedro Dalben).
|
|
@@ -589,9 +715,32 @@ To maintain clear technical boundaries, RailVerdict is explicitly **NOT**:
|
|
|
589
715
|
|
|
590
716
|
---
|
|
591
717
|
|
|
718
|
+
## Try RailVerdict Challenge (5 minutes)
|
|
719
|
+
|
|
720
|
+
1. `bundle add rail_verdict --group development,test --require false && bundle install`
|
|
721
|
+
2. `bundle exec railverdict init && bundle exec railverdict doctor`
|
|
722
|
+
3. `bundle exec railverdict baseline create` (or skip for greenfield) → `bundle exec railverdict check` → first gate
|
|
723
|
+
4. Introduce a controlled RuboCop offense, rerun → `FAIL`, fix → `PASS`, then `railverdict receipt create && railverdict receipt verify`
|
|
724
|
+
|
|
725
|
+
Tell us where it confused (see `docs/troubleshooting.md` and `docs/launch/try-challenge.md`).
|
|
726
|
+
|
|
727
|
+
## FAQ
|
|
728
|
+
|
|
729
|
+
**Is RailVerdict a test framework? Does it replace RSpec/RuboCop/CI?** No — it consumes them; they are evidence, it is the verifier. Run it inside CI.
|
|
730
|
+
|
|
731
|
+
**Does it upload source? Require AI? Does AI decide PASS?** No/no/no — offline by default, AI advisory only off by default (`trust: redacted`).
|
|
732
|
+
|
|
733
|
+
**Legacy apps?** Yes — `no_new_debt` baseline.
|
|
734
|
+
|
|
735
|
+
**INCOMPLETE vs FAIL?** `FAIL` = complete verification rejected new debt; `INCOMPLETE` = required evidence missing → no gate (exit 2), never PASS.
|
|
736
|
+
|
|
737
|
+
**Baseline / waiver / receipt?** Baseline snapshots current debt; waiver is exact-fingerprint, UTC-expired exemption; receipt is an integrity record (`receipt_id = sha256:`) — **not signed**, forgeable via whole-file rewrite — trusted CI is the trust anchor. Mutating after `PASS` makes receipt `stale`.
|
|
738
|
+
|
|
739
|
+
**Brakeman / Ruby / Rails?** Brakeman not in 1.2.0 (HOLD); Ruby `>=3.3`, Rails `>=8.0` bounded.
|
|
740
|
+
|
|
592
741
|
## Contributing & Issues
|
|
593
742
|
|
|
594
|
-
Contributions and issue reports are welcome. Please open an issue on GitHub for:
|
|
743
|
+
Contributions and issue reports are welcome. See `CONTRIBUTING.md` for “how to run tests / add regression”. Issue templates: Bug, Feature request, Compatibility report. Please open an issue on GitHub for:
|
|
595
744
|
|
|
596
745
|
- Analyzer compatibility and version range feedback;
|
|
597
746
|
- False positives or false negatives in evidence normalization;
|
|
@@ -12,7 +12,15 @@ module RailVerdict
|
|
|
12
12
|
def detail_for(run_result)
|
|
13
13
|
detail = run_result.detail.to_s
|
|
14
14
|
stderr = run_result.stderr.to_s.lines.first.to_s.strip
|
|
15
|
-
|
|
15
|
+
combined = [detail, stderr].reject(&:empty?).join(": ")
|
|
16
|
+
if combined.empty?
|
|
17
|
+
combined = if run_result.status == :exited
|
|
18
|
+
"analyzer exited with status #{run_result.exit_code} and produced no diagnostic output"
|
|
19
|
+
else
|
|
20
|
+
"analyzer produced no diagnostic output"
|
|
21
|
+
end
|
|
22
|
+
end
|
|
23
|
+
bounded_message(combined)
|
|
16
24
|
end
|
|
17
25
|
|
|
18
26
|
def execution_message(run_result)
|
|
@@ -40,15 +48,57 @@ module RailVerdict
|
|
|
40
48
|
|
|
41
49
|
def failure_result(analyzer_id:, invocation:, status:, message:, tool_version: nil)
|
|
42
50
|
status = status.to_s
|
|
51
|
+
bounded = bounded_message(message.to_s)
|
|
52
|
+
bounded = "#{status}: analyzer produced no diagnostic output" if bounded.strip.empty?
|
|
53
|
+
normalized_version = normalize_tool_version(tool_version)
|
|
43
54
|
AnalyzerResult.new(
|
|
44
55
|
analyzer: analyzer_id,
|
|
45
|
-
tool_version:
|
|
56
|
+
tool_version: normalized_version,
|
|
46
57
|
invocation: invocation,
|
|
47
58
|
execution_status: status,
|
|
48
59
|
finding_ids: [],
|
|
49
|
-
failure: { "code" => status, "message" =>
|
|
60
|
+
failure: { "code" => status, "message" => bounded }
|
|
50
61
|
)
|
|
51
62
|
end
|
|
63
|
+
|
|
64
|
+
def normalize_tool_version(version)
|
|
65
|
+
return nil if version.nil?
|
|
66
|
+
str = version.to_s.strip
|
|
67
|
+
return nil if str.empty?
|
|
68
|
+
str.encode(Encoding::UTF_8, invalid: :replace, undef: :replace, replace: "?").scrub("?")[0, 128]
|
|
69
|
+
end
|
|
70
|
+
|
|
71
|
+
def canonical_tool_version(version)
|
|
72
|
+
normalized = normalize_tool_version(version)
|
|
73
|
+
normalized.nil? || normalized.empty? ? "unknown" : normalized
|
|
74
|
+
end
|
|
75
|
+
|
|
76
|
+
# Canonical finding message normalization — single path for all analyzers.
|
|
77
|
+
# Guarantees deterministic, bounded, valid non-empty UTF-8.
|
|
78
|
+
FALLBACK_MESSAGE_SUFFIX = "reported a finding without a message"
|
|
79
|
+
|
|
80
|
+
def normalize_finding_message(analyzer_id, raw_message)
|
|
81
|
+
raw = raw_message.to_s.dup
|
|
82
|
+
# Handle invalid UTF-8, null bytes
|
|
83
|
+
raw = raw.encode(Encoding::UTF_8, invalid: :replace, undef: :replace, replace: "\uFFFD")
|
|
84
|
+
raw = raw.scrub("\uFFFD")
|
|
85
|
+
raw = raw.delete("\u0000")
|
|
86
|
+
# Strip ANSI escape sequences
|
|
87
|
+
raw = raw.gsub(/\e\[[0-9;]*[A-Za-z]/, "")
|
|
88
|
+
raw = raw.gsub(/\e\][^\a]*\a/, "")
|
|
89
|
+
raw = raw.gsub(/\e\(B/, "")
|
|
90
|
+
# Remove control characters except tab/newline then normalize whitespace
|
|
91
|
+
raw = raw.gsub(/[\x00-\x08\x0B\x0C\x0E-\x1F\x7F]/, "")
|
|
92
|
+
raw = raw.strip
|
|
93
|
+
# Collapse whitespace including tabs/newlines
|
|
94
|
+
raw = raw.gsub(/\s+/, " ").strip
|
|
95
|
+
if raw.empty?
|
|
96
|
+
"#{analyzer_id} #{FALLBACK_MESSAGE_SUFFIX}"
|
|
97
|
+
else
|
|
98
|
+
raw = raw.encode(Encoding::UTF_8, invalid: :replace, undef: :replace, replace: "\uFFFD").scrub("\uFFFD")
|
|
99
|
+
raw.bytesize > 4096 ? raw.byteslice(0, 4096).scrub("\uFFFD").strip : raw
|
|
100
|
+
end
|
|
101
|
+
end
|
|
52
102
|
end
|
|
53
103
|
end
|
|
54
104
|
end
|
|
@@ -47,7 +47,7 @@ module RailVerdict
|
|
|
47
47
|
Probe.new(status: "malformed", message: Shared.bounded_message(error.message))
|
|
48
48
|
end
|
|
49
49
|
|
|
50
|
-
def run(repository_root, runner: ProcessRunner, timeout_seconds: 30.0, probe_result: nil)
|
|
50
|
+
def run(repository_root, runner: ProcessRunner, timeout_seconds: 30.0, probe_result: nil, configuration: nil)
|
|
51
51
|
command = @command_resolver.call(repository_root)
|
|
52
52
|
probe_result ||= probe(repository_root, runner: runner, timeout_seconds: timeout_seconds)
|
|
53
53
|
version_invocation = Shared.invocation_for(command, ["version"])
|
|
@@ -143,9 +143,8 @@ module RailVerdict
|
|
|
143
143
|
gem_name = "unknown" if gem_name.to_s.empty?
|
|
144
144
|
|
|
145
145
|
severity = map_severity(advisory["criticality"] || advisory["severity"] || entry["criticality"] || entry["severity"])
|
|
146
|
-
|
|
147
|
-
message =
|
|
148
|
-
message = "vulnerability in #{gem_name}" if message.empty?
|
|
146
|
+
raw_msg = advisory["title"] || advisory["description"] || entry["title"] || entry["description"] || "vulnerability in #{gem_name}"
|
|
147
|
+
message = Shared.normalize_finding_message(ANALYZER_ID, raw_msg)
|
|
149
148
|
|
|
150
149
|
rule_id = "bundler_audit/advisory:#{id}"
|
|
151
150
|
category = "dependency"
|
|
@@ -47,7 +47,7 @@ module RailVerdict
|
|
|
47
47
|
Probe.new(status: "malformed", message: Shared.bounded_message(error.message))
|
|
48
48
|
end
|
|
49
49
|
|
|
50
|
-
def run(repository_root, runner: ProcessRunner, timeout_seconds: 30.0, probe_result: nil)
|
|
50
|
+
def run(repository_root, runner: ProcessRunner, timeout_seconds: 30.0, probe_result: nil, configuration: nil)
|
|
51
51
|
probe_result ||= probe(repository_root, runner: runner, timeout_seconds: timeout_seconds)
|
|
52
52
|
|
|
53
53
|
unless probe_result.status == "succeeded"
|
|
@@ -248,12 +248,8 @@ module RailVerdict
|
|
|
248
248
|
severity = status == "errored" ? "critical" : "high"
|
|
249
249
|
category = "test"
|
|
250
250
|
rule_id = "minitest/test:#{class_name}##{method_name}"
|
|
251
|
-
|
|
252
|
-
message =
|
|
253
|
-
message = "test failed: #{method_name}" if message.empty?
|
|
254
|
-
unless message.is_a?(String) && message.valid_encoding? && !message.empty? && message.bytesize <= 4096
|
|
255
|
-
raise MalformedOutput, "Minitest finding message is invalid"
|
|
256
|
-
end
|
|
251
|
+
raw_msg = test["failure_message"] || test["method_name"] || "test failed: #{method_name}"
|
|
252
|
+
message = Shared.normalize_finding_message(ANALYZER_ID, raw_msg)
|
|
257
253
|
|
|
258
254
|
path = normalize_path(test["file"], class_name)
|
|
259
255
|
start_line = test["line"]
|
|
@@ -45,7 +45,7 @@ module RailVerdict
|
|
|
45
45
|
Probe.new(status: "malformed", message: Shared.bounded_message(error.message))
|
|
46
46
|
end
|
|
47
47
|
|
|
48
|
-
def run(repository_root, runner: ProcessRunner, timeout_seconds: 30.0, probe_result: nil)
|
|
48
|
+
def run(repository_root, runner: ProcessRunner, timeout_seconds: 30.0, probe_result: nil, configuration: nil)
|
|
49
49
|
command = @command_resolver.call(repository_root)
|
|
50
50
|
probe_result ||= probe(repository_root, runner: runner, timeout_seconds: timeout_seconds)
|
|
51
51
|
version_invocation = Shared.invocation_for(command, ["--version"])
|
|
@@ -55,11 +55,14 @@ module RailVerdict
|
|
|
55
55
|
end
|
|
56
56
|
|
|
57
57
|
invocation = Shared.invocation_for(command, ["--format", "json"])
|
|
58
|
+
# Real-world large RSpec suites can exceed 4 MiB; use bounded but larger capture.
|
|
59
|
+
max_stdout = resolve_stdout_limit(configuration, repository_root, 16 * 1024 * 1024)
|
|
58
60
|
result = runner.run(
|
|
59
61
|
command.fetch(:executable),
|
|
60
62
|
invocation.fetch("argv"),
|
|
61
63
|
chdir: repository_root,
|
|
62
|
-
timeout_seconds: timeout_seconds
|
|
64
|
+
timeout_seconds: timeout_seconds,
|
|
65
|
+
max_stdout_bytes: max_stdout
|
|
63
66
|
)
|
|
64
67
|
tool_version = probe_result.version
|
|
65
68
|
|
|
@@ -136,9 +139,8 @@ module RailVerdict
|
|
|
136
139
|
return nil
|
|
137
140
|
end
|
|
138
141
|
|
|
139
|
-
|
|
140
|
-
message =
|
|
141
|
-
message = "rspec example failed" if message.empty?
|
|
142
|
+
raw_msg = (example["exception"] && example["exception"]["message"]) || example["full_description"] || example["description"] || nil
|
|
143
|
+
message = Shared.normalize_finding_message(ANALYZER_ID, raw_msg.nil? || raw_msg.to_s.strip.empty? ? "rspec example failed" : raw_msg)
|
|
142
144
|
|
|
143
145
|
rule_id = "rspec/example:#{example['id'] || id_for(example, index)}"
|
|
144
146
|
path = normalize_path(example["file_path"] || example["file"] || "spec/unknown_spec.rb")
|
|
@@ -269,6 +271,21 @@ module RailVerdict
|
|
|
269
271
|
rescue ArgumentError, TypeError
|
|
270
272
|
raise MalformedOutput, "RSpec summary fields have invalid types"
|
|
271
273
|
end
|
|
274
|
+
|
|
275
|
+
def resolve_stdout_limit(configuration, repository_root, default_bytes)
|
|
276
|
+
# Prefer explicit per-analyzer config if present (future-compatible), else default.
|
|
277
|
+
raw_limit = nil
|
|
278
|
+
if configuration
|
|
279
|
+
sel = configuration.analyzers[ANALYZER_ID] rescue nil
|
|
280
|
+
raw_limit = sel && sel["output_limit_bytes"]
|
|
281
|
+
end
|
|
282
|
+
raw_limit ||= default_bytes
|
|
283
|
+
# Clamp to safe ceiling
|
|
284
|
+
limit = Integer(raw_limit) rescue default_bytes
|
|
285
|
+
limit = default_bytes if limit <= 0
|
|
286
|
+
max = RailVerdict::ProcessRunner::MAX_SAFE_STDOUT_BYTES
|
|
287
|
+
[[limit, max].min, 1024].max
|
|
288
|
+
end
|
|
272
289
|
end
|
|
273
290
|
end
|
|
274
291
|
end
|
|
@@ -55,7 +55,7 @@ module RailVerdict
|
|
|
55
55
|
Probe.new(status: "malformed", message: bounded_message(error.message))
|
|
56
56
|
end
|
|
57
57
|
|
|
58
|
-
def run(repository_root, runner: ProcessRunner, timeout_seconds: 30.0, probe_result: nil)
|
|
58
|
+
def run(repository_root, runner: ProcessRunner, timeout_seconds: 30.0, probe_result: nil, configuration: nil)
|
|
59
59
|
command = @command_resolver.call(repository_root)
|
|
60
60
|
probe_result ||= probe(repository_root, runner: runner, timeout_seconds: timeout_seconds)
|
|
61
61
|
version_invocation = invocation_for(command, ["--version"])
|
|
@@ -65,11 +65,13 @@ module RailVerdict
|
|
|
65
65
|
end
|
|
66
66
|
|
|
67
67
|
invocation = invocation_for(command, ["--format", "json"])
|
|
68
|
+
max_stdout = resolve_stdout_limit(configuration, 8 * 1024 * 1024)
|
|
68
69
|
result = runner.run(
|
|
69
70
|
command.fetch(:executable),
|
|
70
71
|
invocation.fetch("argv"),
|
|
71
72
|
chdir: repository_root,
|
|
72
|
-
timeout_seconds: timeout_seconds
|
|
73
|
+
timeout_seconds: timeout_seconds,
|
|
74
|
+
max_stdout_bytes: max_stdout
|
|
73
75
|
)
|
|
74
76
|
tool_version = probe_result.version
|
|
75
77
|
|
|
@@ -255,10 +257,8 @@ module RailVerdict
|
|
|
255
257
|
raise MalformedOutput, "RuboCop offense line range is invalid"
|
|
256
258
|
end
|
|
257
259
|
|
|
258
|
-
|
|
259
|
-
|
|
260
|
-
raise MalformedOutput, "RuboCop offense message is invalid"
|
|
261
|
-
end
|
|
260
|
+
raw_msg = offense["message"]
|
|
261
|
+
message = Shared.normalize_finding_message(ANALYZER_ID, raw_msg)
|
|
262
262
|
|
|
263
263
|
fingerprint = Finding.fingerprint_for(
|
|
264
264
|
analyzer: ANALYZER_ID,
|
|
@@ -284,6 +284,14 @@ module RailVerdict
|
|
|
284
284
|
rescue ArgumentError => error
|
|
285
285
|
raise MalformedOutput, "RuboCop offense #{file_index}:#{offense_index} is malformed: #{error.message}"
|
|
286
286
|
end
|
|
287
|
+
|
|
288
|
+
def resolve_stdout_limit(configuration, default_bytes)
|
|
289
|
+
raw = configuration && configuration.analyzers["rubocop"] && configuration.analyzers["rubocop"]["output_limit_bytes"]
|
|
290
|
+
raw ||= default_bytes
|
|
291
|
+
limit = Integer(raw) rescue default_bytes
|
|
292
|
+
limit = default_bytes if limit <= 0
|
|
293
|
+
[[limit, RailVerdict::ProcessRunner::MAX_SAFE_STDOUT_BYTES].min, 1024].max
|
|
294
|
+
end
|
|
287
295
|
end
|
|
288
296
|
end
|
|
289
297
|
end
|