rail_verdict 1.2.0 → 1.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +151 -18
- data/lib/rail_verdict/cli.rb +15 -14
- data/lib/rail_verdict/mcp/cache.rb +40 -6
- data/lib/rail_verdict/mcp/tools/get_pr_intelligence.rb +16 -3
- data/lib/rail_verdict/mcp/tools/get_verification_receipt.rb +17 -3
- data/lib/rail_verdict/receipt.rb +163 -22
- data/lib/rail_verdict/verification_environment.rb +220 -0
- data/lib/rail_verdict/verification_identity.rb +95 -0
- data/lib/rail_verdict/version.rb +1 -1
- data/lib/rail_verdict.rb +2 -0
- data/schemas/receipt-validation-v1.schema.json +6 -0
- data/schemas/verification-receipt-v1.schema.json +1 -0
- metadata +3 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 63845a29f84295397de20a23b72c7d6e0a9d6c969be35ac8081e2536b0a37cec
|
|
4
|
+
data.tar.gz: 6e778d530c6abe87db5997b9d5fe328ebd750ac1722ede75042195b5d98d92cf
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: beabe5b069a803549a6af55d03f16256e5ebe7e5ab6152672a7a1fa415a75827656108c01207860251fc8936fdc3bfb6397c1f53ca53f342da5ee913051d1c2a
|
|
7
|
+
data.tar.gz: 72e637013c3a4216a90fcb466c947b6b6e97bc9035166bb8b96b76b62579c76f1253eb765d1c56ac1e39dd82197489c3535a411fece8535c6cd340bd0e741ac9
|
data/README.md
CHANGED
|
@@ -78,18 +78,11 @@ The Lab exercises RailVerdict across realistic operational scenarios:
|
|
|
78
78
|
- Exercising fail-closed behavior on missing or malformed analyzer output;
|
|
79
79
|
- Testing release artifact installation from clean environments.
|
|
80
80
|
|
|
81
|
-
###
|
|
81
|
+
### External Validation Campaign
|
|
82
82
|
|
|
83
|
-
The 1.0
|
|
83
|
+
The Lab independently validates the published gem as an external consumer. 1.0 closeout validated **21 / 21** scenarios; **1.2.0** certification validates **81 scenarios** (14 categories, see Lab README) including Verification Receipts and PR Intelligence. See [railverdict-lab](https://github.com/pedro-dalben/railverdict-lab) for the canonical campaign and results.
|
|
84
84
|
|
|
85
|
-
|
|
86
|
-
|---|---:|:---:|
|
|
87
|
-
| **Core Verification** | 12 / 12 | PASS |
|
|
88
|
-
| **Operational & CI** | 6 / 6 | PASS |
|
|
89
|
-
| **Release Closeout** | 3 / 3 | PASS |
|
|
90
|
-
| **Total** | **21 / 21** | **PASS** |
|
|
91
|
-
|
|
92
|
-
These controlled validation scenarios represent rigorous external regression verification rather than a claim of mathematically bug-free software. During development, the Lab uncovered real defects — such as test failure scoping edge cases — which were fixed before the 1.0 release.
|
|
85
|
+
These controlled validation scenarios represent rigorous external regression verification rather than a claim of mathematically bug-free software. During development, the Lab uncovered real defects — such as test failure scoping edge cases and the 1.2 dogfooding hardening — which were fixed before release.
|
|
93
86
|
|
|
94
87
|
---
|
|
95
88
|
|
|
@@ -236,11 +229,73 @@ an explicit value, including every analyzer in older configuration versions,
|
|
|
236
229
|
continues to use 30 seconds. A timeout is incomplete evidence, never a normal
|
|
237
230
|
finding: a required timeout produces `INCOMPLETE` and exit code `2`.
|
|
238
231
|
|
|
239
|
-
|
|
232
|
+
There is no CLI timeout override; the versioned configuration (`version: 1.5`) is
|
|
240
233
|
the supported public surface. SimpleCov accepts the same setting for a uniform
|
|
241
234
|
configuration contract, but reads a local coverage artifact rather than
|
|
242
235
|
starting an analyzer process.
|
|
243
236
|
|
|
237
|
+
### 30-Second Demonstration (real output)
|
|
238
|
+
|
|
239
|
+
```console
|
|
240
|
+
$ railverdict check
|
|
241
|
+
|
|
242
|
+
RailVerdict Verification: PASS
|
|
243
|
+
Policy: no_new_debt (complete)
|
|
244
|
+
Analyzers: 5 run (5 complete)
|
|
245
|
+
Findings: 0 introduced, 0 existing, 0 blocking
|
|
246
|
+
Exit: 0
|
|
247
|
+
```
|
|
248
|
+
|
|
249
|
+
Introduce a controlled defect:
|
|
250
|
+
|
|
251
|
+
```ruby
|
|
252
|
+
# app/models/user.rb:42
|
|
253
|
+
unused = "oops" # Lint/UselessAssignment
|
|
254
|
+
```
|
|
255
|
+
|
|
256
|
+
```console
|
|
257
|
+
$ railverdict check
|
|
258
|
+
|
|
259
|
+
RailVerdict Verification: FAIL
|
|
260
|
+
Policy: no_new_debt (failed)
|
|
261
|
+
Findings: 1 introduced (blocking), 0 existing
|
|
262
|
+
- [rubocop] Lint/UselessAssignment in app/models/user.rb:42 (introduced)
|
|
263
|
+
Exit: 1
|
|
264
|
+
```
|
|
265
|
+
|
|
266
|
+
Fix it, rerun → `PASS`. See `docs/release/1.2-clean-room-rehearsal.md` for automated reproduction.
|
|
267
|
+
|
|
268
|
+
### Why not just CI?
|
|
269
|
+
|
|
270
|
+
CI answers **“did these jobs execute successfully?”** across fragmented outputs. RailVerdict answers **“given required evidence, repository state, baseline and policy, what is the deterministic verification decision for this change?”** — normalized findings, baseline-aware policy (`no_new_debt`), changed-scope (`--changed --base`), fail-closed `INCOMPLETE` on missing evidence, and machine contracts (JSON/SARIF/exits) plus Repository State Identity and Verification Receipts. It complements CI; the example in [GitHub Actions Integration](#github-actions-integration) runs RailVerdict inside CI.
|
|
271
|
+
|
|
272
|
+
### Why not just RSpec + RuboCop?
|
|
273
|
+
|
|
274
|
+
Those are the evidence. RailVerdict is the verifier:
|
|
275
|
+
|
|
276
|
+
```
|
|
277
|
+
RSpec says: 42 tests passed.
|
|
278
|
+
RuboCop says: 3 offenses.
|
|
279
|
+
SimpleCov says: 91.3% coverage.
|
|
280
|
+
Git says: these 7 lines changed.
|
|
281
|
+
RailVerdict says: FAIL — this change introduced a blocking finding.
|
|
282
|
+
```
|
|
283
|
+
|
|
284
|
+
They produce facts; policy owns the gate. `rubocop` + `rspec` + `minitest` + `simplecov` + `bundler-audit` + `Git` are inputs; `PASS/WARN/FAIL/INCOMPLETE` is the `GateResult` (see `docs/contracts.md`).
|
|
285
|
+
|
|
286
|
+
### Why this matters for AI agents
|
|
287
|
+
|
|
288
|
+
An AI agent can write code, run tests, and declare itself finished — but it should not be the authority deciding whether its own work is acceptable.
|
|
289
|
+
|
|
290
|
+
```
|
|
291
|
+
Agent --modifies--> Repository --verify--> RailVerdict
|
|
292
|
+
|--> deterministic evidence
|
|
293
|
+
|--> GateResult
|
|
294
|
+
`--> Verification Receipt
|
|
295
|
+
```
|
|
296
|
+
|
|
297
|
+
The agent proposes; the verification system decides. Receipts let agents (and humans) prove `fresh` vs `stale` after any edit — see below. AI inside RailVerdict (`explain`/`investigate`) is advisory only and never changes `GateResult`.
|
|
298
|
+
|
|
244
299
|
---
|
|
245
300
|
|
|
246
301
|
## The Default Policy Model: No New Debt
|
|
@@ -339,12 +394,28 @@ exit code `2`.
|
|
|
339
394
|
Verification is only meaningful for the exact state that was verified. RailVerdict 1.2 binds every guarded verification to a deterministic **Repository State Identity** (HEAD + Git index snapshot + worktree delta with content hashes + configuration/baseline/waiver digests) and issues a machine-readable **Verification Receipt**:
|
|
340
395
|
|
|
341
396
|
```console
|
|
342
|
-
$ railverdict
|
|
397
|
+
$ railverdict check
|
|
398
|
+
# => RailVerdict Verification: PASS (exit 0)
|
|
399
|
+
$ railverdict receipt create > receipt.json
|
|
400
|
+
# receipt_id = sha256:<64 hex> (no created_at, no timestamps)
|
|
401
|
+
|
|
343
402
|
$ railverdict receipt verify receipt.json --format json
|
|
344
|
-
{"schema_version":"1.0","status":"fresh","reasons":[],"gate":"PASS"
|
|
403
|
+
{"schema_version":"1.0","status":"fresh","reasons":[],"gate":"PASS"}
|
|
404
|
+
```
|
|
405
|
+
|
|
406
|
+
Edit anything afterwards and the same receipt reports `stale`:
|
|
407
|
+
|
|
408
|
+
```console
|
|
409
|
+
$ echo "# changed after verification" >> app/models/user.rb
|
|
410
|
+
$ railverdict receipt verify receipt.json --format json
|
|
411
|
+
{"schema_version":"1.0","status":"stale","reasons":["worktree_changed"],"gate":"PASS"}
|
|
412
|
+
# exit 2 — current state is not what was verified
|
|
413
|
+
|
|
414
|
+
# Same for staged, config, baseline, waiver edits:
|
|
415
|
+
# head_changed | index_changed | worktree_changed | configuration_changed | baseline_changed | waivers_changed
|
|
345
416
|
```
|
|
346
417
|
|
|
347
|
-
|
|
418
|
+
If the repository mutates while analyzers run, receipt issuance fails closed with `repository_changed_during_verification`. Receipts exist for `PASS`, `FAIL`, and `INCOMPLETE`; they are deterministic integrity records — **not signed attestations** — a trusted CI remains the trust anchor when forgery is in scope. Full protocol in [docs/agent-verification.md](docs/agent-verification.md).
|
|
348
419
|
|
|
349
420
|
```
|
|
350
421
|
Deterministic Verification → PR Intelligence → Verification Receipt → Agent Verification Protocol
|
|
@@ -364,7 +435,7 @@ All analyzers in RailVerdict are **external and owned by the target project**. R
|
|
|
364
435
|
| **SimpleCov** | Code and changed-line coverage | `>= 1, < 2` | Ingests versioned public `coverage/coverage.json` v1 (never parses internal `.resultset.json`). |
|
|
365
436
|
| **bundler-audit** | Gem dependency vulnerabilities | `>= 0.9.3, < 1` | Runs `bundle exec bundler-audit check --format json` (never runs automatic updates). Robustly extracts JSON when advisory-DB download notices precede the payload. |
|
|
366
437
|
|
|
367
|
-
> **Brakeman Status:** Brakeman support is **not included** in 1.0 (on HOLD pending legal and licensing review). Third-party analyzers retain their respective upstream licenses.
|
|
438
|
+
> **Brakeman Status:** Brakeman support is **not included** in 1.2.0 (on HOLD pending legal and licensing review). Third-party analyzers retain their respective upstream licenses.
|
|
368
439
|
|
|
369
440
|
---
|
|
370
441
|
|
|
@@ -450,7 +521,44 @@ See [`docs/ai.md`](docs/ai.md) and [`docs/privacy.md`](docs/privacy.md) for deta
|
|
|
450
521
|
|
|
451
522
|
## Coding Agents & The Repair Loop
|
|
452
523
|
|
|
453
|
-
|
|
524
|
+
### Agent Completion Protocol (copyable for AGENTS.md / CLAUDE.md / Codex)
|
|
525
|
+
|
|
526
|
+
Before declaring any code-changing task complete:
|
|
527
|
+
|
|
528
|
+
1. Run the required project tests.
|
|
529
|
+
2. Run `bundle exec railverdict check` (or `check --changed --base <main>` in PRs).
|
|
530
|
+
3. If `FAIL`: remediate findings and rerun until `PASS` or explicitly report the blocker.
|
|
531
|
+
4. If `INCOMPLETE` (exit 2): do not claim verification succeeded — fix the evidence gap (missing analyzer, bad base, shallow history).
|
|
532
|
+
5. After final `PASS`, create `bundle exec railverdict receipt create > receipt.json`.
|
|
533
|
+
6. Do not modify repository state after receipt creation without reverifying — `receipt verify` must stay `fresh`.
|
|
534
|
+
|
|
535
|
+
`FAIL` means remediation required. `INCOMPLETE` must never be represented as `PASS`. See `docs/agent-verification.md` for the formal contract.
|
|
536
|
+
|
|
537
|
+
<details><summary>AGENTS.md snippet (paste into your repo)</summary>
|
|
538
|
+
|
|
539
|
+
```markdown
|
|
540
|
+
## Verification
|
|
541
|
+
|
|
542
|
+
Before completing any code-changing task:
|
|
543
|
+
|
|
544
|
+
bundle exec railverdict check
|
|
545
|
+
|
|
546
|
+
A task may only be reported as verified when RailVerdict returns PASS (exit 0).
|
|
547
|
+
|
|
548
|
+
FAIL (exit 1) means remediation is required.
|
|
549
|
+
|
|
550
|
+
INCOMPLETE (exit 2) means required evidence could not be obtained and must never be represented as PASS.
|
|
551
|
+
|
|
552
|
+
After the final PASS, create a verification receipt: bundle exec railverdict receipt create > receipt.json
|
|
553
|
+
|
|
554
|
+
Do not modify repository state after receipt creation without reverifying (receipt verify must be fresh).
|
|
555
|
+
```
|
|
556
|
+
|
|
557
|
+
Validated against 1.2.0: `init`, `doctor`, `check`, `pr`, `baseline create`, `findings`, `repair`, `receipt create|verify`, `mcp serve`.
|
|
558
|
+
|
|
559
|
+
</details>
|
|
560
|
+
|
|
561
|
+
RailVerdict also provides a structured verification loop for AI coding agents (such as Claude, Codex, or custom agents):
|
|
454
562
|
|
|
455
563
|
```
|
|
456
564
|
Agent modifies code
|
|
@@ -503,6 +611,8 @@ railverdict mcp serve
|
|
|
503
611
|
| `verify_repair` | Reruns verification and classifies whether a repair succeeded or regressed. |
|
|
504
612
|
| `explain` | Generates an advisory AI explanation for a finding. |
|
|
505
613
|
| `investigate` | Investigates top blocking findings across the codebase. |
|
|
614
|
+
| `get_verification_receipt` | Returns the Verification Receipt for the last `verify` without rerunning analyzers (refuses stale cache). |
|
|
615
|
+
| `get_pr_intelligence` | Returns PR Intelligence derived from the last `verify`'s GateResult (no rerun). |
|
|
506
616
|
|
|
507
617
|
### MCP Security Properties
|
|
508
618
|
|
|
@@ -575,7 +685,7 @@ To maintain clear technical boundaries, RailVerdict is explicitly **NOT**:
|
|
|
575
685
|
|
|
576
686
|
## Project Status
|
|
577
687
|
|
|
578
|
-
- **Release Version:** `1.0
|
|
688
|
+
- **Release Version:** `1.2.0`
|
|
579
689
|
- **License:** [MIT](LICENSE) (see [NOTICE](NOTICE))
|
|
580
690
|
- **Trademarks:** [TRADEMARKS.md](TRADEMARKS.md)
|
|
581
691
|
- **Foundation & Legal:** [docs/foundation.md](docs/foundation.md) — preliminary screen found no obvious software/tool conflict; NOT LEGAL CLEARANCE; qualified trademark review NOT PERFORMED — NON-BLOCKING BY MAINTAINER DECISION 2026-08-19 (Pedro Dalben).
|
|
@@ -605,9 +715,32 @@ To maintain clear technical boundaries, RailVerdict is explicitly **NOT**:
|
|
|
605
715
|
|
|
606
716
|
---
|
|
607
717
|
|
|
718
|
+
## Try RailVerdict Challenge (5 minutes)
|
|
719
|
+
|
|
720
|
+
1. `bundle add rail_verdict --group development,test --require false && bundle install`
|
|
721
|
+
2. `bundle exec railverdict init && bundle exec railverdict doctor`
|
|
722
|
+
3. `bundle exec railverdict baseline create` (or skip for greenfield) → `bundle exec railverdict check` → first gate
|
|
723
|
+
4. Introduce a controlled RuboCop offense, rerun → `FAIL`, fix → `PASS`, then `railverdict receipt create && railverdict receipt verify`
|
|
724
|
+
|
|
725
|
+
Tell us where it confused (see `docs/troubleshooting.md` and `docs/launch/try-challenge.md`).
|
|
726
|
+
|
|
727
|
+
## FAQ
|
|
728
|
+
|
|
729
|
+
**Is RailVerdict a test framework? Does it replace RSpec/RuboCop/CI?** No — it consumes them; they are evidence, it is the verifier. Run it inside CI.
|
|
730
|
+
|
|
731
|
+
**Does it upload source? Require AI? Does AI decide PASS?** No/no/no — offline by default, AI advisory only off by default (`trust: redacted`).
|
|
732
|
+
|
|
733
|
+
**Legacy apps?** Yes — `no_new_debt` baseline.
|
|
734
|
+
|
|
735
|
+
**INCOMPLETE vs FAIL?** `FAIL` = complete verification rejected new debt; `INCOMPLETE` = required evidence missing → no gate (exit 2), never PASS.
|
|
736
|
+
|
|
737
|
+
**Baseline / waiver / receipt?** Baseline snapshots current debt; waiver is exact-fingerprint, UTC-expired exemption; receipt is an integrity record (`receipt_id = sha256:`) — **not signed**, forgeable via whole-file rewrite — trusted CI is the trust anchor. Mutating after `PASS` makes receipt `stale`.
|
|
738
|
+
|
|
739
|
+
**Brakeman / Ruby / Rails?** Brakeman not in 1.2.0 (HOLD); Ruby `>=3.3`, Rails `>=8.0` bounded.
|
|
740
|
+
|
|
608
741
|
## Contributing & Issues
|
|
609
742
|
|
|
610
|
-
Contributions and issue reports are welcome. Please open an issue on GitHub for:
|
|
743
|
+
Contributions and issue reports are welcome. See `CONTRIBUTING.md` for “how to run tests / add regression”. Issue templates: Bug, Feature request, Compatibility report. Please open an issue on GitHub for:
|
|
611
744
|
|
|
612
745
|
- Analyzer compatibility and version range feedback;
|
|
613
746
|
- False positives or false negatives in evidence normalization;
|
data/lib/rail_verdict/cli.rb
CHANGED
|
@@ -654,23 +654,24 @@ module RailVerdict
|
|
|
654
654
|
return EXIT_NO_GATE
|
|
655
655
|
end
|
|
656
656
|
|
|
657
|
+
effective_paths = begin
|
|
658
|
+
effective = Check.effective_input_paths(
|
|
659
|
+
root: @working_directory,
|
|
660
|
+
config_path: resolved_override_path(options[:config], File.join(@working_directory, DEFAULT_CONFIG_PATH)),
|
|
661
|
+
baseline_path_override: options[:baseline] && resolved_override_path(options[:baseline], nil),
|
|
662
|
+
waiver_path_override: options[:waiver] && resolved_override_path(options[:waiver], nil)
|
|
663
|
+
)
|
|
664
|
+
{
|
|
665
|
+
config: effective.fetch(:config),
|
|
666
|
+
baseline: effective.fetch(:baseline),
|
|
667
|
+
waivers: effective.fetch(:waivers)
|
|
668
|
+
}
|
|
669
|
+
end
|
|
657
670
|
current_state = RepositoryState.capture(
|
|
658
671
|
repository_root: @working_directory,
|
|
659
|
-
configuration_paths:
|
|
660
|
-
effective = Check.effective_input_paths(
|
|
661
|
-
root: @working_directory,
|
|
662
|
-
config_path: resolved_override_path(options[:config], File.join(@working_directory, DEFAULT_CONFIG_PATH)),
|
|
663
|
-
baseline_path_override: options[:baseline] && resolved_override_path(options[:baseline], nil),
|
|
664
|
-
waiver_path_override: options[:waiver] && resolved_override_path(options[:waiver], nil)
|
|
665
|
-
)
|
|
666
|
-
{
|
|
667
|
-
config: effective.fetch(:config),
|
|
668
|
-
baseline: effective.fetch(:baseline),
|
|
669
|
-
waivers: effective.fetch(:waivers)
|
|
670
|
-
}
|
|
671
|
-
end
|
|
672
|
+
configuration_paths: effective_paths
|
|
672
673
|
)
|
|
673
|
-
document, _receipt = Receipt.evaluate(text, current_state: current_state)
|
|
674
|
+
document, _receipt = Receipt.evaluate(text, current_state: current_state, repository_root: @working_directory, configuration_paths: effective_paths)
|
|
674
675
|
render_receipt_validation(document, options[:format])
|
|
675
676
|
exit_code_for_validation(document)
|
|
676
677
|
rescue RailVerdict::UsageError => error
|
|
@@ -6,6 +6,7 @@ require "json"
|
|
|
6
6
|
|
|
7
7
|
require_relative "../canonical_json"
|
|
8
8
|
require_relative "../repository_state"
|
|
9
|
+
require_relative "../verification_environment"
|
|
9
10
|
|
|
10
11
|
module RailVerdict
|
|
11
12
|
module MCP
|
|
@@ -39,23 +40,32 @@ module RailVerdict
|
|
|
39
40
|
nil
|
|
40
41
|
end
|
|
41
42
|
|
|
42
|
-
# Returns the cached entry only when the CURRENT observable
|
|
43
|
-
#
|
|
44
|
-
|
|
43
|
+
# Returns the cached entry only when the CURRENT observable identity
|
|
44
|
+
# (repository + environment) still matches what was verified. Never reruns analyzers,
|
|
45
|
+
# but does perform lightweight version probes for environment observation.
|
|
46
|
+
def fresh_entry(current_state, current_environment = nil)
|
|
45
47
|
entry = @mutex.synchronize { @entry }
|
|
46
48
|
return nil if entry.nil? || entry.state_digest.nil?
|
|
47
49
|
return nil if current_state.nil? || !current_state.available?
|
|
48
50
|
return nil if current_state.digest != entry.state_digest
|
|
49
51
|
|
|
52
|
+
current_env = current_environment || current_environment_for(entry.outcome)
|
|
53
|
+
return nil if current_env.nil? || !current_env.available?
|
|
54
|
+
return nil if current_env.digest != entry.environment_digest
|
|
55
|
+
|
|
50
56
|
entry
|
|
51
57
|
end
|
|
52
58
|
|
|
53
|
-
def verification_state(current_state)
|
|
59
|
+
def verification_state(current_state, current_environment = nil)
|
|
54
60
|
entry = @mutex.synchronize { @entry }
|
|
55
61
|
return "verification_required" if entry.nil? || entry.state_digest.nil?
|
|
56
62
|
return "state_unavailable" if current_state.nil? || !current_state.available?
|
|
57
63
|
return "stale" if current_state.digest != entry.state_digest
|
|
58
64
|
|
|
65
|
+
current_env = current_environment || current_environment_for(entry.outcome)
|
|
66
|
+
return "state_unavailable" if current_env.nil? || !current_env.available?
|
|
67
|
+
return "stale" if current_env.digest != entry.environment_digest
|
|
68
|
+
|
|
59
69
|
"fresh"
|
|
60
70
|
end
|
|
61
71
|
|
|
@@ -78,8 +88,11 @@ module RailVerdict
|
|
|
78
88
|
current = current_state_for(entry.outcome)
|
|
79
89
|
return false if current.nil? || !current.available?
|
|
80
90
|
|
|
91
|
+
current_env = current_environment_for(entry.outcome)
|
|
92
|
+
return false if current_env.nil? || !current_env.available?
|
|
93
|
+
|
|
81
94
|
current.digest == entry.state_digest &&
|
|
82
|
-
|
|
95
|
+
current_env.digest == entry.environment_digest
|
|
83
96
|
rescue StandardError
|
|
84
97
|
false
|
|
85
98
|
end
|
|
@@ -143,10 +156,31 @@ module RailVerdict
|
|
|
143
156
|
}
|
|
144
157
|
end
|
|
145
158
|
|
|
159
|
+
def current_environment_for(outcome)
|
|
160
|
+
root = outcome&.context&.repository_root
|
|
161
|
+
root ||= outcome&.result&.git&.fetch("repository_root", nil) rescue nil
|
|
162
|
+
return nil unless root.is_a?(String) && File.directory?(root)
|
|
163
|
+
|
|
164
|
+
real = begin
|
|
165
|
+
File.realpath(root)
|
|
166
|
+
rescue StandardError
|
|
167
|
+
return nil
|
|
168
|
+
end
|
|
169
|
+
config = outcome&.configuration
|
|
170
|
+
VerificationEnvironment.capture(repository_root: real, configuration: config)
|
|
171
|
+
rescue StandardError
|
|
172
|
+
nil
|
|
173
|
+
end
|
|
174
|
+
|
|
146
175
|
def environment_digest_for(outcome)
|
|
176
|
+
env = current_environment_for(outcome)
|
|
177
|
+
return env.digest if env&.available? && env.digest
|
|
178
|
+
|
|
179
|
+
# Fallback for cases where current capture fails (e.g., no config)
|
|
147
180
|
payload = {
|
|
148
181
|
"railverdict_version" => RailVerdict::VERSION,
|
|
149
|
-
"
|
|
182
|
+
"ruby_engine" => RUBY_ENGINE.to_s,
|
|
183
|
+
"ruby_version" => RUBY_VERSION.to_s,
|
|
150
184
|
"analyzer_versions" => sorted_analyzer_versions(outcome)
|
|
151
185
|
}
|
|
152
186
|
Digest::SHA256.hexdigest(CanonicalJSON.generate(payload))
|
|
@@ -48,10 +48,23 @@ module RailVerdict
|
|
|
48
48
|
end
|
|
49
49
|
|
|
50
50
|
def call(**_rest)
|
|
51
|
-
|
|
52
|
-
|
|
51
|
+
root = @server.repository_root
|
|
52
|
+
effective_paths = begin
|
|
53
|
+
RailVerdict::Check.effective_input_paths(root: File.realpath(root), config_path: File.join(File.realpath(root), ".railverdict.yml"))
|
|
54
|
+
rescue StandardError
|
|
55
|
+
nil
|
|
56
|
+
end
|
|
57
|
+
current_state = RailVerdict::RepositoryState.capture(repository_root: root, configuration_paths: effective_paths)
|
|
58
|
+
current_env = begin
|
|
59
|
+
outcome = @server.cache.fetch_outcome
|
|
60
|
+
config = outcome&.configuration
|
|
61
|
+
RailVerdict::VerificationEnvironment.capture(repository_root: root, configuration: config)
|
|
62
|
+
rescue StandardError
|
|
63
|
+
nil
|
|
64
|
+
end
|
|
65
|
+
case @server.cache.verification_state(current_state, current_env)
|
|
53
66
|
when "fresh"
|
|
54
|
-
entry = @server.cache.fresh_entry(current_state)
|
|
67
|
+
entry = @server.cache.fresh_entry(current_state, current_env)
|
|
55
68
|
document = entry&.pr_intelligence_document
|
|
56
69
|
if document.is_a?(Hash)
|
|
57
70
|
payload = { "status" => "fresh", "pr_intelligence" => Serializers.scrub(document) }
|
|
@@ -48,10 +48,24 @@ module RailVerdict
|
|
|
48
48
|
end
|
|
49
49
|
|
|
50
50
|
def call(**_rest)
|
|
51
|
-
|
|
52
|
-
|
|
51
|
+
root = @server.repository_root
|
|
52
|
+
effective_paths = begin
|
|
53
|
+
RailVerdict::Check.effective_input_paths(root: File.realpath(root), config_path: File.join(File.realpath(root), ".railverdict.yml"))
|
|
54
|
+
rescue StandardError
|
|
55
|
+
nil
|
|
56
|
+
end
|
|
57
|
+
current_state = RailVerdict::RepositoryState.capture(repository_root: root, configuration_paths: effective_paths)
|
|
58
|
+
current_env = begin
|
|
59
|
+
# Use cached outcome config if available for relevant probing
|
|
60
|
+
outcome = @server.cache.fetch_outcome
|
|
61
|
+
config = outcome&.configuration
|
|
62
|
+
RailVerdict::VerificationEnvironment.capture(repository_root: root, configuration: config)
|
|
63
|
+
rescue StandardError
|
|
64
|
+
nil
|
|
65
|
+
end
|
|
66
|
+
case @server.cache.verification_state(current_state, current_env)
|
|
53
67
|
when "fresh"
|
|
54
|
-
entry = @server.cache.fresh_entry(current_state)
|
|
68
|
+
entry = @server.cache.fresh_entry(current_state, current_env)
|
|
55
69
|
receipt = entry&.receipt_document
|
|
56
70
|
if receipt.is_a?(Hash) && receipt["receipt_id"]
|
|
57
71
|
payload = { "status" => "fresh", "receipt" => Serializers.scrub(receipt) }
|
data/lib/rail_verdict/receipt.rb
CHANGED
|
@@ -5,6 +5,8 @@ require "json"
|
|
|
5
5
|
|
|
6
6
|
require_relative "canonical_json"
|
|
7
7
|
require_relative "repository_state"
|
|
8
|
+
require_relative "verification_environment"
|
|
9
|
+
require_relative "verification_identity"
|
|
8
10
|
require_relative "schema_validator"
|
|
9
11
|
|
|
10
12
|
module RailVerdict
|
|
@@ -28,7 +30,7 @@ module RailVerdict
|
|
|
28
30
|
# Builds a Verification Receipt v1 document from a guarded Check outcome.
|
|
29
31
|
# Fails closed (BuildError with a deterministic code) when repository state
|
|
30
32
|
# cannot be proven stable across the verification.
|
|
31
|
-
def self.build(outcome:, railverdict_version: RailVerdict::VERSION, pr_intelligence_document: nil, repair_packet_id: nil, environment_ruby_version: RUBY_VERSION)
|
|
33
|
+
def self.build(outcome:, railverdict_version: RailVerdict::VERSION, pr_intelligence_document: nil, repair_packet_id: nil, environment_ruby_version: RUBY_VERSION, environment_ruby_engine: RUBY_ENGINE)
|
|
32
34
|
result = outcome&.result
|
|
33
35
|
raise BuildError.new(:receipt_unavailable, "verification outcome is required") if result.nil?
|
|
34
36
|
|
|
@@ -48,6 +50,7 @@ module RailVerdict
|
|
|
48
50
|
"schema_version" => SCHEMA_VERSION,
|
|
49
51
|
"railverdict_version" => railverdict_version.to_s,
|
|
50
52
|
"environment" => {
|
|
53
|
+
"ruby_engine" => environment_ruby_engine.to_s,
|
|
51
54
|
"ruby_version" => environment_ruby_version.to_s,
|
|
52
55
|
"analyzer_versions" => sorted_analyzer_versions(outcome)
|
|
53
56
|
},
|
|
@@ -96,13 +99,19 @@ module RailVerdict
|
|
|
96
99
|
context_versions = outcome.context&.analyzer_versions
|
|
97
100
|
if context_versions.is_a?(Hash) && !context_versions.empty?
|
|
98
101
|
context_versions.each do |key, value|
|
|
99
|
-
|
|
102
|
+
canonical = RailVerdict::Analyzers::Shared.canonical_tool_version(value)
|
|
103
|
+
next if canonical == "unknown"
|
|
104
|
+
|
|
105
|
+
versions[key.to_s] = canonical
|
|
100
106
|
end
|
|
101
107
|
else
|
|
102
108
|
Array(outcome.result.analyzer_results).each do |analyzer|
|
|
103
109
|
next unless analyzer.tool_version.is_a?(String) && !analyzer.tool_version.strip.empty?
|
|
104
110
|
|
|
105
|
-
|
|
111
|
+
canonical = RailVerdict::Analyzers::Shared.canonical_tool_version(analyzer.tool_version)
|
|
112
|
+
next if canonical == "unknown"
|
|
113
|
+
|
|
114
|
+
versions[analyzer.analyzer] = canonical
|
|
106
115
|
end
|
|
107
116
|
end
|
|
108
117
|
versions.sort.to_h
|
|
@@ -238,7 +247,7 @@ module RailVerdict
|
|
|
238
247
|
end
|
|
239
248
|
private :state_components_of
|
|
240
249
|
|
|
241
|
-
Validation = Struct.new(:status, :reasons, :gate, :completion_status, :current_repository_digest, keyword_init: true)
|
|
250
|
+
Validation = Struct.new(:status, :reasons, :gate, :completion_status, :current_repository_digest, :current_environment_digest, keyword_init: true)
|
|
242
251
|
|
|
243
252
|
FRESH = "fresh"
|
|
244
253
|
STALE = "stale"
|
|
@@ -248,21 +257,73 @@ module RailVerdict
|
|
|
248
257
|
# Full black-box evaluation of a serialized receipt document against the
|
|
249
258
|
# current state: integrity first (invalid), then freshness.
|
|
250
259
|
# Returns [validation_document, receipt_or_nil].
|
|
251
|
-
|
|
260
|
+
# When current_environment/current_state are not supplied but repository_root is,
|
|
261
|
+
# they are independently observed via the canonical VerificationIdentity.
|
|
262
|
+
def self.evaluate(document_text, current_state: nil, repository_root: nil, configuration_paths: nil, railverdict_version: RailVerdict::VERSION, environment_ruby_version: RUBY_VERSION, environment_ruby_engine: RUBY_ENGINE, current_analyzer_versions: nil, current_environment: nil)
|
|
252
263
|
receipt, reason = parse(document_text)
|
|
253
264
|
if reason
|
|
254
265
|
return [validation_document(
|
|
255
|
-
Validation.new(status: INVALID, reasons: [reason.to_s], gate: nil, completion_status: nil, current_repository_digest: nil),
|
|
266
|
+
Validation.new(status: INVALID, reasons: [reason.to_s], gate: nil, completion_status: nil, current_repository_digest: nil, current_environment_digest: nil),
|
|
256
267
|
receipt_id: embedded_receipt_id(document_text)
|
|
257
268
|
), nil]
|
|
258
269
|
end
|
|
259
270
|
|
|
271
|
+
# Canonical re-observation if not supplied (fail-closed trust invariant)
|
|
272
|
+
if current_state.nil? && repository_root
|
|
273
|
+
begin
|
|
274
|
+
root_real = File.realpath(repository_root)
|
|
275
|
+
paths = configuration_paths || Check.effective_input_paths(root: root_real, config_path: File.join(root_real, ".railverdict.yml"))
|
|
276
|
+
current_state = RepositoryState.capture(repository_root: root_real, configuration_paths: paths)
|
|
277
|
+
rescue StandardError
|
|
278
|
+
current_state = RepositoryState.unavailable(:repository_root_unavailable)
|
|
279
|
+
end
|
|
280
|
+
end
|
|
281
|
+
|
|
282
|
+
if current_environment.nil? && current_analyzer_versions.nil? && repository_root
|
|
283
|
+
begin
|
|
284
|
+
root_real = File.realpath(repository_root || Dir.pwd)
|
|
285
|
+
stored_analyzer_keys = receipt.document.dig("environment", "analyzer_versions")&.keys || []
|
|
286
|
+
# Resolve configuration for relevant probing
|
|
287
|
+
config = nil
|
|
288
|
+
begin
|
|
289
|
+
paths = configuration_paths || Check.effective_input_paths(root: root_real, config_path: File.join(root_real, ".railverdict.yml"))
|
|
290
|
+
cfg_path = paths[:config]
|
|
291
|
+
config = Configuration.load(cfg_path) if cfg_path && File.file?(cfg_path)
|
|
292
|
+
rescue StandardError
|
|
293
|
+
config = nil
|
|
294
|
+
end
|
|
295
|
+
current_environment = VerificationEnvironment.capture_for_receipt(stored_analyzer_keys, repository_root: root_real, configuration: config)
|
|
296
|
+
rescue StandardError
|
|
297
|
+
current_environment = VerificationEnvironment.new(
|
|
298
|
+
railverdict_version: RailVerdict::VERSION.to_s,
|
|
299
|
+
ruby_engine: RUBY_ENGINE.to_s,
|
|
300
|
+
ruby_version: RUBY_VERSION.to_s,
|
|
301
|
+
analyzer_versions: {},
|
|
302
|
+
digest: nil,
|
|
303
|
+
unavailable_reason: "environment_capture_failed",
|
|
304
|
+
available: false
|
|
305
|
+
)
|
|
306
|
+
end
|
|
307
|
+
elsif current_analyzer_versions.is_a?(Hash) && current_environment.nil?
|
|
308
|
+
# Legacy caller supplied analyzer versions hash directly -> wrap as environment
|
|
309
|
+
current_environment = VerificationEnvironment.new(
|
|
310
|
+
railverdict_version: railverdict_version.to_s,
|
|
311
|
+
ruby_engine: environment_ruby_engine.to_s,
|
|
312
|
+
ruby_version: environment_ruby_version.to_s,
|
|
313
|
+
analyzer_versions: current_analyzer_versions.sort.to_h.transform_values(&:to_s),
|
|
314
|
+
digest: nil,
|
|
315
|
+
available: true
|
|
316
|
+
)
|
|
317
|
+
end
|
|
318
|
+
|
|
260
319
|
validation = validate_freshness(
|
|
261
320
|
receipt: receipt,
|
|
262
321
|
current_state: current_state,
|
|
263
322
|
railverdict_version: railverdict_version,
|
|
264
323
|
environment_ruby_version: environment_ruby_version,
|
|
265
|
-
|
|
324
|
+
environment_ruby_engine: environment_ruby_engine,
|
|
325
|
+
current_analyzer_versions: current_analyzer_versions,
|
|
326
|
+
current_environment: current_environment
|
|
266
327
|
)
|
|
267
328
|
[validation_document(validation, receipt_id: receipt.receipt_id), receipt]
|
|
268
329
|
end
|
|
@@ -288,6 +349,7 @@ module RailVerdict
|
|
|
288
349
|
"completion_status" => validation.completion_status,
|
|
289
350
|
"current_repository_digest" => validation.current_repository_digest
|
|
290
351
|
}
|
|
352
|
+
document["current_environment_digest"] = validation.current_environment_digest if validation.current_environment_digest
|
|
291
353
|
errors = SchemaValidator.validate_receipt_validation(document)
|
|
292
354
|
raise RailVerdict::Error, "receipt-validation-v1 failed: #{errors.join('; ')}" unless errors.empty?
|
|
293
355
|
|
|
@@ -296,18 +358,43 @@ module RailVerdict
|
|
|
296
358
|
private_class_method :validation_document
|
|
297
359
|
|
|
298
360
|
# Validates a parsed receipt against the current observable state.
|
|
299
|
-
#
|
|
300
|
-
#
|
|
301
|
-
#
|
|
302
|
-
|
|
303
|
-
|
|
304
|
-
|
|
361
|
+
# This is the ONE canonical freshness evaluator — CLI and MCP must delegate here.
|
|
362
|
+
# It independently requires current_state and current_environment to be observed,
|
|
363
|
+
# never trusting receipt-provided values as current observation.
|
|
364
|
+
def self.validate_freshness(receipt:, current_state:, railverdict_version: RailVerdict::VERSION, environment_ruby_version: RUBY_VERSION, environment_ruby_engine: RUBY_ENGINE, current_analyzer_versions: nil, current_environment: nil)
|
|
365
|
+
# If current_environment was supplied via legacy analyzer_versions hash, normalize
|
|
366
|
+
if current_environment.nil? && current_analyzer_versions.is_a?(Hash) && !current_analyzer_versions.empty?
|
|
367
|
+
current_environment = VerificationEnvironment.new(
|
|
368
|
+
railverdict_version: railverdict_version.to_s,
|
|
369
|
+
ruby_engine: environment_ruby_engine.to_s,
|
|
370
|
+
ruby_version: environment_ruby_version.to_s,
|
|
371
|
+
analyzer_versions: current_analyzer_versions.sort.to_h.transform_values(&:to_s),
|
|
372
|
+
digest: nil,
|
|
373
|
+
available: true
|
|
374
|
+
)
|
|
375
|
+
end
|
|
376
|
+
|
|
377
|
+
unless current_state&.available?
|
|
378
|
+
reason = current_state&.unavailable_reason || :repository_state_unavailable
|
|
305
379
|
return Validation.new(
|
|
306
380
|
status: UNAVAILABLE,
|
|
307
|
-
reasons: ["repository_state_unavailable:#{
|
|
381
|
+
reasons: ["repository_state_unavailable:#{reason}"],
|
|
308
382
|
gate: receipt&.gate,
|
|
309
383
|
completion_status: receipt&.completion_status,
|
|
310
|
-
current_repository_digest: nil
|
|
384
|
+
current_repository_digest: nil,
|
|
385
|
+
current_environment_digest: nil
|
|
386
|
+
)
|
|
387
|
+
end
|
|
388
|
+
|
|
389
|
+
# Environment must be observable fail-closed
|
|
390
|
+
if current_environment && !current_environment.available?
|
|
391
|
+
return Validation.new(
|
|
392
|
+
status: UNAVAILABLE,
|
|
393
|
+
reasons: ["analyzer_version_unobservable:#{current_environment.unavailable_reason}"],
|
|
394
|
+
gate: receipt.gate,
|
|
395
|
+
completion_status: receipt.completion_status,
|
|
396
|
+
current_repository_digest: current_state.digest,
|
|
397
|
+
current_environment_digest: nil
|
|
311
398
|
)
|
|
312
399
|
end
|
|
313
400
|
|
|
@@ -322,19 +409,73 @@ module RailVerdict
|
|
|
322
409
|
reasons << "waivers_changed" if stored.fetch("waivers_digest") != current.fetch("waivers_digest")
|
|
323
410
|
|
|
324
411
|
environment = receipt.document.fetch("environment")
|
|
325
|
-
|
|
326
|
-
|
|
327
|
-
if
|
|
328
|
-
|
|
329
|
-
|
|
412
|
+
# RailVerdict version is always checked against current (not receipt-provided)
|
|
413
|
+
current_rv = current_environment ? current_environment.railverdict_version.to_s : railverdict_version.to_s
|
|
414
|
+
reasons << "railverdict_version_changed" if receipt.document.fetch("railverdict_version") != current_rv
|
|
415
|
+
|
|
416
|
+
# Ruby version/engine: if environment observable, use it; else use supplied
|
|
417
|
+
current_ruby_version = current_environment ? current_environment.ruby_version.to_s : environment_ruby_version.to_s
|
|
418
|
+
current_ruby_engine = current_environment ? current_environment.ruby_engine.to_s : environment_ruby_engine.to_s
|
|
419
|
+
# Ruby engine: receipt may not have it (1.2 compat) — if missing, treat as mismatch if current engine is present
|
|
420
|
+
stored_ruby_engine = environment["ruby_engine"]
|
|
421
|
+
if stored_ruby_engine.nil?
|
|
422
|
+
# 1.2 receipt without engine: stale if we now track engine and it differs from default? Keep fresh for compat unless engine is not ruby/mri? For deterministic portable contract, we consider missing engine as stale only if current engine != "ruby"
|
|
423
|
+
# To avoid breaking 1.2 compat trivially, we only flag when env explicitly requires engine and receipt lacks it? Current spec says 1.2 receipts should be valid but stale when env stronger — we will flag as stale to surface drift
|
|
424
|
+
# For minimal breakage, do not flag missing engine as stale automatically; document as known limitation. Only check if receipt has engine.
|
|
425
|
+
else
|
|
426
|
+
reasons << "ruby_engine_changed" if stored_ruby_engine.to_s != current_ruby_engine
|
|
427
|
+
end
|
|
428
|
+
if environment.fetch("ruby_version") != current_ruby_version
|
|
429
|
+
reasons << "ruby_version_changed"
|
|
430
|
+
end
|
|
431
|
+
|
|
432
|
+
# Analyzer environment: only relevant analyzers (those in receipt)
|
|
433
|
+
stored_analyzers = environment.fetch("analyzer_versions")
|
|
434
|
+
if stored_analyzers.is_a?(Hash) && !stored_analyzers.empty?
|
|
435
|
+
current_analyzers = if current_environment
|
|
436
|
+
# Only compare relevant keys
|
|
437
|
+
filtered = current_environment.analyzer_versions.select { |k, _| stored_analyzers.key?(k) }
|
|
438
|
+
# Also include any stored key missing in current -> drift (probed enabled set may have disabled it)
|
|
439
|
+
# If stored key not in current, we need to probe it specifically; for now treat missing as stale
|
|
440
|
+
stored_analyzers.keys.each do |k|
|
|
441
|
+
filtered[k] ||= "__missing__"
|
|
442
|
+
end
|
|
443
|
+
filtered
|
|
444
|
+
else
|
|
445
|
+
current_analyzer_versions.is_a?(Hash) ? current_analyzer_versions.sort.to_h.transform_values(&:to_s) : nil
|
|
446
|
+
end
|
|
447
|
+
if current_analyzers
|
|
448
|
+
# Detect unknown in current as unavailable handled above; now compare relevant
|
|
449
|
+
relevant_stored = stored_analyzers.sort.to_h.transform_values(&:to_s)
|
|
450
|
+
relevant_current = current_analyzers.sort.to_h.transform_values(&:to_s)
|
|
451
|
+
# If any stored "unknown" exists, treat as unavailable (fail-closed)
|
|
452
|
+
if relevant_stored.values.include?("unknown") || relevant_current.values.include?("unknown")
|
|
453
|
+
return Validation.new(
|
|
454
|
+
status: UNAVAILABLE,
|
|
455
|
+
reasons: ["analyzer_version_unobservable:unknown_or_missing"],
|
|
456
|
+
gate: receipt.gate,
|
|
457
|
+
completion_status: receipt.completion_status,
|
|
458
|
+
current_repository_digest: current_state.digest,
|
|
459
|
+
current_environment_digest: current_environment&.digest
|
|
460
|
+
)
|
|
461
|
+
end
|
|
462
|
+
# Missing relevant analyzer in current env is drift -> stale, not unavailable
|
|
463
|
+
if relevant_current.values.include?("__missing__")
|
|
464
|
+
reasons << "analyzer_environment_changed"
|
|
465
|
+
elsif relevant_stored != relevant_current
|
|
466
|
+
reasons << "analyzer_environment_changed"
|
|
467
|
+
end
|
|
468
|
+
end
|
|
330
469
|
end
|
|
331
470
|
|
|
471
|
+
status = reasons.empty? ? FRESH : STALE
|
|
332
472
|
Validation.new(
|
|
333
|
-
status:
|
|
473
|
+
status: status,
|
|
334
474
|
reasons: reasons.sort,
|
|
335
475
|
gate: receipt.gate,
|
|
336
476
|
completion_status: receipt.completion_status,
|
|
337
|
-
current_repository_digest: current_state.digest
|
|
477
|
+
current_repository_digest: current_state.digest,
|
|
478
|
+
current_environment_digest: current_environment&.digest
|
|
338
479
|
)
|
|
339
480
|
end
|
|
340
481
|
end
|
|
@@ -0,0 +1,220 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "digest"
|
|
4
|
+
require_relative "canonical_json"
|
|
5
|
+
require_relative "process_runner"
|
|
6
|
+
|
|
7
|
+
module RailVerdict
|
|
8
|
+
class VerificationEnvironment
|
|
9
|
+
MAX_ENV_PROBE_BYTES = 64 * 1024
|
|
10
|
+
MAX_PROBE_TIMEOUT = 5.0
|
|
11
|
+
|
|
12
|
+
attr_reader :railverdict_version, :ruby_engine, :ruby_version, :analyzer_versions, :digest, :unavailable_reason, :available
|
|
13
|
+
|
|
14
|
+
def initialize(railverdict_version:, ruby_engine:, ruby_version:, analyzer_versions:, digest: nil, unavailable_reason: nil, available: true)
|
|
15
|
+
@railverdict_version = railverdict_version
|
|
16
|
+
@ruby_engine = ruby_engine
|
|
17
|
+
@ruby_version = ruby_version
|
|
18
|
+
@analyzer_versions = analyzer_versions.freeze
|
|
19
|
+
@digest = digest
|
|
20
|
+
@unavailable_reason = unavailable_reason
|
|
21
|
+
@available = available
|
|
22
|
+
freeze
|
|
23
|
+
end
|
|
24
|
+
|
|
25
|
+
def available?
|
|
26
|
+
@available
|
|
27
|
+
end
|
|
28
|
+
|
|
29
|
+
def self.capture(repository_root:, configuration: nil, runner: ProcessRunner, timeout_seconds: MAX_PROBE_TIMEOUT)
|
|
30
|
+
railverdict_version = RailVerdict::VERSION.to_s
|
|
31
|
+
ruby_engine = RUBY_ENGINE.to_s
|
|
32
|
+
ruby_version = RUBY_VERSION.to_s
|
|
33
|
+
|
|
34
|
+
config = configuration
|
|
35
|
+
if config.nil? && repository_root
|
|
36
|
+
begin
|
|
37
|
+
root_real = File.realpath(repository_root)
|
|
38
|
+
config_path = File.join(root_real, ".railverdict.yml")
|
|
39
|
+
config = Configuration.load(config_path) if File.file?(config_path)
|
|
40
|
+
rescue StandardError
|
|
41
|
+
config = nil
|
|
42
|
+
end
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
analyzer_versions = {}
|
|
46
|
+
probes_failed = []
|
|
47
|
+
if config
|
|
48
|
+
config.analyzers.each do |name, selection|
|
|
49
|
+
next unless selection.fetch("enabled")
|
|
50
|
+
|
|
51
|
+
adapter_class = Check::REGISTRY[name]
|
|
52
|
+
next unless adapter_class
|
|
53
|
+
|
|
54
|
+
adapter = build_adapter(name)
|
|
55
|
+
next unless adapter
|
|
56
|
+
|
|
57
|
+
probe_timeout = resolve_probe_timeout(config, name, timeout_seconds)
|
|
58
|
+
probe = probe_adapter(adapter, repository_root, runner, probe_timeout)
|
|
59
|
+
# Fallback for bundle exec failures in cloned fixtures without bundle install
|
|
60
|
+
if probe.nil? || probe.status != "succeeded"
|
|
61
|
+
fallback = probe_fallback(name, repository_root, runner, probe_timeout)
|
|
62
|
+
probe = fallback if fallback && fallback.status == "succeeded"
|
|
63
|
+
end
|
|
64
|
+
if name.to_s == "simplecov" && probe && probe.status == "unavailable" && probe.message.to_s.include?("coverage file is absent")
|
|
65
|
+
# SimpleCov without coverage file is not a version-probe failure; skip it
|
|
66
|
+
next
|
|
67
|
+
end
|
|
68
|
+
if probe.nil? || probe.status != "succeeded" || probe.version.nil? || probe.version.to_s.strip.empty?
|
|
69
|
+
# Non-required analyzers failing probe should not make environment unavailable; they are advisory
|
|
70
|
+
if selection["required"] == false
|
|
71
|
+
next
|
|
72
|
+
end
|
|
73
|
+
probes_failed << name
|
|
74
|
+
next
|
|
75
|
+
end
|
|
76
|
+
version = RailVerdict::Analyzers::Shared.canonical_tool_version(probe.version)
|
|
77
|
+
analyzer_versions[name.to_s] = version
|
|
78
|
+
end
|
|
79
|
+
else
|
|
80
|
+
# No configuration observable -> try all known analyzers that are present?
|
|
81
|
+
# Conservative: probe all registry entries cheaply; only succeeded probes contribute.
|
|
82
|
+
# Missing probes do not make unavailable if no config exists; use empty relevant set.
|
|
83
|
+
end
|
|
84
|
+
|
|
85
|
+
# If any enabled analyzer probe failed, environment is unobservable fail-closed
|
|
86
|
+
unless probes_failed.empty?
|
|
87
|
+
return new(
|
|
88
|
+
railverdict_version: railverdict_version,
|
|
89
|
+
ruby_engine: ruby_engine,
|
|
90
|
+
ruby_version: ruby_version,
|
|
91
|
+
analyzer_versions: analyzer_versions.sort.to_h,
|
|
92
|
+
digest: nil,
|
|
93
|
+
unavailable_reason: "analyzer_version_unobservable:#{probes_failed.sort.join(',')}",
|
|
94
|
+
available: false
|
|
95
|
+
)
|
|
96
|
+
end
|
|
97
|
+
|
|
98
|
+
# Detect "unknown" canonicalized versions as unobservable as well
|
|
99
|
+
unknown_keys = analyzer_versions.select { |_, v| v == "unknown" }.keys
|
|
100
|
+
unless unknown_keys.empty?
|
|
101
|
+
return new(
|
|
102
|
+
railverdict_version: railverdict_version,
|
|
103
|
+
ruby_engine: ruby_engine,
|
|
104
|
+
ruby_version: ruby_version,
|
|
105
|
+
analyzer_versions: analyzer_versions.sort.to_h,
|
|
106
|
+
digest: nil,
|
|
107
|
+
unavailable_reason: "analyzer_version_unobservable:#{unknown_keys.sort.join(',')}",
|
|
108
|
+
available: false
|
|
109
|
+
)
|
|
110
|
+
end
|
|
111
|
+
|
|
112
|
+
sorted = analyzer_versions.sort.to_h
|
|
113
|
+
payload = {
|
|
114
|
+
"railverdict_version" => railverdict_version,
|
|
115
|
+
"ruby_engine" => ruby_engine,
|
|
116
|
+
"ruby_version" => ruby_version,
|
|
117
|
+
"analyzer_versions" => sorted
|
|
118
|
+
}
|
|
119
|
+
digest = "sha256:#{Digest::SHA256.hexdigest(CanonicalJSON.generate(payload))}"
|
|
120
|
+
new(
|
|
121
|
+
railverdict_version: railverdict_version,
|
|
122
|
+
ruby_engine: ruby_engine,
|
|
123
|
+
ruby_version: ruby_version,
|
|
124
|
+
analyzer_versions: sorted,
|
|
125
|
+
digest: digest,
|
|
126
|
+
available: true
|
|
127
|
+
)
|
|
128
|
+
rescue StandardError => error
|
|
129
|
+
new(
|
|
130
|
+
railverdict_version: RailVerdict::VERSION.to_s,
|
|
131
|
+
ruby_engine: RUBY_ENGINE.to_s,
|
|
132
|
+
ruby_version: RUBY_VERSION.to_s,
|
|
133
|
+
analyzer_versions: {},
|
|
134
|
+
digest: nil,
|
|
135
|
+
unavailable_reason: "environment_capture_failed:#{error.class}",
|
|
136
|
+
available: false
|
|
137
|
+
)
|
|
138
|
+
end
|
|
139
|
+
|
|
140
|
+
def self.capture_for_receipt(receipt_analyzer_keys, repository_root:, configuration: nil, runner: ProcessRunner, timeout_seconds: MAX_PROBE_TIMEOUT)
|
|
141
|
+
# Only probe analyzers that are relevant (present in receipt)
|
|
142
|
+
keys = Array(receipt_analyzer_keys).map(&:to_s)
|
|
143
|
+
return capture(repository_root: repository_root, configuration: configuration, runner: runner, timeout_seconds: timeout_seconds) if keys.empty?
|
|
144
|
+
|
|
145
|
+
env = capture(repository_root: repository_root, configuration: configuration, runner: runner, timeout_seconds: timeout_seconds)
|
|
146
|
+
return env unless env.available?
|
|
147
|
+
|
|
148
|
+
# Filter to only relevant keys, but if any relevant key missing from current env -> stale (drift)
|
|
149
|
+
# The unavailable check already handled probe failures for enabled analyzers.
|
|
150
|
+
# Here we need to ensure we return only relevant versions for comparison.
|
|
151
|
+
filtered = env.analyzer_versions.select { |k, _| keys.include?(k) }
|
|
152
|
+
# If a relevant analyzer is now disabled in config but was in receipt, it's drift -> but we keep filtered empty; caller will detect mismatch
|
|
153
|
+
payload = {
|
|
154
|
+
"railverdict_version" => env.railverdict_version,
|
|
155
|
+
"ruby_engine" => env.ruby_engine,
|
|
156
|
+
"ruby_version" => env.ruby_version,
|
|
157
|
+
"analyzer_versions" => filtered.sort.to_h
|
|
158
|
+
}
|
|
159
|
+
digest = "sha256:#{Digest::SHA256.hexdigest(CanonicalJSON.generate(payload))}"
|
|
160
|
+
new(
|
|
161
|
+
railverdict_version: env.railverdict_version,
|
|
162
|
+
ruby_engine: env.ruby_engine,
|
|
163
|
+
ruby_version: env.ruby_version,
|
|
164
|
+
analyzer_versions: filtered.sort.to_h,
|
|
165
|
+
digest: digest,
|
|
166
|
+
available: true
|
|
167
|
+
)
|
|
168
|
+
end
|
|
169
|
+
|
|
170
|
+
private_class_method def self.build_adapter(name)
|
|
171
|
+
case name
|
|
172
|
+
when "rubocop" then RailVerdict::Analyzers::RuboCop.new
|
|
173
|
+
when "minitest" then RailVerdict::Analyzers::Minitest.new
|
|
174
|
+
when "rspec" then RailVerdict::Analyzers::RSpec.new
|
|
175
|
+
when "simplecov" then RailVerdict::Analyzers::SimpleCov.new
|
|
176
|
+
when "bundler_audit" then RailVerdict::Analyzers::BundlerAudit.new
|
|
177
|
+
end
|
|
178
|
+
rescue StandardError
|
|
179
|
+
nil
|
|
180
|
+
end
|
|
181
|
+
|
|
182
|
+
private_class_method def self.probe_adapter(adapter, root, runner, timeout)
|
|
183
|
+
adapter.probe(root, runner: runner, timeout_seconds: timeout)
|
|
184
|
+
rescue StandardError
|
|
185
|
+
nil
|
|
186
|
+
end
|
|
187
|
+
|
|
188
|
+
private_class_method def self.probe_fallback(name, root, runner, timeout)
|
|
189
|
+
# Try direct executable without bundle for version probing
|
|
190
|
+
cmd = case name.to_s
|
|
191
|
+
when "rubocop" then { executable: "rubocop", args_prefix: [] }
|
|
192
|
+
when "rspec" then { executable: "rspec", args_prefix: [] }
|
|
193
|
+
when "bundler_audit" then { executable: "bundler-audit", args_prefix: [] }
|
|
194
|
+
when "minitest" then { executable: "ruby", args_prefix: ["-rminitest", "-e", "puts Minitest::VERSION"] }
|
|
195
|
+
else nil
|
|
196
|
+
end
|
|
197
|
+
return nil unless cmd
|
|
198
|
+
|
|
199
|
+
# Create a temporary adapter with fallback command resolver
|
|
200
|
+
fallback_adapter = case name.to_s
|
|
201
|
+
when "rubocop" then RailVerdict::Analyzers::RuboCop.new(command_resolver: ->(_) { cmd })
|
|
202
|
+
when "rspec" then RailVerdict::Analyzers::RSpec.new(command_resolver: ->(_) { cmd })
|
|
203
|
+
when "bundler_audit" then RailVerdict::Analyzers::BundlerAudit.new(command_resolver: ->(_) { cmd })
|
|
204
|
+
when "minitest" then RailVerdict::Analyzers::Minitest.new(command_resolver: ->(_) { cmd })
|
|
205
|
+
else nil
|
|
206
|
+
end
|
|
207
|
+
return nil unless fallback_adapter
|
|
208
|
+
|
|
209
|
+
fallback_adapter.probe(root, runner: runner, timeout_seconds: timeout)
|
|
210
|
+
rescue StandardError
|
|
211
|
+
nil
|
|
212
|
+
end
|
|
213
|
+
|
|
214
|
+
private_class_method def self.resolve_probe_timeout(config, name, default)
|
|
215
|
+
config.analyzer_timeout_seconds(name.to_s) || [default, 5.0].min
|
|
216
|
+
rescue StandardError
|
|
217
|
+
[default, 5.0].min
|
|
218
|
+
end
|
|
219
|
+
end
|
|
220
|
+
end
|
|
@@ -0,0 +1,95 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "digest"
|
|
4
|
+
require_relative "canonical_json"
|
|
5
|
+
require_relative "repository_state"
|
|
6
|
+
require_relative "verification_environment"
|
|
7
|
+
|
|
8
|
+
module RailVerdict
|
|
9
|
+
class VerificationIdentity
|
|
10
|
+
attr_reader :repository_state, :environment, :digest, :unavailable_reason, :available
|
|
11
|
+
|
|
12
|
+
def initialize(repository_state:, environment:, digest: nil, unavailable_reason: nil, available: true)
|
|
13
|
+
@repository_state = repository_state
|
|
14
|
+
@environment = environment
|
|
15
|
+
@digest = digest
|
|
16
|
+
@unavailable_reason = unavailable_reason
|
|
17
|
+
@available = available
|
|
18
|
+
freeze
|
|
19
|
+
end
|
|
20
|
+
|
|
21
|
+
def available?
|
|
22
|
+
@available && @repository_state&.available? && @environment&.available?
|
|
23
|
+
end
|
|
24
|
+
|
|
25
|
+
def self.capture(repository_root:, configuration_paths: nil, configuration: nil, runner: ProcessRunner, timeout_seconds: 5.0, receipt_analyzer_keys: nil)
|
|
26
|
+
repo_state = RepositoryState.capture(repository_root: repository_root, configuration_paths: configuration_paths)
|
|
27
|
+
|
|
28
|
+
unless repo_state.available?
|
|
29
|
+
return new(
|
|
30
|
+
repository_state: repo_state,
|
|
31
|
+
environment: VerificationEnvironment.new(
|
|
32
|
+
railverdict_version: RailVerdict::VERSION.to_s,
|
|
33
|
+
ruby_engine: RUBY_ENGINE.to_s,
|
|
34
|
+
ruby_version: RUBY_VERSION.to_s,
|
|
35
|
+
analyzer_versions: {},
|
|
36
|
+
digest: nil,
|
|
37
|
+
unavailable_reason: repo_state.unavailable_reason,
|
|
38
|
+
available: false
|
|
39
|
+
),
|
|
40
|
+
digest: nil,
|
|
41
|
+
unavailable_reason: "repository_state_unavailable:#{repo_state.unavailable_reason}",
|
|
42
|
+
available: false
|
|
43
|
+
)
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
config = configuration
|
|
47
|
+
if config.nil? && repository_root
|
|
48
|
+
begin
|
|
49
|
+
# Try to load config for environment capture to know enabled analyzers
|
|
50
|
+
paths = configuration_paths || Check.effective_input_paths(root: File.realpath(repository_root), config_path: File.join(File.realpath(repository_root), ".railverdict.yml"))
|
|
51
|
+
config_path = paths[:config]
|
|
52
|
+
config = Configuration.load(config_path) if config_path && File.file?(config_path)
|
|
53
|
+
rescue StandardError
|
|
54
|
+
config = nil
|
|
55
|
+
end
|
|
56
|
+
end
|
|
57
|
+
|
|
58
|
+
env = if receipt_analyzer_keys
|
|
59
|
+
VerificationEnvironment.capture_for_receipt(receipt_analyzer_keys, repository_root: repository_root, configuration: config, runner: runner, timeout_seconds: timeout_seconds)
|
|
60
|
+
else
|
|
61
|
+
VerificationEnvironment.capture(repository_root: repository_root, configuration: config, runner: runner, timeout_seconds: timeout_seconds)
|
|
62
|
+
end
|
|
63
|
+
|
|
64
|
+
unless env.available?
|
|
65
|
+
return new(
|
|
66
|
+
repository_state: repo_state,
|
|
67
|
+
environment: env,
|
|
68
|
+
digest: nil,
|
|
69
|
+
unavailable_reason: env.unavailable_reason,
|
|
70
|
+
available: false
|
|
71
|
+
)
|
|
72
|
+
end
|
|
73
|
+
|
|
74
|
+
# Digest over both repository and environment digests
|
|
75
|
+
payload = {
|
|
76
|
+
"repository_digest" => repo_state.digest,
|
|
77
|
+
"environment_digest" => env.digest
|
|
78
|
+
}
|
|
79
|
+
digest = "sha256:#{Digest::SHA256.hexdigest(CanonicalJSON.generate(payload))}"
|
|
80
|
+
new(repository_state: repo_state, environment: env, digest: digest, available: true)
|
|
81
|
+
rescue StandardError => error
|
|
82
|
+
repo_unavailable = RepositoryState.unavailable(:capture_failed)
|
|
83
|
+
env_unavailable = VerificationEnvironment.new(
|
|
84
|
+
railverdict_version: RailVerdict::VERSION.to_s,
|
|
85
|
+
ruby_engine: RUBY_ENGINE.to_s,
|
|
86
|
+
ruby_version: RUBY_VERSION.to_s,
|
|
87
|
+
analyzer_versions: {},
|
|
88
|
+
digest: nil,
|
|
89
|
+
unavailable_reason: error.class.to_s,
|
|
90
|
+
available: false
|
|
91
|
+
)
|
|
92
|
+
new(repository_state: repo_unavailable, environment: env_unavailable, digest: nil, unavailable_reason: "capture_failed:#{error.class}", available: false)
|
|
93
|
+
end
|
|
94
|
+
end
|
|
95
|
+
end
|
data/lib/rail_verdict/version.rb
CHANGED
data/lib/rail_verdict.rb
CHANGED
|
@@ -41,6 +41,8 @@ require_relative "rail_verdict/pr_intelligence"
|
|
|
41
41
|
require_relative "rail_verdict/path_safety"
|
|
42
42
|
require_relative "rail_verdict/canonical_json"
|
|
43
43
|
require_relative "rail_verdict/repository_state"
|
|
44
|
+
require_relative "rail_verdict/verification_environment"
|
|
45
|
+
require_relative "rail_verdict/verification_identity"
|
|
44
46
|
require_relative "rail_verdict/receipt"
|
|
45
47
|
require_relative "rail_verdict/cli"
|
|
46
48
|
require_relative "rail_verdict/intelligence"
|
|
@@ -52,6 +52,7 @@
|
|
|
52
52
|
"additionalProperties": false,
|
|
53
53
|
"required": ["ruby_version", "analyzer_versions"],
|
|
54
54
|
"properties": {
|
|
55
|
+
"ruby_engine": { "type": "string", "minLength": 1, "maxLength": 32 },
|
|
55
56
|
"ruby_version": { "type": "string", "minLength": 1, "maxLength": 32 },
|
|
56
57
|
"analyzer_versions": {
|
|
57
58
|
"type": "object",
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: rail_verdict
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 1.
|
|
4
|
+
version: 1.3.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Pedro Dalben
|
|
@@ -154,6 +154,8 @@ files:
|
|
|
154
154
|
- lib/rail_verdict/schema_validator.rb
|
|
155
155
|
- lib/rail_verdict/strict_yaml.rb
|
|
156
156
|
- lib/rail_verdict/verification/policy.rb
|
|
157
|
+
- lib/rail_verdict/verification_environment.rb
|
|
158
|
+
- lib/rail_verdict/verification_identity.rb
|
|
157
159
|
- lib/rail_verdict/version.rb
|
|
158
160
|
- lib/rail_verdict/waiver.rb
|
|
159
161
|
- lib/rail_verdict/waiver_store.rb
|