reeve 0.4.0 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +71 -1
- data/README.md +90 -7
- data/lib/reeve/authorization.rb +1 -11
- data/lib/reeve/configuration.rb +25 -1
- data/lib/reeve/decision.rb +6 -1
- data/lib/reeve/errors.rb +13 -0
- data/lib/reeve/inventory/report.rb +146 -0
- data/lib/reeve/inventory.rb +212 -0
- data/lib/reeve/invocation.rb +3 -1
- data/lib/reeve/testing/assertions.rb +2 -2
- data/lib/reeve/testing/checks/endpoint_coverage.rb +133 -0
- data/lib/reeve/testing/checks.rb +17 -6
- data/lib/reeve/testing/compliance_assertions.rb +3 -1
- data/lib/reeve/testing/compliance_suite.rb +4 -1
- data/lib/reeve/testing/matchers/pass_reeve_check.rb +1 -1
- data/lib/reeve/testing/report.rb +22 -6
- data/lib/reeve/testing/result.rb +28 -8
- data/lib/reeve/testing.rb +19 -3
- data/lib/reeve/version.rb +1 -1
- data/lib/reeve.rb +1 -0
- metadata +5 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 67bc82ca855e0385d021e54477cbcfcef697a1134d3c93e4fc073fea2ff3b748
|
|
4
|
+
data.tar.gz: 0c11093e604b6a056a23bec7ebd419620d4dd1780f93ff7e30fc8a08bdbd92d3
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 0524a6cd7376dc9cc8b7f2611d4a0a6a8b1fa81b6b5ff59ab662d3e5f5f313b4f47e7ad5b2d52d272da8bab1f1c19042ae937300beb382f57804f2fc72b77d3e
|
|
7
|
+
data.tar.gz: 4ead6c668edcde8b543c1e4ead9817e08015ecbf42adfbc34d44c5fa31426206284efb3ffdd88dad4cd983033be7708e256ac9cc2e5eb4758a6fc92f2ac14a19
|
data/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,76 @@ All notable changes are recorded here. This project follows [Semantic
|
|
|
4
4
|
Versioning](https://semver.org), with one rule specific to what it does — see
|
|
5
5
|
[Versioning policy](#versioning-policy).
|
|
6
6
|
|
|
7
|
+
## [0.5.0] - 2026-10-01
|
|
8
|
+
|
|
9
|
+
Reeve protects the tools routed through it, and only those. A server that registers its
|
|
10
|
+
tools as names and handler blocks can send some through `Reeve.invoke` and run the rest
|
|
11
|
+
directly — and the direct ones are not unguarded, they are invisible: no denial, no scope,
|
|
12
|
+
no ledger row, and nothing that says so. The compliance suite could not see them either,
|
|
13
|
+
because it walks tools reeve was told about. A green run certified the subset reeve
|
|
14
|
+
happened to know, and read exactly like a green run over the whole endpoint.
|
|
15
|
+
|
|
16
|
+
**The audit-entry contract is unchanged at `2`, so no migration is required.** One new
|
|
17
|
+
`rule` value, `unbound_tool`, can appear — only for a host that dispatches through an
|
|
18
|
+
inventory.
|
|
19
|
+
|
|
20
|
+
Behaviour changes worth reading before upgrading:
|
|
21
|
+
|
|
22
|
+
- Both compliance suites have one more check, `EndpointCoverage`. Until `config.inventory`
|
|
23
|
+
is set it is **skipped** — pending in RSpec, a skip in Minitest — with a message saying
|
|
24
|
+
the endpoint was not verified. Nothing that passed before fails.
|
|
25
|
+
- A `Report` now counts and prints skipped results (`8 checks, 7 passed, 0 failed, 1
|
|
26
|
+
skipped`, followed by `SKIP EndpointCoverage` and its reason). `passed?` is unchanged: a
|
|
27
|
+
skip is not a failure.
|
|
28
|
+
- A multi-line check message is now indented as one block under its `FAIL` heading.
|
|
29
|
+
|
|
30
|
+
**On Rails 7.0, keep `json` below 3.0.** json 3.0 dropped the `quirks_mode:` keyword that
|
|
31
|
+
ActiveSupport 7.0 still passes to `JSON.generate`, so the ledger cannot serialize a row.
|
|
32
|
+
Under the default `audit_failure_mode = :fail`, reeve fails closed: every invocation raises
|
|
33
|
+
`Reeve::AuditWriteError` rather than running unrecorded — nothing goes unaudited, and
|
|
34
|
+
nothing runs. Under `:warn`, every call runs and none is recorded. The defect is ActiveSupport's, not
|
|
35
|
+
reeve's, and no reeve version works around it; pin `gem "json", "< 3"` until the
|
|
36
|
+
application is on a Rails that supports json 3. Reeve's own CI pins it on the Ruby 3.0
|
|
37
|
+
job for the same reason.
|
|
38
|
+
|
|
39
|
+
### Added
|
|
40
|
+
|
|
41
|
+
- **`Reeve::Inventory`** — the host's registered tool names, which of them route through
|
|
42
|
+
reeve and to which class, and which are exempt and why. Every part takes a value or a
|
|
43
|
+
callable, so a Rails initializer can declare it without autoloading application
|
|
44
|
+
constants. No MCP library involved: a list of names is the whole protocol.
|
|
45
|
+
|
|
46
|
+
- `#report` classifies every registered name as protected and guarded, routed but
|
|
47
|
+
unguarded, bypassing reeve, or exempted — plus anything the inventory names that the
|
|
48
|
+
server does not register, which is usually a rename that left the real name unbound.
|
|
49
|
+
The output names tools, classes and exemption reasons, never an argument, a principal
|
|
50
|
+
or a record, so it is safe in a CI log.
|
|
51
|
+
- `#verify!` raises `Reeve::IncompleteInventoryError`, carrying the report, unless every
|
|
52
|
+
registered name is guarded or exempt. For boot, a deploy step, or CI.
|
|
53
|
+
- `#dispatch(name, ...) { handler }` is the dispatch boundary. A routed name runs through
|
|
54
|
+
`Reeve.invoke`, with the handler block as the body; an exempt name runs as written; any
|
|
55
|
+
other name is treated as a tool with no guard — refused with the new `unbound_tool`
|
|
56
|
+
rule under `unguarded_tools = :deny`, or run unscoped with `guard: "none"` under
|
|
57
|
+
`:allow_with_warning` — and recorded either way. A host that dispatches through it
|
|
58
|
+
cannot have a tool that bypasses reeve.
|
|
59
|
+
- An exemption needs a non-empty reason, and is listed in every report, complete or not.
|
|
60
|
+
|
|
61
|
+
- **`config.inventory`**, and **`Checks::EndpointCoverage`**, which compares a compliance
|
|
62
|
+
run against it. Passes only when the inventory is complete and the run covered all of
|
|
63
|
+
it; fails when the run claimed the whole endpoint and a registered tool bypasses reeve
|
|
64
|
+
or reaches it unguarded; skipped when no inventory is declared, or when
|
|
65
|
+
`compliance_tools` or `tools:` narrowed the run, naming what was left out. The
|
|
66
|
+
incremental path stays open, and a subset can no longer pass for the endpoint.
|
|
67
|
+
|
|
68
|
+
- **Skipped results.** `Testing::Result.skipped`, `Result#skipped?`, `Report#skips` and
|
|
69
|
+
`Report#skipped?`. Only `EndpointCoverage` produces one.
|
|
70
|
+
|
|
71
|
+
### Changed
|
|
72
|
+
|
|
73
|
+
- The compliance suites' default population includes every class the inventory routes to.
|
|
74
|
+
A registry of blocks has no reason to `include Reeve::Guard` anywhere, so a routed class
|
|
75
|
+
with no guard would otherwise never reach `GuardDeclared`.
|
|
76
|
+
|
|
7
77
|
## [0.4.0] - 2026-09-03
|
|
8
78
|
|
|
9
79
|
Everything here came out of a second application integrating the gem — one that had put a
|
|
@@ -342,4 +412,4 @@ is for:
|
|
|
342
412
|
- Adding a nullable ledger column is minor and leaves the audit-entry contract version
|
|
343
413
|
alone. Removing or renaming a column, or changing what a value means, is major and bumps
|
|
344
414
|
it.
|
|
345
|
-
- The audit-entry contract version is `
|
|
415
|
+
- The audit-entry contract version is `2`.
|
data/README.md
CHANGED
|
@@ -7,7 +7,7 @@ A Ruby gem that makes it safe for a Rails application to expose MCP tools to AI
|
|
|
7
7
|
declarative per-record authorization, an append-only audit ledger, and a testing kit that
|
|
8
8
|
proves both hold.
|
|
9
9
|
|
|
10
|
-
> **Status: 0.
|
|
10
|
+
> **Status: 0.5.0.** Read the known limitations in
|
|
11
11
|
> [CHANGELOG.md](CHANGELOG.md) before adopting it — particularly the one about a
|
|
12
12
|
> transaction wrapped around an invocation, if your application wraps requests in one.
|
|
13
13
|
|
|
@@ -242,12 +242,10 @@ config.compliance_tools = -> { [InvoiceSearchTool, InvoiceShowTool] }
|
|
|
242
242
|
```
|
|
243
243
|
|
|
244
244
|
Reeve can only see tools that reached it. If your MCP server dispatches tools Reeve has
|
|
245
|
-
never been told about — a custom controller with its own registry —
|
|
246
|
-
inventory, or
|
|
247
|
-
|
|
248
|
-
|
|
249
|
-
Reeve::Checks.run_all(principals: [alice, bob], tools: McpToolRegistry.tool_classes)
|
|
250
|
-
```
|
|
245
|
+
never been told about — a custom controller with its own registry — declare the server's
|
|
246
|
+
real inventory ([below](#every-tool-the-endpoint-dispatches)), or the run certifies the
|
|
247
|
+
subset Reeve happens to know. Until you do, the suite says so: `EndpointCoverage` is
|
|
248
|
+
reported as skipped, never as passed.
|
|
251
249
|
|
|
252
250
|
As a Rails rake task:
|
|
253
251
|
|
|
@@ -380,6 +378,91 @@ Reeve::Checks.run_all(
|
|
|
380
378
|
That is the whole integration: one call site, your auth untouched, and the same three
|
|
381
379
|
guarantees the fast-mcp adapter gets.
|
|
382
380
|
|
|
381
|
+
## Every tool the endpoint dispatches
|
|
382
|
+
|
|
383
|
+
Reeve protects the tools routed through it, and only those. A server that registers tools
|
|
384
|
+
as names and handler blocks can send some through `Reeve.invoke` and run the rest directly
|
|
385
|
+
— and the direct ones are not unguarded, they are invisible: no denial, no scope, no ledger
|
|
386
|
+
row, and nothing that says so. Deny-by-default cannot deny a call it never sees.
|
|
387
|
+
|
|
388
|
+
`Reeve::Inventory` makes the server's own list of names the thing that gets checked:
|
|
389
|
+
|
|
390
|
+
```ruby
|
|
391
|
+
# config/initializers/reeve.rb
|
|
392
|
+
McpInventory = Reeve::Inventory.new(
|
|
393
|
+
registered: -> { McpServer.tool_names },
|
|
394
|
+
routed: -> { { "search_invoices" => InvoiceSearchTool, "get_invoice" => InvoiceShowTool } },
|
|
395
|
+
exemptions: { "healthcheck" => { reason: "returns no application data" } }
|
|
396
|
+
)
|
|
397
|
+
|
|
398
|
+
Reeve.configure { |config| config.inventory = McpInventory }
|
|
399
|
+
```
|
|
400
|
+
|
|
401
|
+
Each part takes a value or a callable. Callables are read when asked, so the initializer
|
|
402
|
+
neither autoloads application constants nor runs before the server has registered its
|
|
403
|
+
tools.
|
|
404
|
+
|
|
405
|
+
```text
|
|
406
|
+
reeve inventory: 4 registered
|
|
407
|
+
2 protected and guarded
|
|
408
|
+
0 routed through reeve but unguarded
|
|
409
|
+
1 bypasses reeve
|
|
410
|
+
1 exempted
|
|
411
|
+
coverage: incomplete
|
|
412
|
+
|
|
413
|
+
BYPASS export_invoices
|
|
414
|
+
registered with the server but neither routed through reeve nor exempted — route it to
|
|
415
|
+
a guarded tool, or exempt it with a reason
|
|
416
|
+
|
|
417
|
+
EXEMPT healthcheck
|
|
418
|
+
returns no application data
|
|
419
|
+
```
|
|
420
|
+
|
|
421
|
+
The report names tools, classes and exemption reasons — never an argument, a principal or
|
|
422
|
+
a record — so it is safe to print in CI. An exemption without a reason is refused, and one
|
|
423
|
+
with a reason is listed every time, complete or not.
|
|
424
|
+
|
|
425
|
+
**Fail boot or CI on a gap:**
|
|
426
|
+
|
|
427
|
+
```ruby
|
|
428
|
+
McpInventory.verify! # raises Reeve::IncompleteInventoryError, carrying the report
|
|
429
|
+
```
|
|
430
|
+
|
|
431
|
+
**Dispatch through it, and a bypass stops being possible:**
|
|
432
|
+
|
|
433
|
+
```ruby
|
|
434
|
+
def call_tool
|
|
435
|
+
name = params.dig(:params, :name)
|
|
436
|
+
arguments = params.dig(:params, :arguments).to_h.symbolize_keys
|
|
437
|
+
|
|
438
|
+
result = McpInventory.dispatch(name, arguments: arguments,
|
|
439
|
+
agent: { id: request.headers["X-MCP-Client"] }) do
|
|
440
|
+
McpServer.handlers.fetch(name).call(arguments)
|
|
441
|
+
end
|
|
442
|
+
|
|
443
|
+
render json: { jsonrpc: "2.0", id: params[:id], result: serialize(result) }
|
|
444
|
+
rescue Reeve::DeniedError => e
|
|
445
|
+
render json: { jsonrpc: "2.0", id: params[:id],
|
|
446
|
+
error: { code: -32_003, message: e.message } }
|
|
447
|
+
end
|
|
448
|
+
```
|
|
449
|
+
|
|
450
|
+
- A **routed** name runs through `Reeve.invoke` under its class's guard, with the handler
|
|
451
|
+
block as the body. A registry of blocks needs one guarded class per tool to name its
|
|
452
|
+
policy, and nothing else — the block still does the work, and what it returns is scoped.
|
|
453
|
+
Handlers dispatched this way should not call `Reeve.invoke` themselves.
|
|
454
|
+
- An **exempt** name runs the block as written.
|
|
455
|
+
- **Any other name** is a tool with no guard, and `unguarded_tools` decides: refused with
|
|
456
|
+
`unbound_tool` under `:deny`, the default; run unscoped and recorded with `guard: "none"`
|
|
457
|
+
under `:allow_with_warning`. Recorded either way, so a name the inventory missed lands on
|
|
458
|
+
the same ledger worklist as an unguarded class.
|
|
459
|
+
|
|
460
|
+
**In the compliance suite,** `EndpointCoverage` compares the run against the inventory. It
|
|
461
|
+
passes only when every registered tool is guarded or exempt and the run covered all of
|
|
462
|
+
them. It fails when the run claimed the whole endpoint and something bypasses Reeve. A run
|
|
463
|
+
narrowed by `compliance_tools` is reported as a subset — skipped, naming what it left out —
|
|
464
|
+
so a retrofit stays green without passing for complete coverage.
|
|
465
|
+
|
|
383
466
|
Policies are plain objects unless you want Pundit (`authorize` and `scope`, two methods).
|
|
384
467
|
The ledger is an ActiveRecord table unless you supply your own recorder. Records are
|
|
385
468
|
ActiveRecord unless they are not — a plain object with an `id` works.
|
data/lib/reeve/authorization.rb
CHANGED
|
@@ -42,7 +42,7 @@ module Reeve
|
|
|
42
42
|
registry: registry,
|
|
43
43
|
authorizer: Authorization::Authorizer.new,
|
|
44
44
|
scoper: Authorization::Scoper.new,
|
|
45
|
-
config:
|
|
45
|
+
config: config.with_principal(principal)
|
|
46
46
|
) { body ? body.call : run(tool, arguments) }
|
|
47
47
|
end
|
|
48
48
|
end
|
|
@@ -62,15 +62,5 @@ module Reeve
|
|
|
62
62
|
|
|
63
63
|
Authorization::Declaration.new(tool_class: klass, policy: nil, action: :index).tool_name
|
|
64
64
|
end
|
|
65
|
-
|
|
66
|
-
# An explicitly supplied principal is just a resolver that returns it. Routing it
|
|
67
|
-
# through the same resolution step rather than around it keeps one answer to "where
|
|
68
|
-
# did this principal come from" — the envelope still resolves, records and denies
|
|
69
|
-
# identically, and a nil passed in still denies with `no_principal`.
|
|
70
|
-
def configuration_for(principal)
|
|
71
|
-
return config if principal == :unset
|
|
72
|
-
|
|
73
|
-
config.dup.tap { |scoped| scoped.principal_resolver = ->(_context) { principal } }
|
|
74
|
-
end
|
|
75
65
|
end
|
|
76
66
|
end
|
data/lib/reeve/configuration.rb
CHANGED
|
@@ -24,7 +24,7 @@ module Reeve
|
|
|
24
24
|
SETTINGS = %i[
|
|
25
25
|
principal_resolver unguarded_tools audit_failure_mode redact_arguments
|
|
26
26
|
max_recorded_ids policy_adapter default_action audit_recorder logger
|
|
27
|
-
compliance_principals compliance_tools
|
|
27
|
+
compliance_principals compliance_tools inventory
|
|
28
28
|
].freeze
|
|
29
29
|
|
|
30
30
|
attr_reader(*SETTINGS)
|
|
@@ -41,6 +41,30 @@ module Reeve
|
|
|
41
41
|
@logger = nil
|
|
42
42
|
@compliance_principals = nil
|
|
43
43
|
@compliance_tools = nil
|
|
44
|
+
@inventory = nil
|
|
45
|
+
end
|
|
46
|
+
|
|
47
|
+
# The host's whole tool inventory (Reeve::Inventory). Optional. Setting it is what lets
|
|
48
|
+
# the compliance suite say whether it covered the endpoint or only the tools reeve
|
|
49
|
+
# happened to be told about.
|
|
50
|
+
def inventory=(inventory)
|
|
51
|
+
unless inventory.nil? || inventory.is_a?(Inventory)
|
|
52
|
+
raise ArgumentError, "inventory must be a Reeve::Inventory, got #{inventory.inspect}"
|
|
53
|
+
end
|
|
54
|
+
|
|
55
|
+
@inventory = inventory
|
|
56
|
+
end
|
|
57
|
+
|
|
58
|
+
# This configuration with the principal fixed for one invocation.
|
|
59
|
+
#
|
|
60
|
+
# An explicitly supplied principal is just a resolver that returns it. Routing it
|
|
61
|
+
# through the same resolution step rather than around it keeps one answer to "where
|
|
62
|
+
# did this principal come from" — the envelope still resolves, records and denies
|
|
63
|
+
# identically, and a nil passed in still denies with `no_principal`.
|
|
64
|
+
def with_principal(principal)
|
|
65
|
+
return self if principal == :unset
|
|
66
|
+
|
|
67
|
+
dup.tap { |scoped| scoped.principal_resolver = ->(_context) { principal } }
|
|
44
68
|
end
|
|
45
69
|
|
|
46
70
|
# Two fixture principals with disjoint records — the only host setup the compliance
|
data/lib/reeve/decision.rb
CHANGED
|
@@ -18,6 +18,10 @@ module Reeve
|
|
|
18
18
|
# ledger records it as a deny — the trace of a call that blew up is the one most
|
|
19
19
|
# worth having (R5).
|
|
20
20
|
TOOL_ERROR = "tool_error"
|
|
21
|
+
# A name dispatched through an Inventory that neither routes it through reeve nor
|
|
22
|
+
# exempts it. Not `no_guard_declared`: there may be no class to declare one on, and
|
|
23
|
+
# the fix is in the inventory.
|
|
24
|
+
UNBOUND_TOOL = "unbound_tool"
|
|
21
25
|
|
|
22
26
|
# The one reserved *allow* rule: a tool with no guard, permitted because the host
|
|
23
27
|
# opted into :allow_with_warning. Kept out of RESERVED_RULES, which names deny paths.
|
|
@@ -33,7 +37,8 @@ module Reeve
|
|
|
33
37
|
UNSCOPED_DERIVED_RESULT,
|
|
34
38
|
OUT_OF_SCOPE_RECORD,
|
|
35
39
|
AUDIT_WRITE_FAILED,
|
|
36
|
-
TOOL_ERROR
|
|
40
|
+
TOOL_ERROR,
|
|
41
|
+
UNBOUND_TOOL
|
|
37
42
|
].freeze
|
|
38
43
|
|
|
39
44
|
OUTCOMES = %i[allow deny].freeze
|
data/lib/reeve/errors.rb
CHANGED
|
@@ -57,6 +57,19 @@ module Reeve
|
|
|
57
57
|
end
|
|
58
58
|
end
|
|
59
59
|
|
|
60
|
+
# Raised by Inventory#verify! when a registered tool is not protected: routed through
|
|
61
|
+
# reeve with no guard, not routed at all, or declared in the inventory but not registered
|
|
62
|
+
# with the server. The message is the whole report — tool names, classes and exemption
|
|
63
|
+
# reasons, never an argument, a principal or a record.
|
|
64
|
+
class IncompleteInventoryError < Error
|
|
65
|
+
attr_reader :report
|
|
66
|
+
|
|
67
|
+
def initialize(report)
|
|
68
|
+
@report = report
|
|
69
|
+
super(report.to_s)
|
|
70
|
+
end
|
|
71
|
+
end
|
|
72
|
+
|
|
60
73
|
# Raised when the ledger write fails and audit_failure_mode is :fail (FR-012).
|
|
61
74
|
#
|
|
62
75
|
# It carries a rule like any other denial, because the failure is not recorded anywhere
|
|
@@ -0,0 +1,146 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Reeve
|
|
4
|
+
class Inventory
|
|
5
|
+
# Every registered name, in one of four states, plus whatever the inventory declares
|
|
6
|
+
# that the server does not register.
|
|
7
|
+
#
|
|
8
|
+
# The output names tools, tool classes and exemption reasons, and nothing else. It is
|
|
9
|
+
# built before any invocation and knows no arguments, principals or records, so it is
|
|
10
|
+
# safe to print in a CI log or a boot failure — there is nothing in it to redact.
|
|
11
|
+
#
|
|
12
|
+
# reeve inventory: 6 registered
|
|
13
|
+
# 4 protected and guarded
|
|
14
|
+
# 1 routed through reeve but unguarded
|
|
15
|
+
# 1 bypasses reeve
|
|
16
|
+
# 0 exempted
|
|
17
|
+
# coverage: incomplete
|
|
18
|
+
class Report
|
|
19
|
+
attr_reader :entries
|
|
20
|
+
|
|
21
|
+
def initialize(registered, routes, reasons)
|
|
22
|
+
@entries = (classify(registered, routes, reasons) +
|
|
23
|
+
unregistered(registered, routes, reasons)).freeze
|
|
24
|
+
freeze
|
|
25
|
+
end
|
|
26
|
+
|
|
27
|
+
# Every registered name is either guarded or deliberately exempt, and the inventory
|
|
28
|
+
# names nothing the server does not register.
|
|
29
|
+
#
|
|
30
|
+
# That last part is not tidiness. A route for a name the server does not register is
|
|
31
|
+
# almost always a rename or a typo — and the real name, the one being dispatched, is
|
|
32
|
+
# then the one with no route.
|
|
33
|
+
def complete?
|
|
34
|
+
unguarded.empty? && bypassing.empty? && unregistered_entries.empty?
|
|
35
|
+
end
|
|
36
|
+
|
|
37
|
+
def registered
|
|
38
|
+
entries.reject { |entry| entry.status == :unregistered }
|
|
39
|
+
end
|
|
40
|
+
|
|
41
|
+
def guarded
|
|
42
|
+
select(:guarded)
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
def unguarded
|
|
46
|
+
select(:unguarded)
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
def bypassing
|
|
50
|
+
select(:bypass)
|
|
51
|
+
end
|
|
52
|
+
|
|
53
|
+
def exempted
|
|
54
|
+
select(:exempt)
|
|
55
|
+
end
|
|
56
|
+
|
|
57
|
+
def unregistered_entries
|
|
58
|
+
select(:unregistered)
|
|
59
|
+
end
|
|
60
|
+
|
|
61
|
+
# The classes routed to a registered name.
|
|
62
|
+
def routed_tools
|
|
63
|
+
entries.filter_map(&:tool).uniq
|
|
64
|
+
end
|
|
65
|
+
|
|
66
|
+
def to_s
|
|
67
|
+
[summary, *entries.filter_map { |entry| detail(entry) }].join("\n")
|
|
68
|
+
end
|
|
69
|
+
|
|
70
|
+
def inspect
|
|
71
|
+
"#<Reeve::Inventory::Report #{registered.size} registered, " \
|
|
72
|
+
"coverage: #{complete? ? 'complete' : 'incomplete'}>"
|
|
73
|
+
end
|
|
74
|
+
|
|
75
|
+
private
|
|
76
|
+
|
|
77
|
+
def select(status)
|
|
78
|
+
entries.select { |entry| entry.status == status }
|
|
79
|
+
end
|
|
80
|
+
|
|
81
|
+
def classify(registered, routes, reasons)
|
|
82
|
+
registered.map do |name|
|
|
83
|
+
tool = routes[name]
|
|
84
|
+
if tool
|
|
85
|
+
status = Reeve.registry.for_class(tool) ? :guarded : :unguarded
|
|
86
|
+
Entry.new(name: name, status: status, tool: tool)
|
|
87
|
+
elsif reasons.key?(name)
|
|
88
|
+
Entry.new(name: name, status: :exempt, reason: reasons[name])
|
|
89
|
+
else
|
|
90
|
+
Entry.new(name: name, status: :bypass)
|
|
91
|
+
end
|
|
92
|
+
end
|
|
93
|
+
end
|
|
94
|
+
|
|
95
|
+
def unregistered(registered, routes, reasons)
|
|
96
|
+
(routes.keys | reasons.keys).reject { |name| registered.include?(name) }.map do |name|
|
|
97
|
+
Entry.new(name: name, status: :unregistered, tool: routes[name],
|
|
98
|
+
reason: reasons[name])
|
|
99
|
+
end
|
|
100
|
+
end
|
|
101
|
+
|
|
102
|
+
def summary
|
|
103
|
+
["reeve inventory: #{registered.size} registered", *counts.map { |line| " #{line}" },
|
|
104
|
+
"coverage: #{complete? ? 'complete' : 'incomplete'}"].join("\n")
|
|
105
|
+
end
|
|
106
|
+
|
|
107
|
+
def counts
|
|
108
|
+
lines = [
|
|
109
|
+
"#{guarded.size} protected and guarded",
|
|
110
|
+
"#{unguarded.size} routed through reeve but unguarded",
|
|
111
|
+
"#{bypassing.size} #{bypassing.size == 1 ? 'bypasses' : 'bypass'} reeve",
|
|
112
|
+
"#{exempted.size} exempted"
|
|
113
|
+
]
|
|
114
|
+
return lines if unregistered_entries.empty?
|
|
115
|
+
|
|
116
|
+
lines << "#{unregistered_entries.size} declared but not registered"
|
|
117
|
+
end
|
|
118
|
+
|
|
119
|
+
# One line saying what is wrong and one saying what to do, for everything but a
|
|
120
|
+
# guarded tool — which needs neither. Exemptions are listed even when coverage is
|
|
121
|
+
# complete: a deliberate hole is still a hole, and it should be read every time.
|
|
122
|
+
def detail(entry)
|
|
123
|
+
case entry.status
|
|
124
|
+
when :unguarded
|
|
125
|
+
"\nUNGUARDED #{entry.name}\n routed to #{label(entry.tool)}, which has no " \
|
|
126
|
+
"guard_with declaration — reeve denies it, or runs it unscoped under " \
|
|
127
|
+
":allow_with_warning"
|
|
128
|
+
when :bypass
|
|
129
|
+
"\nBYPASS #{entry.name}\n registered with the server but neither routed " \
|
|
130
|
+
"through reeve nor exempted — route it to a guarded tool, or exempt it with a " \
|
|
131
|
+
"reason"
|
|
132
|
+
when :exempt
|
|
133
|
+
"\nEXEMPT #{entry.name}\n #{entry.reason}"
|
|
134
|
+
when :unregistered
|
|
135
|
+
what = entry.tool ? "routed to #{label(entry.tool)}" : "exempted"
|
|
136
|
+
"\nNOT REGISTERED #{entry.name}\n #{what}, but the server registers no tool " \
|
|
137
|
+
"by that name — if it was renamed, the new name is unbound"
|
|
138
|
+
end
|
|
139
|
+
end
|
|
140
|
+
|
|
141
|
+
def label(tool)
|
|
142
|
+
tool.name || tool.inspect
|
|
143
|
+
end
|
|
144
|
+
end
|
|
145
|
+
end
|
|
146
|
+
end
|
|
@@ -0,0 +1,212 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Reeve
|
|
4
|
+
# The host's whole tool inventory, stated against what reeve was told about.
|
|
5
|
+
#
|
|
6
|
+
# Reeve only sees an invocation that reaches `Reeve.invoke`. A server that registers its
|
|
7
|
+
# tools as names and handler blocks can route some of them through the envelope and
|
|
8
|
+
# dispatch the rest directly, and the direct ones are invisible: neither guarded nor
|
|
9
|
+
# reported as unguarded. Every guarantee reeve makes — deny by default, scoping, the
|
|
10
|
+
# ledger — is skipped together for them, and nothing anywhere says so. The registry
|
|
11
|
+
# cannot answer "is every tool this endpoint dispatches protected?" because the tools
|
|
12
|
+
# that are not are, by construction, the ones it never heard of.
|
|
13
|
+
#
|
|
14
|
+
# The inventory makes the host's list of registered names the thing being checked:
|
|
15
|
+
#
|
|
16
|
+
# McpInventory = Reeve::Inventory.new(
|
|
17
|
+
# registered: -> { McpServer.tool_names },
|
|
18
|
+
# routed: -> { { "search_invoices" => InvoiceSearchTool } },
|
|
19
|
+
# exemptions: { "healthcheck" => { reason: "returns no application data" } }
|
|
20
|
+
# )
|
|
21
|
+
#
|
|
22
|
+
# McpInventory.verify! # at boot, or in CI
|
|
23
|
+
#
|
|
24
|
+
# and, optionally, the place the host dispatches through, so that a name it has no
|
|
25
|
+
# entry for is refused rather than run:
|
|
26
|
+
#
|
|
27
|
+
# McpInventory.dispatch(name, arguments: args) { handlers.fetch(name).call(args) }
|
|
28
|
+
#
|
|
29
|
+
# Every part takes a value or a callable returning one. A Rails initializer cannot
|
|
30
|
+
# reference application constants, and a server registers its tools after the
|
|
31
|
+
# initializer runs, so the lists are read when asked for rather than when declared.
|
|
32
|
+
#
|
|
33
|
+
# Nothing here depends on an MCP library. A list of names is the whole protocol.
|
|
34
|
+
class Inventory
|
|
35
|
+
# One registered name, classified. +tool+ is the class it is routed to, if any;
|
|
36
|
+
# +reason+ is the stated reason for an exemption.
|
|
37
|
+
Entry = Struct.new(:name, :status, :tool, :reason, keyword_init: true)
|
|
38
|
+
|
|
39
|
+
def initialize(registered:, routed: {}, exemptions: {})
|
|
40
|
+
@registered = registered
|
|
41
|
+
@routed = routed
|
|
42
|
+
@exemptions = exemptions
|
|
43
|
+
|
|
44
|
+
# Literal values are checked now, so a blank exemption reason fails on the line that
|
|
45
|
+
# declared it. Callables cannot be until they are asked; #report checks those.
|
|
46
|
+
resolve unless [registered, routed, exemptions].any? { |part| part.respond_to?(:call) }
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
# The classification, read fresh. Raises ArgumentError if the inventory is malformed:
|
|
50
|
+
# a report built from a list it could not read would be a report about nothing.
|
|
51
|
+
def report
|
|
52
|
+
Report.new(*resolve)
|
|
53
|
+
end
|
|
54
|
+
|
|
55
|
+
# The strict gate: the report when coverage is complete, IncompleteInventoryError
|
|
56
|
+
# otherwise. Call it where failing is what you want — boot, a deploy step, CI.
|
|
57
|
+
def verify!
|
|
58
|
+
report.tap do |result|
|
|
59
|
+
raise IncompleteInventoryError, result unless result.complete?
|
|
60
|
+
end
|
|
61
|
+
end
|
|
62
|
+
|
|
63
|
+
# The tool classes the inventory routes through reeve. The testing kit certifies these.
|
|
64
|
+
def routed_tools
|
|
65
|
+
routes.values.uniq
|
|
66
|
+
end
|
|
67
|
+
|
|
68
|
+
# The dispatch boundary. A host that sends every tool call through here cannot have a
|
|
69
|
+
# tool that bypasses reeve, because the three answers are exhaustive:
|
|
70
|
+
#
|
|
71
|
+
# * **routed** — runs through `Reeve.invoke` with the class it is routed to. A handler
|
|
72
|
+
# block, if given, is the body, so a registry of blocks needs one guarded class per
|
|
73
|
+
# tool to say which policy governs it and nothing else.
|
|
74
|
+
# * **exempt** — the handler runs as it would have without reeve. That is what the
|
|
75
|
+
# exemption says, and why it needs a reason.
|
|
76
|
+
# * **neither** — governed by `config.unguarded_tools`, exactly like a tool with no
|
|
77
|
+
# guard. Under `:deny`, the default, the call is refused with `unbound_tool`; under
|
|
78
|
+
# `:allow_with_warning` it runs unscoped. Either way it is recorded, so a name the
|
|
79
|
+
# inventory missed shows up in the ledger rather than nowhere.
|
|
80
|
+
#
|
|
81
|
+
# +registered+ is not consulted. The host's own dispatcher already decided the name
|
|
82
|
+
# exists; what this decides is whether reeve stands in front of it.
|
|
83
|
+
def dispatch(name, arguments: {}, principal: :unset, agent: nil, metadata: {}, &handler)
|
|
84
|
+
name = name.to_s
|
|
85
|
+
tool = routes[name]
|
|
86
|
+
if tool
|
|
87
|
+
return Reeve.invoke(tool: tool, arguments: arguments, principal: principal,
|
|
88
|
+
agent: agent, metadata: metadata, &handler)
|
|
89
|
+
end
|
|
90
|
+
|
|
91
|
+
if handler.nil?
|
|
92
|
+
raise ArgumentError, "#{name} is not routed to a tool, so dispatch needs the " \
|
|
93
|
+
"handler block that runs it"
|
|
94
|
+
end
|
|
95
|
+
|
|
96
|
+
return handler.call if exemption_reasons.key?(name)
|
|
97
|
+
|
|
98
|
+
dispatch_unbound(name, arguments: arguments, principal: principal, agent: agent,
|
|
99
|
+
metadata: metadata, &handler)
|
|
100
|
+
end
|
|
101
|
+
|
|
102
|
+
def inspect
|
|
103
|
+
"#<Reeve::Inventory>"
|
|
104
|
+
end
|
|
105
|
+
|
|
106
|
+
private
|
|
107
|
+
|
|
108
|
+
def resolve
|
|
109
|
+
names = registered_names
|
|
110
|
+
routes = self.routes
|
|
111
|
+
reasons = exemption_reasons
|
|
112
|
+
|
|
113
|
+
both = routes.keys & reasons.keys
|
|
114
|
+
unless both.empty?
|
|
115
|
+
raise ArgumentError, "a tool is either routed through reeve or exempt from it, " \
|
|
116
|
+
"not both: #{both.join(', ')}"
|
|
117
|
+
end
|
|
118
|
+
|
|
119
|
+
[names, routes, reasons]
|
|
120
|
+
end
|
|
121
|
+
|
|
122
|
+
def registered_names
|
|
123
|
+
names = read(:registered, @registered)
|
|
124
|
+
unless names.respond_to?(:to_a) && !names.is_a?(Hash)
|
|
125
|
+
raise ArgumentError, "registered must list tool names, got #{names.inspect}"
|
|
126
|
+
end
|
|
127
|
+
|
|
128
|
+
names.to_a.map { |name| name_from(:registered, name) }.uniq
|
|
129
|
+
end
|
|
130
|
+
|
|
131
|
+
def routes
|
|
132
|
+
read_hash(:routed, @routed).to_h do |name, tool|
|
|
133
|
+
unless tool.is_a?(Class)
|
|
134
|
+
raise ArgumentError, "routed must map each name to a tool class; " \
|
|
135
|
+
"#{name} maps to #{tool.inspect}"
|
|
136
|
+
end
|
|
137
|
+
|
|
138
|
+
[name_from(:routed, name), tool]
|
|
139
|
+
end
|
|
140
|
+
end
|
|
141
|
+
|
|
142
|
+
# An exemption with no reason is a hole someone forgot about. With one, it is a
|
|
143
|
+
# decision someone can be asked about.
|
|
144
|
+
def exemption_reasons
|
|
145
|
+
read_hash(:exemptions, @exemptions).to_h do |name, options|
|
|
146
|
+
name = name_from(:exemptions, name)
|
|
147
|
+
reason = options.is_a?(Hash) ? (options[:reason] || options["reason"]) : nil
|
|
148
|
+
if reason.nil? || reason.to_s.strip.empty?
|
|
149
|
+
raise ArgumentError, "the exemption for #{name} needs a reason, e.g. " \
|
|
150
|
+
"{ \"#{name}\" => { reason: \"returns no application data\" } }"
|
|
151
|
+
end
|
|
152
|
+
|
|
153
|
+
[name, reason.to_s.strip]
|
|
154
|
+
end
|
|
155
|
+
end
|
|
156
|
+
|
|
157
|
+
def read_hash(part, source)
|
|
158
|
+
value = read(part, source)
|
|
159
|
+
return value if value.is_a?(Hash)
|
|
160
|
+
|
|
161
|
+
raise ArgumentError, "#{part} must be a Hash keyed by tool name, got #{value.inspect}"
|
|
162
|
+
end
|
|
163
|
+
|
|
164
|
+
def read(part, source)
|
|
165
|
+
value = source.respond_to?(:call) ? source.call : source
|
|
166
|
+
raise ArgumentError, "#{part} is nil" if value.nil?
|
|
167
|
+
|
|
168
|
+
value
|
|
169
|
+
end
|
|
170
|
+
|
|
171
|
+
def name_from(part, name)
|
|
172
|
+
string = name.respond_to?(:to_str) || name.is_a?(Symbol) ? name.to_s.strip : nil
|
|
173
|
+
return string unless string.nil? || string.empty?
|
|
174
|
+
|
|
175
|
+
raise ArgumentError, "#{part} contains a blank or non-string tool name: #{name.inspect}"
|
|
176
|
+
end
|
|
177
|
+
|
|
178
|
+
# A name the inventory has no entry for still goes through the envelope, under its
|
|
179
|
+
# own name and with nothing to look up. The registry answer below is what makes the
|
|
180
|
+
# refusal name the real cause — the inventory, not a missing `guard_with` on a class
|
|
181
|
+
# that may not exist.
|
|
182
|
+
def dispatch_unbound(name, arguments:, principal:, agent:, metadata:, &handler)
|
|
183
|
+
config = Reeve.config.with_principal(principal)
|
|
184
|
+
context = Context.new(tool_name: name, agent: agent, arguments: arguments,
|
|
185
|
+
metadata: metadata)
|
|
186
|
+
|
|
187
|
+
Invocation.call(context, registry: Unbound.new(name, config), config: config, &handler)
|
|
188
|
+
end
|
|
189
|
+
|
|
190
|
+
# The registry the envelope consults for an unbound name. Under `:deny` it answers
|
|
191
|
+
# with the refusal itself; under `:allow_with_warning` it has no guard to offer, and
|
|
192
|
+
# the envelope runs the tool unscoped and records `guard: "none"` — the same worklist
|
|
193
|
+
# an unguarded class lands on.
|
|
194
|
+
class Unbound
|
|
195
|
+
def initialize(name, config)
|
|
196
|
+
@name = name
|
|
197
|
+
@config = config
|
|
198
|
+
end
|
|
199
|
+
|
|
200
|
+
def guard_for(_tool_name)
|
|
201
|
+
return nil if @config.unguarded_tools == :allow_with_warning
|
|
202
|
+
|
|
203
|
+
Decision.deny(
|
|
204
|
+
rule: Decision::UNBOUND_TOOL,
|
|
205
|
+
detail: "#{@name} is neither routed through reeve nor exempted in the inventory"
|
|
206
|
+
)
|
|
207
|
+
end
|
|
208
|
+
end
|
|
209
|
+
end
|
|
210
|
+
end
|
|
211
|
+
|
|
212
|
+
require_relative "inventory/report"
|
data/lib/reeve/invocation.rb
CHANGED
|
@@ -90,7 +90,9 @@ module Reeve
|
|
|
90
90
|
context.principal = principal
|
|
91
91
|
|
|
92
92
|
guard = look_up_guard
|
|
93
|
-
|
|
93
|
+
# A registry may answer with a refusal instead of a guard: the lookup raised, or an
|
|
94
|
+
# Inventory is refusing a name it has no entry for.
|
|
95
|
+
return guard if guard.is_a?(Decision)
|
|
94
96
|
|
|
95
97
|
if guard.nil?
|
|
96
98
|
if deny_unguarded?
|
|
@@ -56,14 +56,14 @@ module Reeve
|
|
|
56
56
|
)
|
|
57
57
|
end
|
|
58
58
|
|
|
59
|
-
# SC-009's escape hatch: any
|
|
59
|
+
# SC-009's escape hatch: any check, straight from Minitest.
|
|
60
60
|
def assert_reeve_check(check)
|
|
61
61
|
result = check.call
|
|
62
62
|
assert result.passed?, result.message
|
|
63
63
|
result
|
|
64
64
|
end
|
|
65
65
|
|
|
66
|
-
# A whole Report at once, for a host that would rather have one test than
|
|
66
|
+
# A whole Report at once, for a host that would rather have one test than one per check.
|
|
67
67
|
def assert_reeve_compliance(principals: nil, **options)
|
|
68
68
|
report = Checks.run_all(
|
|
69
69
|
principals: principals || Testing.compliance_principals, **options
|
|
@@ -0,0 +1,133 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Reeve
|
|
4
|
+
module Testing
|
|
5
|
+
module Checks
|
|
6
|
+
# Whether the run covered the endpoint, or only the tools reeve was told about.
|
|
7
|
+
#
|
|
8
|
+
# Every other check walks a population reeve assembled itself: the classes that
|
|
9
|
+
# included `Reeve::Guard`. A server that dispatches some tools without ever handing
|
|
10
|
+
# them to reeve leaves them out of that population, and the run goes green on the
|
|
11
|
+
# subset it could see. This check compares the run against the host's own list of
|
|
12
|
+
# registered names (`config.inventory`), so a green run can only mean the endpoint.
|
|
13
|
+
#
|
|
14
|
+
# Four answers, and only one of them is a pass:
|
|
15
|
+
#
|
|
16
|
+
# * **no inventory declared** — skipped. Reeve cannot know what the server registers
|
|
17
|
+
# unless told, and a pass here would be the green that means "did not look".
|
|
18
|
+
# * **a subset** — skipped, when the host narrowed the run with `compliance_tools` or
|
|
19
|
+
# `tools:`. That is the incremental-adoption path; the run says what it certified
|
|
20
|
+
# and that it is not the endpoint, without turning the build red mid-retrofit.
|
|
21
|
+
# * **incomplete** — failed: a registered tool bypasses reeve or reaches it unguarded,
|
|
22
|
+
# and the run claimed to cover everything.
|
|
23
|
+
# * **complete** — passed.
|
|
24
|
+
#
|
|
25
|
+
# Reeve::Checks::EndpointCoverage.new.call
|
|
26
|
+
# Reeve::Checks::EndpointCoverage.new(inventory: McpInventory).call
|
|
27
|
+
class EndpointCoverage < Base
|
|
28
|
+
def initialize(inventory: :configured, tools: nil, ledger: nil)
|
|
29
|
+
super(ledger: ledger)
|
|
30
|
+
@inventory = inventory == :configured ? Reeve.config.inventory : inventory
|
|
31
|
+
@tools = tools
|
|
32
|
+
end
|
|
33
|
+
|
|
34
|
+
def call
|
|
35
|
+
return unverified if inventory.nil?
|
|
36
|
+
|
|
37
|
+
report = inventory.report
|
|
38
|
+
if narrowed?
|
|
39
|
+
subset(report)
|
|
40
|
+
elsif report.complete?
|
|
41
|
+
complete(report)
|
|
42
|
+
else
|
|
43
|
+
incomplete(report)
|
|
44
|
+
end
|
|
45
|
+
rescue StandardError => e
|
|
46
|
+
failed("the endpoint inventory could not be read, so its coverage is unknown: " \
|
|
47
|
+
"#{e.class}: #{e.message}", coverage: :unknown)
|
|
48
|
+
end
|
|
49
|
+
|
|
50
|
+
private
|
|
51
|
+
|
|
52
|
+
attr_reader :inventory
|
|
53
|
+
|
|
54
|
+
def narrowed?
|
|
55
|
+
!@tools.nil? || Testing.compliance_tools_narrowed?
|
|
56
|
+
end
|
|
57
|
+
|
|
58
|
+
def certified
|
|
59
|
+
@certified ||= @tools || Testing.compliance_tools
|
|
60
|
+
end
|
|
61
|
+
|
|
62
|
+
def unverified
|
|
63
|
+
skipped(
|
|
64
|
+
"no endpoint inventory is declared, so this run certifies the " \
|
|
65
|
+
"#{pluralize(certified.size, 'tool')} reeve knows about and cannot say whether " \
|
|
66
|
+
"the server dispatches others outside it — set config.inventory to a " \
|
|
67
|
+
"Reeve::Inventory to verify the endpoint",
|
|
68
|
+
coverage: :unverified
|
|
69
|
+
)
|
|
70
|
+
end
|
|
71
|
+
|
|
72
|
+
def complete(report)
|
|
73
|
+
passed(
|
|
74
|
+
"every one of the #{pluralize(report.registered.size, 'tool')} the server " \
|
|
75
|
+
"registers is protected and guarded (#{report.guarded.size}) or exempted " \
|
|
76
|
+
"(#{report.exempted.size})",
|
|
77
|
+
coverage: :complete
|
|
78
|
+
)
|
|
79
|
+
end
|
|
80
|
+
|
|
81
|
+
def incomplete(report)
|
|
82
|
+
failed(
|
|
83
|
+
"expected every tool the server registers to be protected by reeve or " \
|
|
84
|
+
"exempted, but #{gaps(report)}\n\n#{report}",
|
|
85
|
+
coverage: :incomplete
|
|
86
|
+
)
|
|
87
|
+
end
|
|
88
|
+
|
|
89
|
+
# Complete coverage with a narrowed run is still a subset: the inventory is sound,
|
|
90
|
+
# but this run did not certify all of it.
|
|
91
|
+
def subset(report)
|
|
92
|
+
routed = report.routed_tools
|
|
93
|
+
missing = routed - certified
|
|
94
|
+
lines = [
|
|
95
|
+
"this run certifies a subset, not the endpoint: " \
|
|
96
|
+
"#{(routed - missing).size} of the #{pluralize(routed.size, 'tool')} routed " \
|
|
97
|
+
"through reeve (compliance_tools or tools: narrowed it)"
|
|
98
|
+
]
|
|
99
|
+
lines << "not certified: #{missing.map { |tool| tool.name || tool.inspect }.join(', ')}" \
|
|
100
|
+
unless missing.empty?
|
|
101
|
+
lines << "" << report.to_s unless report.complete?
|
|
102
|
+
|
|
103
|
+
skipped(lines.join("\n"), coverage: :subset)
|
|
104
|
+
end
|
|
105
|
+
|
|
106
|
+
def gaps(report)
|
|
107
|
+
[
|
|
108
|
+
gap(report.bypassing, "bypasses it", "bypass it"),
|
|
109
|
+
gap(report.unguarded, "is routed through it unguarded",
|
|
110
|
+
"are routed through it unguarded"),
|
|
111
|
+
gap(report.unregistered_entries, "declared in the inventory is not registered",
|
|
112
|
+
"declared in the inventory are not registered")
|
|
113
|
+
].compact.join(", ")
|
|
114
|
+
end
|
|
115
|
+
|
|
116
|
+
def gap(entries, one, many)
|
|
117
|
+
return nil if entries.empty?
|
|
118
|
+
|
|
119
|
+
"#{entries.size} #{entries.size == 1 ? one : many}"
|
|
120
|
+
end
|
|
121
|
+
|
|
122
|
+
# Only this check can come back with nothing established either way.
|
|
123
|
+
def skipped(message, details = {})
|
|
124
|
+
Result.skipped(check: check_name, message: message, details: details)
|
|
125
|
+
end
|
|
126
|
+
|
|
127
|
+
def base_details
|
|
128
|
+
{}
|
|
129
|
+
end
|
|
130
|
+
end
|
|
131
|
+
end
|
|
132
|
+
end
|
|
133
|
+
end
|
data/lib/reeve/testing/checks.rb
CHANGED
|
@@ -8,10 +8,11 @@ require_relative "checks/rule_present"
|
|
|
8
8
|
require_relative "checks/redaction_holds"
|
|
9
9
|
require_relative "checks/principal_required"
|
|
10
10
|
require_relative "checks/contract_version"
|
|
11
|
+
require_relative "checks/endpoint_coverage"
|
|
11
12
|
|
|
12
13
|
module Reeve
|
|
13
14
|
module Testing
|
|
14
|
-
# The
|
|
15
|
+
# The guarantees, as objects.
|
|
15
16
|
#
|
|
16
17
|
# This is the whole of the testing kit's logic. The RSpec matchers and the Minitest
|
|
17
18
|
# assertions are adapters over it and contain no assertions of their own, which is why
|
|
@@ -29,11 +30,13 @@ module Reeve
|
|
|
29
30
|
RulePresent,
|
|
30
31
|
RedactionHolds,
|
|
31
32
|
PrincipalRequired,
|
|
32
|
-
ContractVersion
|
|
33
|
+
ContractVersion,
|
|
34
|
+
EndpointCoverage
|
|
33
35
|
].freeze
|
|
34
36
|
|
|
35
|
-
# The checks that are asked once about the ledger
|
|
36
|
-
|
|
37
|
+
# The checks that are asked once — about the ledger, or about the endpoint as a
|
|
38
|
+
# whole — rather than once per tool.
|
|
39
|
+
GLOBAL = [ContractVersion, EndpointCoverage].freeze
|
|
37
40
|
|
|
38
41
|
# The compliance suite's engine (FR-018): every check, against every registered
|
|
39
42
|
# guarded tool, in one Report.
|
|
@@ -53,7 +56,10 @@ module Reeve
|
|
|
53
56
|
# test method out of, so that a failing suite names the guarantee that broke rather
|
|
54
57
|
# than reporting "compliance" as one undifferentiated red.
|
|
55
58
|
def self.run(check, principals:, tools: nil, arguments: {}, invoke: nil, ledger: nil)
|
|
56
|
-
|
|
59
|
+
if GLOBAL.include?(check)
|
|
60
|
+
return Report.new([build(check, tool: nil, principals: principals, tools: tools,
|
|
61
|
+
ledger: ledger).call])
|
|
62
|
+
end
|
|
57
63
|
|
|
58
64
|
subjects = tools || Testing.compliance_tools
|
|
59
65
|
Report.new(
|
|
@@ -74,12 +80,17 @@ module Reeve
|
|
|
74
80
|
|
|
75
81
|
# The one place that knows what each check's constructor wants. Front-ends and hosts
|
|
76
82
|
# ask for a check by class and get a configured one back.
|
|
77
|
-
|
|
83
|
+
#
|
|
84
|
+
# +tools+ is the population the run was given, which only EndpointCoverage reads: an
|
|
85
|
+
# explicit list is a narrowed run, and it must say so.
|
|
86
|
+
def self.build(check, tool:, principals:, arguments: {}, invoke: nil, ledger: nil,
|
|
87
|
+
tools: nil)
|
|
78
88
|
common = { tool: tool, arguments: arguments, invoke: invoke, ledger: ledger }
|
|
79
89
|
|
|
80
90
|
case check.check_name
|
|
81
91
|
when "GuardDeclared" then GuardDeclared.new(tool: tool, ledger: ledger)
|
|
82
92
|
when "ContractVersion" then ContractVersion.new(ledger: ledger)
|
|
93
|
+
when "EndpointCoverage" then EndpointCoverage.new(tools: tools, ledger: ledger)
|
|
83
94
|
when "PrincipalRequired" then PrincipalRequired.new(**common)
|
|
84
95
|
when "CrossPrincipalLeak" then CrossPrincipalLeak.new(principals: Array(principals),
|
|
85
96
|
**common)
|
|
@@ -15,7 +15,8 @@ module Reeve
|
|
|
15
15
|
#
|
|
16
16
|
# One method per check, matching the RSpec shared example group one for one, so a red
|
|
17
17
|
# build names the guarantee that broke rather than reporting "compliance" as one
|
|
18
|
-
# undifferentiated failure.
|
|
18
|
+
# undifferentiated failure. A check that could establish nothing either way is
|
|
19
|
+
# skipped with its reason, never passed.
|
|
19
20
|
module ComplianceAssertions
|
|
20
21
|
include Assertions
|
|
21
22
|
|
|
@@ -25,6 +26,7 @@ module Reeve
|
|
|
25
26
|
|
|
26
27
|
define_method(method_name) do
|
|
27
28
|
report = Checks.run(check, principals: Testing.compliance_principals)
|
|
29
|
+
skip(report.to_s) if report.skipped?
|
|
28
30
|
assert report.passed?, report.to_s
|
|
29
31
|
report
|
|
30
32
|
end
|
|
@@ -11,11 +11,14 @@
|
|
|
11
11
|
# Reeve::Testing.compliance_principals = -> { [users(:alice), users(:bob)] }
|
|
12
12
|
#
|
|
13
13
|
# One example per check rather than one for the lot, so a red build names the guarantee
|
|
14
|
-
# that broke — "CrossPrincipalLeak" is an incident, "compliance" is a shrug.
|
|
14
|
+
# that broke — "CrossPrincipalLeak" is an incident, "compliance" is a shrug. A check that
|
|
15
|
+
# could establish nothing either way (EndpointCoverage with no inventory) is skipped, with
|
|
16
|
+
# its reason, rather than passed.
|
|
15
17
|
RSpec.shared_examples "a reeve-compliant server" do
|
|
16
18
|
Reeve::Checks::ALL.each do |check|
|
|
17
19
|
it "satisfies #{check.check_name} for every registered guarded tool" do
|
|
18
20
|
report = Reeve::Checks.run(check, principals: Reeve::Testing.compliance_principals)
|
|
21
|
+
skip(report.to_s) if report.skipped?
|
|
19
22
|
|
|
20
23
|
expect(report).to be_passed, report.to_s
|
|
21
24
|
end
|
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
module Reeve
|
|
4
4
|
module Testing
|
|
5
5
|
module Matchers
|
|
6
|
-
# The escape hatch that keeps SC-009 honest: every
|
|
6
|
+
# The escape hatch that keeps SC-009 honest: every check is
|
|
7
7
|
# assertable from RSpec, not only the two with names of their own.
|
|
8
8
|
#
|
|
9
9
|
# expect(Reeve::Checks::RedactionHolds.new(tool: T, principal: alice))
|
data/lib/reeve/testing/report.rb
CHANGED
|
@@ -15,6 +15,8 @@ module Reeve
|
|
|
15
15
|
freeze
|
|
16
16
|
end
|
|
17
17
|
|
|
18
|
+
# Nothing failed. A skip is not a failure — and it is printed, so it is not
|
|
19
|
+
# mistaken for a pass either.
|
|
18
20
|
def passed?
|
|
19
21
|
failures.empty?
|
|
20
22
|
end
|
|
@@ -23,20 +25,31 @@ module Reeve
|
|
|
23
25
|
!passed?
|
|
24
26
|
end
|
|
25
27
|
|
|
28
|
+
# Every result was a skip: the run established nothing either way. The front-ends
|
|
29
|
+
# report this as a skipped test rather than a green one.
|
|
30
|
+
def skipped?
|
|
31
|
+
!results.empty? && results.all?(&:skipped?)
|
|
32
|
+
end
|
|
33
|
+
|
|
26
34
|
def failures
|
|
27
|
-
results.
|
|
35
|
+
results.select(&:failed?)
|
|
28
36
|
end
|
|
29
37
|
|
|
30
38
|
def passes
|
|
31
39
|
results.select(&:passed?)
|
|
32
40
|
end
|
|
33
41
|
|
|
42
|
+
def skips
|
|
43
|
+
results.select(&:skipped?)
|
|
44
|
+
end
|
|
45
|
+
|
|
34
46
|
def size
|
|
35
47
|
results.size
|
|
36
48
|
end
|
|
37
49
|
|
|
38
50
|
def to_s
|
|
39
|
-
[summary, *failures.map { |result| detail(result) }
|
|
51
|
+
[summary, *failures.map { |result| detail("FAIL", result) },
|
|
52
|
+
*skips.map { |result| detail("SKIP", result) }].join("\n")
|
|
40
53
|
end
|
|
41
54
|
|
|
42
55
|
def inspect
|
|
@@ -46,12 +59,15 @@ module Reeve
|
|
|
46
59
|
private
|
|
47
60
|
|
|
48
61
|
def summary
|
|
49
|
-
"reeve compliance: #{size} #{size == 1 ? 'check' : 'checks'}, " \
|
|
50
|
-
|
|
62
|
+
line = "reeve compliance: #{size} #{size == 1 ? 'check' : 'checks'}, " \
|
|
63
|
+
"#{passes.size} passed, #{failures.size} failed"
|
|
64
|
+
skips.empty? ? line : "#{line}, #{skips.size} skipped"
|
|
51
65
|
end
|
|
52
66
|
|
|
53
|
-
|
|
54
|
-
|
|
67
|
+
# Every line of a multi-line message indented, so it reads as one block under its
|
|
68
|
+
# heading rather than as stray lines of the report.
|
|
69
|
+
def detail(heading, result)
|
|
70
|
+
"\n#{heading} #{result.check}\n #{result.message.gsub("\n", "\n ")}"
|
|
55
71
|
end
|
|
56
72
|
end
|
|
57
73
|
end
|
data/lib/reeve/testing/result.rb
CHANGED
|
@@ -9,31 +9,51 @@ module Reeve
|
|
|
9
9
|
# Minitest assertion, a rake task — reads +message+ and prints it; none of them
|
|
10
10
|
# composes their own (FR-019). That is what makes the same violation read identically
|
|
11
11
|
# from all three.
|
|
12
|
+
#
|
|
13
|
+
# A result is passed, failed, or skipped. Skipped is for a guarantee the run could not
|
|
14
|
+
# establish either way — endpoint coverage with no inventory declared. Reporting that
|
|
15
|
+
# as a pass would be the green that means "we did not look"; reporting it as a failure
|
|
16
|
+
# would turn every build red over a question the host never asked. A skip is printed,
|
|
17
|
+
# and shows up yellow in both test frameworks, so it is neither.
|
|
12
18
|
class Result
|
|
13
|
-
|
|
19
|
+
STATUSES = %i[passed failed skipped].freeze
|
|
20
|
+
|
|
21
|
+
attr_reader :check, :message, :details, :status
|
|
14
22
|
|
|
15
23
|
def self.passed(check:, message:, details: {})
|
|
16
|
-
new(check: check,
|
|
24
|
+
new(check: check, status: :passed, message: message, details: details)
|
|
17
25
|
end
|
|
18
26
|
|
|
19
27
|
def self.failed(check:, message:, details: {})
|
|
20
|
-
new(check: check,
|
|
28
|
+
new(check: check, status: :failed, message: message, details: details)
|
|
29
|
+
end
|
|
30
|
+
|
|
31
|
+
def self.skipped(check:, message:, details: {})
|
|
32
|
+
new(check: check, status: :skipped, message: message, details: details)
|
|
21
33
|
end
|
|
22
34
|
|
|
23
|
-
def initialize(check:,
|
|
35
|
+
def initialize(check:, status:, message:, details: {})
|
|
36
|
+
unless STATUSES.include?(status)
|
|
37
|
+
raise ArgumentError, "status must be one of #{STATUSES.inspect}, got #{status.inspect}"
|
|
38
|
+
end
|
|
39
|
+
|
|
24
40
|
@check = check.to_s
|
|
25
|
-
@
|
|
41
|
+
@status = status
|
|
26
42
|
@message = message.to_s
|
|
27
43
|
@details = details.freeze
|
|
28
44
|
freeze
|
|
29
45
|
end
|
|
30
46
|
|
|
31
47
|
def passed?
|
|
32
|
-
|
|
48
|
+
status == :passed
|
|
33
49
|
end
|
|
34
50
|
|
|
35
51
|
def failed?
|
|
36
|
-
|
|
52
|
+
status == :failed
|
|
53
|
+
end
|
|
54
|
+
|
|
55
|
+
def skipped?
|
|
56
|
+
status == :skipped
|
|
37
57
|
end
|
|
38
58
|
|
|
39
59
|
def to_s
|
|
@@ -41,7 +61,7 @@ module Reeve
|
|
|
41
61
|
end
|
|
42
62
|
|
|
43
63
|
def inspect
|
|
44
|
-
"#<Reeve::Testing::Result #{check} #{
|
|
64
|
+
"#<Reeve::Testing::Result #{check} #{status} " \
|
|
45
65
|
"#{message.inspect}>"
|
|
46
66
|
end
|
|
47
67
|
end
|
data/lib/reeve/testing.rb
CHANGED
|
@@ -41,17 +41,23 @@ module Reeve
|
|
|
41
41
|
end
|
|
42
42
|
|
|
43
43
|
# The tools the suite walks. Unset means every tool reeve knows about — the classes
|
|
44
|
-
# that included the DSL, guarded or not
|
|
45
|
-
#
|
|
44
|
+
# that included the DSL, guarded or not, plus every class the endpoint inventory
|
|
45
|
+
# routes to, which need not have included anything. That default is what makes an
|
|
46
|
+
# unguarded tool visible rather than merely absent.
|
|
46
47
|
attr_writer :compliance_tools
|
|
47
48
|
|
|
48
49
|
def compliance_tools
|
|
49
50
|
source = @compliance_tools || Reeve.config.compliance_tools
|
|
50
|
-
return Reeve.registry.tool_classes if source.nil?
|
|
51
|
+
return Reeve.registry.tool_classes | inventory_tools if source.nil?
|
|
51
52
|
|
|
52
53
|
Array(source.respond_to?(:call) ? source.call : source)
|
|
53
54
|
end
|
|
54
55
|
|
|
56
|
+
# The host said which tools it is certifying, so a run is a subset by its own account.
|
|
57
|
+
def compliance_tools_narrowed?
|
|
58
|
+
!(@compliance_tools || Reeve.config.compliance_tools).nil?
|
|
59
|
+
end
|
|
60
|
+
|
|
55
61
|
def compliance_principals?
|
|
56
62
|
!(@compliance_principals || Reeve.config.compliance_principals).nil?
|
|
57
63
|
end
|
|
@@ -63,6 +69,16 @@ module Reeve
|
|
|
63
69
|
|
|
64
70
|
private
|
|
65
71
|
|
|
72
|
+
# An inventory that cannot be read adds nothing here; EndpointCoverage fails on it
|
|
73
|
+
# by name, which is a better place to learn about it than an exception in every
|
|
74
|
+
# per-tool check.
|
|
75
|
+
def inventory_tools
|
|
76
|
+
inventory = Reeve.config.inventory
|
|
77
|
+
inventory ? inventory.routed_tools : []
|
|
78
|
+
rescue StandardError
|
|
79
|
+
[]
|
|
80
|
+
end
|
|
81
|
+
|
|
66
82
|
def missing_principals_message
|
|
67
83
|
"the reeve compliance suite needs two fixture principals with disjoint records. " \
|
|
68
84
|
"Set them in your test helper:\n\n " \
|
data/lib/reeve/version.rb
CHANGED
data/lib/reeve.rb
CHANGED
|
@@ -8,6 +8,7 @@ require_relative "reeve/context"
|
|
|
8
8
|
require_relative "reeve/scope_result"
|
|
9
9
|
require_relative "reeve/invocation"
|
|
10
10
|
require_relative "reeve/authorization"
|
|
11
|
+
require_relative "reeve/inventory"
|
|
11
12
|
|
|
12
13
|
# Reeve — per-record authorization and an append-only audit ledger for the MCP tools
|
|
13
14
|
# a Rails application exposes to AI agents.
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: reeve
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.5.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Victor Velazquez
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2026-
|
|
11
|
+
date: 2026-10-01 00:00:00.000000000 Z
|
|
12
12
|
dependencies: []
|
|
13
13
|
description: |
|
|
14
14
|
Reeve makes it safe for a Rails application to expose MCP (Model Context Protocol)
|
|
@@ -56,6 +56,8 @@ files:
|
|
|
56
56
|
- lib/reeve/fast_mcp.rb
|
|
57
57
|
- lib/reeve/integrations/fast_mcp/context_builder.rb
|
|
58
58
|
- lib/reeve/integrations/fast_mcp/tool_extension.rb
|
|
59
|
+
- lib/reeve/inventory.rb
|
|
60
|
+
- lib/reeve/inventory/report.rb
|
|
59
61
|
- lib/reeve/invocation.rb
|
|
60
62
|
- lib/reeve/minitest.rb
|
|
61
63
|
- lib/reeve/rspec.rb
|
|
@@ -67,6 +69,7 @@ files:
|
|
|
67
69
|
- lib/reeve/testing/checks/base.rb
|
|
68
70
|
- lib/reeve/testing/checks/contract_version.rb
|
|
69
71
|
- lib/reeve/testing/checks/cross_principal_leak.rb
|
|
72
|
+
- lib/reeve/testing/checks/endpoint_coverage.rb
|
|
70
73
|
- lib/reeve/testing/checks/guard_declared.rb
|
|
71
74
|
- lib/reeve/testing/checks/principal_required.rb
|
|
72
75
|
- lib/reeve/testing/checks/redaction_holds.rb
|