mcp-evidence-validator 0.4.3__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- mcp_evidence_validator-0.4.3/LICENSE +201 -0
- mcp_evidence_validator-0.4.3/NOTICE +17 -0
- mcp_evidence_validator-0.4.3/PKG-INFO +274 -0
- mcp_evidence_validator-0.4.3/README.md +246 -0
- mcp_evidence_validator-0.4.3/pyproject.toml +52 -0
- mcp_evidence_validator-0.4.3/setup.cfg +4 -0
- mcp_evidence_validator-0.4.3/src/mcp_evidence_validator/__init__.py +33 -0
- mcp_evidence_validator-0.4.3/src/mcp_evidence_validator/__main__.py +8 -0
- mcp_evidence_validator-0.4.3/src/mcp_evidence_validator/cli.py +148 -0
- mcp_evidence_validator-0.4.3/src/mcp_evidence_validator/fingerprint.py +31 -0
- mcp_evidence_validator-0.4.3/src/mcp_evidence_validator/ledger.py +116 -0
- mcp_evidence_validator-0.4.3/src/mcp_evidence_validator/validator.py +279 -0
- mcp_evidence_validator-0.4.3/src/mcp_evidence_validator.egg-info/PKG-INFO +274 -0
- mcp_evidence_validator-0.4.3/src/mcp_evidence_validator.egg-info/SOURCES.txt +20 -0
- mcp_evidence_validator-0.4.3/src/mcp_evidence_validator.egg-info/dependency_links.txt +1 -0
- mcp_evidence_validator-0.4.3/src/mcp_evidence_validator.egg-info/entry_points.txt +2 -0
- mcp_evidence_validator-0.4.3/src/mcp_evidence_validator.egg-info/top_level.txt +1 -0
- mcp_evidence_validator-0.4.3/tests/test_filesystem_example.py +334 -0
- mcp_evidence_validator-0.4.3/tests/test_fingerprint.py +36 -0
- mcp_evidence_validator-0.4.3/tests/test_ledger.py +65 -0
- mcp_evidence_validator-0.4.3/tests/test_ledger_forgery.py +84 -0
- mcp_evidence_validator-0.4.3/tests/test_validator.py +516 -0
|
@@ -0,0 +1,201 @@
|
|
|
1
|
+
Apache License
|
|
2
|
+
Version 2.0, January 2004
|
|
3
|
+
http://www.apache.org/licenses/
|
|
4
|
+
|
|
5
|
+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
|
6
|
+
|
|
7
|
+
1. Definitions.
|
|
8
|
+
|
|
9
|
+
"License" shall mean the terms and conditions for use, reproduction,
|
|
10
|
+
and distribution as defined by Sections 1 through 9 of this document.
|
|
11
|
+
|
|
12
|
+
"Licensor" shall mean the copyright owner or entity authorized by
|
|
13
|
+
the copyright owner that is granting the License.
|
|
14
|
+
|
|
15
|
+
"Legal Entity" shall mean the union of the acting entity and all
|
|
16
|
+
other entities that control, are controlled by, or are under common
|
|
17
|
+
control with that entity. For the purposes of this definition,
|
|
18
|
+
"control" means (i) the power, direct or indirect, to cause the
|
|
19
|
+
direction or management of such entity, whether by contract or
|
|
20
|
+
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
|
21
|
+
outstanding shares, or (iii) beneficial ownership of such entity.
|
|
22
|
+
|
|
23
|
+
"You" (or "Your") shall mean an individual or Legal Entity
|
|
24
|
+
exercising permissions granted by this License.
|
|
25
|
+
|
|
26
|
+
"Source" form shall mean the preferred form for making modifications,
|
|
27
|
+
including but not limited to software source code, documentation
|
|
28
|
+
source, and configuration files.
|
|
29
|
+
|
|
30
|
+
"Object" form shall mean any form resulting from mechanical
|
|
31
|
+
transformation or translation of a Source form, including but
|
|
32
|
+
not limited to compiled object code, generated documentation,
|
|
33
|
+
and conversions to other media types.
|
|
34
|
+
|
|
35
|
+
"Work" shall mean the work of authorship, whether in Source or
|
|
36
|
+
Object form, made available under the License, as indicated by a
|
|
37
|
+
copyright notice that is included in or attached to the work
|
|
38
|
+
(an example is provided in the Appendix below).
|
|
39
|
+
|
|
40
|
+
"Derivative Works" shall mean any work, whether in Source or Object
|
|
41
|
+
form, that is based on (or derived from) the Work and for which the
|
|
42
|
+
editorial revisions, annotations, elaborations, or other modifications
|
|
43
|
+
represent, as a whole, an original work of authorship. For the purposes
|
|
44
|
+
of this License, Derivative Works shall not include works that remain
|
|
45
|
+
separable from, or merely link (or bind by name) to the interfaces of,
|
|
46
|
+
the Work and Derivative Works thereof.
|
|
47
|
+
|
|
48
|
+
"Contribution" shall mean any work of authorship, including
|
|
49
|
+
the original version of the Work and any modifications or additions
|
|
50
|
+
to that Work or Derivative Works thereof, that is intentionally
|
|
51
|
+
submitted to Licensor for inclusion in the Work by the copyright owner
|
|
52
|
+
or by an individual or Legal Entity authorized to submit on behalf of
|
|
53
|
+
the copyright owner. For the purposes of this definition, "submitted"
|
|
54
|
+
means any form of electronic, verbal, or written communication sent
|
|
55
|
+
to the Licensor or its representatives, including but not limited to
|
|
56
|
+
communication on electronic mailing lists, source code control systems,
|
|
57
|
+
and issue tracking systems that are managed by, or on behalf of, the
|
|
58
|
+
Licensor for the purpose of discussing and improving the Work, but
|
|
59
|
+
excluding communication that is conspicuously marked or otherwise
|
|
60
|
+
designated in writing by the copyright owner as "Not a Contribution."
|
|
61
|
+
|
|
62
|
+
"Contributor" shall mean Licensor and any individual or Legal Entity
|
|
63
|
+
on behalf of whom a Contribution has been received by Licensor and
|
|
64
|
+
subsequently incorporated within the Work.
|
|
65
|
+
|
|
66
|
+
2. Grant of Copyright License. Subject to the terms and conditions of
|
|
67
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
68
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
69
|
+
copyright license to reproduce, prepare Derivative Works of,
|
|
70
|
+
publicly display, publicly perform, sublicense, and distribute the
|
|
71
|
+
Work and such Derivative Works in Source or Object form.
|
|
72
|
+
|
|
73
|
+
3. Grant of Patent License. Subject to the terms and conditions of
|
|
74
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
75
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
76
|
+
(except as stated in this section) patent license to make, have made,
|
|
77
|
+
use, offer to sell, sell, import, and otherwise transfer the Work,
|
|
78
|
+
where such license applies only to those patent claims licensable
|
|
79
|
+
by such Contributor that are necessarily infringed by their
|
|
80
|
+
Contribution(s) alone or by combination of their Contribution(s)
|
|
81
|
+
with the Work to which such Contribution(s) was submitted. If You
|
|
82
|
+
institute patent litigation against any entity (including a
|
|
83
|
+
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
|
84
|
+
or a Contribution incorporated within the Work constitutes direct
|
|
85
|
+
or contributory patent infringement, then any patent licenses
|
|
86
|
+
granted to You under this License for that Work shall terminate
|
|
87
|
+
as of the date such litigation is filed.
|
|
88
|
+
|
|
89
|
+
4. Redistribution. You may reproduce and distribute copies of the
|
|
90
|
+
Work or Derivative Works thereof in any medium, with or without
|
|
91
|
+
modifications, and in Source or Object form, provided that You
|
|
92
|
+
meet the following conditions:
|
|
93
|
+
|
|
94
|
+
(a) You must give any other recipients of the Work or
|
|
95
|
+
Derivative Works a copy of this License; and
|
|
96
|
+
|
|
97
|
+
(b) You must cause any modified files to carry prominent notices
|
|
98
|
+
stating that You changed the files; and
|
|
99
|
+
|
|
100
|
+
(c) You must retain, in the Source form of any Derivative Works
|
|
101
|
+
that You distribute, all copyright, patent, trademark, and
|
|
102
|
+
attribution notices from the Source form of the Work,
|
|
103
|
+
excluding those notices that do not pertain to any part of
|
|
104
|
+
the Derivative Works; and
|
|
105
|
+
|
|
106
|
+
(d) If the Work includes a "NOTICE" text file as part of its
|
|
107
|
+
distribution, then any Derivative Works that You distribute must
|
|
108
|
+
include a readable copy of the attribution notices contained
|
|
109
|
+
within such NOTICE file, excluding those notices that do not
|
|
110
|
+
pertain to any part of the Derivative Works, in at least one
|
|
111
|
+
of the following places: within a NOTICE text file distributed
|
|
112
|
+
as part of the Derivative Works; within the Source form or
|
|
113
|
+
documentation, if provided along with the Derivative Works; or,
|
|
114
|
+
within a display generated by the Derivative Works, if and
|
|
115
|
+
wherever such third-party notices normally appear. The contents
|
|
116
|
+
of the NOTICE file are for informational purposes only and
|
|
117
|
+
do not modify the License. You may add Your own attribution
|
|
118
|
+
notices within Derivative Works that You distribute, alongside
|
|
119
|
+
or as an addendum to the NOTICE text from the Work, provided
|
|
120
|
+
that such additional attribution notices cannot be construed
|
|
121
|
+
as modifying the License.
|
|
122
|
+
|
|
123
|
+
You may add Your own copyright statement to Your modifications and
|
|
124
|
+
may provide additional or different license terms and conditions
|
|
125
|
+
for use, reproduction, or distribution of Your modifications, or
|
|
126
|
+
for any such Derivative Works as a whole, provided Your use,
|
|
127
|
+
reproduction, and distribution of the Work otherwise complies with
|
|
128
|
+
the conditions stated in this License.
|
|
129
|
+
|
|
130
|
+
5. Submission of Contributions. Unless You explicitly state otherwise,
|
|
131
|
+
any Contribution intentionally submitted for inclusion in the Work
|
|
132
|
+
by You to the Licensor shall be under the terms and conditions of
|
|
133
|
+
this License, without any additional terms or conditions.
|
|
134
|
+
Notwithstanding the above, nothing herein shall supersede or modify
|
|
135
|
+
the terms of any separate license agreement you may have executed
|
|
136
|
+
with Licensor regarding such Contributions.
|
|
137
|
+
|
|
138
|
+
6. Trademarks. This License does not grant permission to use the trade
|
|
139
|
+
names, trademarks, service marks, or product names of the Licensor,
|
|
140
|
+
except as required for reasonable and customary use in describing the
|
|
141
|
+
origin of the Work and reproducing the content of the NOTICE file.
|
|
142
|
+
|
|
143
|
+
7. Disclaimer of Warranty. Unless required by applicable law or
|
|
144
|
+
agreed to in writing, Licensor provides the Work (and each
|
|
145
|
+
Contributor provides its Contributions) on an "AS IS" BASIS,
|
|
146
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
|
147
|
+
implied, including, without limitation, any warranties or conditions
|
|
148
|
+
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
|
149
|
+
PARTICULAR PURPOSE. You are solely responsible for determining the
|
|
150
|
+
appropriateness of using or redistributing the Work and assume any
|
|
151
|
+
risks associated with Your exercise of permissions under this License.
|
|
152
|
+
|
|
153
|
+
8. Limitation of Liability. In no event and under no legal theory,
|
|
154
|
+
whether in tort (including negligence), contract, or otherwise,
|
|
155
|
+
unless required by applicable law (such as deliberate and grossly
|
|
156
|
+
negligent acts) or agreed to in writing, shall any Contributor be
|
|
157
|
+
liable to You for damages, including any direct, indirect, special,
|
|
158
|
+
incidental, or consequential damages of any character arising as a
|
|
159
|
+
result of this License or out of the use or inability to use the
|
|
160
|
+
Work (including but not limited to damages for loss of goodwill,
|
|
161
|
+
work stoppage, computer failure or malfunction, or any and all
|
|
162
|
+
other commercial damages or losses), even if such Contributor
|
|
163
|
+
has been advised of the possibility of such damages.
|
|
164
|
+
|
|
165
|
+
9. Accepting Warranty or Additional Liability. While redistributing
|
|
166
|
+
the Work or Derivative Works thereof, You may choose to offer,
|
|
167
|
+
and charge a fee for, acceptance of support, warranty, indemnity,
|
|
168
|
+
or other liability obligations and/or rights consistent with this
|
|
169
|
+
License. However, in accepting such obligations, You may act only
|
|
170
|
+
on Your own behalf and on Your sole responsibility, not on behalf
|
|
171
|
+
of any other Contributor, and only if You agree to indemnify,
|
|
172
|
+
defend, and hold each Contributor harmless for any liability
|
|
173
|
+
incurred by, or claims asserted against, such Contributor by reason
|
|
174
|
+
of your accepting any such warranty or additional liability.
|
|
175
|
+
|
|
176
|
+
END OF TERMS AND CONDITIONS
|
|
177
|
+
|
|
178
|
+
APPENDIX: How to apply the Apache License to your work.
|
|
179
|
+
|
|
180
|
+
To apply the Apache License to your work, attach the following
|
|
181
|
+
boilerplate notice, with the fields enclosed by brackets "[]"
|
|
182
|
+
replaced with your own identifying information. (Don't include
|
|
183
|
+
the brackets!) The text should be enclosed in the appropriate
|
|
184
|
+
comment syntax for the file format. We also recommend that a
|
|
185
|
+
file or class name and description of purpose be included on the
|
|
186
|
+
same "printed page" as the copyright notice for easier
|
|
187
|
+
identification within third-party archives.
|
|
188
|
+
|
|
189
|
+
Copyright [yyyy] [name of copyright owner]
|
|
190
|
+
|
|
191
|
+
Licensed under the Apache License, Version 2.0 (the "License");
|
|
192
|
+
you may not use this file except in compliance with the License.
|
|
193
|
+
You may obtain a copy of the License at
|
|
194
|
+
|
|
195
|
+
http://www.apache.org/licenses/LICENSE-2.0
|
|
196
|
+
|
|
197
|
+
Unless required by applicable law or agreed to in writing, software
|
|
198
|
+
distributed under the License is distributed on an "AS IS" BASIS,
|
|
199
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
200
|
+
See the License for the specific language governing permissions and
|
|
201
|
+
limitations under the License.
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
MCP Evidence Validator
|
|
2
|
+
Copyright 2026 Empire Labs Pty Ltd
|
|
3
|
+
|
|
4
|
+
This product includes software developed at Empire Labs Pty Ltd
|
|
5
|
+
(https://empirelabs.com.au).
|
|
6
|
+
|
|
7
|
+
Licensed under the Apache License, Version 2.0 (the "License");
|
|
8
|
+
you may not use this file except in compliance with the License.
|
|
9
|
+
You may obtain a copy of the License at
|
|
10
|
+
|
|
11
|
+
http://www.apache.org/licenses/LICENSE-2.0
|
|
12
|
+
|
|
13
|
+
Unless required by applicable law or agreed to in writing, software
|
|
14
|
+
distributed under the License is distributed on an "AS IS" BASIS,
|
|
15
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
16
|
+
See the License for the specific language governing permissions and
|
|
17
|
+
limitations under the License.
|
|
@@ -0,0 +1,274 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: mcp-evidence-validator
|
|
3
|
+
Version: 0.4.3
|
|
4
|
+
Summary: Validate what an MCP server declares against what it actually does, with a tamper-evident SHA-256 evidence ledger.
|
|
5
|
+
Author-email: Empire Labs Pty Ltd <contact@empirelabs.com.au>
|
|
6
|
+
License: Apache-2.0
|
|
7
|
+
Project-URL: Homepage, https://github.com/narko4u/mcp-evidence-validator
|
|
8
|
+
Project-URL: Source, https://github.com/narko4u/mcp-evidence-validator
|
|
9
|
+
Project-URL: Documentation, https://github.com/narko4u/mcp-evidence-validator/tree/main/docs
|
|
10
|
+
Project-URL: Issues, https://github.com/narko4u/mcp-evidence-validator/issues
|
|
11
|
+
Keywords: mcp,model-context-protocol,evidence,audit,observability,agentic-ai
|
|
12
|
+
Classifier: Development Status :: 4 - Beta
|
|
13
|
+
Classifier: Intended Audience :: Developers
|
|
14
|
+
Classifier: Intended Audience :: Information Technology
|
|
15
|
+
Classifier: License :: OSI Approved :: Apache Software License
|
|
16
|
+
Classifier: Operating System :: OS Independent
|
|
17
|
+
Classifier: Programming Language :: Python :: 3
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
19
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
20
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
21
|
+
Classifier: Topic :: Security
|
|
22
|
+
Classifier: Topic :: Software Development :: Quality Assurance
|
|
23
|
+
Requires-Python: >=3.10
|
|
24
|
+
Description-Content-Type: text/markdown
|
|
25
|
+
License-File: LICENSE
|
|
26
|
+
License-File: NOTICE
|
|
27
|
+
Dynamic: license-file
|
|
28
|
+
|
|
29
|
+
# MCP Evidence Validator
|
|
30
|
+
|
|
31
|
+
[](https://www.bestpractices.dev/projects/14122)
|
|
32
|
+
[](https://www.bestpractices.dev/projects/14122)
|
|
33
|
+
|
|
34
|
+
Validate what an MCP server *declares* against what it *actually does*, and produce a tamper-evident evidence record you can hand to an auditor.
|
|
35
|
+
|
|
36
|
+
**Status:** v0.4 (installable) · **License:** Apache-2.0 · **Language:** Python 3.10+ (stdlib only, zero dependencies)
|
|
37
|
+
|
|
38
|
+
**Built by [Empire Labs Pty Ltd](https://empirelabs.com.au)** and published as part of the Empire Stack — the evidence layer for agent actions. Related public work:
|
|
39
|
+
|
|
40
|
+
- [witnessos](https://github.com/narko4u/witnessos) — credential-brokered enforcement and cryptographic evidence for agent actions; the evidence-grade ladder these checks descend from (E0 Declared → E1 Observed → E2 Enforced → E3 Corroborated → E4 Anchored).
|
|
41
|
+
- [evidence-record-spec](https://github.com/narko4u/evidence-record-spec) — portable, evidence-grade record format for agent actions.
|
|
42
|
+
|
|
43
|
+
---
|
|
44
|
+
|
|
45
|
+
## Why
|
|
46
|
+
|
|
47
|
+
Model Context Protocol (MCP) is the industry standard for connecting AI agents to tools and data. It is now governed by the Agentic AI Foundation (AAIF) under the Linux Foundation, with 97M+ monthly SDK downloads and 10,000+ active public servers.
|
|
48
|
+
|
|
49
|
+
But the protocol leaves the *evidence* problem open:
|
|
50
|
+
|
|
51
|
+
- A server declares tool schemas and permissions, but nothing verifies those declarations against observed runtime behaviour.
|
|
52
|
+
- An annotation can be accurate at declaration time and silently stale minutes later, when the contract it was bound to mutates underneath it.
|
|
53
|
+
- MCP has already seen the first malicious server (Sept 2025), CVE-2025-6514 (CVSS 9.6, RCE), and a hosting-platform breach affecting 3,000+ downstream applications.
|
|
54
|
+
|
|
55
|
+
The **declared-vs-observed gap** is the measurement finding that matters. This validator is a reference implementation for closing it.
|
|
56
|
+
|
|
57
|
+
## What it does
|
|
58
|
+
|
|
59
|
+
1. **Captures declarations** - the tool schemas, permissions, and annotations an MCP server publishes.
|
|
60
|
+
2. **Observes reality** - the tool invocations, argument shapes, and contract hashes seen at runtime.
|
|
61
|
+
3. **Checks the gap** - declared annotation still bound? Contract mutated since declaration? Privilege use inside declared scope?
|
|
62
|
+
4. **Produces an anchored evidence ledger** - every check result is committed to a SHA-256 hash chain, and `validate` prints the head digest of that chain. Record the head somewhere the ledger file's holder cannot edit, and `verify` refuses any ledger that does not reach it. A chain read on its own proves ordering to whoever holds the file and nothing to anyone else: an editor who can change a record can replay the chain over the change, drop the last block, or publish any `prev_hash` and `index` they like.
|
|
63
|
+
|
|
64
|
+
## Install
|
|
65
|
+
|
|
66
|
+
```bash
|
|
67
|
+
pip install mcp-evidence-validator # from PyPI once published
|
|
68
|
+
# or from source:
|
|
69
|
+
git clone https://github.com/narko4u/mcp-evidence-validator.git
|
|
70
|
+
cd mcp-evidence-validator
|
|
71
|
+
pip install .
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
No dependencies — Python 3.10+ standard library only.
|
|
75
|
+
|
|
76
|
+
## Quick start
|
|
77
|
+
|
|
78
|
+
After installing from source above, run these commands from the repository root
|
|
79
|
+
so the bundled example files are available.
|
|
80
|
+
|
|
81
|
+
```bash
|
|
82
|
+
# Compare a declared manifest against observed runtime records, and write the
|
|
83
|
+
# head digest of the resulting ledger to a separate file
|
|
84
|
+
mcp-ev-validate validate --declared examples/fictional-server-declared.json --observed examples/fictional-server-observed.json --out evidence.json --head-out evidence.head
|
|
85
|
+
|
|
86
|
+
# Audit a ledger later, against the head you kept elsewhere
|
|
87
|
+
mcp-ev-validate verify --ledger evidence.json --expected-head "$(cat evidence.head)"
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
`--expected-head` is required. Keep the head where the ledger's holder cannot reach it: a signature, a commit in another repository, a transparency log entry, or a line in the auditor's own notes. Both files land in the current directory, so the commands run unchanged on Windows.
|
|
91
|
+
|
|
92
|
+
The `validate` command prints a JSON report. Its `summary` contains:
|
|
93
|
+
|
|
94
|
+
```json
|
|
95
|
+
{
|
|
96
|
+
"declared_tools": 2,
|
|
97
|
+
"observations": 3,
|
|
98
|
+
"findings": 2,
|
|
99
|
+
"contract_recipes": ["1"],
|
|
100
|
+
"checks": [
|
|
101
|
+
"bound_unmutated",
|
|
102
|
+
"contract_mutated",
|
|
103
|
+
"scope_violation",
|
|
104
|
+
"recipe_mismatch"
|
|
105
|
+
]
|
|
106
|
+
}
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
`contract_recipes` names the recipe each contract hash was computed under — `["1"]` here because the constructed example predates the current recipe and does not state one, so `validate` also notes that on stderr. See [Contract recipes](#contract-recipes).
|
|
110
|
+
|
|
111
|
+
The report's `findings` array describes the deliberately undeclared `admin_token` argument in observation 2 and the changed contract in observation 3. Validation exits **0** when the report is produced, even when it contains findings; inspect the report to assess the observed behaviour.
|
|
112
|
+
|
|
113
|
+
Record the head somewhere the ledger file's holder cannot edit. `verify` checks the chain against that head and, for this example, exits **0** with:
|
|
114
|
+
|
|
115
|
+
```text
|
|
116
|
+
ledger intact: 3 blocks, chain verified against the expected head
|
|
117
|
+
block types: {"declaration": 1, "observation_batch": 1, "report": 1}
|
|
118
|
+
contract recipes: 1
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
The `contract recipes:` line is the verification being explicit about how the evidence it just checked was produced: a contract hash means nothing without the recipe that produced it.
|
|
122
|
+
|
|
123
|
+
`verify` exits **1** when the ledger does not reach the expected head, and **2** when it is invoked without one. It checks the ledger against the head you pass in, so it does not vouch for the observations being free of findings: a chain read on its own proves ordering to whoever holds the file, and nothing to anyone else.
|
|
124
|
+
|
|
125
|
+
Or run as a module: `python -m mcp_evidence_validator validate --declared ...`
|
|
126
|
+
|
|
127
|
+
Output is a machine-readable evidence record with a `chain` of hash-linked entries plus a human-readable `findings` summary. The `verify` subcommand replays the chain, checks the published `prev_hash` and `index` of every block against that walk, and compares the head it reaches to the one you pass in. Try editing `evidence.json` and re-verifying against the original head: the edit is refused.
|
|
128
|
+
|
|
129
|
+
## Covered server types
|
|
130
|
+
|
|
131
|
+
| Example | Server | Declared | Observed | Recipe | Shows |
|
|
132
|
+
|---------|--------|----------|----------|--------|-------|
|
|
133
|
+
| `examples/fictional-server-*.json` | fictional weather server | — | — | 1 (legacy) | all three checks against constructed data |
|
|
134
|
+
| `examples/filesystem-server-*.json` | `@modelcontextprotocol/server-filesystem` | 2026.1.14 | 2026.8.31 | 2 | three real contract mutations between two releases, two of which recipe 1 could not see |
|
|
135
|
+
|
|
136
|
+
The second pair is a real capture, not a constructed one:
|
|
137
|
+
|
|
138
|
+
```bash
|
|
139
|
+
mcp-ev-validate validate --declared examples/filesystem-server-declared.json --observed examples/filesystem-server-observed.json --out fs-evidence.json --head-out fs-evidence.head
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
It reports three findings, one per observed call. `read_media_file`'s contract moved between the two versions in a way that matters to a caller: the description went from "Read an image or audio file" to reading *any* file, "returned as an embedded resource" when it is neither image nor audio, and the output schema gained an `anyOf` branch whose second case is a `resource` carrying a `blob`, while the first case lost `blob` from its `type` enum. A client that bound a handler to the earlier output shape at declaration time is now holding a stale annotation; the check says so, and the two hashes it prints are the two real contracts.
|
|
143
|
+
|
|
144
|
+
The other two, `read_text_file` and `get_file_info`, are the more interesting findings: the `2026.8.31` release added an `openWorldHint` annotation to every tool it serves, and those two changed *nothing else*. Under recipe 1 they were byte-identical and reported healthy. Under recipe 2 their contracts moved, correctly, because what a tool declares about its own side effects is part of what it declares.
|
|
145
|
+
|
|
146
|
+
The three calls in the observed file were made against the running server and every `contract_hash` is the validator's own fingerprint over what that server served. The raw replies are committed in `examples/captures/` and `tests/test_filesystem_example.py` recomputes each hash from them, so the pair cannot silently drift from its evidence. To recapture:
|
|
147
|
+
|
|
148
|
+
```bash
|
|
149
|
+
python3 examples/capture_mcp_server.py \
|
|
150
|
+
--package @modelcontextprotocol/server-filesystem \
|
|
151
|
+
--declared-version 2026.1.14 --observed-version 2026.8.31 \
|
|
152
|
+
--root /tmp/mcp-capture-root --label filesystem-server
|
|
153
|
+
```
|
|
154
|
+
|
|
155
|
+
That needs Node.js (`npx`) and registry access. The committed files are static, so neither the tests nor the validator need either.
|
|
156
|
+
|
|
157
|
+
To rebuild the pair from the committed captures — which is how a pair is migrated to a new contract recipe, with no Node.js and no network:
|
|
158
|
+
|
|
159
|
+
```bash
|
|
160
|
+
python3 examples/rebuild_pair.py --recipe 2 # rewrite the pair and its derived hashes
|
|
161
|
+
python3 examples/rebuild_pair.py --check # report differences, write nothing
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
## Concepts
|
|
165
|
+
|
|
166
|
+
| Term | Meaning |
|
|
167
|
+
|------|---------|
|
|
168
|
+
| Declaration | What a server publishes: tool name, description, input schema, permission scope |
|
|
169
|
+
| Contract | A canonical, hashable form of a declaration (canonical JSON → SHA-256) |
|
|
170
|
+
| Contract recipe | Which declaration fields a contract hash covers. Carried in the evidence next to the hash, never assumed — see [Contract recipes](#contract-recipes) |
|
|
171
|
+
| Annotation | A statement binding a declaration to a contract (e.g. "this tool is scoped to read-only"). Distinct from a *tool's* MCP `annotations` hints (`readOnlyHint`, `destructiveHint`, …), which recipe 2 folds into the contract itself |
|
|
172
|
+
| Observation | A runtime fact: invocation record, argument values, contract hash at call time |
|
|
173
|
+
| Finding | A measurable gap between declaration and observation |
|
|
174
|
+
| Ledger | An append-only SHA-256 hash chain over every captured record |
|
|
175
|
+
|
|
176
|
+
## Contract recipes
|
|
177
|
+
|
|
178
|
+
A contract hash answers "has this declaration changed?" — which is only useful if you also know *which parts of the declaration it covers*. That field set is the recipe, and it travels with the evidence instead of being assumed:
|
|
179
|
+
|
|
180
|
+
| Recipe | Covers | Applies to |
|
|
181
|
+
|--------|--------|-----------|
|
|
182
|
+
| `1` | name, description, input schema, permissions | every ledger written before v0.4.0, and any declaration that does not state a recipe |
|
|
183
|
+
| `2` | recipe 1, plus the declared output schema and the tool's MCP annotation hints | the current default for new declarations |
|
|
184
|
+
|
|
185
|
+
Where it lives:
|
|
186
|
+
|
|
187
|
+
- a declaration states `contract_recipe` at the manifest level, and an individual tool can override it;
|
|
188
|
+
- an observation may state the `contract_recipe` its `contract_hash` was computed under; one that carries a hash but states none is read as recipe 1 — the same default a silent declaration gets — and one that carries no `contract_hash` at all (a scope-only observation) states no recipe either way;
|
|
189
|
+
- the report's `summary` lists every recipe in play, including one stated on an observation for a tool the declaration does not carry, and `verify` prints them for a ledger.
|
|
190
|
+
|
|
191
|
+
Four rules make that safe rather than decorative:
|
|
192
|
+
|
|
193
|
+
1. **No silent reinterpretation.** A declaration that states no recipe is hashed under recipe 1, exactly as before — so ledgers issued before the recipe existed still verify, and nothing already published changes meaning. New declarations opt up by stating the recipe.
|
|
194
|
+
2. **No guessing across recipes.** If an observation states a different recipe from the declaration's, the comparison is refused with a `recipe_mismatch` finding rather than reported as drift. Two hashes computed different ways are not evidence of change in either direction.
|
|
195
|
+
3. **Silence means legacy, not unknown — where there is a hash to interpret.** An observation that carries a `contract_hash` but states no recipe was produced before recipes existed, so it is read as recipe 1 — and a pre-v0.4.0 ledger compared against a recipe-2 declaration is therefore refused, not reported as drift. The finding says so, and states the one-line change (`contract_recipe: "1"` on the declaration) that judges that ledger under the recipe that produced it. The inference stops at the hash: an observation that carries no `contract_hash` states no recipe either way, so none is inferred for it and no finding claims a hash the evidence does not hold. Arguments on such an observation are still judged against the declared input schema.
|
|
196
|
+
4. **A recipe is read wherever it appears.** An observation for a tool the declaration does not carry is still parsed and counted, so the summary cannot describe a ledger as holding one recipe while the evidence also holds a hash made under another. An unknown recipe is rejected there too, rather than escaping validation by virtue of the tool being undeclared.
|
|
197
|
+
|
|
198
|
+
Migrating a pair between recipes re-derives every contract hash from the raw captures, never from a stored hash — which is why the example pair can be rebuilt offline:
|
|
199
|
+
|
|
200
|
+
```bash
|
|
201
|
+
python3 examples/rebuild_pair.py --recipe 2
|
|
202
|
+
```
|
|
203
|
+
|
|
204
|
+
The example in this repository shows what the wider recipe buys: two of its three findings are contract changes that recipe 1 reported as healthy.
|
|
205
|
+
|
|
206
|
+
## Check types
|
|
207
|
+
|
|
208
|
+
- **Bound, contract unmutated** - healthy baseline: annotation still matches the contract it was declared against.
|
|
209
|
+
- **Bound, contract since mutated** - finding: the annotation was accurate at declaration time but is stale because the contract moved underneath it. Pairs with a review-scheduling gate.
|
|
210
|
+
- **Observed outside declared scope** - finding: runtime behaviour exceeds what the declaration permits (arguments, tools, or permissions not present in the declaration).
|
|
211
|
+
- **Recipe mismatch** - finding: the observed contract hash was computed under a different recipe from the declaration's (including an observation that states none, and is read as recipe 1), so the comparison is refused. High severity because it means the check could not run — not because the server did anything wrong.
|
|
212
|
+
|
|
213
|
+
## Roadmap
|
|
214
|
+
|
|
215
|
+
- [x] Installable package (v0.2, `mcp-ev-validate` CLI, `verify` subcommand)
|
|
216
|
+
- [x] Test suite + CI (Python 3.10–3.12)
|
|
217
|
+
- [x] Anchored evidence ledger — `verify` refuses a ledger that does not reach the head digest recorded outside it (v0.3)
|
|
218
|
+
- [x] Contract recipes — a contract hash states which declaration fields it covers, and a recipe disagreement is refused rather than guessed (v0.4)
|
|
219
|
+
- [ ] **A2A agent-card validation** — validate A2A agent cards
|
|
220
|
+
(`.well-known/agent-card.json`) against observed agent behaviour:
|
|
221
|
+
declared capabilities vs runtime delegation, auth requirements honoured,
|
|
222
|
+
signed-card identity checks. A2A is an AAIF-hosted project (joined
|
|
223
|
+
Aug 2026); this extends the declared-vs-observed evidence ladder to the
|
|
224
|
+
agent-to-agent boundary. Directly addresses the open A2A identity
|
|
225
|
+
verification gap (a2aproject/A2A issue #1672).
|
|
226
|
+
- [ ] MCP client integration (intercept tool-call records via a lightweight proxy)
|
|
227
|
+
- [ ] Automated review-scheduling gate (re-validate annotations on contract change)
|
|
228
|
+
- [ ] Report renderers (HTML, PDF)
|
|
229
|
+
- [ ] Policy pack support (declare what "acceptable" means per environment)
|
|
230
|
+
|
|
231
|
+
## Project status
|
|
232
|
+
|
|
233
|
+
A reference implementation of declared-vs-observed evidence for MCP servers, built and maintained by **[Empire Labs Pty Ltd](https://empirelabs.com.au)** as the open layer of its evidence-engineering work. The enforcement and cryptographic-evidence side of that work is [witnessos](https://github.com/narko4u/witnessos); the record format is [evidence-record-spec](https://github.com/narko4u/evidence-record-spec).
|
|
234
|
+
|
|
235
|
+
It is also the implementation this project puts behind the argument when the declared-vs-observed gap comes up in standards venues — including the Agentic AI Foundation's Security & Privacy work on tamper-evident evidence for agent actions — because a gap is easier to settle with a working validator than with a slide.
|
|
236
|
+
|
|
237
|
+
The core ideas are exercised in production-grade systems elsewhere in the organisation. This repository contains no proprietary code and no client data: the fictional example is constructed to exercise the checks, and the filesystem example is a capture of a public open-source MCP reference server, committed together with the raw replies it was derived from.
|
|
238
|
+
|
|
239
|
+
## Security
|
|
240
|
+
|
|
241
|
+
See [SECURITY.md](SECURITY.md) for the vulnerability reporting policy. Security issues are handled privately — do not open a public issue.
|
|
242
|
+
|
|
243
|
+
## Verifying releases
|
|
244
|
+
|
|
245
|
+
Releases are signed with **sigstore keyless signing** (GitHub Actions
|
|
246
|
+
workload identity). Each release contains:
|
|
247
|
+
|
|
248
|
+
- `SHA256SUMS` — hashes of every release asset
|
|
249
|
+
- `SHA256SUMS.sig` — the cosign signature over `SHA256SUMS`
|
|
250
|
+
- `SHA256SUMS.pem` — the ephemeral signing certificate
|
|
251
|
+
|
|
252
|
+
To verify a release (requires the [cosign CLI](https://docs.sigstore.dev/cosign/installation/)):
|
|
253
|
+
|
|
254
|
+
```sh
|
|
255
|
+
cosign verify-blob \
|
|
256
|
+
--cert SHA256SUMS.pem \
|
|
257
|
+
--signature SHA256SUMS.sig \
|
|
258
|
+
--certificate-identity "https://github.com/narko4u/mcp-evidence-validator/.github/workflows/release.yml@refs/tags/v0.2.1" \
|
|
259
|
+
--certificate-oidc-issuer "https://token.actions.githubusercontent.com" \
|
|
260
|
+
SHA256SUMS
|
|
261
|
+
```
|
|
262
|
+
|
|
263
|
+
The expected signer identity is the repository's `release.yml` workflow
|
|
264
|
+
running under the release tag (GitHub Actions OIDC, issuer
|
|
265
|
+
`https://token.actions.githubusercontent.com`). After the signature
|
|
266
|
+
verifies, check the asset hashes:
|
|
267
|
+
|
|
268
|
+
```sh
|
|
269
|
+
sha256sum -c SHA256SUMS
|
|
270
|
+
```
|
|
271
|
+
|
|
272
|
+
## Contributing
|
|
273
|
+
|
|
274
|
+
See [CONTRIBUTING.md](CONTRIBUTING.md). Contributions welcome under the Apache-2.0 licence; please follow the Code of Conduct.
|