synapsek-claims 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- synapsek_claims-0.1.0/LICENSE +202 -0
- synapsek_claims-0.1.0/MANIFEST.in +8 -0
- synapsek_claims-0.1.0/PKG-INFO +158 -0
- synapsek_claims-0.1.0/README.md +140 -0
- synapsek_claims-0.1.0/RELEASE-CHECKLIST.md +65 -0
- synapsek_claims-0.1.0/docs/DESIGN.md +98 -0
- synapsek_claims-0.1.0/examples/TRUST.md +17 -0
- synapsek_claims-0.1.0/examples/cartridge-v11/attestation/v1.1.json +779 -0
- synapsek_claims-0.1.0/examples/cartridge-v11/claim.json +306 -0
- synapsek_claims-0.1.0/examples/cartridge-v11/evidence/A_seed1.jsonl +100 -0
- synapsek_claims-0.1.0/examples/cartridge-v11/evidence/A_seed2.jsonl +100 -0
- synapsek_claims-0.1.0/examples/cartridge-v11/evidence/A_seed3.jsonl +100 -0
- synapsek_claims-0.1.0/examples/cartridge-v11/evidence/B_seed1.jsonl +100 -0
- synapsek_claims-0.1.0/examples/cartridge-v11/evidence/B_seed2.jsonl +100 -0
- synapsek_claims-0.1.0/examples/cartridge-v11/evidence/B_seed3.jsonl +100 -0
- synapsek_claims-0.1.0/examples/pipeline-dedup/claim.json +147 -0
- synapsek_claims-0.1.0/examples/pipeline-dedup/code/config.py +30 -0
- synapsek_claims-0.1.0/examples/pipeline-dedup/code/encoder.py +77 -0
- synapsek_claims-0.1.0/examples/pipeline-dedup/code/eval_dedup.py +144 -0
- synapsek_claims-0.1.0/examples/pipeline-dedup/code/layers.py +31 -0
- synapsek_claims-0.1.0/examples/pipeline-dedup/code/tokenizer/merges.txt +151387 -0
- synapsek_claims-0.1.0/examples/pipeline-dedup/code/tokenizer/tokenizer.json +303282 -0
- synapsek_claims-0.1.0/examples/pipeline-dedup/code/tokenizer/tokenizer_config.json +207 -0
- synapsek_claims-0.1.0/examples/pipeline-dedup/code/tokenizer/vocab.json +1 -0
- synapsek_claims-0.1.0/examples/pipeline-dedup/data/docs.jsonl +100 -0
- synapsek_claims-0.1.0/examples/pipeline-dedup/evidence/pairs.jsonl +4950 -0
- synapsek_claims-0.1.0/examples/pipeline-dedup/evidence/summary.json +1 -0
- synapsek_claims-0.1.0/examples/trusted-issuer.pub +1 -0
- synapsek_claims-0.1.0/pyproject.toml +28 -0
- synapsek_claims-0.1.0/setup.cfg +4 -0
- synapsek_claims-0.1.0/src/svc/__init__.py +18 -0
- synapsek_claims-0.1.0/src/svc/canonical.py +55 -0
- synapsek_claims-0.1.0/src/svc/claim.py +73 -0
- synapsek_claims-0.1.0/src/svc/cli.py +142 -0
- synapsek_claims-0.1.0/src/svc/crypto.py +57 -0
- synapsek_claims-0.1.0/src/svc/registry.py +63 -0
- synapsek_claims-0.1.0/src/svc/rerun.py +77 -0
- synapsek_claims-0.1.0/src/svc/rules.py +48 -0
- synapsek_claims-0.1.0/src/svc/schema.py +105 -0
- synapsek_claims-0.1.0/src/svc/verify.py +218 -0
- synapsek_claims-0.1.0/src/synapsek_claims.egg-info/PKG-INFO +158 -0
- synapsek_claims-0.1.0/src/synapsek_claims.egg-info/SOURCES.txt +49 -0
- synapsek_claims-0.1.0/src/synapsek_claims.egg-info/dependency_links.txt +1 -0
- synapsek_claims-0.1.0/src/synapsek_claims.egg-info/entry_points.txt +2 -0
- synapsek_claims-0.1.0/src/synapsek_claims.egg-info/requires.txt +4 -0
- synapsek_claims-0.1.0/src/synapsek_claims.egg-info/top_level.txt +1 -0
- synapsek_claims-0.1.0/tests/conftest.py +102 -0
- synapsek_claims-0.1.0/tests/test_canonical.py +33 -0
- synapsek_claims-0.1.0/tests/test_registry.py +37 -0
- synapsek_claims-0.1.0/tests/test_roundtrip.py +20 -0
- synapsek_claims-0.1.0/tests/test_tamper.py +83 -0
|
@@ -0,0 +1,202 @@
|
|
|
1
|
+
|
|
2
|
+
Apache License
|
|
3
|
+
Version 2.0, January 2004
|
|
4
|
+
http://www.apache.org/licenses/
|
|
5
|
+
|
|
6
|
+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
|
7
|
+
|
|
8
|
+
1. Definitions.
|
|
9
|
+
|
|
10
|
+
"License" shall mean the terms and conditions for use, reproduction,
|
|
11
|
+
and distribution as defined by Sections 1 through 9 of this document.
|
|
12
|
+
|
|
13
|
+
"Licensor" shall mean the copyright owner or entity authorized by
|
|
14
|
+
the copyright owner that is granting the License.
|
|
15
|
+
|
|
16
|
+
"Legal Entity" shall mean the union of the acting entity and all
|
|
17
|
+
other entities that control, are controlled by, or are under common
|
|
18
|
+
control with that entity. For the purposes of this definition,
|
|
19
|
+
"control" means (i) the power, direct or indirect, to cause the
|
|
20
|
+
direction or management of such entity, whether by contract or
|
|
21
|
+
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
|
22
|
+
outstanding shares, or (iii) beneficial ownership of such entity.
|
|
23
|
+
|
|
24
|
+
"You" (or "Your") shall mean an individual or Legal Entity
|
|
25
|
+
exercising permissions granted by this License.
|
|
26
|
+
|
|
27
|
+
"Source" form shall mean the preferred form for making modifications,
|
|
28
|
+
including but not limited to software source code, documentation
|
|
29
|
+
source, and configuration files.
|
|
30
|
+
|
|
31
|
+
"Object" form shall mean any form resulting from mechanical
|
|
32
|
+
transformation or translation of a Source form, including but
|
|
33
|
+
not limited to compiled object code, generated documentation,
|
|
34
|
+
and conversions to other media types.
|
|
35
|
+
|
|
36
|
+
"Work" shall mean the work of authorship, whether in Source or
|
|
37
|
+
Object form, made available under the License, as indicated by a
|
|
38
|
+
copyright notice that is included in or attached to the work
|
|
39
|
+
(an example is provided in the Appendix below).
|
|
40
|
+
|
|
41
|
+
"Derivative Works" shall mean any work, whether in Source or Object
|
|
42
|
+
form, that is based on (or derived from) the Work and for which the
|
|
43
|
+
editorial revisions, annotations, elaborations, or other modifications
|
|
44
|
+
represent, as a whole, an original work of authorship. For the purposes
|
|
45
|
+
of this License, Derivative Works shall not include works that remain
|
|
46
|
+
separable from, or merely link (or bind by name) to the interfaces of,
|
|
47
|
+
the Work and Derivative Works thereof.
|
|
48
|
+
|
|
49
|
+
"Contribution" shall mean any work of authorship, including
|
|
50
|
+
the original version of the Work and any modifications or additions
|
|
51
|
+
to that Work or Derivative Works thereof, that is intentionally
|
|
52
|
+
submitted to Licensor for inclusion in the Work by the copyright owner
|
|
53
|
+
or by an individual or Legal Entity authorized to submit on behalf of
|
|
54
|
+
the copyright owner. For the purposes of this definition, "submitted"
|
|
55
|
+
means any form of electronic, verbal, or written communication sent
|
|
56
|
+
to the Licensor or its representatives, including but not limited to
|
|
57
|
+
communication on electronic mailing lists, source code control systems,
|
|
58
|
+
and issue tracking systems that are managed by, or on behalf of, the
|
|
59
|
+
Licensor for the purpose of discussing and improving the Work, but
|
|
60
|
+
excluding communication that is conspicuously marked or otherwise
|
|
61
|
+
designated in writing by the copyright owner as "Not a Contribution."
|
|
62
|
+
|
|
63
|
+
"Contributor" shall mean Licensor and any individual or Legal Entity
|
|
64
|
+
on behalf of whom a Contribution has been received by Licensor and
|
|
65
|
+
subsequently incorporated within the Work.
|
|
66
|
+
|
|
67
|
+
2. Grant of Copyright License. Subject to the terms and conditions of
|
|
68
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
69
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
70
|
+
copyright license to reproduce, prepare Derivative Works of,
|
|
71
|
+
publicly display, publicly perform, sublicense, and distribute the
|
|
72
|
+
Work and such Derivative Works in Source or Object form.
|
|
73
|
+
|
|
74
|
+
3. Grant of Patent License. Subject to the terms and conditions of
|
|
75
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
76
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
77
|
+
(except as stated in this section) patent license to make, have made,
|
|
78
|
+
use, offer to sell, sell, import, and otherwise transfer the Work,
|
|
79
|
+
where such license applies only to those patent claims licensable
|
|
80
|
+
by such Contributor that are necessarily infringed by their
|
|
81
|
+
Contribution(s) alone or by combination of their Contribution(s)
|
|
82
|
+
with the Work to which such Contribution(s) was submitted. If You
|
|
83
|
+
institute patent litigation against any entity (including a
|
|
84
|
+
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
|
85
|
+
or a Contribution incorporated within the Work constitutes direct
|
|
86
|
+
or contributory patent infringement, then any patent licenses
|
|
87
|
+
granted to You under this License for that Work shall terminate
|
|
88
|
+
as of the date such litigation is filed.
|
|
89
|
+
|
|
90
|
+
4. Redistribution. You may reproduce and distribute copies of the
|
|
91
|
+
Work or Derivative Works thereof in any medium, with or without
|
|
92
|
+
modifications, and in Source or Object form, provided that You
|
|
93
|
+
meet the following conditions:
|
|
94
|
+
|
|
95
|
+
(a) You must give any other recipients of the Work or
|
|
96
|
+
Derivative Works a copy of this License; and
|
|
97
|
+
|
|
98
|
+
(b) You must cause any modified files to carry prominent notices
|
|
99
|
+
stating that You changed the files; and
|
|
100
|
+
|
|
101
|
+
(c) You must retain, in the Source form of any Derivative Works
|
|
102
|
+
that You distribute, all copyright, patent, trademark, and
|
|
103
|
+
attribution notices from the Source form of the Work,
|
|
104
|
+
excluding those notices that do not pertain to any part of
|
|
105
|
+
the Derivative Works; and
|
|
106
|
+
|
|
107
|
+
(d) If the Work includes a "NOTICE" text file as part of its
|
|
108
|
+
distribution, then any Derivative Works that You distribute must
|
|
109
|
+
include a readable copy of the attribution notices contained
|
|
110
|
+
within such NOTICE file, excluding those notices that do not
|
|
111
|
+
pertain to any part of the Derivative Works, in at least one
|
|
112
|
+
of the following places: within a NOTICE text file distributed
|
|
113
|
+
as part of the Derivative Works; within the Source form or
|
|
114
|
+
documentation, if provided along with the Derivative Works; or,
|
|
115
|
+
within a display generated by the Derivative Works, if and
|
|
116
|
+
wherever such third-party notices normally appear. The contents
|
|
117
|
+
of the NOTICE file are for informational purposes only and
|
|
118
|
+
do not modify the License. You may add Your own attribution
|
|
119
|
+
notices within Derivative Works that You distribute, alongside
|
|
120
|
+
or as an addendum to the NOTICE text from the Work, provided
|
|
121
|
+
that such additional attribution notices cannot be construed
|
|
122
|
+
as modifying the License.
|
|
123
|
+
|
|
124
|
+
You may add Your own copyright statement to Your modifications and
|
|
125
|
+
may provide additional or different license terms and conditions
|
|
126
|
+
for use, reproduction, or distribution of Your modifications, or
|
|
127
|
+
for any such Derivative Works as a whole, provided Your use,
|
|
128
|
+
reproduction, and distribution of the Work otherwise complies with
|
|
129
|
+
the conditions stated in this License.
|
|
130
|
+
|
|
131
|
+
5. Submission of Contributions. Unless You explicitly state otherwise,
|
|
132
|
+
any Contribution intentionally submitted for inclusion in the Work
|
|
133
|
+
by You to the Licensor shall be under the terms and conditions of
|
|
134
|
+
this License, without any additional terms or conditions.
|
|
135
|
+
Notwithstanding the above, nothing herein shall supersede or modify
|
|
136
|
+
the terms of any separate license agreement you may have executed
|
|
137
|
+
with Licensor regarding such Contributions.
|
|
138
|
+
|
|
139
|
+
6. Trademarks. This License does not grant permission to use the trade
|
|
140
|
+
names, trademarks, service marks, or product names of the Licensor,
|
|
141
|
+
except as required for reasonable and customary use in describing the
|
|
142
|
+
origin of the Work and reproducing the content of the NOTICE file.
|
|
143
|
+
|
|
144
|
+
7. Disclaimer of Warranty. Unless required by applicable law or
|
|
145
|
+
agreed to in writing, Licensor provides the Work (and each
|
|
146
|
+
Contributor provides its Contributions) on an "AS IS" BASIS,
|
|
147
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
|
148
|
+
implied, including, without limitation, any warranties or conditions
|
|
149
|
+
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
|
150
|
+
PARTICULAR PURPOSE. You are solely responsible for determining the
|
|
151
|
+
appropriateness of using or redistributing the Work and assume any
|
|
152
|
+
risks associated with Your exercise of permissions under this License.
|
|
153
|
+
|
|
154
|
+
8. Limitation of Liability. In no event and under no legal theory,
|
|
155
|
+
whether in tort (including negligence), contract, or otherwise,
|
|
156
|
+
unless required by applicable law (such as deliberate and grossly
|
|
157
|
+
negligent acts) or agreed to in writing, shall any Contributor be
|
|
158
|
+
liable to You for damages, including any direct, indirect, special,
|
|
159
|
+
incidental, or consequential damages of any character arising as a
|
|
160
|
+
result of this License or out of the use or inability to use the
|
|
161
|
+
Work (including but not limited to damages for loss of goodwill,
|
|
162
|
+
work stoppage, computer failure or malfunction, or any and all
|
|
163
|
+
other commercial damages or losses), even if such Contributor
|
|
164
|
+
has been advised of the possibility of such damages.
|
|
165
|
+
|
|
166
|
+
9. Accepting Warranty or Additional Liability. While redistributing
|
|
167
|
+
the Work or Derivative Works thereof, You may choose to offer,
|
|
168
|
+
and charge a fee for, acceptance of support, warranty, indemnity,
|
|
169
|
+
or other liability obligations and/or rights consistent with this
|
|
170
|
+
License. However, in accepting such obligations, You may act only
|
|
171
|
+
on Your own behalf and on Your sole responsibility, not on behalf
|
|
172
|
+
of any other Contributor, and only if You agree to indemnify,
|
|
173
|
+
defend, and hold each Contributor harmless for any liability
|
|
174
|
+
incurred by, or claims asserted against, such Contributor by reason
|
|
175
|
+
of your accepting any such warranty or additional liability.
|
|
176
|
+
|
|
177
|
+
END OF TERMS AND CONDITIONS
|
|
178
|
+
|
|
179
|
+
APPENDIX: How to apply the Apache License to your work.
|
|
180
|
+
|
|
181
|
+
To apply the Apache License to your work, attach the following
|
|
182
|
+
boilerplate notice, with the fields enclosed by brackets "[]"
|
|
183
|
+
replaced with your own identifying information. (Don't include
|
|
184
|
+
the brackets!) The text should be enclosed in the appropriate
|
|
185
|
+
comment syntax for the file format. We also recommend that a
|
|
186
|
+
file or class name and description of purpose be included on the
|
|
187
|
+
same "printed page" as the copyright notice for easier
|
|
188
|
+
identification within third-party archives.
|
|
189
|
+
|
|
190
|
+
Copyright [yyyy] [name of copyright owner]
|
|
191
|
+
|
|
192
|
+
Licensed under the Apache License, Version 2.0 (the "License");
|
|
193
|
+
you may not use this file except in compliance with the License.
|
|
194
|
+
You may obtain a copy of the License at
|
|
195
|
+
|
|
196
|
+
http://www.apache.org/licenses/LICENSE-2.0
|
|
197
|
+
|
|
198
|
+
Unless required by applicable law or agreed to in writing, software
|
|
199
|
+
distributed under the License is distributed on an "AS IS" BASIS,
|
|
200
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
201
|
+
See the License for the specific language governing permissions and
|
|
202
|
+
limitations under the License.
|
|
@@ -0,0 +1,158 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: synapsek-claims
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: SVC — Synapsek Verified Claims: neutral, cryptographic verification of measured claims about AI artifacts and pipelines
|
|
5
|
+
Author: Synapsek Labs
|
|
6
|
+
License: Apache-2.0
|
|
7
|
+
Keywords: ai,attestation,verification,reproducibility,llm,supply-chain
|
|
8
|
+
Classifier: License :: OSI Approved :: Apache Software License
|
|
9
|
+
Classifier: Programming Language :: Python :: 3
|
|
10
|
+
Classifier: Topic :: Security
|
|
11
|
+
Requires-Python: >=3.10
|
|
12
|
+
Description-Content-Type: text/markdown
|
|
13
|
+
License-File: LICENSE
|
|
14
|
+
Requires-Dist: cryptography>=41
|
|
15
|
+
Provides-Extra: test
|
|
16
|
+
Requires-Dist: pytest>=7; extra == "test"
|
|
17
|
+
Dynamic: license-file
|
|
18
|
+
|
|
19
|
+
# synapsek-claims (svc)
|
|
20
|
+
|
|
21
|
+
**Verified claims for AI artifacts.** A framework-agnostic, cryptographically
|
|
22
|
+
verifiable layer that signs *measured quality claims* about any AI artifact or
|
|
23
|
+
pipeline — and lets a buyer **re-derive them with one command**.
|
|
24
|
+
|
|
25
|
+
You have seen vendor-reported benchmark numbers. You may have re-run one and
|
|
26
|
+
found it did not come out the same. `svc` is the missing referee:
|
|
27
|
+
|
|
28
|
+
- the issuer signs a claim document: measured values, the exact evidence
|
|
29
|
+
traces, the exact artifacts and code used, the pinned environment;
|
|
30
|
+
- the buyer runs **one command** that (1) checks the signature against a
|
|
31
|
+
trusted key, (2) re-hashes every pinned file, (3) re-checks every evidence
|
|
32
|
+
trace (sha256 + Merkle), (4) **recomputes every claimed number from the
|
|
33
|
+
evidence**, optionally **re-executes the pinned evaluation**, and (5)
|
|
34
|
+
re-derives every pass/fail status from the documented thresholds.
|
|
35
|
+
|
|
36
|
+
If the issuer claimed more than the numbers support, verification **fails**.
|
|
37
|
+
That last property — anti-overclaiming by construction — is what makes this
|
|
38
|
+
different from a signed eval dashboard.
|
|
39
|
+
|
|
40
|
+
```
|
|
41
|
+
$ svc verify claim.json --trusted-pubkey issuer.pub --base-dir . --rerun
|
|
42
|
+
=== SVC VERIFY synapsek.claim/knowledge-encoder-dedup/v3b-seed1 ===
|
|
43
|
+
[PASS] L1 authenticity: valid
|
|
44
|
+
[PASS] L2 integrity: 10 files match
|
|
45
|
+
[PASS] L3 evidence: 2 traces match
|
|
46
|
+
[PASS] L4 soundness: 1/1 values recomputed
|
|
47
|
+
[PASS] L4b rerun: 1 claims re-executed
|
|
48
|
+
[PASS] L5 claims: statuses + recommendations re-derived
|
|
49
|
+
claim dedup_spearman: value=0.4172424238214698 recomputed=0.4172424238214698 status=proven [OK]
|
|
50
|
+
RESULT: PASS — all layers verified; claims re-derived
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
## What a claim is
|
|
54
|
+
|
|
55
|
+
A claim is a JSON document:
|
|
56
|
+
|
|
57
|
+
- **subject** — what is being claimed about: files (hashed) + evidence traces
|
|
58
|
+
(hashed + Merkle-rooted);
|
|
59
|
+
- **pinned** — the environment, dataset, and config the measurement used;
|
|
60
|
+
- **claims** — each has a metric, a measured `value`, a documented
|
|
61
|
+
`threshold`, a `status` (`proven` / `not_proven`), and a `derivation`
|
|
62
|
+
(a rule that tells the buyer how to recompute the value from the evidence);
|
|
63
|
+
- **recommendation** — which labels are recommended (only the proven ones may
|
|
64
|
+
be) and which are explicitly not;
|
|
65
|
+
- **disclosures** — honest limitations;
|
|
66
|
+
- **signature** — Ed25519 over the canonical form of everything above.
|
|
67
|
+
|
|
68
|
+
Derivation rules are a tiny grammar, **no arbitrary code in rules**:
|
|
69
|
+
|
|
70
|
+
| rule | meaning |
|
|
71
|
+
|---|---|
|
|
72
|
+
| `field:<name>\|mean` / `field:<name>\|last` | aggregate a field over an evidence trace |
|
|
73
|
+
| `file:<path>#<field>` | read a field from a pinned JSON file |
|
|
74
|
+
| `script:<path>#<id>` | run `python3 <path>` in the claim dir; read `id` from its final JSON stdout |
|
|
75
|
+
| `attested:<json.path>` | read a value from a pinned attestation document |
|
|
76
|
+
|
|
77
|
+
`script:` is the only rule that executes code — it is the issuer's pinned
|
|
78
|
+
evaluation script, fully visible to the buyer, and it only runs if the buyer
|
|
79
|
+
passes `--rerun` (or explicitly accepts it). The buyer reads the script before
|
|
80
|
+
it can ever run.
|
|
81
|
+
|
|
82
|
+
## Quickstart
|
|
83
|
+
|
|
84
|
+
```bash
|
|
85
|
+
pip install synapsek-claims # or: pip install -e .
|
|
86
|
+
|
|
87
|
+
# buyer side
|
|
88
|
+
svc verify examples/pipeline-dedup/claim.json \
|
|
89
|
+
--trusted-pubkey examples/trusted-issuer.pub \
|
|
90
|
+
--base-dir examples/pipeline-dedup \
|
|
91
|
+
--rerun # also re-execute the pinned eval (CPU)
|
|
92
|
+
|
|
93
|
+
# issuer side
|
|
94
|
+
svc keygen --out-dir ./keys # new attestation authority
|
|
95
|
+
svc attest skeleton.json --key keys/issuer_ed25519 --base-dir ./claimdir
|
|
96
|
+
svc registry add registry/registry.json --claim claimdir/claim.json --issuer-id me
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
Exit code 0 = PASS, 1 = FAIL. `--json` gives a machine-readable report.
|
|
100
|
+
|
|
101
|
+
## The two working examples
|
|
102
|
+
|
|
103
|
+
Both are real, measured, and verify PASS as shipped:
|
|
104
|
+
|
|
105
|
+
1. **`examples/pipeline-dedup`** — a knowledge encoder (2-layer transformer +
|
|
106
|
+
Perceiver resampler, K=128 latents). Claims Spearman 0.4172 between latent
|
|
107
|
+
similarity and token Jaccard over a pinned 100-doc holdout (4950 pairs,
|
|
108
|
+
full pair-level evidence trace). `--rerun` re-executes the pinned eval on
|
|
109
|
+
CPU and reproduces the value **bit-exactly**.
|
|
110
|
+
2. **`examples/cartridge-v11`** — the Synapsek knowledge cartridge v1.1,
|
|
111
|
+
wrapped through the same pipeline as its full attestation. 13 claims: the
|
|
112
|
+
4 v0 gist dimensions recorded as **not_proven** (below pre-registered bars —
|
|
113
|
+
an honest claim is allowed to say "no"), and 9 QA-fidelity numbers
|
|
114
|
+
re-derived from the pinned per-item traces, including cross-family results
|
|
115
|
+
(a 7B model decoding latents produced by a small encoder).
|
|
116
|
+
|
|
117
|
+
## What this is NOT
|
|
118
|
+
|
|
119
|
+
- **Not a model-signing tool.** OpenSSF/Sigstore-style integrity signing is a
|
|
120
|
+
commodity and a good one; `svc` is complementary. It signs *quality claims*
|
|
121
|
+
and proves they re-derive, not just that the bytes did not change.
|
|
122
|
+
- **Not a benchmarking service.** We pin *your* evaluation. `svc` cannot make
|
|
123
|
+
a bad number look good, and it is structurally hostile to one.
|
|
124
|
+
- **Not a quality certifier.** A `proven` claim means: this measured value is
|
|
125
|
+
reproducible from the pinned evidence at/above the documented threshold.
|
|
126
|
+
The threshold is the issuer's documented commitment, and the buyer is free
|
|
127
|
+
to think it is low.
|
|
128
|
+
|
|
129
|
+
## Security model
|
|
130
|
+
|
|
131
|
+
- Ed25519 over deterministic canonical bytes (sorted keys, compact JSON).
|
|
132
|
+
- Only the **public** key ships with claims; the private key stays with the
|
|
133
|
+
issuer. Buyers verify against a trusted public key held out-of-band;
|
|
134
|
+
`issuer.public_key` must equal the trusted key (no rogue self-issuance).
|
|
135
|
+
- The registry is a trust anchor: claim id → issuer key → claim hash.
|
|
136
|
+
- Every layer is independently checkable; the report says which layer failed
|
|
137
|
+
and why.
|
|
138
|
+
|
|
139
|
+
## Tests
|
|
140
|
+
|
|
141
|
+
```bash
|
|
142
|
+
python3 -m pytest tests/ -q # 16 tests: roundtrip, tamper L1/L2/L3/L4b/L5, registry
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
## Layout
|
|
146
|
+
|
|
147
|
+
```
|
|
148
|
+
src/svc/ canonical, crypto, schema, rules, rerun, claim, verify, registry, cli
|
|
149
|
+
tests/ 16-test suite
|
|
150
|
+
examples/pipeline-dedup/ real claim, rerunnable on CPU (artifact/encoder.pt is a release asset)
|
|
151
|
+
examples/cartridge-v11/ real claim wrapping the v1.1 attestation + QA traces
|
|
152
|
+
docs/DESIGN.md schema, threat model, design decisions
|
|
153
|
+
RELEASE-CHECKLIST.md 10-minute publish handoff
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
## License
|
|
157
|
+
|
|
158
|
+
Apache-2.0. See LICENSE.
|
|
@@ -0,0 +1,140 @@
|
|
|
1
|
+
# synapsek-claims (svc)
|
|
2
|
+
|
|
3
|
+
**Verified claims for AI artifacts.** A framework-agnostic, cryptographically
|
|
4
|
+
verifiable layer that signs *measured quality claims* about any AI artifact or
|
|
5
|
+
pipeline — and lets a buyer **re-derive them with one command**.
|
|
6
|
+
|
|
7
|
+
You have seen vendor-reported benchmark numbers. You may have re-run one and
|
|
8
|
+
found it did not come out the same. `svc` is the missing referee:
|
|
9
|
+
|
|
10
|
+
- the issuer signs a claim document: measured values, the exact evidence
|
|
11
|
+
traces, the exact artifacts and code used, the pinned environment;
|
|
12
|
+
- the buyer runs **one command** that (1) checks the signature against a
|
|
13
|
+
trusted key, (2) re-hashes every pinned file, (3) re-checks every evidence
|
|
14
|
+
trace (sha256 + Merkle), (4) **recomputes every claimed number from the
|
|
15
|
+
evidence**, optionally **re-executes the pinned evaluation**, and (5)
|
|
16
|
+
re-derives every pass/fail status from the documented thresholds.
|
|
17
|
+
|
|
18
|
+
If the issuer claimed more than the numbers support, verification **fails**.
|
|
19
|
+
That last property — anti-overclaiming by construction — is what makes this
|
|
20
|
+
different from a signed eval dashboard.
|
|
21
|
+
|
|
22
|
+
```
|
|
23
|
+
$ svc verify claim.json --trusted-pubkey issuer.pub --base-dir . --rerun
|
|
24
|
+
=== SVC VERIFY synapsek.claim/knowledge-encoder-dedup/v3b-seed1 ===
|
|
25
|
+
[PASS] L1 authenticity: valid
|
|
26
|
+
[PASS] L2 integrity: 10 files match
|
|
27
|
+
[PASS] L3 evidence: 2 traces match
|
|
28
|
+
[PASS] L4 soundness: 1/1 values recomputed
|
|
29
|
+
[PASS] L4b rerun: 1 claims re-executed
|
|
30
|
+
[PASS] L5 claims: statuses + recommendations re-derived
|
|
31
|
+
claim dedup_spearman: value=0.4172424238214698 recomputed=0.4172424238214698 status=proven [OK]
|
|
32
|
+
RESULT: PASS — all layers verified; claims re-derived
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
## What a claim is
|
|
36
|
+
|
|
37
|
+
A claim is a JSON document:
|
|
38
|
+
|
|
39
|
+
- **subject** — what is being claimed about: files (hashed) + evidence traces
|
|
40
|
+
(hashed + Merkle-rooted);
|
|
41
|
+
- **pinned** — the environment, dataset, and config the measurement used;
|
|
42
|
+
- **claims** — each has a metric, a measured `value`, a documented
|
|
43
|
+
`threshold`, a `status` (`proven` / `not_proven`), and a `derivation`
|
|
44
|
+
(a rule that tells the buyer how to recompute the value from the evidence);
|
|
45
|
+
- **recommendation** — which labels are recommended (only the proven ones may
|
|
46
|
+
be) and which are explicitly not;
|
|
47
|
+
- **disclosures** — honest limitations;
|
|
48
|
+
- **signature** — Ed25519 over the canonical form of everything above.
|
|
49
|
+
|
|
50
|
+
Derivation rules are a tiny grammar, **no arbitrary code in rules**:
|
|
51
|
+
|
|
52
|
+
| rule | meaning |
|
|
53
|
+
|---|---|
|
|
54
|
+
| `field:<name>\|mean` / `field:<name>\|last` | aggregate a field over an evidence trace |
|
|
55
|
+
| `file:<path>#<field>` | read a field from a pinned JSON file |
|
|
56
|
+
| `script:<path>#<id>` | run `python3 <path>` in the claim dir; read `id` from its final JSON stdout |
|
|
57
|
+
| `attested:<json.path>` | read a value from a pinned attestation document |
|
|
58
|
+
|
|
59
|
+
`script:` is the only rule that executes code — it is the issuer's pinned
|
|
60
|
+
evaluation script, fully visible to the buyer, and it only runs if the buyer
|
|
61
|
+
passes `--rerun` (or explicitly accepts it). The buyer reads the script before
|
|
62
|
+
it can ever run.
|
|
63
|
+
|
|
64
|
+
## Quickstart
|
|
65
|
+
|
|
66
|
+
```bash
|
|
67
|
+
pip install synapsek-claims # or: pip install -e .
|
|
68
|
+
|
|
69
|
+
# buyer side
|
|
70
|
+
svc verify examples/pipeline-dedup/claim.json \
|
|
71
|
+
--trusted-pubkey examples/trusted-issuer.pub \
|
|
72
|
+
--base-dir examples/pipeline-dedup \
|
|
73
|
+
--rerun # also re-execute the pinned eval (CPU)
|
|
74
|
+
|
|
75
|
+
# issuer side
|
|
76
|
+
svc keygen --out-dir ./keys # new attestation authority
|
|
77
|
+
svc attest skeleton.json --key keys/issuer_ed25519 --base-dir ./claimdir
|
|
78
|
+
svc registry add registry/registry.json --claim claimdir/claim.json --issuer-id me
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
Exit code 0 = PASS, 1 = FAIL. `--json` gives a machine-readable report.
|
|
82
|
+
|
|
83
|
+
## The two working examples
|
|
84
|
+
|
|
85
|
+
Both are real, measured, and verify PASS as shipped:
|
|
86
|
+
|
|
87
|
+
1. **`examples/pipeline-dedup`** — a knowledge encoder (2-layer transformer +
|
|
88
|
+
Perceiver resampler, K=128 latents). Claims Spearman 0.4172 between latent
|
|
89
|
+
similarity and token Jaccard over a pinned 100-doc holdout (4950 pairs,
|
|
90
|
+
full pair-level evidence trace). `--rerun` re-executes the pinned eval on
|
|
91
|
+
CPU and reproduces the value **bit-exactly**.
|
|
92
|
+
2. **`examples/cartridge-v11`** — the Synapsek knowledge cartridge v1.1,
|
|
93
|
+
wrapped through the same pipeline as its full attestation. 13 claims: the
|
|
94
|
+
4 v0 gist dimensions recorded as **not_proven** (below pre-registered bars —
|
|
95
|
+
an honest claim is allowed to say "no"), and 9 QA-fidelity numbers
|
|
96
|
+
re-derived from the pinned per-item traces, including cross-family results
|
|
97
|
+
(a 7B model decoding latents produced by a small encoder).
|
|
98
|
+
|
|
99
|
+
## What this is NOT
|
|
100
|
+
|
|
101
|
+
- **Not a model-signing tool.** OpenSSF/Sigstore-style integrity signing is a
|
|
102
|
+
commodity and a good one; `svc` is complementary. It signs *quality claims*
|
|
103
|
+
and proves they re-derive, not just that the bytes did not change.
|
|
104
|
+
- **Not a benchmarking service.** We pin *your* evaluation. `svc` cannot make
|
|
105
|
+
a bad number look good, and it is structurally hostile to one.
|
|
106
|
+
- **Not a quality certifier.** A `proven` claim means: this measured value is
|
|
107
|
+
reproducible from the pinned evidence at/above the documented threshold.
|
|
108
|
+
The threshold is the issuer's documented commitment, and the buyer is free
|
|
109
|
+
to think it is low.
|
|
110
|
+
|
|
111
|
+
## Security model
|
|
112
|
+
|
|
113
|
+
- Ed25519 over deterministic canonical bytes (sorted keys, compact JSON).
|
|
114
|
+
- Only the **public** key ships with claims; the private key stays with the
|
|
115
|
+
issuer. Buyers verify against a trusted public key held out-of-band;
|
|
116
|
+
`issuer.public_key` must equal the trusted key (no rogue self-issuance).
|
|
117
|
+
- The registry is a trust anchor: claim id → issuer key → claim hash.
|
|
118
|
+
- Every layer is independently checkable; the report says which layer failed
|
|
119
|
+
and why.
|
|
120
|
+
|
|
121
|
+
## Tests
|
|
122
|
+
|
|
123
|
+
```bash
|
|
124
|
+
python3 -m pytest tests/ -q # 16 tests: roundtrip, tamper L1/L2/L3/L4b/L5, registry
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
## Layout
|
|
128
|
+
|
|
129
|
+
```
|
|
130
|
+
src/svc/ canonical, crypto, schema, rules, rerun, claim, verify, registry, cli
|
|
131
|
+
tests/ 16-test suite
|
|
132
|
+
examples/pipeline-dedup/ real claim, rerunnable on CPU (artifact/encoder.pt is a release asset)
|
|
133
|
+
examples/cartridge-v11/ real claim wrapping the v1.1 attestation + QA traces
|
|
134
|
+
docs/DESIGN.md schema, threat model, design decisions
|
|
135
|
+
RELEASE-CHECKLIST.md 10-minute publish handoff
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
## License
|
|
139
|
+
|
|
140
|
+
Apache-2.0. See LICENSE.
|
|
@@ -0,0 +1,65 @@
|
|
|
1
|
+
# Release checklist — synapsek-claims v0.1.0
|
|
2
|
+
|
|
3
|
+
Prepared 2026-10-17. All build/verify steps below were already executed on
|
|
4
|
+
Cherry; the remaining ~10 minutes are YOUR calls (org, account, publish).
|
|
5
|
+
|
|
6
|
+
## Already done (on Cherry, /root/synapsek-verified-claims)
|
|
7
|
+
|
|
8
|
+
- [x] package builds: `pip install -e .` clean, `python3 -m compileall` clean
|
|
9
|
+
- [x] 16/16 tests pass (roundtrip, tamper L1/L2/L3/L4b/L5, registry, canonical)
|
|
10
|
+
- [x] example 1 sealed + verified PASS (incl. bit-exact CPU rerun) + tamper-tested
|
|
11
|
+
- [x] example 2 sealed + verified PASS (13 claims: 4 not_proven + 9 proven) + tamper-tested
|
|
12
|
+
- [x] both claims registered in registry/registry.json under issuer
|
|
13
|
+
synapsek.compiler/v1 (public key MHYuRK-KEKtx3w-MQYnEJ7oiHWWJsJfP0WzNwn_Fi1M)
|
|
14
|
+
- [x] leak scan: no private key material anywhere in the tree (public key only)
|
|
15
|
+
|
|
16
|
+
## Your steps (~10 min)
|
|
17
|
+
|
|
18
|
+
1. **Review** (2 min): open `examples/*/claim.json` and the two
|
|
19
|
+
`svc verify ... --rerun` outputs above. The claim text is the public face
|
|
20
|
+
of the project — read the disclosures.
|
|
21
|
+
2. **GitHub** (3 min): create the org/repo `synapsek/synapsek-verified-claims`
|
|
22
|
+
(public). From the tree:
|
|
23
|
+
```
|
|
24
|
+
cd /root/synapsek-verified-claims
|
|
25
|
+
git init -b main && git add -A && git commit -m "synapsek-claims v0.1.0"
|
|
26
|
+
git remote add origin git@github.com:synapsek/synapsek-verified-claims.git
|
|
27
|
+
git push -u origin main
|
|
28
|
+
git tag v0.1.0 && git push origin v0.1.0
|
|
29
|
+
```
|
|
30
|
+
(`.gitignore` already excludes the 160MB `encoder.pt`, Python cache,
|
|
31
|
+
`.svc_*` rerun logs.)
|
|
32
|
+
3. **Release asset** (2 min): GitHub → repo → Releases → `v0.1.0` → attach
|
|
33
|
+
`examples/pipeline-dedup/artifact/encoder.pt` (160MB). Fresh clones get a
|
|
34
|
+
README pointer; verify of example 1 needs that asset placed back into
|
|
35
|
+
`artifact/encoder.pt`.
|
|
36
|
+
4. **PyPI** (3 min): create the account/owner (suggested: user `synapsek` or
|
|
37
|
+
`synapsek-claims`; project is `synapsek-claims`, already named in
|
|
38
|
+
pyproject). Then:
|
|
39
|
+
```
|
|
40
|
+
pip install build twine
|
|
41
|
+
python -m build # -> dist/synapsek_claims-0.1.0.*
|
|
42
|
+
twine upload dist/*
|
|
43
|
+
```
|
|
44
|
+
Verify: `pip install synapsek-claims --dry-run` from a clean venv, or
|
|
45
|
+
`pip download synapsek-claims`.
|
|
46
|
+
5. **Key ceremony (optional but recommended)**: publish the trusted issuer
|
|
47
|
+
public key + the exact `svc verify` commands as a repo page or
|
|
48
|
+
`TRUST.md`, so buyers know which key to trust and how to get it out-of-band
|
|
49
|
+
(this file itself is that artifact: `examples/trusted-issuer.pub`).
|
|
50
|
+
6. **Announce** (1 min): one-paragraph blurb — "verified claims for AI
|
|
51
|
+
artifacts; one command re-derives the numbers" + the two example links.
|
|
52
|
+
|
|
53
|
+
## Non-goals for v0.1.0 (do not promise in the announcement)
|
|
54
|
+
|
|
55
|
+
- No public registry *service* yet (the registry is a portable JSON file;
|
|
56
|
+
endpoint comes later).
|
|
57
|
+
- No GPU/CPU cross-device tolerance guarantees (example 1's rerun is
|
|
58
|
+
CPU-bit-exact on the pinned env; other envs may need a documented tolerance).
|
|
59
|
+
- No agent-execution attestation yet (roadmap, v0.2+).
|
|
60
|
+
|
|
61
|
+
## Rollback
|
|
62
|
+
|
|
63
|
+
Unpublish is one `twine`/GitHub action per surface; the tag is the anchor.
|
|
64
|
+
Nothing here touches the RunPod pod, the compiler key, or any model weights
|
|
65
|
+
beyond the one example artifact.
|
|
@@ -0,0 +1,98 @@
|
|
|
1
|
+
# SVC design notes (schema v0.1)
|
|
2
|
+
|
|
3
|
+
## Threat model
|
|
4
|
+
|
|
5
|
+
Attacker/defect goals, and the layer that stops each:
|
|
6
|
+
|
|
7
|
+
| threat | stopped by |
|
|
8
|
+
|---|---|
|
|
9
|
+
| claim document edited after signing (value, status, threshold) | L1 (signature) |
|
|
10
|
+
| issuer swap (rogue key self-attesting) | L1 (`issuer.public_key` must equal trusted key) |
|
|
11
|
+
| artifact/weights/code replaced in the claim dir | L2 (per-file sha256) |
|
|
12
|
+
| evidence trace edited, reordered, truncated | L3 (file sha256 + item count + Merkle root) |
|
|
13
|
+
| claimed value not actually supported by the evidence | L4 (value recomputed from evidence by the derivation rule) |
|
|
14
|
+
| claimed value not actually reproducible by re-executing the pinned eval | L4b (`--rerun`) |
|
|
15
|
+
| status overstated vs value+threshold | L5 (status re-derived) |
|
|
16
|
+
| recommendation includes a not-proven label / omits a not-proven label | L5 (recommendation set re-derived) |
|
|
17
|
+
| claim circulated that was never registered | `--registry` hash match |
|
|
18
|
+
|
|
19
|
+
What the layers do **not** stop: a buyer running a `script:` rule whose
|
|
20
|
+
contents they did not read; an evaluation script that measures something
|
|
21
|
+
different from what the metric name implies (mitigation: the script is pinned,
|
|
22
|
+
visible, and small by convention); a threshold chosen to be trivially low
|
|
23
|
+
(mitigation: it is disclosed and the buyer judges it).
|
|
24
|
+
|
|
25
|
+
## Canonical form
|
|
26
|
+
|
|
27
|
+
`canonical_bytes(obj) = json.dumps(obj, sort_keys=True, separators=(",", ":"),
|
|
28
|
+
ensure_ascii=False).encode("utf-8")`. The signature covers everything except
|
|
29
|
+
the `signature` field itself. Same byte format as the Synapsek v1.1
|
|
30
|
+
attestation engine — an SVC claim and a v1.1 attestation over the same JSON
|
|
31
|
+
object share canonical bytes, so the ecosystems interoperate.
|
|
32
|
+
|
|
33
|
+
## Evidence model
|
|
34
|
+
|
|
35
|
+
Evidence is JSONL (one JSON object per line; a single-line JSON file is a
|
|
36
|
+
valid 1-item trace). Each trace carries `sha256` (whole file), `n_items`, and
|
|
37
|
+
`merkle_root` (order-sensitive: leaf = sha256(canonical(item)), odd nodes
|
|
38
|
+
duplicated). Per-item traces (e.g. one row per doc pair, or one row per QA
|
|
39
|
+
item) give the buyer auditable granularity; the Merkle root makes even a
|
|
40
|
+
single swapped row visible.
|
|
41
|
+
|
|
42
|
+
## Claim lifecycle
|
|
43
|
+
|
|
44
|
+
1. **measure** — issuer runs the pinned eval on the pinned dataset/artifacts;
|
|
45
|
+
the eval writes evidence traces (per-item rows) and prints the summary.
|
|
46
|
+
2. **seal** — `svc attest` fills every file hash, trace hash, Merkle root and
|
|
47
|
+
item count from disk, re-derives every claim `status` from `value` +
|
|
48
|
+
`threshold` (the skeleton's status is overwritten — the issuer cannot
|
|
49
|
+
hand-pick), hashes the config, and signs.
|
|
50
|
+
3. **register** — `svc registry add` stores claim id → issuer key → claim
|
|
51
|
+
hash. The registry file is plain JSON and portable (same shape as the v11
|
|
52
|
+
attestation registry).
|
|
53
|
+
4. **verify** — `svc verify` (buyer). With `--registry`, the claim document is
|
|
54
|
+
first matched against the registry entry, then the 5 layers run. With
|
|
55
|
+
`--rerun`, pinned eval commands are re-executed and their outputs compared
|
|
56
|
+
within the claim's tolerance.
|
|
57
|
+
|
|
58
|
+
## Rule grammar (deliberately tiny)
|
|
59
|
+
|
|
60
|
+
No code evaluation inside rules. Four forms (see README table). `script:`
|
|
61
|
+
runs `python3 <path>` with cwd = claim dir and a timeout; its **final stdout
|
|
62
|
+
line must be one JSON object** and the named field must be numeric. This
|
|
63
|
+
keeps the recompute path auditable: the buyer sees exactly which file runs,
|
|
64
|
+
with which cwd, and what field is compared.
|
|
65
|
+
|
|
66
|
+
## Recommendation semantics
|
|
67
|
+
|
|
68
|
+
- `recommended_for` must **exactly** equal the sorted set of labels of
|
|
69
|
+
proven claims (over-recommending fails, under-recommending fails);
|
|
70
|
+
- `not_recommended_for` must be a **superset** of the labels of not-proven
|
|
71
|
+
claims (extra entries allowed — use them for honest scope limits).
|
|
72
|
+
|
|
73
|
+
This is the anti-overclaiming core: a vendor cannot ship a claim whose
|
|
74
|
+
numbers fail its own thresholds while still marketing the label.
|
|
75
|
+
|
|
76
|
+
## Scale honesty
|
|
77
|
+
|
|
78
|
+
Values and thresholds live in whatever scale the derivation rule produces.
|
|
79
|
+
If the evidence is 0-1 and the claim value is percent-based (or vice versa),
|
|
80
|
+
L4 fails — a good failure, because scale confusion is exactly how vendor
|
|
81
|
+
numbers get misread. Claim `unit` fields document the scale.
|
|
82
|
+
|
|
83
|
+
## Relationship to the v1.1 attestation
|
|
84
|
+
|
|
85
|
+
The cartridge attestation signs the whole cartridge qualification
|
|
86
|
+
(provenance, fidelity, 24 evidence traces). An SVC claim about the same
|
|
87
|
+
cartridge *wraps* it: the attestation document is a pinned file (role
|
|
88
|
+
`attestation`), `attested:` rules re-derive values from it, and QA-fidelity
|
|
89
|
+
claims re-derive directly from the same per-item traces. Chain of trust:
|
|
90
|
+
SVC claim → (hash) → v1.1 attestation → (hash) → weights.
|
|
91
|
+
|
|
92
|
+
## v0.2 ideas (not shipped)
|
|
93
|
+
|
|
94
|
+
- public registry endpoint + claim URI scheme
|
|
95
|
+
- verifiable agent-execution records (tool calls, approvals, audit log
|
|
96
|
+
entries as evidence traces)
|
|
97
|
+
- batch verify (many claims, one report)
|
|
98
|
+
- tolerance policies for GPU/CPU drift
|