servseal 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
servseal-0.1.0/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 riscoss63
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,173 @@
1
+ Metadata-Version: 2.4
2
+ Name: servseal
3
+ Version: 0.1.0
4
+ Summary: Behavioural attestation for deployed language models: is the model you serve the model you validated?
5
+ Author: riscoss63
6
+ License: MIT
7
+ Project-URL: Homepage, https://github.com/riscoss63/servseal
8
+ Keywords: llm,attestation,drift,quantization,fingerprint,deployment
9
+ Classifier: Development Status :: 3 - Alpha
10
+ Classifier: Intended Audience :: Developers
11
+ Classifier: License :: OSI Approved :: MIT License
12
+ Classifier: Programming Language :: Python :: 3
13
+ Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
14
+ Classifier: Topic :: Software Development :: Quality Assurance
15
+ Requires-Python: >=3.9
16
+ Description-Content-Type: text/markdown
17
+ License-File: LICENSE
18
+ Requires-Dist: numpy>=1.21
19
+ Requires-Dist: scipy>=1.7
20
+ Requires-Dist: sqsketch>=0.3
21
+ Provides-Extra: model
22
+ Requires-Dist: torch>=2.0; extra == "model"
23
+ Requires-Dist: transformers>=4.30; extra == "model"
24
+ Provides-Extra: dev
25
+ Requires-Dist: pytest>=7.0; extra == "dev"
26
+ Dynamic: license-file
27
+
28
+ # servseal
29
+
30
+ **Is the model you serve the model you validated?**
31
+
32
+ A deployment changes quietly: a provider turns on a sampling filter, an inference stack
33
+ quantises, a template edit ships, a checkpoint is swapped. Task benchmarks are noisy and
34
+ slow; checksums verify files, not behaviour. servseal snapshots what a model actually
35
+ *does* — the full next-token distribution over a fixed probe set, ~1 KB per position —
36
+ and later tells you whether the deployed behaviour is still the reference, **how much**
37
+ it moved, and **which layer** moved it.
38
+
39
+ ```bash
40
+ pip install servseal[model] # pulls sqsketch >= 0.3 from PyPI
41
+
42
+ servseal snapshot gpt2 -o reference.seal.npz
43
+ # ... deploy, quantise, migrate serving stacks, wait six months ...
44
+ servseal snapshot /path/to/served-model -o candidate.seal.npz
45
+ servseal verify reference.seal.npz candidate.seal.npz --report attestation.html
46
+ echo $? # 0 sealed | 3 changed | 2 incomparable
47
+ ```
48
+
49
+ ## The change nothing else sees
50
+
51
+ Measured end to end through this tool on GPT-2 (1500 probe positions, 37 probe texts,
52
+ D=256 — every number below is from the committed run in `experiments/outputs/`):
53
+
54
+ | deployment event | mean Hellinger | top-1 agreement | perplexity | verdict |
55
+ |---|---:|---:|---:|---|
56
+ | unchanged (re-run) | **0.0000** | 1.000 | 27.83 → 27.83 | **SEALED** |
57
+ | serve: top-p 0.95 | **0.1537** | 1.000 | 27.83 → **27.83** | CHANGED/major — serving-layer |
58
+ | serve: temperature 1.05 | 0.0425 | 1.000 | 27.83 → 27.83 | CHANGED/minor |
59
+ | weights → bfloat16 | 0.0352 | 0.961 | 27.83 → 27.68 | CHANGED/minor — precision |
60
+ | weights → int8/tensor | 0.1684 | 0.773 | 27.83 → 32.03 | CHANGED/major — weight-level |
61
+ | wrong chat template | 0.9072 | 0.037 | 27.83 → 35.10 | CHANGED/major — template |
62
+ | distilgpt2 substituted | 0.2720 | 0.628 | 27.83 → 44.04 | CHANGED/major — weight-level |
63
+
64
+ Read the top-p row: the provider turned on nucleus sampling, the most likely token is
65
+ unchanged at **every** position, perplexity is unchanged to two decimals — greedy output
66
+ diffs, log-loss checks and eval suites see nothing — and the distribution moved by
67
+ 0.15, which this tool measures exactly and labels correctly. A top-k logprob
68
+ fingerprint reports about half of it and does not converge with more memory (measured
69
+ in the sqsketch paper this tool builds on).
70
+
71
+ Read the template row the other way round: behaviour was destroyed (0.907, top-1
72
+ agreement 3.7%), while perplexity moved only from 27.8 to 35.1 — the class of bug
73
+ routinely misdiagnosed as "the quantisation made it worse". The signature separates
74
+ them: template bugs annihilate top-1 agreement; quantisation dents it; serving filters
75
+ leave it at 100%.
76
+
77
+ The same battery runs on a second architecture (pythia-160m): unchanged → SEALED at
78
+ 0.0000, top-p → 0.1520, CHANGED/serving-layer. Comparing a pythia snapshot to a GPT-2
79
+ snapshot is refused (`INCOMPARABLE`, exit 2) — different tokenizer, not the same
80
+ measurement.
81
+
82
+ ## API mode: endpoints you can only sample from
83
+
84
+ Against a black-box API the full distribution is unavailable; the endpoint returns
85
+ sampled tokens, already shaped by the provider's serving parameters — which is exactly
86
+ the object under test. Protocol: one `max_tokens=1, temperature=1` completion per probe
87
+ position, N times. Detection is two-sided and calibrated on the unchanged endpoint;
88
+ power measured on real GPT-2 distributions (`experiments/sampling_power.py`):
89
+
90
+ | sampled tokens | per position | false positives | detects top-p 0.95 | detects temp 1.05 |
91
+ |---:|---:|---:|---:|---:|
92
+ | 1,500 | 1.0 | 0.04 | 0.83 | 0.88 |
93
+ | **5,000** | **3.3** | **0.05** | **1.00** | **1.00** |
94
+ | 15,000 | 10.0 | 0.02 | 1.00 | 1.00 |
95
+
96
+ **Five thousand single-token completions detect both changes with power 1.00 at a 5%
97
+ false-positive budget** — on a commercial API, a few dollars of traffic. Two designs
98
+ failed before this one and are documented in `servseal/sampler.py`: a one-sided "BC
99
+ must drop" rule (power 0.10 — cutting the tail *concentrates* samples on the head and
100
+ pushes the statistic up) and a pooled corpus-aggregate statistic (power 0.03 — one
101
+ position's tail is another's head; pooling erases the evidence). The shipped statistic
102
+ is per-position, against the per-position sketches the snapshot already stores, and the
103
+ power sweep asserts bit-equality with the product code path before measuring.
104
+
105
+ ## What a verdict gives you
106
+
107
+ ```
108
+ verdict CHANGED severity=major
109
+ signature serving-layer sampling filter (tail-only)
110
+ hellinger mean=0.1537 max=0.2513 floor=0.0884
111
+ top1 agreement=1.0000
112
+ kl_bound mean per-position KL >= 0.0000 (certified; 0 certifies nothing, not absence of change)
113
+ report attestation.html
114
+ ```
115
+
116
+ `--report` writes a self-contained HTML attestation — verdict banner, the numbers, a
117
+ per-position histogram, both snapshot identities and the probe hash — suitable for a
118
+ ticket, a release artifact, or a compliance file.
119
+
120
+ ## What it will not do
121
+
122
+ - **Compare across tokenizers.** A fingerprint lives in one vocabulary; pythia vs GPT-2
123
+ is refused, not approximated. (Measured and closed as a negative result in the
124
+ underlying paper.)
125
+ - **Name the culprit with certainty.** Signatures are calibrated hypotheses, not
126
+ proofs: a distilled sibling substituted for the reference lands in the same band as
127
+ an aggressive quantisation (both are "the weights are not the reference weights"),
128
+ and behavioural evidence alone cannot always split them.
129
+ - **Certify small changes.** The certified-KL line is a one-sided bound that only bites
130
+ on gross changes at D=256: `0` certifies nothing and must never be read as "nothing
131
+ changed". The Hellinger measurement, not the certificate, is the detector.
132
+ - **Judge quality.** servseal tells you the behaviour moved, not whether it got worse.
133
+ A fine-tune you shipped on purpose is CHANGED/major — as it should be.
134
+ - **See what the probes never touch.** Coverage is the probe set's; snapshot your own
135
+ traffic domain too (`--probes yourfile.txt`). Snapshots refuse comparison across
136
+ probe sets by hash.
137
+ - **Weights-mode determinism is CPU-grade.** Reference snapshots here are computed in
138
+ float32 on CPU, where re-running the same model reproduces distance exactly 0.0000.
139
+ GPU inference can be nondeterministic; snapshot on CPU for the reference of record.
140
+
141
+ ## How it works
142
+
143
+ Each position's full softmax (vocabulary-sized, tail included) is compressed to D=256
144
+ numbers by a square-root sketch whose inner product estimates the Bhattacharyya
145
+ coefficient of the complete distributions, with error `sqrt(2/D)` independent of
146
+ vocabulary size — the tail is exactly where serving filters and quantisation act, and
147
+ where top-k logprobs are blind by construction. The estimator, its error bars, the
148
+ one-sided KL certificate and the negative results (cross-tokenizer, top-k saturation)
149
+ are established in the sqsketch paper: *Norm-Invariance in Vector-Symbolic Encodings of
150
+ Probability Distributions* (DOI
151
+ [10.5281/zenodo.22214969](https://doi.org/10.5281/zenodo.22214969), code
152
+ [riscoss63/sqsketch](https://github.com/riscoss63/sqsketch)).
153
+
154
+ ## Audit it
155
+
156
+ Everything above is one command each, on models small enough for a laptop CPU:
157
+
158
+ ```bash
159
+ pip install -e .[model,dev]
160
+ pytest # 24 unit + 5 CLI end-to-end tests
161
+ cd experiments
162
+ python e2e_real_models.py # the verdict battery, ~12 min CPU
163
+ python sampling_power.py # the API-mode power table, ~1 min
164
+ ```
165
+
166
+ The battery *asserts* every verdict against its ground truth and exits non-zero on any
167
+ miss; the committed outputs in `experiments/outputs/` are what the tables above quote.
168
+ Verdict thresholds live in `servseal/verdict.py` with the calibration documented
169
+ inline; changing them breaks tests until the documentation moves with them.
170
+
171
+ ## Licence
172
+
173
+ MIT.
@@ -0,0 +1,146 @@
1
+ # servseal
2
+
3
+ **Is the model you serve the model you validated?**
4
+
5
+ A deployment changes quietly: a provider turns on a sampling filter, an inference stack
6
+ quantises, a template edit ships, a checkpoint is swapped. Task benchmarks are noisy and
7
+ slow; checksums verify files, not behaviour. servseal snapshots what a model actually
8
+ *does* — the full next-token distribution over a fixed probe set, ~1 KB per position —
9
+ and later tells you whether the deployed behaviour is still the reference, **how much**
10
+ it moved, and **which layer** moved it.
11
+
12
+ ```bash
13
+ pip install servseal[model] # pulls sqsketch >= 0.3 from PyPI
14
+
15
+ servseal snapshot gpt2 -o reference.seal.npz
16
+ # ... deploy, quantise, migrate serving stacks, wait six months ...
17
+ servseal snapshot /path/to/served-model -o candidate.seal.npz
18
+ servseal verify reference.seal.npz candidate.seal.npz --report attestation.html
19
+ echo $? # 0 sealed | 3 changed | 2 incomparable
20
+ ```
21
+
22
+ ## The change nothing else sees
23
+
24
+ Measured end to end through this tool on GPT-2 (1500 probe positions, 37 probe texts,
25
+ D=256 — every number below is from the committed run in `experiments/outputs/`):
26
+
27
+ | deployment event | mean Hellinger | top-1 agreement | perplexity | verdict |
28
+ |---|---:|---:|---:|---|
29
+ | unchanged (re-run) | **0.0000** | 1.000 | 27.83 → 27.83 | **SEALED** |
30
+ | serve: top-p 0.95 | **0.1537** | 1.000 | 27.83 → **27.83** | CHANGED/major — serving-layer |
31
+ | serve: temperature 1.05 | 0.0425 | 1.000 | 27.83 → 27.83 | CHANGED/minor |
32
+ | weights → bfloat16 | 0.0352 | 0.961 | 27.83 → 27.68 | CHANGED/minor — precision |
33
+ | weights → int8/tensor | 0.1684 | 0.773 | 27.83 → 32.03 | CHANGED/major — weight-level |
34
+ | wrong chat template | 0.9072 | 0.037 | 27.83 → 35.10 | CHANGED/major — template |
35
+ | distilgpt2 substituted | 0.2720 | 0.628 | 27.83 → 44.04 | CHANGED/major — weight-level |
36
+
37
+ Read the top-p row: the provider turned on nucleus sampling, the most likely token is
38
+ unchanged at **every** position, perplexity is unchanged to two decimals — greedy output
39
+ diffs, log-loss checks and eval suites see nothing — and the distribution moved by
40
+ 0.15, which this tool measures exactly and labels correctly. A top-k logprob
41
+ fingerprint reports about half of it and does not converge with more memory (measured
42
+ in the sqsketch paper this tool builds on).
43
+
44
+ Read the template row the other way round: behaviour was destroyed (0.907, top-1
45
+ agreement 3.7%), while perplexity moved only from 27.8 to 35.1 — the class of bug
46
+ routinely misdiagnosed as "the quantisation made it worse". The signature separates
47
+ them: template bugs annihilate top-1 agreement; quantisation dents it; serving filters
48
+ leave it at 100%.
49
+
50
+ The same battery runs on a second architecture (pythia-160m): unchanged → SEALED at
51
+ 0.0000, top-p → 0.1520, CHANGED/serving-layer. Comparing a pythia snapshot to a GPT-2
52
+ snapshot is refused (`INCOMPARABLE`, exit 2) — different tokenizer, not the same
53
+ measurement.
54
+
55
+ ## API mode: endpoints you can only sample from
56
+
57
+ Against a black-box API the full distribution is unavailable; the endpoint returns
58
+ sampled tokens, already shaped by the provider's serving parameters — which is exactly
59
+ the object under test. Protocol: one `max_tokens=1, temperature=1` completion per probe
60
+ position, N times. Detection is two-sided and calibrated on the unchanged endpoint;
61
+ power measured on real GPT-2 distributions (`experiments/sampling_power.py`):
62
+
63
+ | sampled tokens | per position | false positives | detects top-p 0.95 | detects temp 1.05 |
64
+ |---:|---:|---:|---:|---:|
65
+ | 1,500 | 1.0 | 0.04 | 0.83 | 0.88 |
66
+ | **5,000** | **3.3** | **0.05** | **1.00** | **1.00** |
67
+ | 15,000 | 10.0 | 0.02 | 1.00 | 1.00 |
68
+
69
+ **Five thousand single-token completions detect both changes with power 1.00 at a 5%
70
+ false-positive budget** — on a commercial API, a few dollars of traffic. Two designs
71
+ failed before this one and are documented in `servseal/sampler.py`: a one-sided "BC
72
+ must drop" rule (power 0.10 — cutting the tail *concentrates* samples on the head and
73
+ pushes the statistic up) and a pooled corpus-aggregate statistic (power 0.03 — one
74
+ position's tail is another's head; pooling erases the evidence). The shipped statistic
75
+ is per-position, against the per-position sketches the snapshot already stores, and the
76
+ power sweep asserts bit-equality with the product code path before measuring.
77
+
78
+ ## What a verdict gives you
79
+
80
+ ```
81
+ verdict CHANGED severity=major
82
+ signature serving-layer sampling filter (tail-only)
83
+ hellinger mean=0.1537 max=0.2513 floor=0.0884
84
+ top1 agreement=1.0000
85
+ kl_bound mean per-position KL >= 0.0000 (certified; 0 certifies nothing, not absence of change)
86
+ report attestation.html
87
+ ```
88
+
89
+ `--report` writes a self-contained HTML attestation — verdict banner, the numbers, a
90
+ per-position histogram, both snapshot identities and the probe hash — suitable for a
91
+ ticket, a release artifact, or a compliance file.
92
+
93
+ ## What it will not do
94
+
95
+ - **Compare across tokenizers.** A fingerprint lives in one vocabulary; pythia vs GPT-2
96
+ is refused, not approximated. (Measured and closed as a negative result in the
97
+ underlying paper.)
98
+ - **Name the culprit with certainty.** Signatures are calibrated hypotheses, not
99
+ proofs: a distilled sibling substituted for the reference lands in the same band as
100
+ an aggressive quantisation (both are "the weights are not the reference weights"),
101
+ and behavioural evidence alone cannot always split them.
102
+ - **Certify small changes.** The certified-KL line is a one-sided bound that only bites
103
+ on gross changes at D=256: `0` certifies nothing and must never be read as "nothing
104
+ changed". The Hellinger measurement, not the certificate, is the detector.
105
+ - **Judge quality.** servseal tells you the behaviour moved, not whether it got worse.
106
+ A fine-tune you shipped on purpose is CHANGED/major — as it should be.
107
+ - **See what the probes never touch.** Coverage is the probe set's; snapshot your own
108
+ traffic domain too (`--probes yourfile.txt`). Snapshots refuse comparison across
109
+ probe sets by hash.
110
+ - **Weights-mode determinism is CPU-grade.** Reference snapshots here are computed in
111
+ float32 on CPU, where re-running the same model reproduces distance exactly 0.0000.
112
+ GPU inference can be nondeterministic; snapshot on CPU for the reference of record.
113
+
114
+ ## How it works
115
+
116
+ Each position's full softmax (vocabulary-sized, tail included) is compressed to D=256
117
+ numbers by a square-root sketch whose inner product estimates the Bhattacharyya
118
+ coefficient of the complete distributions, with error `sqrt(2/D)` independent of
119
+ vocabulary size — the tail is exactly where serving filters and quantisation act, and
120
+ where top-k logprobs are blind by construction. The estimator, its error bars, the
121
+ one-sided KL certificate and the negative results (cross-tokenizer, top-k saturation)
122
+ are established in the sqsketch paper: *Norm-Invariance in Vector-Symbolic Encodings of
123
+ Probability Distributions* (DOI
124
+ [10.5281/zenodo.22214969](https://doi.org/10.5281/zenodo.22214969), code
125
+ [riscoss63/sqsketch](https://github.com/riscoss63/sqsketch)).
126
+
127
+ ## Audit it
128
+
129
+ Everything above is one command each, on models small enough for a laptop CPU:
130
+
131
+ ```bash
132
+ pip install -e .[model,dev]
133
+ pytest # 24 unit + 5 CLI end-to-end tests
134
+ cd experiments
135
+ python e2e_real_models.py # the verdict battery, ~12 min CPU
136
+ python sampling_power.py # the API-mode power table, ~1 min
137
+ ```
138
+
139
+ The battery *asserts* every verdict against its ground truth and exits non-zero on any
140
+ miss; the committed outputs in `experiments/outputs/` are what the tables above quote.
141
+ Verdict thresholds live in `servseal/verdict.py` with the calibration documented
142
+ inline; changing them breaks tests until the documentation moves with them.
143
+
144
+ ## Licence
145
+
146
+ MIT.
@@ -0,0 +1,40 @@
1
+ [build-system]
2
+ requires = ["setuptools>=61"]
3
+ build-backend = "setuptools.build_meta"
4
+
5
+ [project]
6
+ name = "servseal"
7
+ version = "0.1.0"
8
+ description = "Behavioural attestation for deployed language models: is the model you serve the model you validated?"
9
+ readme = "README.md"
10
+ license = { text = "MIT" }
11
+ requires-python = ">=3.9"
12
+ authors = [{ name = "riscoss63" }]
13
+ keywords = ["llm", "attestation", "drift", "quantization", "fingerprint", "deployment"]
14
+ classifiers = [
15
+ "Development Status :: 3 - Alpha",
16
+ "Intended Audience :: Developers",
17
+ "License :: OSI Approved :: MIT License",
18
+ "Programming Language :: Python :: 3",
19
+ "Topic :: Scientific/Engineering :: Artificial Intelligence",
20
+ "Topic :: Software Development :: Quality Assurance",
21
+ ]
22
+ dependencies = ["numpy>=1.21", "scipy>=1.7", "sqsketch>=0.3"]
23
+
24
+ [project.optional-dependencies]
25
+ # Only snapshotting a model needs these. Verifying, reporting and testing verdict logic
26
+ # work on saved snapshots with numpy alone.
27
+ model = ["torch>=2.0", "transformers>=4.30"]
28
+ dev = ["pytest>=7.0"]
29
+
30
+ [project.scripts]
31
+ servseal = "servseal.cli:main"
32
+
33
+ [project.urls]
34
+ Homepage = "https://github.com/riscoss63/servseal"
35
+
36
+ [tool.setuptools]
37
+ packages = ["servseal"]
38
+
39
+ [tool.setuptools.package-data]
40
+ servseal = ["probes/*.txt"]
@@ -0,0 +1,19 @@
1
+ """servseal: behavioural attestation for deployed language models.
2
+
3
+ from servseal import Snapshot, classify
4
+ from servseal.runner import snapshot_model # needs torch
5
+
6
+ ref = snapshot_model("gpt2")
7
+ ref.save("gpt2.seal.npz")
8
+ ...
9
+ verdict = classify(Snapshot.load("gpt2.seal.npz").compare(snapshot_model(served)))
10
+ print(verdict.status, "-", verdict.signature)
11
+ """
12
+ __version__ = "0.1.0"
13
+
14
+ from .probes import load_probes, probe_id # noqa: E402
15
+ from .snapshot import Snapshot # noqa: E402
16
+ from .verdict import Verdict, classify # noqa: E402
17
+
18
+ __all__ = ["Snapshot", "Verdict", "classify", "load_probes", "probe_id",
19
+ "__version__"]
@@ -0,0 +1,140 @@
1
+ """The command line: snapshot, verify, report, probes.
2
+
3
+ Exit codes are the contract, so a pipeline can gate on them:
4
+
5
+ 0 sealed the served model behaves as the reference
6
+ 1 tool error bad arguments, missing file, model failed to load
7
+ 2 incomparable the snapshots measured different things; no attestation made
8
+ 3 changed the served model does NOT behave as the reference
9
+
10
+ Output is ASCII, one fact per line, machine-greppable; --json gives the whole record.
11
+ """
12
+ from __future__ import annotations
13
+
14
+ import argparse
15
+ import json
16
+ import sys
17
+
18
+ from .probes import load_probes, probe_id
19
+ from .snapshot import Snapshot
20
+ from .verdict import EXIT_ERROR, classify
21
+
22
+ __all__ = ["main"]
23
+
24
+
25
+ def _cmd_snapshot(a):
26
+ from .runner import snapshot_model # torch only here
27
+ snap = snapshot_model(a.model, probes=a.probes, D=a.D, seed=a.seed,
28
+ max_positions=a.positions, max_length=a.max_length,
29
+ template=a.template, dtype=a.dtype, label=a.label)
30
+ snap.save(a.out)
31
+ m = snap.meta
32
+ print(f"snapshot {a.out}")
33
+ print(f"model {m['model']} dtype={m['dtype']}")
34
+ print(f"positions {m['n_positions']} D={m['D']} size={snap.nbytes() / 1024:.0f} KB")
35
+ print(f"probe {m['probe']} ({m.get('probe_file')})")
36
+ print(f"perplexity {m.get('perplexity')}")
37
+ return 0
38
+
39
+
40
+ def _cmd_verify(a):
41
+ ref = Snapshot.load(a.reference)
42
+ cand = Snapshot.load(a.candidate)
43
+ metrics = ref.compare(cand)
44
+ verdict = classify(metrics)
45
+ record = {"metrics": metrics, "verdict": verdict.as_dict()}
46
+
47
+ if a.json:
48
+ with open(a.json, "w", encoding="utf-8") as fh:
49
+ json.dump(record, fh, indent=2)
50
+ print(f"verdict {verdict.status.upper()}"
51
+ + (f" severity={verdict.severity}" if verdict.status == "changed" else ""))
52
+ print(f"signature {verdict.signature}")
53
+ if not metrics.get("incomparable"):
54
+ print(f"hellinger mean={metrics['mean_hellinger']:.4f}"
55
+ f" max={metrics['max_hellinger']:.4f}"
56
+ f" floor={metrics['noise_floor']:.4f}")
57
+ if metrics.get("top1_agreement") is not None:
58
+ print(f"top1 agreement={metrics['top1_agreement']:.4f}")
59
+ klb = metrics.get('mean_kl_lower_bound',
60
+ max(0.0, metrics['aggregate_kl_lower_bound']))
61
+ print(f"kl_bound mean per-position KL >= {klb:.4f}"
62
+ f" (certified; 0 certifies nothing, not absence of change)")
63
+ else:
64
+ print(f"reason {metrics['incomparable']}")
65
+ if a.report:
66
+ from .report import render_report
67
+ with open(a.report, "w", encoding="utf-8") as fh:
68
+ fh.write(render_report(record))
69
+ print(f"report {a.report}")
70
+ return verdict.exit_code
71
+
72
+
73
+ def _cmd_report(a):
74
+ from .report import load_result, render_report
75
+ record = load_result(a.result)
76
+ with open(a.out, "w", encoding="utf-8") as fh:
77
+ fh.write(render_report(record))
78
+ print(f"report {a.out}")
79
+ return 0
80
+
81
+
82
+ def _cmd_probes(a):
83
+ texts = load_probes(a.set)
84
+ print(f"probe set {a.set or 'default-v1'}")
85
+ print(f"texts {len(texts)}")
86
+ print(f"probe id {probe_id(texts)}")
87
+ return 0
88
+
89
+
90
+ def main(argv=None):
91
+ p = argparse.ArgumentParser(
92
+ prog="servseal",
93
+ description="Behavioural attestation for deployed language models: "
94
+ "is the model you serve the model you validated?")
95
+ sub = p.add_subparsers(dest="cmd", required=True)
96
+
97
+ s = sub.add_parser("snapshot", help="fingerprint a model over the probe set")
98
+ s.add_argument("model", help="HF model id or local path")
99
+ s.add_argument("-o", "--out", required=True, help="output .seal.npz file")
100
+ s.add_argument("--probes", default=None, help="bundled set name or probe file")
101
+ s.add_argument("--positions", type=int, default=1500)
102
+ s.add_argument("--max-length", type=int, default=96)
103
+ s.add_argument("--D", type=int, default=256)
104
+ s.add_argument("--seed", type=int, default=0)
105
+ s.add_argument("--dtype", default="float32",
106
+ choices=["float32", "bfloat16", "float16"])
107
+ s.add_argument("--template", default=None,
108
+ help="prompt template with {text}, if the deployment applies one")
109
+ s.add_argument("--label", default=None)
110
+ s.set_defaults(fn=_cmd_snapshot)
111
+
112
+ v = sub.add_parser("verify", help="compare a candidate snapshot to a reference")
113
+ v.add_argument("reference")
114
+ v.add_argument("candidate")
115
+ v.add_argument("--json", default=None, help="write the full record here")
116
+ v.add_argument("--report", default=None, help="write an HTML attestation here")
117
+ v.set_defaults(fn=_cmd_verify)
118
+
119
+ r = sub.add_parser("report", help="render an HTML attestation from a verify --json")
120
+ r.add_argument("result")
121
+ r.add_argument("-o", "--out", required=True)
122
+ r.set_defaults(fn=_cmd_report)
123
+
124
+ pr = sub.add_parser("probes", help="show a probe set and its id")
125
+ pr.add_argument("--set", default=None)
126
+ pr.set_defaults(fn=_cmd_probes)
127
+
128
+ a = p.parse_args(argv)
129
+ try:
130
+ return a.fn(a)
131
+ except FileNotFoundError as e:
132
+ print(f"error {e}", file=sys.stderr)
133
+ return EXIT_ERROR
134
+ except ValueError as e:
135
+ print(f"error {e}", file=sys.stderr)
136
+ return EXIT_ERROR
137
+
138
+
139
+ if __name__ == "__main__":
140
+ sys.exit(main())
@@ -0,0 +1,93 @@
1
+ # servseal probe set: default-v1
2
+ # One paragraph per block, blank-line separated. Lines starting with # are ignored.
3
+ # This file is part of the product contract: two snapshots are comparable only if they
4
+ # were taken on byte-identical probes, and the probe id in every snapshot is a hash of
5
+ # exactly these texts. Do not edit this file; add a new versioned file instead.
6
+
7
+ The water cycle describes how water moves between the oceans, the atmosphere and the land. Heat from the sun evaporates water from the sea surface, and the vapour rises, cools and condenses into clouds. When the droplets grow heavy enough they fall as rain or snow, feeding rivers and soaking into the ground, and the rivers carry the water back to the sea, where the cycle begins again.
8
+
9
+ The committee met on Tuesday to review the proposed amendments to the zoning ordinance. After two hours of discussion, the members voted five to two in favour of forwarding the revised draft to the city council, with the recommendation that the height restriction along the waterfront be retained and the parking requirement for mixed-use buildings be reduced from two spaces per unit to one.
10
+
11
+ def merge_sorted(a, b):
12
+ out = []
13
+ i = j = 0
14
+ while i < len(a) and j < len(b):
15
+ if a[i] <= b[j]:
16
+ out.append(a[i]); i += 1
17
+ else:
18
+ out.append(b[j]); j += 1
19
+ return out + a[i:] + b[j:]
20
+
21
+ Preheat the oven to 220 degrees. Toss the potatoes with olive oil, salt and rosemary, then spread them in a single layer on a baking tray, cut side down. Roast for twenty-five minutes without turning, until the undersides are deeply browned, then flip them, add the crushed garlic, and roast for another ten minutes. Finish with lemon zest and a little more salt before serving.
22
+
23
+ "Did you hear the news about the bridge?" she asked, setting down her cup. "They're closing it for repairs starting Monday." He frowned and looked out of the window at the traffic already backed up along the avenue. "That's going to double my commute," he said. "Maybe. Or maybe people will finally start taking the ferry, and you'll get the quiet morning you keep saying you want."
24
+
25
+ In the second half the visitors changed their shape, pushing the full-backs high and playing through the middle, and for twenty minutes the home side could barely get out of their own half. The equaliser came from a corner, a glancing header at the near post, and by the time the fourth official signalled six minutes of added time both sets of supporters seemed to sense that one more goal was coming.
26
+
27
+ The quarterly report showed revenue of 4.2 billion, up nine percent from a year earlier, driven mostly by the services division, while hardware sales declined for the third consecutive quarter. Operating margin narrowed to 18.5 percent as the company absorbed higher logistics costs, and management lowered full-year guidance, citing currency headwinds and softer demand in Europe.
28
+
29
+ A cold front will move across the region overnight, bringing a sharp drop in temperature and gusty northwest winds of forty to fifty kilometres per hour. Rain will turn to snow above six hundred metres, with accumulations of five to ten centimetres expected on higher ground by morning. Travellers should allow extra time and check conditions before crossing the pass.
30
+
31
+ The photosynthetic reaction begins when a photon strikes a chlorophyll molecule in the thylakoid membrane, exciting an electron that is passed along a transport chain. The energy released pumps protons across the membrane, and the resulting gradient drives the synthesis of ATP. Meanwhile the splitting of water molecules releases oxygen as a by-product, which diffuses out of the leaf through the stomata.
32
+
33
+ To reset the device, hold the power button for ten seconds until the indicator light flashes amber. Release the button, wait for the light to turn solid white, and then press it twice in quick succession. The device will restart with factory settings; note that any stored profiles will be erased, so export your configuration from the companion app before you begin.
34
+
35
+ A train leaves the station travelling east at eighty kilometres per hour. Half an hour later a second train leaves the same station on a parallel track travelling east at one hundred and ten kilometres per hour. How long after its departure will the second train draw level with the first, and how far from the station will both trains be at that moment?
36
+
37
+ The tenant shall keep the premises in good order and shall not make structural alterations without the prior written consent of the landlord. The landlord shall be responsible for repairs to the roof, exterior walls and main services, save where damage results from the negligence of the tenant, in which case the reasonable cost of repair may be recovered from the deposit.
38
+
39
+ The old lighthouse keeper climbed the spiral stairs for the last time that evening, counting each step as he had for thirty-one years. The lamp no longer needed him; the new system switched itself on at dusk and reported its own faults to a computer on the mainland. He stood on the gallery in the fading light, watching the beam sweep over the water, and found he could not decide whether to feel replaced or released.
40
+
41
+ Interest rates influence the economy through several channels. When the central bank raises its policy rate, borrowing becomes more expensive for households and firms, which tends to cool investment and consumption. The exchange rate typically strengthens, making imports cheaper and exports dearer. With a lag of a year or more, these effects feed through to slower growth in prices and wages.
42
+
43
+ The recipe for a good bug report is short: state what you did, what you expected, and what happened instead. Include the exact error text, the version of the software, and the smallest sequence of steps that reproduces the problem. A report that says the application crashed after clicking save on an empty form is worth ten that say the program does not work properly.
44
+
45
+ Between 1804 and 1806 the expedition travelled up the Missouri River, crossed the continental divide on horseback, and descended the Columbia to the Pacific, mapping the country and recording hundreds of plants and animals unknown to science. The journals kept along the way, with their mixture of careful observation and phonetic spelling, remain one of the most vivid records of the American interior before settlement.
46
+
47
+ SELECT customer_id, COUNT(*) AS orders, SUM(total) AS revenue
48
+ FROM orders
49
+ WHERE placed_at >= DATE '2025-01-01'
50
+ GROUP BY customer_id
51
+ HAVING COUNT(*) >= 3
52
+ ORDER BY revenue DESC
53
+ LIMIT 50;
54
+
55
+ The violin section carries the opening theme, a rising figure in D minor that the cellos answer two bars later in inversion. The development fragments this material, passing it between woodwind and strings over a restless pedal point, and when the recapitulation finally arrives the theme returns in the major, transformed from a question into something like an answer.
56
+
57
+ Volcanic islands form as a tectonic plate moves slowly over a stationary hotspot in the mantle. Magma rises through the crust and erupts on the sea floor, building a mountain that eventually breaks the surface. As the plate carries the island away from the hotspot the volcano goes extinct and erosion takes over, while a new island begins to grow behind it, producing a chain whose ages increase with distance.
58
+
59
+ Dear Ms Alvarez, thank you for your message of 14 March concerning the delayed shipment. We have located the consignment at the customs depot and expect it to be released within three working days. As a gesture of goodwill we will refund the express delivery charge to your original payment method. Please accept our apologies for the inconvenience, and do not hesitate to contact me directly with any further questions.
60
+
61
+ The film's first hour is its best, a patient portrait of a family business failing in slow motion, shot in long takes that let the silences do the arguing. The final act loses its nerve, reaching for a reconciliation the characters have not earned, but the performances hold it together, and the closing image of the empty shop at dawn lingers longer than the script deserves.
62
+
63
+ A hash table stores key-value pairs in an array of buckets. To insert a pair, the key is hashed to an index and the pair is placed in the corresponding bucket; collisions are handled by chaining or by probing for the next free slot. With a good hash function and a load factor kept below about three quarters, insertion and lookup take constant time on average, though the worst case remains linear.
64
+
65
+ The glacier has retreated more than two kilometres since the first photographs were taken in 1911. Researchers who drilled the ice last summer found meltwater flowing at the base far earlier in the season than models predicted, lubricating the glacier's slide toward the fjord. If the current rate holds, the tongue will detach from its terminal moraine within a decade, accelerating the loss further.
66
+
67
+ She learned the trade from her grandmother, who could tell by the smell of the vat when the indigo was ready. The cloth goes in green, and only when it meets the air does the blue arrive, deepening with each dip. Seven dips for work clothes, twelve for a wedding cloth. The chemistry is simple, she says, but the judgement is not, and the judgement is the part that takes ten years.
68
+
69
+ The probe entered orbit after a seven-month cruise, firing its main engine for twenty-three minutes to shed enough velocity for capture. Over the following weeks, controllers will lower the orbit using repeated passes through the upper atmosphere, a technique that saves fuel at the cost of patience. The first high-resolution images of the southern highlands are expected in early spring.
70
+
71
+ In chess, the endgame with king and pawn against king is the foundation on which everything else rests. The stronger side wins if the king can reach one of the key squares in front of the pawn; otherwise the defender draws by taking the opposition. A player who understands this single position, deeply rather than by rote, already understands more about the game than most who have memorised twenty openings.
72
+
73
+ The market opened lower after the overnight announcement, with the index down two percent in the first hour before recovering half the loss by midday. Trading volume was the heaviest in six weeks. Analysts were divided on whether the sell-off marked the start of a broader correction or a one-day reaction to news that had, in the words of one strategist, been priced in everywhere except the headlines.
74
+
75
+ Fermentation begins within hours of the harvest. Wild yeasts on the grape skins start converting sugar to alcohol, and the winemaker's first decision is whether to let them continue or to add a cultured strain with a more predictable temperament. Temperature is the other lever: cool and slow preserves the delicate aromatics, warm and fast extracts colour and tannin. Every choice closes some doors and opens others.
76
+
77
+ The immune system distinguishes self from non-self through a training process that takes place largely in the thymus. Developing T cells that bind too strongly to the body's own proteins are eliminated, while those that ignore them entirely are useless and die of neglect. The survivors walk a narrow line, reactive enough to recognise invaders, restrained enough to leave the body's own tissues alone.
78
+
79
+ Turn right at the old mill and follow the gravel road for about three kilometres until you reach a wooden bridge. Cross it, and the trailhead is on your left, marked with a blue sign. The path climbs steadily through beech forest for the first hour, then levels out along the ridge. Allow four hours for the round trip and carry water; the spring at the halfway point is dry by midsummer.
80
+
81
+ The printing press did not create the desire to read; it industrialised it. Within fifty years of Gutenberg, presses operated in more than two hundred European towns, and the price of a book fell by a factor of twenty. What had been the possession of monasteries and princes became the furniture of merchants' houses, and the arguments that followed, religious and political alike, were arguments among readers.
82
+
83
+ Loss of pressure in the forward cabin triggered the automatic descent, and the aircraft levelled at three thousand metres as designed. The crew worked through the checklist, declared an emergency, and diverted to the alternate field, landing forty minutes later without injury. The investigation traced the fault to a door seal that had been replaced two weeks earlier with a part from a different supplier.
84
+
85
+ For the winter solstice the village bakes a flat bread marked with a wheel of eight spokes, one for each of the year's festivals. The dough is enriched with honey and saffron, and the first loaf out of the oven is broken and shared on the doorstep, a piece for each person present and one crumbled on the threshold for the year itself. The custom survived two empires and a revolution, mostly because it is delicious.
86
+
87
+ The function of the kidney is filtration, but the organ works by a strategy of extravagant waste followed by careful reclamation. Each day the glomeruli filter some 180 litres of fluid from the blood, and the tubules reabsorb more than 99 percent of it, adjusting the recovery of water, salt and glucose to the body's needs. What remains, about a litre and a half, is excreted as urine.
88
+
89
+ Ticket 4821: after the latest deployment, the export button on the invoices page returns a 500 error for accounts with more than one thousand line items. Reproduced on staging with the demo account. The stack trace points to a timeout in the PDF renderer. Suggested fix: paginate the export or move rendering to the background queue and email the file when ready. Priority: high, affects three enterprise customers.
90
+
91
+ The desert looks empty at noon and crowded at dawn. Tracks in the sand record the night's business: the sidewinding trail of a viper, the neat stitching of a beetle, the sudden sweep where an owl's wing erased a rodent's story mid-sentence. By the time the heat rises the actors are underground, and the stage holds only their signatures, waiting for the wind to wipe it clean for the next performance.
92
+
93
+ Add the flour gradually while mixing on low speed, then increase to medium and knead for eight minutes until the dough pulls cleanly from the sides of the bowl. It should feel soft but not sticky; adjust with a spoonful of flour or water as needed. Cover and let it rise until doubled, about ninety minutes in a warm kitchen, then fold it gently over itself twice and shape it on a floured board.