servseal 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- servseal-0.1.0/LICENSE +21 -0
- servseal-0.1.0/PKG-INFO +173 -0
- servseal-0.1.0/README.md +146 -0
- servseal-0.1.0/pyproject.toml +40 -0
- servseal-0.1.0/servseal/__init__.py +19 -0
- servseal-0.1.0/servseal/cli.py +140 -0
- servseal-0.1.0/servseal/probes/default-v1.txt +93 -0
- servseal-0.1.0/servseal/probes.py +66 -0
- servseal-0.1.0/servseal/report.py +206 -0
- servseal-0.1.0/servseal/runner.py +81 -0
- servseal-0.1.0/servseal/sampler.py +117 -0
- servseal-0.1.0/servseal/snapshot.py +160 -0
- servseal-0.1.0/servseal/verdict.py +131 -0
- servseal-0.1.0/servseal.egg-info/PKG-INFO +173 -0
- servseal-0.1.0/servseal.egg-info/SOURCES.txt +22 -0
- servseal-0.1.0/servseal.egg-info/dependency_links.txt +1 -0
- servseal-0.1.0/servseal.egg-info/entry_points.txt +2 -0
- servseal-0.1.0/servseal.egg-info/requires.txt +10 -0
- servseal-0.1.0/servseal.egg-info/top_level.txt +1 -0
- servseal-0.1.0/setup.cfg +4 -0
- servseal-0.1.0/tests/test_cli_e2e.py +77 -0
- servseal-0.1.0/tests/test_probes_and_sampler.py +58 -0
- servseal-0.1.0/tests/test_snapshot.py +97 -0
- servseal-0.1.0/tests/test_verdict.py +83 -0
servseal-0.1.0/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 riscoss63
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
servseal-0.1.0/PKG-INFO
ADDED
|
@@ -0,0 +1,173 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: servseal
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: Behavioural attestation for deployed language models: is the model you serve the model you validated?
|
|
5
|
+
Author: riscoss63
|
|
6
|
+
License: MIT
|
|
7
|
+
Project-URL: Homepage, https://github.com/riscoss63/servseal
|
|
8
|
+
Keywords: llm,attestation,drift,quantization,fingerprint,deployment
|
|
9
|
+
Classifier: Development Status :: 3 - Alpha
|
|
10
|
+
Classifier: Intended Audience :: Developers
|
|
11
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
12
|
+
Classifier: Programming Language :: Python :: 3
|
|
13
|
+
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
|
14
|
+
Classifier: Topic :: Software Development :: Quality Assurance
|
|
15
|
+
Requires-Python: >=3.9
|
|
16
|
+
Description-Content-Type: text/markdown
|
|
17
|
+
License-File: LICENSE
|
|
18
|
+
Requires-Dist: numpy>=1.21
|
|
19
|
+
Requires-Dist: scipy>=1.7
|
|
20
|
+
Requires-Dist: sqsketch>=0.3
|
|
21
|
+
Provides-Extra: model
|
|
22
|
+
Requires-Dist: torch>=2.0; extra == "model"
|
|
23
|
+
Requires-Dist: transformers>=4.30; extra == "model"
|
|
24
|
+
Provides-Extra: dev
|
|
25
|
+
Requires-Dist: pytest>=7.0; extra == "dev"
|
|
26
|
+
Dynamic: license-file
|
|
27
|
+
|
|
28
|
+
# servseal
|
|
29
|
+
|
|
30
|
+
**Is the model you serve the model you validated?**
|
|
31
|
+
|
|
32
|
+
A deployment changes quietly: a provider turns on a sampling filter, an inference stack
|
|
33
|
+
quantises, a template edit ships, a checkpoint is swapped. Task benchmarks are noisy and
|
|
34
|
+
slow; checksums verify files, not behaviour. servseal snapshots what a model actually
|
|
35
|
+
*does* — the full next-token distribution over a fixed probe set, ~1 KB per position —
|
|
36
|
+
and later tells you whether the deployed behaviour is still the reference, **how much**
|
|
37
|
+
it moved, and **which layer** moved it.
|
|
38
|
+
|
|
39
|
+
```bash
|
|
40
|
+
pip install servseal[model] # pulls sqsketch >= 0.3 from PyPI
|
|
41
|
+
|
|
42
|
+
servseal snapshot gpt2 -o reference.seal.npz
|
|
43
|
+
# ... deploy, quantise, migrate serving stacks, wait six months ...
|
|
44
|
+
servseal snapshot /path/to/served-model -o candidate.seal.npz
|
|
45
|
+
servseal verify reference.seal.npz candidate.seal.npz --report attestation.html
|
|
46
|
+
echo $? # 0 sealed | 3 changed | 2 incomparable
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
## The change nothing else sees
|
|
50
|
+
|
|
51
|
+
Measured end to end through this tool on GPT-2 (1500 probe positions, 37 probe texts,
|
|
52
|
+
D=256 — every number below is from the committed run in `experiments/outputs/`):
|
|
53
|
+
|
|
54
|
+
| deployment event | mean Hellinger | top-1 agreement | perplexity | verdict |
|
|
55
|
+
|---|---:|---:|---:|---|
|
|
56
|
+
| unchanged (re-run) | **0.0000** | 1.000 | 27.83 → 27.83 | **SEALED** |
|
|
57
|
+
| serve: top-p 0.95 | **0.1537** | 1.000 | 27.83 → **27.83** | CHANGED/major — serving-layer |
|
|
58
|
+
| serve: temperature 1.05 | 0.0425 | 1.000 | 27.83 → 27.83 | CHANGED/minor |
|
|
59
|
+
| weights → bfloat16 | 0.0352 | 0.961 | 27.83 → 27.68 | CHANGED/minor — precision |
|
|
60
|
+
| weights → int8/tensor | 0.1684 | 0.773 | 27.83 → 32.03 | CHANGED/major — weight-level |
|
|
61
|
+
| wrong chat template | 0.9072 | 0.037 | 27.83 → 35.10 | CHANGED/major — template |
|
|
62
|
+
| distilgpt2 substituted | 0.2720 | 0.628 | 27.83 → 44.04 | CHANGED/major — weight-level |
|
|
63
|
+
|
|
64
|
+
Read the top-p row: the provider turned on nucleus sampling, the most likely token is
|
|
65
|
+
unchanged at **every** position, perplexity is unchanged to two decimals — greedy output
|
|
66
|
+
diffs, log-loss checks and eval suites see nothing — and the distribution moved by
|
|
67
|
+
0.15, which this tool measures exactly and labels correctly. A top-k logprob
|
|
68
|
+
fingerprint reports about half of it and does not converge with more memory (measured
|
|
69
|
+
in the sqsketch paper this tool builds on).
|
|
70
|
+
|
|
71
|
+
Read the template row the other way round: behaviour was destroyed (0.907, top-1
|
|
72
|
+
agreement 3.7%), while perplexity moved only from 27.8 to 35.1 — the class of bug
|
|
73
|
+
routinely misdiagnosed as "the quantisation made it worse". The signature separates
|
|
74
|
+
them: template bugs annihilate top-1 agreement; quantisation dents it; serving filters
|
|
75
|
+
leave it at 100%.
|
|
76
|
+
|
|
77
|
+
The same battery runs on a second architecture (pythia-160m): unchanged → SEALED at
|
|
78
|
+
0.0000, top-p → 0.1520, CHANGED/serving-layer. Comparing a pythia snapshot to a GPT-2
|
|
79
|
+
snapshot is refused (`INCOMPARABLE`, exit 2) — different tokenizer, not the same
|
|
80
|
+
measurement.
|
|
81
|
+
|
|
82
|
+
## API mode: endpoints you can only sample from
|
|
83
|
+
|
|
84
|
+
Against a black-box API the full distribution is unavailable; the endpoint returns
|
|
85
|
+
sampled tokens, already shaped by the provider's serving parameters — which is exactly
|
|
86
|
+
the object under test. Protocol: one `max_tokens=1, temperature=1` completion per probe
|
|
87
|
+
position, N times. Detection is two-sided and calibrated on the unchanged endpoint;
|
|
88
|
+
power measured on real GPT-2 distributions (`experiments/sampling_power.py`):
|
|
89
|
+
|
|
90
|
+
| sampled tokens | per position | false positives | detects top-p 0.95 | detects temp 1.05 |
|
|
91
|
+
|---:|---:|---:|---:|---:|
|
|
92
|
+
| 1,500 | 1.0 | 0.04 | 0.83 | 0.88 |
|
|
93
|
+
| **5,000** | **3.3** | **0.05** | **1.00** | **1.00** |
|
|
94
|
+
| 15,000 | 10.0 | 0.02 | 1.00 | 1.00 |
|
|
95
|
+
|
|
96
|
+
**Five thousand single-token completions detect both changes with power 1.00 at a 5%
|
|
97
|
+
false-positive budget** — on a commercial API, a few dollars of traffic. Two designs
|
|
98
|
+
failed before this one and are documented in `servseal/sampler.py`: a one-sided "BC
|
|
99
|
+
must drop" rule (power 0.10 — cutting the tail *concentrates* samples on the head and
|
|
100
|
+
pushes the statistic up) and a pooled corpus-aggregate statistic (power 0.03 — one
|
|
101
|
+
position's tail is another's head; pooling erases the evidence). The shipped statistic
|
|
102
|
+
is per-position, against the per-position sketches the snapshot already stores, and the
|
|
103
|
+
power sweep asserts bit-equality with the product code path before measuring.
|
|
104
|
+
|
|
105
|
+
## What a verdict gives you
|
|
106
|
+
|
|
107
|
+
```
|
|
108
|
+
verdict CHANGED severity=major
|
|
109
|
+
signature serving-layer sampling filter (tail-only)
|
|
110
|
+
hellinger mean=0.1537 max=0.2513 floor=0.0884
|
|
111
|
+
top1 agreement=1.0000
|
|
112
|
+
kl_bound mean per-position KL >= 0.0000 (certified; 0 certifies nothing, not absence of change)
|
|
113
|
+
report attestation.html
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
`--report` writes a self-contained HTML attestation — verdict banner, the numbers, a
|
|
117
|
+
per-position histogram, both snapshot identities and the probe hash — suitable for a
|
|
118
|
+
ticket, a release artifact, or a compliance file.
|
|
119
|
+
|
|
120
|
+
## What it will not do
|
|
121
|
+
|
|
122
|
+
- **Compare across tokenizers.** A fingerprint lives in one vocabulary; pythia vs GPT-2
|
|
123
|
+
is refused, not approximated. (Measured and closed as a negative result in the
|
|
124
|
+
underlying paper.)
|
|
125
|
+
- **Name the culprit with certainty.** Signatures are calibrated hypotheses, not
|
|
126
|
+
proofs: a distilled sibling substituted for the reference lands in the same band as
|
|
127
|
+
an aggressive quantisation (both are "the weights are not the reference weights"),
|
|
128
|
+
and behavioural evidence alone cannot always split them.
|
|
129
|
+
- **Certify small changes.** The certified-KL line is a one-sided bound that only bites
|
|
130
|
+
on gross changes at D=256: `0` certifies nothing and must never be read as "nothing
|
|
131
|
+
changed". The Hellinger measurement, not the certificate, is the detector.
|
|
132
|
+
- **Judge quality.** servseal tells you the behaviour moved, not whether it got worse.
|
|
133
|
+
A fine-tune you shipped on purpose is CHANGED/major — as it should be.
|
|
134
|
+
- **See what the probes never touch.** Coverage is the probe set's; snapshot your own
|
|
135
|
+
traffic domain too (`--probes yourfile.txt`). Snapshots refuse comparison across
|
|
136
|
+
probe sets by hash.
|
|
137
|
+
- **Weights-mode determinism is CPU-grade.** Reference snapshots here are computed in
|
|
138
|
+
float32 on CPU, where re-running the same model reproduces distance exactly 0.0000.
|
|
139
|
+
GPU inference can be nondeterministic; snapshot on CPU for the reference of record.
|
|
140
|
+
|
|
141
|
+
## How it works
|
|
142
|
+
|
|
143
|
+
Each position's full softmax (vocabulary-sized, tail included) is compressed to D=256
|
|
144
|
+
numbers by a square-root sketch whose inner product estimates the Bhattacharyya
|
|
145
|
+
coefficient of the complete distributions, with error `sqrt(2/D)` independent of
|
|
146
|
+
vocabulary size — the tail is exactly where serving filters and quantisation act, and
|
|
147
|
+
where top-k logprobs are blind by construction. The estimator, its error bars, the
|
|
148
|
+
one-sided KL certificate and the negative results (cross-tokenizer, top-k saturation)
|
|
149
|
+
are established in the sqsketch paper: *Norm-Invariance in Vector-Symbolic Encodings of
|
|
150
|
+
Probability Distributions* (DOI
|
|
151
|
+
[10.5281/zenodo.22214969](https://doi.org/10.5281/zenodo.22214969), code
|
|
152
|
+
[riscoss63/sqsketch](https://github.com/riscoss63/sqsketch)).
|
|
153
|
+
|
|
154
|
+
## Audit it
|
|
155
|
+
|
|
156
|
+
Everything above is one command each, on models small enough for a laptop CPU:
|
|
157
|
+
|
|
158
|
+
```bash
|
|
159
|
+
pip install -e .[model,dev]
|
|
160
|
+
pytest # 24 unit + 5 CLI end-to-end tests
|
|
161
|
+
cd experiments
|
|
162
|
+
python e2e_real_models.py # the verdict battery, ~12 min CPU
|
|
163
|
+
python sampling_power.py # the API-mode power table, ~1 min
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
The battery *asserts* every verdict against its ground truth and exits non-zero on any
|
|
167
|
+
miss; the committed outputs in `experiments/outputs/` are what the tables above quote.
|
|
168
|
+
Verdict thresholds live in `servseal/verdict.py` with the calibration documented
|
|
169
|
+
inline; changing them breaks tests until the documentation moves with them.
|
|
170
|
+
|
|
171
|
+
## Licence
|
|
172
|
+
|
|
173
|
+
MIT.
|
servseal-0.1.0/README.md
ADDED
|
@@ -0,0 +1,146 @@
|
|
|
1
|
+
# servseal
|
|
2
|
+
|
|
3
|
+
**Is the model you serve the model you validated?**
|
|
4
|
+
|
|
5
|
+
A deployment changes quietly: a provider turns on a sampling filter, an inference stack
|
|
6
|
+
quantises, a template edit ships, a checkpoint is swapped. Task benchmarks are noisy and
|
|
7
|
+
slow; checksums verify files, not behaviour. servseal snapshots what a model actually
|
|
8
|
+
*does* — the full next-token distribution over a fixed probe set, ~1 KB per position —
|
|
9
|
+
and later tells you whether the deployed behaviour is still the reference, **how much**
|
|
10
|
+
it moved, and **which layer** moved it.
|
|
11
|
+
|
|
12
|
+
```bash
|
|
13
|
+
pip install servseal[model] # pulls sqsketch >= 0.3 from PyPI
|
|
14
|
+
|
|
15
|
+
servseal snapshot gpt2 -o reference.seal.npz
|
|
16
|
+
# ... deploy, quantise, migrate serving stacks, wait six months ...
|
|
17
|
+
servseal snapshot /path/to/served-model -o candidate.seal.npz
|
|
18
|
+
servseal verify reference.seal.npz candidate.seal.npz --report attestation.html
|
|
19
|
+
echo $? # 0 sealed | 3 changed | 2 incomparable
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
## The change nothing else sees
|
|
23
|
+
|
|
24
|
+
Measured end to end through this tool on GPT-2 (1500 probe positions, 37 probe texts,
|
|
25
|
+
D=256 — every number below is from the committed run in `experiments/outputs/`):
|
|
26
|
+
|
|
27
|
+
| deployment event | mean Hellinger | top-1 agreement | perplexity | verdict |
|
|
28
|
+
|---|---:|---:|---:|---|
|
|
29
|
+
| unchanged (re-run) | **0.0000** | 1.000 | 27.83 → 27.83 | **SEALED** |
|
|
30
|
+
| serve: top-p 0.95 | **0.1537** | 1.000 | 27.83 → **27.83** | CHANGED/major — serving-layer |
|
|
31
|
+
| serve: temperature 1.05 | 0.0425 | 1.000 | 27.83 → 27.83 | CHANGED/minor |
|
|
32
|
+
| weights → bfloat16 | 0.0352 | 0.961 | 27.83 → 27.68 | CHANGED/minor — precision |
|
|
33
|
+
| weights → int8/tensor | 0.1684 | 0.773 | 27.83 → 32.03 | CHANGED/major — weight-level |
|
|
34
|
+
| wrong chat template | 0.9072 | 0.037 | 27.83 → 35.10 | CHANGED/major — template |
|
|
35
|
+
| distilgpt2 substituted | 0.2720 | 0.628 | 27.83 → 44.04 | CHANGED/major — weight-level |
|
|
36
|
+
|
|
37
|
+
Read the top-p row: the provider turned on nucleus sampling, the most likely token is
|
|
38
|
+
unchanged at **every** position, perplexity is unchanged to two decimals — greedy output
|
|
39
|
+
diffs, log-loss checks and eval suites see nothing — and the distribution moved by
|
|
40
|
+
0.15, which this tool measures exactly and labels correctly. A top-k logprob
|
|
41
|
+
fingerprint reports about half of it and does not converge with more memory (measured
|
|
42
|
+
in the sqsketch paper this tool builds on).
|
|
43
|
+
|
|
44
|
+
Read the template row the other way round: behaviour was destroyed (0.907, top-1
|
|
45
|
+
agreement 3.7%), while perplexity moved only from 27.8 to 35.1 — the class of bug
|
|
46
|
+
routinely misdiagnosed as "the quantisation made it worse". The signature separates
|
|
47
|
+
them: template bugs annihilate top-1 agreement; quantisation dents it; serving filters
|
|
48
|
+
leave it at 100%.
|
|
49
|
+
|
|
50
|
+
The same battery runs on a second architecture (pythia-160m): unchanged → SEALED at
|
|
51
|
+
0.0000, top-p → 0.1520, CHANGED/serving-layer. Comparing a pythia snapshot to a GPT-2
|
|
52
|
+
snapshot is refused (`INCOMPARABLE`, exit 2) — different tokenizer, not the same
|
|
53
|
+
measurement.
|
|
54
|
+
|
|
55
|
+
## API mode: endpoints you can only sample from
|
|
56
|
+
|
|
57
|
+
Against a black-box API the full distribution is unavailable; the endpoint returns
|
|
58
|
+
sampled tokens, already shaped by the provider's serving parameters — which is exactly
|
|
59
|
+
the object under test. Protocol: one `max_tokens=1, temperature=1` completion per probe
|
|
60
|
+
position, N times. Detection is two-sided and calibrated on the unchanged endpoint;
|
|
61
|
+
power measured on real GPT-2 distributions (`experiments/sampling_power.py`):
|
|
62
|
+
|
|
63
|
+
| sampled tokens | per position | false positives | detects top-p 0.95 | detects temp 1.05 |
|
|
64
|
+
|---:|---:|---:|---:|---:|
|
|
65
|
+
| 1,500 | 1.0 | 0.04 | 0.83 | 0.88 |
|
|
66
|
+
| **5,000** | **3.3** | **0.05** | **1.00** | **1.00** |
|
|
67
|
+
| 15,000 | 10.0 | 0.02 | 1.00 | 1.00 |
|
|
68
|
+
|
|
69
|
+
**Five thousand single-token completions detect both changes with power 1.00 at a 5%
|
|
70
|
+
false-positive budget** — on a commercial API, a few dollars of traffic. Two designs
|
|
71
|
+
failed before this one and are documented in `servseal/sampler.py`: a one-sided "BC
|
|
72
|
+
must drop" rule (power 0.10 — cutting the tail *concentrates* samples on the head and
|
|
73
|
+
pushes the statistic up) and a pooled corpus-aggregate statistic (power 0.03 — one
|
|
74
|
+
position's tail is another's head; pooling erases the evidence). The shipped statistic
|
|
75
|
+
is per-position, against the per-position sketches the snapshot already stores, and the
|
|
76
|
+
power sweep asserts bit-equality with the product code path before measuring.
|
|
77
|
+
|
|
78
|
+
## What a verdict gives you
|
|
79
|
+
|
|
80
|
+
```
|
|
81
|
+
verdict CHANGED severity=major
|
|
82
|
+
signature serving-layer sampling filter (tail-only)
|
|
83
|
+
hellinger mean=0.1537 max=0.2513 floor=0.0884
|
|
84
|
+
top1 agreement=1.0000
|
|
85
|
+
kl_bound mean per-position KL >= 0.0000 (certified; 0 certifies nothing, not absence of change)
|
|
86
|
+
report attestation.html
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
`--report` writes a self-contained HTML attestation — verdict banner, the numbers, a
|
|
90
|
+
per-position histogram, both snapshot identities and the probe hash — suitable for a
|
|
91
|
+
ticket, a release artifact, or a compliance file.
|
|
92
|
+
|
|
93
|
+
## What it will not do
|
|
94
|
+
|
|
95
|
+
- **Compare across tokenizers.** A fingerprint lives in one vocabulary; pythia vs GPT-2
|
|
96
|
+
is refused, not approximated. (Measured and closed as a negative result in the
|
|
97
|
+
underlying paper.)
|
|
98
|
+
- **Name the culprit with certainty.** Signatures are calibrated hypotheses, not
|
|
99
|
+
proofs: a distilled sibling substituted for the reference lands in the same band as
|
|
100
|
+
an aggressive quantisation (both are "the weights are not the reference weights"),
|
|
101
|
+
and behavioural evidence alone cannot always split them.
|
|
102
|
+
- **Certify small changes.** The certified-KL line is a one-sided bound that only bites
|
|
103
|
+
on gross changes at D=256: `0` certifies nothing and must never be read as "nothing
|
|
104
|
+
changed". The Hellinger measurement, not the certificate, is the detector.
|
|
105
|
+
- **Judge quality.** servseal tells you the behaviour moved, not whether it got worse.
|
|
106
|
+
A fine-tune you shipped on purpose is CHANGED/major — as it should be.
|
|
107
|
+
- **See what the probes never touch.** Coverage is the probe set's; snapshot your own
|
|
108
|
+
traffic domain too (`--probes yourfile.txt`). Snapshots refuse comparison across
|
|
109
|
+
probe sets by hash.
|
|
110
|
+
- **Weights-mode determinism is CPU-grade.** Reference snapshots here are computed in
|
|
111
|
+
float32 on CPU, where re-running the same model reproduces distance exactly 0.0000.
|
|
112
|
+
GPU inference can be nondeterministic; snapshot on CPU for the reference of record.
|
|
113
|
+
|
|
114
|
+
## How it works
|
|
115
|
+
|
|
116
|
+
Each position's full softmax (vocabulary-sized, tail included) is compressed to D=256
|
|
117
|
+
numbers by a square-root sketch whose inner product estimates the Bhattacharyya
|
|
118
|
+
coefficient of the complete distributions, with error `sqrt(2/D)` independent of
|
|
119
|
+
vocabulary size — the tail is exactly where serving filters and quantisation act, and
|
|
120
|
+
where top-k logprobs are blind by construction. The estimator, its error bars, the
|
|
121
|
+
one-sided KL certificate and the negative results (cross-tokenizer, top-k saturation)
|
|
122
|
+
are established in the sqsketch paper: *Norm-Invariance in Vector-Symbolic Encodings of
|
|
123
|
+
Probability Distributions* (DOI
|
|
124
|
+
[10.5281/zenodo.22214969](https://doi.org/10.5281/zenodo.22214969), code
|
|
125
|
+
[riscoss63/sqsketch](https://github.com/riscoss63/sqsketch)).
|
|
126
|
+
|
|
127
|
+
## Audit it
|
|
128
|
+
|
|
129
|
+
Everything above is one command each, on models small enough for a laptop CPU:
|
|
130
|
+
|
|
131
|
+
```bash
|
|
132
|
+
pip install -e .[model,dev]
|
|
133
|
+
pytest # 24 unit + 5 CLI end-to-end tests
|
|
134
|
+
cd experiments
|
|
135
|
+
python e2e_real_models.py # the verdict battery, ~12 min CPU
|
|
136
|
+
python sampling_power.py # the API-mode power table, ~1 min
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
The battery *asserts* every verdict against its ground truth and exits non-zero on any
|
|
140
|
+
miss; the committed outputs in `experiments/outputs/` are what the tables above quote.
|
|
141
|
+
Verdict thresholds live in `servseal/verdict.py` with the calibration documented
|
|
142
|
+
inline; changing them breaks tests until the documentation moves with them.
|
|
143
|
+
|
|
144
|
+
## Licence
|
|
145
|
+
|
|
146
|
+
MIT.
|
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
[build-system]
|
|
2
|
+
requires = ["setuptools>=61"]
|
|
3
|
+
build-backend = "setuptools.build_meta"
|
|
4
|
+
|
|
5
|
+
[project]
|
|
6
|
+
name = "servseal"
|
|
7
|
+
version = "0.1.0"
|
|
8
|
+
description = "Behavioural attestation for deployed language models: is the model you serve the model you validated?"
|
|
9
|
+
readme = "README.md"
|
|
10
|
+
license = { text = "MIT" }
|
|
11
|
+
requires-python = ">=3.9"
|
|
12
|
+
authors = [{ name = "riscoss63" }]
|
|
13
|
+
keywords = ["llm", "attestation", "drift", "quantization", "fingerprint", "deployment"]
|
|
14
|
+
classifiers = [
|
|
15
|
+
"Development Status :: 3 - Alpha",
|
|
16
|
+
"Intended Audience :: Developers",
|
|
17
|
+
"License :: OSI Approved :: MIT License",
|
|
18
|
+
"Programming Language :: Python :: 3",
|
|
19
|
+
"Topic :: Scientific/Engineering :: Artificial Intelligence",
|
|
20
|
+
"Topic :: Software Development :: Quality Assurance",
|
|
21
|
+
]
|
|
22
|
+
dependencies = ["numpy>=1.21", "scipy>=1.7", "sqsketch>=0.3"]
|
|
23
|
+
|
|
24
|
+
[project.optional-dependencies]
|
|
25
|
+
# Only snapshotting a model needs these. Verifying, reporting and testing verdict logic
|
|
26
|
+
# work on saved snapshots with numpy alone.
|
|
27
|
+
model = ["torch>=2.0", "transformers>=4.30"]
|
|
28
|
+
dev = ["pytest>=7.0"]
|
|
29
|
+
|
|
30
|
+
[project.scripts]
|
|
31
|
+
servseal = "servseal.cli:main"
|
|
32
|
+
|
|
33
|
+
[project.urls]
|
|
34
|
+
Homepage = "https://github.com/riscoss63/servseal"
|
|
35
|
+
|
|
36
|
+
[tool.setuptools]
|
|
37
|
+
packages = ["servseal"]
|
|
38
|
+
|
|
39
|
+
[tool.setuptools.package-data]
|
|
40
|
+
servseal = ["probes/*.txt"]
|
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
"""servseal: behavioural attestation for deployed language models.
|
|
2
|
+
|
|
3
|
+
from servseal import Snapshot, classify
|
|
4
|
+
from servseal.runner import snapshot_model # needs torch
|
|
5
|
+
|
|
6
|
+
ref = snapshot_model("gpt2")
|
|
7
|
+
ref.save("gpt2.seal.npz")
|
|
8
|
+
...
|
|
9
|
+
verdict = classify(Snapshot.load("gpt2.seal.npz").compare(snapshot_model(served)))
|
|
10
|
+
print(verdict.status, "-", verdict.signature)
|
|
11
|
+
"""
|
|
12
|
+
__version__ = "0.1.0"
|
|
13
|
+
|
|
14
|
+
from .probes import load_probes, probe_id # noqa: E402
|
|
15
|
+
from .snapshot import Snapshot # noqa: E402
|
|
16
|
+
from .verdict import Verdict, classify # noqa: E402
|
|
17
|
+
|
|
18
|
+
__all__ = ["Snapshot", "Verdict", "classify", "load_probes", "probe_id",
|
|
19
|
+
"__version__"]
|
|
@@ -0,0 +1,140 @@
|
|
|
1
|
+
"""The command line: snapshot, verify, report, probes.
|
|
2
|
+
|
|
3
|
+
Exit codes are the contract, so a pipeline can gate on them:
|
|
4
|
+
|
|
5
|
+
0 sealed the served model behaves as the reference
|
|
6
|
+
1 tool error bad arguments, missing file, model failed to load
|
|
7
|
+
2 incomparable the snapshots measured different things; no attestation made
|
|
8
|
+
3 changed the served model does NOT behave as the reference
|
|
9
|
+
|
|
10
|
+
Output is ASCII, one fact per line, machine-greppable; --json gives the whole record.
|
|
11
|
+
"""
|
|
12
|
+
from __future__ import annotations
|
|
13
|
+
|
|
14
|
+
import argparse
|
|
15
|
+
import json
|
|
16
|
+
import sys
|
|
17
|
+
|
|
18
|
+
from .probes import load_probes, probe_id
|
|
19
|
+
from .snapshot import Snapshot
|
|
20
|
+
from .verdict import EXIT_ERROR, classify
|
|
21
|
+
|
|
22
|
+
__all__ = ["main"]
|
|
23
|
+
|
|
24
|
+
|
|
25
|
+
def _cmd_snapshot(a):
|
|
26
|
+
from .runner import snapshot_model # torch only here
|
|
27
|
+
snap = snapshot_model(a.model, probes=a.probes, D=a.D, seed=a.seed,
|
|
28
|
+
max_positions=a.positions, max_length=a.max_length,
|
|
29
|
+
template=a.template, dtype=a.dtype, label=a.label)
|
|
30
|
+
snap.save(a.out)
|
|
31
|
+
m = snap.meta
|
|
32
|
+
print(f"snapshot {a.out}")
|
|
33
|
+
print(f"model {m['model']} dtype={m['dtype']}")
|
|
34
|
+
print(f"positions {m['n_positions']} D={m['D']} size={snap.nbytes() / 1024:.0f} KB")
|
|
35
|
+
print(f"probe {m['probe']} ({m.get('probe_file')})")
|
|
36
|
+
print(f"perplexity {m.get('perplexity')}")
|
|
37
|
+
return 0
|
|
38
|
+
|
|
39
|
+
|
|
40
|
+
def _cmd_verify(a):
|
|
41
|
+
ref = Snapshot.load(a.reference)
|
|
42
|
+
cand = Snapshot.load(a.candidate)
|
|
43
|
+
metrics = ref.compare(cand)
|
|
44
|
+
verdict = classify(metrics)
|
|
45
|
+
record = {"metrics": metrics, "verdict": verdict.as_dict()}
|
|
46
|
+
|
|
47
|
+
if a.json:
|
|
48
|
+
with open(a.json, "w", encoding="utf-8") as fh:
|
|
49
|
+
json.dump(record, fh, indent=2)
|
|
50
|
+
print(f"verdict {verdict.status.upper()}"
|
|
51
|
+
+ (f" severity={verdict.severity}" if verdict.status == "changed" else ""))
|
|
52
|
+
print(f"signature {verdict.signature}")
|
|
53
|
+
if not metrics.get("incomparable"):
|
|
54
|
+
print(f"hellinger mean={metrics['mean_hellinger']:.4f}"
|
|
55
|
+
f" max={metrics['max_hellinger']:.4f}"
|
|
56
|
+
f" floor={metrics['noise_floor']:.4f}")
|
|
57
|
+
if metrics.get("top1_agreement") is not None:
|
|
58
|
+
print(f"top1 agreement={metrics['top1_agreement']:.4f}")
|
|
59
|
+
klb = metrics.get('mean_kl_lower_bound',
|
|
60
|
+
max(0.0, metrics['aggregate_kl_lower_bound']))
|
|
61
|
+
print(f"kl_bound mean per-position KL >= {klb:.4f}"
|
|
62
|
+
f" (certified; 0 certifies nothing, not absence of change)")
|
|
63
|
+
else:
|
|
64
|
+
print(f"reason {metrics['incomparable']}")
|
|
65
|
+
if a.report:
|
|
66
|
+
from .report import render_report
|
|
67
|
+
with open(a.report, "w", encoding="utf-8") as fh:
|
|
68
|
+
fh.write(render_report(record))
|
|
69
|
+
print(f"report {a.report}")
|
|
70
|
+
return verdict.exit_code
|
|
71
|
+
|
|
72
|
+
|
|
73
|
+
def _cmd_report(a):
|
|
74
|
+
from .report import load_result, render_report
|
|
75
|
+
record = load_result(a.result)
|
|
76
|
+
with open(a.out, "w", encoding="utf-8") as fh:
|
|
77
|
+
fh.write(render_report(record))
|
|
78
|
+
print(f"report {a.out}")
|
|
79
|
+
return 0
|
|
80
|
+
|
|
81
|
+
|
|
82
|
+
def _cmd_probes(a):
|
|
83
|
+
texts = load_probes(a.set)
|
|
84
|
+
print(f"probe set {a.set or 'default-v1'}")
|
|
85
|
+
print(f"texts {len(texts)}")
|
|
86
|
+
print(f"probe id {probe_id(texts)}")
|
|
87
|
+
return 0
|
|
88
|
+
|
|
89
|
+
|
|
90
|
+
def main(argv=None):
|
|
91
|
+
p = argparse.ArgumentParser(
|
|
92
|
+
prog="servseal",
|
|
93
|
+
description="Behavioural attestation for deployed language models: "
|
|
94
|
+
"is the model you serve the model you validated?")
|
|
95
|
+
sub = p.add_subparsers(dest="cmd", required=True)
|
|
96
|
+
|
|
97
|
+
s = sub.add_parser("snapshot", help="fingerprint a model over the probe set")
|
|
98
|
+
s.add_argument("model", help="HF model id or local path")
|
|
99
|
+
s.add_argument("-o", "--out", required=True, help="output .seal.npz file")
|
|
100
|
+
s.add_argument("--probes", default=None, help="bundled set name or probe file")
|
|
101
|
+
s.add_argument("--positions", type=int, default=1500)
|
|
102
|
+
s.add_argument("--max-length", type=int, default=96)
|
|
103
|
+
s.add_argument("--D", type=int, default=256)
|
|
104
|
+
s.add_argument("--seed", type=int, default=0)
|
|
105
|
+
s.add_argument("--dtype", default="float32",
|
|
106
|
+
choices=["float32", "bfloat16", "float16"])
|
|
107
|
+
s.add_argument("--template", default=None,
|
|
108
|
+
help="prompt template with {text}, if the deployment applies one")
|
|
109
|
+
s.add_argument("--label", default=None)
|
|
110
|
+
s.set_defaults(fn=_cmd_snapshot)
|
|
111
|
+
|
|
112
|
+
v = sub.add_parser("verify", help="compare a candidate snapshot to a reference")
|
|
113
|
+
v.add_argument("reference")
|
|
114
|
+
v.add_argument("candidate")
|
|
115
|
+
v.add_argument("--json", default=None, help="write the full record here")
|
|
116
|
+
v.add_argument("--report", default=None, help="write an HTML attestation here")
|
|
117
|
+
v.set_defaults(fn=_cmd_verify)
|
|
118
|
+
|
|
119
|
+
r = sub.add_parser("report", help="render an HTML attestation from a verify --json")
|
|
120
|
+
r.add_argument("result")
|
|
121
|
+
r.add_argument("-o", "--out", required=True)
|
|
122
|
+
r.set_defaults(fn=_cmd_report)
|
|
123
|
+
|
|
124
|
+
pr = sub.add_parser("probes", help="show a probe set and its id")
|
|
125
|
+
pr.add_argument("--set", default=None)
|
|
126
|
+
pr.set_defaults(fn=_cmd_probes)
|
|
127
|
+
|
|
128
|
+
a = p.parse_args(argv)
|
|
129
|
+
try:
|
|
130
|
+
return a.fn(a)
|
|
131
|
+
except FileNotFoundError as e:
|
|
132
|
+
print(f"error {e}", file=sys.stderr)
|
|
133
|
+
return EXIT_ERROR
|
|
134
|
+
except ValueError as e:
|
|
135
|
+
print(f"error {e}", file=sys.stderr)
|
|
136
|
+
return EXIT_ERROR
|
|
137
|
+
|
|
138
|
+
|
|
139
|
+
if __name__ == "__main__":
|
|
140
|
+
sys.exit(main())
|
|
@@ -0,0 +1,93 @@
|
|
|
1
|
+
# servseal probe set: default-v1
|
|
2
|
+
# One paragraph per block, blank-line separated. Lines starting with # are ignored.
|
|
3
|
+
# This file is part of the product contract: two snapshots are comparable only if they
|
|
4
|
+
# were taken on byte-identical probes, and the probe id in every snapshot is a hash of
|
|
5
|
+
# exactly these texts. Do not edit this file; add a new versioned file instead.
|
|
6
|
+
|
|
7
|
+
The water cycle describes how water moves between the oceans, the atmosphere and the land. Heat from the sun evaporates water from the sea surface, and the vapour rises, cools and condenses into clouds. When the droplets grow heavy enough they fall as rain or snow, feeding rivers and soaking into the ground, and the rivers carry the water back to the sea, where the cycle begins again.
|
|
8
|
+
|
|
9
|
+
The committee met on Tuesday to review the proposed amendments to the zoning ordinance. After two hours of discussion, the members voted five to two in favour of forwarding the revised draft to the city council, with the recommendation that the height restriction along the waterfront be retained and the parking requirement for mixed-use buildings be reduced from two spaces per unit to one.
|
|
10
|
+
|
|
11
|
+
def merge_sorted(a, b):
|
|
12
|
+
out = []
|
|
13
|
+
i = j = 0
|
|
14
|
+
while i < len(a) and j < len(b):
|
|
15
|
+
if a[i] <= b[j]:
|
|
16
|
+
out.append(a[i]); i += 1
|
|
17
|
+
else:
|
|
18
|
+
out.append(b[j]); j += 1
|
|
19
|
+
return out + a[i:] + b[j:]
|
|
20
|
+
|
|
21
|
+
Preheat the oven to 220 degrees. Toss the potatoes with olive oil, salt and rosemary, then spread them in a single layer on a baking tray, cut side down. Roast for twenty-five minutes without turning, until the undersides are deeply browned, then flip them, add the crushed garlic, and roast for another ten minutes. Finish with lemon zest and a little more salt before serving.
|
|
22
|
+
|
|
23
|
+
"Did you hear the news about the bridge?" she asked, setting down her cup. "They're closing it for repairs starting Monday." He frowned and looked out of the window at the traffic already backed up along the avenue. "That's going to double my commute," he said. "Maybe. Or maybe people will finally start taking the ferry, and you'll get the quiet morning you keep saying you want."
|
|
24
|
+
|
|
25
|
+
In the second half the visitors changed their shape, pushing the full-backs high and playing through the middle, and for twenty minutes the home side could barely get out of their own half. The equaliser came from a corner, a glancing header at the near post, and by the time the fourth official signalled six minutes of added time both sets of supporters seemed to sense that one more goal was coming.
|
|
26
|
+
|
|
27
|
+
The quarterly report showed revenue of 4.2 billion, up nine percent from a year earlier, driven mostly by the services division, while hardware sales declined for the third consecutive quarter. Operating margin narrowed to 18.5 percent as the company absorbed higher logistics costs, and management lowered full-year guidance, citing currency headwinds and softer demand in Europe.
|
|
28
|
+
|
|
29
|
+
A cold front will move across the region overnight, bringing a sharp drop in temperature and gusty northwest winds of forty to fifty kilometres per hour. Rain will turn to snow above six hundred metres, with accumulations of five to ten centimetres expected on higher ground by morning. Travellers should allow extra time and check conditions before crossing the pass.
|
|
30
|
+
|
|
31
|
+
The photosynthetic reaction begins when a photon strikes a chlorophyll molecule in the thylakoid membrane, exciting an electron that is passed along a transport chain. The energy released pumps protons across the membrane, and the resulting gradient drives the synthesis of ATP. Meanwhile the splitting of water molecules releases oxygen as a by-product, which diffuses out of the leaf through the stomata.
|
|
32
|
+
|
|
33
|
+
To reset the device, hold the power button for ten seconds until the indicator light flashes amber. Release the button, wait for the light to turn solid white, and then press it twice in quick succession. The device will restart with factory settings; note that any stored profiles will be erased, so export your configuration from the companion app before you begin.
|
|
34
|
+
|
|
35
|
+
A train leaves the station travelling east at eighty kilometres per hour. Half an hour later a second train leaves the same station on a parallel track travelling east at one hundred and ten kilometres per hour. How long after its departure will the second train draw level with the first, and how far from the station will both trains be at that moment?
|
|
36
|
+
|
|
37
|
+
The tenant shall keep the premises in good order and shall not make structural alterations without the prior written consent of the landlord. The landlord shall be responsible for repairs to the roof, exterior walls and main services, save where damage results from the negligence of the tenant, in which case the reasonable cost of repair may be recovered from the deposit.
|
|
38
|
+
|
|
39
|
+
The old lighthouse keeper climbed the spiral stairs for the last time that evening, counting each step as he had for thirty-one years. The lamp no longer needed him; the new system switched itself on at dusk and reported its own faults to a computer on the mainland. He stood on the gallery in the fading light, watching the beam sweep over the water, and found he could not decide whether to feel replaced or released.
|
|
40
|
+
|
|
41
|
+
Interest rates influence the economy through several channels. When the central bank raises its policy rate, borrowing becomes more expensive for households and firms, which tends to cool investment and consumption. The exchange rate typically strengthens, making imports cheaper and exports dearer. With a lag of a year or more, these effects feed through to slower growth in prices and wages.
|
|
42
|
+
|
|
43
|
+
The recipe for a good bug report is short: state what you did, what you expected, and what happened instead. Include the exact error text, the version of the software, and the smallest sequence of steps that reproduces the problem. A report that says the application crashed after clicking save on an empty form is worth ten that say the program does not work properly.
|
|
44
|
+
|
|
45
|
+
Between 1804 and 1806 the expedition travelled up the Missouri River, crossed the continental divide on horseback, and descended the Columbia to the Pacific, mapping the country and recording hundreds of plants and animals unknown to science. The journals kept along the way, with their mixture of careful observation and phonetic spelling, remain one of the most vivid records of the American interior before settlement.
|
|
46
|
+
|
|
47
|
+
SELECT customer_id, COUNT(*) AS orders, SUM(total) AS revenue
|
|
48
|
+
FROM orders
|
|
49
|
+
WHERE placed_at >= DATE '2025-01-01'
|
|
50
|
+
GROUP BY customer_id
|
|
51
|
+
HAVING COUNT(*) >= 3
|
|
52
|
+
ORDER BY revenue DESC
|
|
53
|
+
LIMIT 50;
|
|
54
|
+
|
|
55
|
+
The violin section carries the opening theme, a rising figure in D minor that the cellos answer two bars later in inversion. The development fragments this material, passing it between woodwind and strings over a restless pedal point, and when the recapitulation finally arrives the theme returns in the major, transformed from a question into something like an answer.
|
|
56
|
+
|
|
57
|
+
Volcanic islands form as a tectonic plate moves slowly over a stationary hotspot in the mantle. Magma rises through the crust and erupts on the sea floor, building a mountain that eventually breaks the surface. As the plate carries the island away from the hotspot the volcano goes extinct and erosion takes over, while a new island begins to grow behind it, producing a chain whose ages increase with distance.
|
|
58
|
+
|
|
59
|
+
Dear Ms Alvarez, thank you for your message of 14 March concerning the delayed shipment. We have located the consignment at the customs depot and expect it to be released within three working days. As a gesture of goodwill we will refund the express delivery charge to your original payment method. Please accept our apologies for the inconvenience, and do not hesitate to contact me directly with any further questions.
|
|
60
|
+
|
|
61
|
+
The film's first hour is its best, a patient portrait of a family business failing in slow motion, shot in long takes that let the silences do the arguing. The final act loses its nerve, reaching for a reconciliation the characters have not earned, but the performances hold it together, and the closing image of the empty shop at dawn lingers longer than the script deserves.
|
|
62
|
+
|
|
63
|
+
A hash table stores key-value pairs in an array of buckets. To insert a pair, the key is hashed to an index and the pair is placed in the corresponding bucket; collisions are handled by chaining or by probing for the next free slot. With a good hash function and a load factor kept below about three quarters, insertion and lookup take constant time on average, though the worst case remains linear.
|
|
64
|
+
|
|
65
|
+
The glacier has retreated more than two kilometres since the first photographs were taken in 1911. Researchers who drilled the ice last summer found meltwater flowing at the base far earlier in the season than models predicted, lubricating the glacier's slide toward the fjord. If the current rate holds, the tongue will detach from its terminal moraine within a decade, accelerating the loss further.
|
|
66
|
+
|
|
67
|
+
She learned the trade from her grandmother, who could tell by the smell of the vat when the indigo was ready. The cloth goes in green, and only when it meets the air does the blue arrive, deepening with each dip. Seven dips for work clothes, twelve for a wedding cloth. The chemistry is simple, she says, but the judgement is not, and the judgement is the part that takes ten years.
|
|
68
|
+
|
|
69
|
+
The probe entered orbit after a seven-month cruise, firing its main engine for twenty-three minutes to shed enough velocity for capture. Over the following weeks, controllers will lower the orbit using repeated passes through the upper atmosphere, a technique that saves fuel at the cost of patience. The first high-resolution images of the southern highlands are expected in early spring.
|
|
70
|
+
|
|
71
|
+
In chess, the endgame with king and pawn against king is the foundation on which everything else rests. The stronger side wins if the king can reach one of the key squares in front of the pawn; otherwise the defender draws by taking the opposition. A player who understands this single position, deeply rather than by rote, already understands more about the game than most who have memorised twenty openings.
|
|
72
|
+
|
|
73
|
+
The market opened lower after the overnight announcement, with the index down two percent in the first hour before recovering half the loss by midday. Trading volume was the heaviest in six weeks. Analysts were divided on whether the sell-off marked the start of a broader correction or a one-day reaction to news that had, in the words of one strategist, been priced in everywhere except the headlines.
|
|
74
|
+
|
|
75
|
+
Fermentation begins within hours of the harvest. Wild yeasts on the grape skins start converting sugar to alcohol, and the winemaker's first decision is whether to let them continue or to add a cultured strain with a more predictable temperament. Temperature is the other lever: cool and slow preserves the delicate aromatics, warm and fast extracts colour and tannin. Every choice closes some doors and opens others.
|
|
76
|
+
|
|
77
|
+
The immune system distinguishes self from non-self through a training process that takes place largely in the thymus. Developing T cells that bind too strongly to the body's own proteins are eliminated, while those that ignore them entirely are useless and die of neglect. The survivors walk a narrow line, reactive enough to recognise invaders, restrained enough to leave the body's own tissues alone.
|
|
78
|
+
|
|
79
|
+
Turn right at the old mill and follow the gravel road for about three kilometres until you reach a wooden bridge. Cross it, and the trailhead is on your left, marked with a blue sign. The path climbs steadily through beech forest for the first hour, then levels out along the ridge. Allow four hours for the round trip and carry water; the spring at the halfway point is dry by midsummer.
|
|
80
|
+
|
|
81
|
+
The printing press did not create the desire to read; it industrialised it. Within fifty years of Gutenberg, presses operated in more than two hundred European towns, and the price of a book fell by a factor of twenty. What had been the possession of monasteries and princes became the furniture of merchants' houses, and the arguments that followed, religious and political alike, were arguments among readers.
|
|
82
|
+
|
|
83
|
+
Loss of pressure in the forward cabin triggered the automatic descent, and the aircraft levelled at three thousand metres as designed. The crew worked through the checklist, declared an emergency, and diverted to the alternate field, landing forty minutes later without injury. The investigation traced the fault to a door seal that had been replaced two weeks earlier with a part from a different supplier.
|
|
84
|
+
|
|
85
|
+
For the winter solstice the village bakes a flat bread marked with a wheel of eight spokes, one for each of the year's festivals. The dough is enriched with honey and saffron, and the first loaf out of the oven is broken and shared on the doorstep, a piece for each person present and one crumbled on the threshold for the year itself. The custom survived two empires and a revolution, mostly because it is delicious.
|
|
86
|
+
|
|
87
|
+
The function of the kidney is filtration, but the organ works by a strategy of extravagant waste followed by careful reclamation. Each day the glomeruli filter some 180 litres of fluid from the blood, and the tubules reabsorb more than 99 percent of it, adjusting the recovery of water, salt and glucose to the body's needs. What remains, about a litre and a half, is excreted as urine.
|
|
88
|
+
|
|
89
|
+
Ticket 4821: after the latest deployment, the export button on the invoices page returns a 500 error for accounts with more than one thousand line items. Reproduced on staging with the demo account. The stack trace points to a timeout in the PDF renderer. Suggested fix: paginate the export or move rendering to the background queue and email the file when ready. Priority: high, affects three enterprise customers.
|
|
90
|
+
|
|
91
|
+
The desert looks empty at noon and crowded at dawn. Tracks in the sand record the night's business: the sidewinding trail of a viper, the neat stitching of a beetle, the sudden sweep where an owl's wing erased a rodent's story mid-sentence. By the time the heat rises the actors are underground, and the stage holds only their signatures, waiting for the wind to wipe it clean for the next performance.
|
|
92
|
+
|
|
93
|
+
Add the flour gradually while mixing on low speed, then increase to medium and knead for eight minutes until the dough pulls cleanly from the sides of the bowl. It should feel soft but not sticky; adjust with a spoonful of flour or water as needed. Cover and let it rise until doubled, about ninety minutes in a warm kitchen, then fold it gently over itself twice and shape it on a floured board.
|