veil-pii 0.3.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- veil_pii-0.3.1/.gitignore +22 -0
- veil_pii-0.3.1/CHANGELOG.md +107 -0
- veil_pii-0.3.1/LICENSE +21 -0
- veil_pii-0.3.1/PKG-INFO +409 -0
- veil_pii-0.3.1/README.md +370 -0
- veil_pii-0.3.1/pyproject.toml +121 -0
- veil_pii-0.3.1/src/veil/__init__.py +52 -0
- veil_pii-0.3.1/src/veil/audit.py +84 -0
- veil_pii-0.3.1/src/veil/backends/__init__.py +25 -0
- veil_pii-0.3.1/src/veil/backends/base.py +24 -0
- veil_pii-0.3.1/src/veil/backends/known_entities.py +67 -0
- veil_pii-0.3.1/src/veil/backends/spacy_backend.py +74 -0
- veil_pii-0.3.1/src/veil/cli.py +130 -0
- veil_pii-0.3.1/src/veil/detectors/__init__.py +61 -0
- veil_pii-0.3.1/src/veil/detectors/address.py +46 -0
- veil_pii-0.3.1/src/veil/detectors/base.py +24 -0
- veil_pii-0.3.1/src/veil/detectors/card.py +82 -0
- veil_pii-0.3.1/src/veil/detectors/dob.py +55 -0
- veil_pii-0.3.1/src/veil/detectors/email.py +50 -0
- veil_pii-0.3.1/src/veil/detectors/iban.py +72 -0
- veil_pii-0.3.1/src/veil/detectors/ip.py +67 -0
- veil_pii-0.3.1/src/veil/detectors/phone.py +128 -0
- veil_pii-0.3.1/src/veil/detectors/secrets.py +119 -0
- veil_pii-0.3.1/src/veil/detectors/ssn.py +68 -0
- veil_pii-0.3.1/src/veil/detectors/url.py +45 -0
- veil_pii-0.3.1/src/veil/integrations/__init__.py +8 -0
- veil_pii-0.3.1/src/veil/integrations/_common.py +124 -0
- veil_pii-0.3.1/src/veil/integrations/anthropic.py +235 -0
- veil_pii-0.3.1/src/veil/integrations/openai.py +293 -0
- veil_pii-0.3.1/src/veil/integrations/openai_responses.py +278 -0
- veil_pii-0.3.1/src/veil/masker.py +79 -0
- veil_pii-0.3.1/src/veil/py.typed +0 -0
- veil_pii-0.3.1/src/veil/restore.py +185 -0
- veil_pii-0.3.1/src/veil/surrogates.py +143 -0
- veil_pii-0.3.1/src/veil/types.py +79 -0
- veil_pii-0.3.1/src/veil/vault.py +191 -0
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
__pycache__/
|
|
2
|
+
*.py[cod]
|
|
3
|
+
*.egg-info/
|
|
4
|
+
.venv/
|
|
5
|
+
.pytest_cache/
|
|
6
|
+
.mypy_cache/
|
|
7
|
+
.ruff_cache/
|
|
8
|
+
.hypothesis/
|
|
9
|
+
.coverage
|
|
10
|
+
htmlcov/
|
|
11
|
+
dist/
|
|
12
|
+
build/
|
|
13
|
+
node_modules/
|
|
14
|
+
docs/site/
|
|
15
|
+
*.log
|
|
16
|
+
.DS_Store
|
|
17
|
+
|
|
18
|
+
# Generated, not committed: it's reproducible from a fixed seed (see
|
|
19
|
+
# README "Measured results"), and its secret-detector fixtures are
|
|
20
|
+
# synthetic strings shaped enough like real API keys that public secret
|
|
21
|
+
# scanners flag committed copies as leaked credentials.
|
|
22
|
+
benchmarks/*.jsonl
|
|
@@ -0,0 +1,107 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project are documented in this file.
|
|
4
|
+
The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
5
|
+
and this project adheres to [Semantic Versioning](https://semver.org/).
|
|
6
|
+
|
|
7
|
+
## [0.3.1] - 2026-10-01
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
|
|
11
|
+
- Published to PyPI as `veil-pii`: `pip install veil-pii`, with the extras
|
|
12
|
+
`veil-pii[anthropic]`, `[openai]`, `[spacy]` and `[vault-crypto]`. The README's
|
|
13
|
+
images and links are rewritten to absolute URLs at build time so they work
|
|
14
|
+
on the project page.
|
|
15
|
+
- `veil --version`.
|
|
16
|
+
|
|
17
|
+
## [0.3.0] - 2026-09-30
|
|
18
|
+
|
|
19
|
+
### Added
|
|
20
|
+
|
|
21
|
+
- OpenAI Responses API support: `OpenAIVeil.create_response` and
|
|
22
|
+
`stream_response` mask `instructions` and every message, `function_call`,
|
|
23
|
+
`function_call_output` and custom tool item in `input`, and restore message
|
|
24
|
+
text, refusals, tool-call arguments and reasoning summaries in `output`
|
|
25
|
+
(`response.output_text` included). Stream deltas are restored as they
|
|
26
|
+
arrive; output items fed back as `input` are replayed exactly as the model
|
|
27
|
+
produced them. Tested against the `openai` SDK's own response and event
|
|
28
|
+
types.
|
|
29
|
+
|
|
30
|
+
### Fixed
|
|
31
|
+
|
|
32
|
+
- At the end of a Chat Completions stream, text held back because it might
|
|
33
|
+
begin a surrogate came out as a plain dict, so code reading
|
|
34
|
+
`chunk.choices[0].delta.content` failed on it. It is now a copy of the last
|
|
35
|
+
real chunk, with only the held-back text in its delta.
|
|
36
|
+
|
|
37
|
+
## [0.2.0] - 2026-09-30
|
|
38
|
+
|
|
39
|
+
### Security
|
|
40
|
+
|
|
41
|
+
- `AnthropicVeil` sent real values back to the API: the `tool_use` input it
|
|
42
|
+
restored for the application to execute was not re-masked when the
|
|
43
|
+
assistant turn was resent as history. `tool_use` input is now masked, and
|
|
44
|
+
both wrappers replay each assistant turn they restored exactly as the model
|
|
45
|
+
produced it (`ReplayCache`), so restored values never leave in a later
|
|
46
|
+
request.
|
|
47
|
+
|
|
48
|
+
- A PEM private key (`-----BEGIN ... PRIVATE KEY-----` through its END line,
|
|
49
|
+
or through its body when truncated) was not masked at all; it is now one
|
|
50
|
+
`SECRET` entity.
|
|
51
|
+
- Credentials in non-HTTP URLs were missed: in
|
|
52
|
+
`postgres://admin:hunter2@db.internal/app` only `hunter2@db.internal` was
|
|
53
|
+
caught, as an email address. Userinfo credentials are now detected in URLs
|
|
54
|
+
of any scheme (`postgres`, `redis`, including the password-only
|
|
55
|
+
`redis://:secret@host`, `mongodb+srv`, `amqp`, ...).
|
|
56
|
+
|
|
57
|
+
### Fixed
|
|
58
|
+
|
|
59
|
+
- A dotted quad labelled as a version (`version 10.2.14.3`, `v2.4.1.0`,
|
|
60
|
+
`build 1.0.0.7`) was masked as an IPv4 address.
|
|
61
|
+
- Resent assistant turns no longer differ from what the model produced
|
|
62
|
+
(re-masking a restored turn gave model-written, PII-looking text a fresh
|
|
63
|
+
surrogate). A changed earlier turn restarts the prompt cache and, on
|
|
64
|
+
current Claude models, invalidates the signatures of later thinking blocks.
|
|
65
|
+
- OpenAI tool-call arguments whose surrogate the model wrote with `\uXXXX`
|
|
66
|
+
escapes were not restored. Complete arguments are parsed and restored value
|
|
67
|
+
by value; streamed fragments go through a JSON-string-mode `Restorer` that
|
|
68
|
+
recognizes the escaped form and escapes restored originals, so the
|
|
69
|
+
arguments stay valid JSON.
|
|
70
|
+
- Install hints pointed at `veil-pii` on PyPI, where the package isn't
|
|
71
|
+
published; they now use the Git URL or name the actual dependency.
|
|
72
|
+
|
|
73
|
+
### Added
|
|
74
|
+
|
|
75
|
+
- `veil.restore.restore_json_text` and `Restorer(vault, json_string=True)`.
|
|
76
|
+
- `veil.integrations._common.ReplayCache` (bounded, 10,000 entries by default).
|
|
77
|
+
|
|
78
|
+
## [0.1.0] - 2026-09-24
|
|
79
|
+
|
|
80
|
+
### Added
|
|
81
|
+
|
|
82
|
+
- Core detectors: email, phone (NANP or `+`-prefixed international only,
|
|
83
|
+
with a context guard against order/invoice/ticket/ZIP numbers), payment
|
|
84
|
+
card (Luhn + issuer ranges), IBAN (mod-97), US SSN (consistent
|
|
85
|
+
separators required, to avoid matching a ZIP+4 code), IPv4/IPv6, URLs
|
|
86
|
+
with embedded credentials, API keys/secrets (known prefixes + entropy
|
|
87
|
+
heuristic), date-of-birth (context-gated), and an opt-in US street
|
|
88
|
+
address heuristic.
|
|
89
|
+
- Pluggable name/org/location backends: a first-class "known entities"
|
|
90
|
+
backend and an optional spaCy backend.
|
|
91
|
+
- Configurable surrogates: placeholder tokens and realistic
|
|
92
|
+
format-preserving fakes (names, `example.com` emails, 555-01xx phone
|
|
93
|
+
numbers, Luhn-valid test card numbers).
|
|
94
|
+
- Exact and tolerant restore, plus a streaming, trie-based `Restorer`
|
|
95
|
+
proven equivalent to non-streamed restore under arbitrary chunking via
|
|
96
|
+
Hypothesis property tests.
|
|
97
|
+
- Leak audit: detects original values that leaked into outgoing text and
|
|
98
|
+
new PII a model introduced.
|
|
99
|
+
- In-memory vault with JSON serialization and optional Fernet encryption.
|
|
100
|
+
- `veil` CLI: `mask`, `restore`, `audit`.
|
|
101
|
+
- Thin Anthropic and OpenAI SDK wrapper integrations, including streaming.
|
|
102
|
+
- Synthetic, labelled evaluation corpus and per-detector precision/recall
|
|
103
|
+
report, plus a benign-text category measuring false positives per 1,000
|
|
104
|
+
words on ordinary numeric text (dates, order/invoice/ticket IDs, ZIP+4,
|
|
105
|
+
tracking numbers, prices, room/build numbers).
|
|
106
|
+
- Browser demo (`web/`, Vite + TypeScript) running the real package via
|
|
107
|
+
Pyodide, deployed to GitHub Pages.
|
veil_pii-0.3.1/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Anton Soloviev
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
veil_pii-0.3.1/PKG-INFO
ADDED
|
@@ -0,0 +1,409 @@
|
|
|
1
|
+
Metadata-Version: 2.5
|
|
2
|
+
Name: veil-pii
|
|
3
|
+
Version: 0.3.1
|
|
4
|
+
Summary: Reversible PII masking for LLM calls: mask before the prompt leaves your network, restore in the answer, streaming included.
|
|
5
|
+
Project-URL: Homepage, https://antonsoo.github.io/veil/
|
|
6
|
+
Project-URL: Repository, https://github.com/antonsoo/veil
|
|
7
|
+
Project-URL: Issues, https://github.com/antonsoo/veil/issues
|
|
8
|
+
Project-URL: Changelog, https://github.com/antonsoo/veil/blob/main/CHANGELOG.md
|
|
9
|
+
Author-email: Anton Soloviev <anton@praviel.com>
|
|
10
|
+
License-Expression: MIT
|
|
11
|
+
License-File: LICENSE
|
|
12
|
+
Keywords: anonymization,data-masking,gdpr,llm,pii,privacy,pseudonymization,redaction
|
|
13
|
+
Classifier: Development Status :: 4 - Beta
|
|
14
|
+
Classifier: Intended Audience :: Developers
|
|
15
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
16
|
+
Classifier: Programming Language :: Python :: 3
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
19
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
20
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
21
|
+
Classifier: Topic :: Security
|
|
22
|
+
Classifier: Topic :: Software Development :: Libraries :: Python Modules
|
|
23
|
+
Classifier: Typing :: Typed
|
|
24
|
+
Requires-Python: >=3.10
|
|
25
|
+
Provides-Extra: all
|
|
26
|
+
Requires-Dist: anthropic>=0.40; extra == 'all'
|
|
27
|
+
Requires-Dist: cryptography>=42; extra == 'all'
|
|
28
|
+
Requires-Dist: openai>=1.50; extra == 'all'
|
|
29
|
+
Requires-Dist: spacy>=3.7; extra == 'all'
|
|
30
|
+
Provides-Extra: anthropic
|
|
31
|
+
Requires-Dist: anthropic>=0.40; extra == 'anthropic'
|
|
32
|
+
Provides-Extra: openai
|
|
33
|
+
Requires-Dist: openai>=1.50; extra == 'openai'
|
|
34
|
+
Provides-Extra: spacy
|
|
35
|
+
Requires-Dist: spacy>=3.7; extra == 'spacy'
|
|
36
|
+
Provides-Extra: vault-crypto
|
|
37
|
+
Requires-Dist: cryptography>=42; extra == 'vault-crypto'
|
|
38
|
+
Description-Content-Type: text/markdown
|
|
39
|
+
|
|
40
|
+
# veil
|
|
41
|
+
|
|
42
|
+
**Reversible PII masking for LLM calls.** Mask personal data before the
|
|
43
|
+
prompt leaves your network, let the model reason over consistent
|
|
44
|
+
surrogates, and restore the originals in the answer — streaming included.
|
|
45
|
+
|
|
46
|
+
[](https://github.com/antonsoo/veil/blob/main/LICENSE)
|
|
47
|
+
[](https://antonsoo.github.io/veil/)
|
|
48
|
+
[](https://huggingface.co/spaces/antonsoloviev/veil)
|
|
49
|
+
|
|
50
|
+

|
|
51
|
+
|
|
52
|
+
**[Try the live demo →](https://antonsoo.github.io/veil/)** — runs the
|
|
53
|
+
real Python package in your browser via [Pyodide](https://pyodide.org),
|
|
54
|
+
nothing leaves the page. (`web/`; see [Web demo](#web-demo) below.)
|
|
55
|
+
|
|
56
|
+
## Why this exists
|
|
57
|
+
|
|
58
|
+
Teams in healthcare, legal, finance, and support want to use hosted LLMs,
|
|
59
|
+
but their prompts contain customer names, emails, phone numbers, card
|
|
60
|
+
numbers, account IDs, and API keys. The usual answer is "redact it" —
|
|
61
|
+
replace the value with `[REDACTED]`. That breaks the response: the model
|
|
62
|
+
can't say "Dear [REDACTED], your order [REDACTED] ships tomorrow."
|
|
63
|
+
|
|
64
|
+
Reversible pseudonymization fixes that instead: replace each sensitive
|
|
65
|
+
value with a consistent surrogate, let the model reason over the
|
|
66
|
+
surrogates, and map them back in the response. The model never sees the
|
|
67
|
+
real data; the human on the other end never sees a redaction. The part
|
|
68
|
+
almost nobody handles is doing that restoration on a *stream* of tokens,
|
|
69
|
+
where a surrogate can be split across chunk boundaries — that's most of
|
|
70
|
+
what this repository is about.
|
|
71
|
+
|
|
72
|
+
## How it works
|
|
73
|
+
|
|
74
|
+
```mermaid
|
|
75
|
+
flowchart LR
|
|
76
|
+
A["Prompt\n(has real PII)"] -->|"Masker.mask()"| B["Masked prompt\n(surrogates only)"]
|
|
77
|
+
B --> C["LLM"]
|
|
78
|
+
C --> D["Response\n(surrogates only)"]
|
|
79
|
+
D -->|"Masker.restore()\nor Restorer.feed()"| E["Restored response\n(real PII back)"]
|
|
80
|
+
B -.->|"stores mapping"| V[("Vault")]
|
|
81
|
+
E -.->|"looks up mapping"| V
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
1. **Detect.** Regex-and-validation detectors (Luhn, mod-97, structural
|
|
85
|
+
checks) find emails, phone numbers, cards, IBANs, SSNs, IPs,
|
|
86
|
+
credential-bearing URLs, API keys, and dates of birth. A pluggable
|
|
87
|
+
backend finds names/orgs/locations — see "Names, orgs, and locations"
|
|
88
|
+
under [Features](#features) below.
|
|
89
|
+
2. **Mask.** Each detected value gets a surrogate — either a placeholder
|
|
90
|
+
token (`⟨EMAIL_1⟩`) or a realistic fake (`user1@example.com`) — and the
|
|
91
|
+
`(original, surrogate)` pair is recorded in a `Vault`. The same
|
|
92
|
+
original always maps to the same surrogate for the life of the vault.
|
|
93
|
+
3. **Send.** The masked text goes to the LLM. It never sees the real
|
|
94
|
+
values.
|
|
95
|
+
4. **Restore.** The response comes back full of surrogates. `restore_exact`
|
|
96
|
+
/ `restore_tolerant` swap them back for a buffered response;
|
|
97
|
+
`Restorer.feed()` does it token-by-token for a stream.
|
|
98
|
+
|
|
99
|
+
### The streaming problem, concretely
|
|
100
|
+
|
|
101
|
+
Say the vault maps `⟨EMAIL_1⟩` to `alice@example.com`, and the model
|
|
102
|
+
streams its reply in small chunks:
|
|
103
|
+
|
|
104
|
+
```
|
|
105
|
+
chunk 1: "Sure, I'll email ⟨EM"
|
|
106
|
+
chunk 2: "AIL_1⟩ right away."
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
A restorer that looks at each chunk in isolation emits `⟨EM` verbatim (the
|
|
110
|
+
placeholder leaks into what the user sees) and then has no memory of it
|
|
111
|
+
when `AIL_1⟩` arrives in the next chunk. `veil.restore.Restorer` instead
|
|
112
|
+
recognizes that `⟨EM` is a valid *prefix* of a known surrogate, holds back
|
|
113
|
+
only that trailing fragment, and completes the match once `AIL_1⟩`
|
|
114
|
+
arrives — emitting `alice@example.com right away.` with nothing leaked
|
|
115
|
+
and no extra latency beyond the one held-back fragment. It's built on a
|
|
116
|
+
trie over the vault's surrogate strings (see `veil/restore.py`), and
|
|
117
|
+
Hypothesis property tests prove that *any* way you split a given text
|
|
118
|
+
into chunks produces output identical to restoring it unsplit
|
|
119
|
+
(`tests/test_restore_streaming.py`).
|
|
120
|
+
|
|
121
|
+
## Quickstart
|
|
122
|
+
|
|
123
|
+
```bash
|
|
124
|
+
pip install veil-pii
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
```python
|
|
128
|
+
from veil import Masker
|
|
129
|
+
|
|
130
|
+
masker = Masker()
|
|
131
|
+
masked = masker.mask("Email alice@example.com about invoice #4471.")
|
|
132
|
+
print(masked) # "Email ⟨EMAIL_1⟩ about invoice #4471."
|
|
133
|
+
|
|
134
|
+
# ... send `masked` to your LLM of choice, get `reply` back ...
|
|
135
|
+
reply = "Sure, I've noted it for ⟨EMAIL_1⟩."
|
|
136
|
+
print(masker.restore(reply)) # "Sure, I've noted it for alice@example.com."
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
### With the Anthropic SDK
|
|
140
|
+
|
|
141
|
+
```bash
|
|
142
|
+
pip install "veil-pii[anthropic]"
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
```python
|
|
146
|
+
from anthropic import Anthropic
|
|
147
|
+
from veil.integrations.anthropic import AnthropicVeil
|
|
148
|
+
|
|
149
|
+
client = AnthropicVeil(Anthropic())
|
|
150
|
+
|
|
151
|
+
response = client.create(
|
|
152
|
+
model="claude-opus-5-5",
|
|
153
|
+
max_tokens=1024,
|
|
154
|
+
messages=[{"role": "user", "content": "Email alice@example.com about invoice #4471."}],
|
|
155
|
+
)
|
|
156
|
+
print(response.content[0].text) # the real address, restored — Claude only ever saw a surrogate
|
|
157
|
+
|
|
158
|
+
# Streaming:
|
|
159
|
+
with client.stream(model="claude-opus-5-5", max_tokens=1024, messages=[...]) as stream:
|
|
160
|
+
for text in stream.text_stream: # already restored, split-token-safe
|
|
161
|
+
print(text, end="", flush=True)
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
### With the OpenAI SDK
|
|
165
|
+
|
|
166
|
+
`veil.integrations.openai.OpenAIVeil` (the `openai` extra) wraps both of
|
|
167
|
+
OpenAI's APIs: `create` / `stream` for Chat Completions, and
|
|
168
|
+
`create_response` / `stream_response` for the Responses API.
|
|
169
|
+
|
|
170
|
+
```python
|
|
171
|
+
from openai import OpenAI
|
|
172
|
+
from veil.integrations.openai import OpenAIVeil
|
|
173
|
+
|
|
174
|
+
client = OpenAIVeil(OpenAI())
|
|
175
|
+
|
|
176
|
+
response = client.create_response(
|
|
177
|
+
model="gpt-6-sol",
|
|
178
|
+
instructions="You draft customer emails.",
|
|
179
|
+
input="Email alice@example.com about invoice #4471.",
|
|
180
|
+
)
|
|
181
|
+
print(response.output_text) # restored; the model only saw ⟨EMAIL_1⟩
|
|
182
|
+
|
|
183
|
+
for event in client.stream_response(model="gpt-6-sol", input=[...]):
|
|
184
|
+
if event.type == "response.output_text.delta":
|
|
185
|
+
print(event.delta, end="", flush=True) # restored, split-token-safe
|
|
186
|
+
```
|
|
187
|
+
|
|
188
|
+
For the Responses API, veil masks `instructions` and every message,
|
|
189
|
+
`function_call`, `function_call_output` and custom tool item in `input`. It
|
|
190
|
+
restores message text, refusals, tool-call arguments and reasoning summaries
|
|
191
|
+
in `output`, and `response.output_text` with them, since the SDK computes it
|
|
192
|
+
from `output`. Reasoning items pass through untouched. In a stream, text and
|
|
193
|
+
argument deltas are restored as they arrive, and any text held back because
|
|
194
|
+
it might begin a surrogate is released, as one more delta event, before the
|
|
195
|
+
matching `.done` event. With `previous_response_id` the server keeps the
|
|
196
|
+
masked history, so keep the same `OpenAIVeil` (and its vault) for the whole
|
|
197
|
+
conversation. Tested against the `openai` SDK's own response and event types.
|
|
198
|
+
|
|
199
|
+
### Multi-turn conversations and tool calls
|
|
200
|
+
|
|
201
|
+
Keep your history the normal way: append the assistant turn veil handed
|
|
202
|
+
you (real values restored, including in the `tool_use` input your code is
|
|
203
|
+
about to execute) and send the whole list on the next call. The wrapper
|
|
204
|
+
remembers the exact blocks the model produced for everything it restored,
|
|
205
|
+
and swaps them back in before the request leaves, so:
|
|
206
|
+
|
|
207
|
+
- **No real value goes back to the provider.** Restored tool-call
|
|
208
|
+
arguments are the easy thing to leak: they are real by design, because
|
|
209
|
+
your code has to act on them.
|
|
210
|
+
- **The history the API sees never changes.** Re-masking a restored turn
|
|
211
|
+
only approximates the original (a support address the model wrote
|
|
212
|
+
itself looks like PII and would get a fresh surrogate). Any difference
|
|
213
|
+
is an edit to an earlier turn: the prompt cache restarts from there, and
|
|
214
|
+
on current Claude models every later thinking block's signature stops
|
|
215
|
+
matching its conversation, which accounts that enforce the check reject
|
|
216
|
+
with a 400. Thinking blocks are never masked or restored for the same
|
|
217
|
+
reason.
|
|
218
|
+
|
|
219
|
+
A turn veil didn't produce (or one you edited) is masked like any other
|
|
220
|
+
message. OpenAI tool-call arguments arrive as a JSON string; veil parses
|
|
221
|
+
them before restoring, so a surrogate the model wrote with `\u27e8`-style
|
|
222
|
+
escapes is still found, and restored values are escaped so the arguments
|
|
223
|
+
stay valid JSON.
|
|
224
|
+
|
|
225
|
+
## Features
|
|
226
|
+
|
|
227
|
+
- **Detectors** (`veil.detectors`): email, phone (NANP, or international
|
|
228
|
+
with an explicit `+` country code — no unprefixed generic fallback; see
|
|
229
|
+
the module docstring for why),
|
|
230
|
+
payment card (Luhn + issuer ranges), IBAN (mod-97 + per-country length),
|
|
231
|
+
US SSN (excluding never-issued ranges), IPv4/IPv6 (a dotted quad labelled
|
|
232
|
+
as a version is left alone), credential-bearing URLs of any scheme
|
|
233
|
+
(including `postgres://`/`redis://`-style connection strings), PEM private
|
|
234
|
+
key blocks, API keys/secrets (`AKIA…`, `ghp_…`, `github_pat_…`, `xox…`,
|
|
235
|
+
`sk_live_…`, `sk-ant-…`, `sk-…`, plus a Shannon-entropy heuristic),
|
|
236
|
+
context-gated date-of-birth, and an opt-in US street-address heuristic.
|
|
237
|
+
- **Names, orgs, and locations** via a pluggable `NameBackend`: a
|
|
238
|
+
first-class `KnownEntitiesBackend` (you already know the customer's name
|
|
239
|
+
from your own database — this is the most reliable option in practice)
|
|
240
|
+
and an optional `SpacyBackend`.
|
|
241
|
+
- **Surrogates**: placeholder tokens or realistic format-preserving fakes
|
|
242
|
+
(fake names, `example.com`/`example.org` emails per RFC 2606, `555-01xx`
|
|
243
|
+
NANP phone numbers, Luhn-valid test-range card numbers), consistent
|
|
244
|
+
within a session.
|
|
245
|
+
- **Restore**: exact (byte-for-byte) and tolerant (case changes,
|
|
246
|
+
possessives, a surrogate split across a line wrap) — see
|
|
247
|
+
`veil/restore.py` for exactly what "tolerant" does and doesn't cover.
|
|
248
|
+
- **Streaming restore**: `Restorer.feed(chunk) -> str` / `.flush() -> str`,
|
|
249
|
+
proven equal to non-streamed restore under arbitrary chunking.
|
|
250
|
+
- **Leak audit** (`veil.audit`): checks outgoing text for original values
|
|
251
|
+
that should never reappear, and flags *new* PII the model introduced.
|
|
252
|
+
- **Vault**: in-memory by default, JSON-serializable, optional Fernet
|
|
253
|
+
encryption at rest (`veil-pii[vault-crypto]`) — see the threat model in
|
|
254
|
+
`veil/vault.py`.
|
|
255
|
+
- **CLI**: `veil mask`, `veil restore`, `veil audit`.
|
|
256
|
+
- **SDK integrations**: thin wrappers for the Anthropic Messages API and
|
|
257
|
+
OpenAI's Chat Completions and Responses APIs that mask every outgoing
|
|
258
|
+
message (tool calls included), restore responses and streams, and replay
|
|
259
|
+
earlier assistant turns exactly as the model produced them.
|
|
260
|
+
- **Zero runtime dependencies in the core.** Everything above the
|
|
261
|
+
detectors/surrogates/restore/vault/CLI layer is an optional extra.
|
|
262
|
+
|
|
263
|
+
## CLI
|
|
264
|
+
|
|
265
|
+
```bash
|
|
266
|
+
$ veil mask examples/support_ticket.txt --vault vault.json
|
|
267
|
+
Subject: Can't access my account
|
|
268
|
+
...
|
|
269
|
+
My account email is ⟨EMAIL_1⟩ and my phone is ⟨PHONE_1⟩. I tried to
|
|
270
|
+
update the card on file (⟨CARD_1⟩) but the charge failed.
|
|
271
|
+
...
|
|
272
|
+
|
|
273
|
+
$ veil restore masked_reply.txt --vault vault.json
|
|
274
|
+
$ veil audit outgoing_reply.txt --vault vault.json # exit 1 if anything leaked
|
|
275
|
+
```
|
|
276
|
+
|
|
277
|
+

|
|
278
|
+
|
|
279
|
+
## Web demo
|
|
280
|
+
|
|
281
|
+
[antonsoo.github.io/veil](https://antonsoo.github.io/veil/) runs the
|
|
282
|
+
*actual* `veil` Python package in the browser via
|
|
283
|
+
[Pyodide](https://pyodide.org) (loaded from cdn.jsdelivr.net) — the
|
|
284
|
+
`web/scripts/copy-veil-src.mjs` build step bundles `src/veil`'s real
|
|
285
|
+
source (zero runtime dependencies makes this possible with no wheel
|
|
286
|
+
build) so the demo is never a JS reimplementation drifting from the
|
|
287
|
+
library. Paste a prompt, optionally list names your app already knows
|
|
288
|
+
(wired to `KnownEntitiesBackend`), mask it, and watch a simulated reply
|
|
289
|
+
stream back through the real `Restorer`, one random-sized chunk at a
|
|
290
|
+
time — dark mode:
|
|
291
|
+
|
|
292
|
+

|
|
293
|
+
|
|
294
|
+
Everything — Pyodide, the package source, the whole interaction — stays
|
|
295
|
+
in that browser tab; nothing is sent anywhere, which the page itself
|
|
296
|
+
says. Built with Vite + TypeScript in `web/`; `npm run dev` there for
|
|
297
|
+
local development.
|
|
298
|
+
|
|
299
|
+
## Measured results
|
|
300
|
+
|
|
301
|
+
Measured on this machine (14 vCPU WSL2 Linux, 48 GB RAM) by running
|
|
302
|
+
`scripts/evaluate.py` against `benchmarks/corpus.jsonl` — a 240-document,
|
|
303
|
+
6-category **synthetic** corpus generated by `scripts/generate_corpus.py`
|
|
304
|
+
with a fixed seed (reproduce with
|
|
305
|
+
`uv run python scripts/generate_corpus.py 40 > benchmarks/corpus.jsonl`).
|
|
306
|
+
|
|
307
|
+
| Detector | Precision | Recall | F1 |
|
|
308
|
+
|---|---|---|---|
|
|
309
|
+
| Address (heuristic) | 1.000 | 1.000 | 1.000 |
|
|
310
|
+
| Card | 1.000 | 1.000 | 1.000 |
|
|
311
|
+
| DOB | 1.000 | 1.000 | 1.000 |
|
|
312
|
+
| Email | 1.000 | 1.000 | 1.000 |
|
|
313
|
+
| IBAN | 1.000 | 1.000 | 1.000 |
|
|
314
|
+
| IPv4 | 1.000 | 1.000 | 1.000 |
|
|
315
|
+
| Phone | 1.000 | 1.000 | 1.000 |
|
|
316
|
+
| Secret | 1.000 | 1.000 | 1.000 |
|
|
317
|
+
| SSN | 1.000 | 1.000 | 1.000 |
|
|
318
|
+
| **Total** | **1.000** | **1.000** | **1.000** |
|
|
319
|
+
|
|
320
|
+
**Benign false positives: 0.00 per 1,000 words**, measured over the
|
|
321
|
+
corpus's `clean_negative` and `benign_numbers` categories (80 documents,
|
|
322
|
+
4,155 words of ordinary business text with zero PII, deliberately
|
|
323
|
+
saturated with the numeric shapes that most look like phone numbers or
|
|
324
|
+
SSNs: ISO/DMY dates, times, `#order-1234` and `INV-`/ticket IDs, ZIP+4,
|
|
325
|
+
carrier tracking numbers, prices, room numbers, and version/build
|
|
326
|
+
strings — the exact categories a naive digit-group detector over-masks).
|
|
327
|
+
0.00 here means specifically "no false positive on *these* known-tricky
|
|
328
|
+
shapes", not "no false positives on arbitrary text" — see below.
|
|
329
|
+
|
|
330
|
+
**Masking throughput:** ~1.6-2.1 MB/s (single-threaded, all nine
|
|
331
|
+
detectors run on every document; varies run to run — see `elapsed_s` in
|
|
332
|
+
the script's output).
|
|
333
|
+
|
|
334
|
+
**Streaming-restore overhead:** within roughly ±20% of non-streamed
|
|
335
|
+
restore at a 24-character chunk size, on the same corpus — noise-level on
|
|
336
|
+
this machine, not a meaningful cost.
|
|
337
|
+
|
|
338
|
+
**Read the "1.000" row honestly** — see [Accuracy and
|
|
339
|
+
limitations](#accuracy-and-limitations). This corpus was built to
|
|
340
|
+
independently verify each detector's *validated* claims (a real Luhn
|
|
341
|
+
check, a real mod-97 check, real NANP/E.164 structure, real never-issued
|
|
342
|
+
SSN ranges — see [How it works](#how-it-works)), so a clean score mostly
|
|
343
|
+
means those checks are implemented correctly, not that real-world text is
|
|
344
|
+
this easy. The corpus generator and the benign-numbers check both found
|
|
345
|
+
and fixed real precision bugs during development — most recently the
|
|
346
|
+
phone detector masking an ISO date and an order number as phone numbers
|
|
347
|
+
in ordinary support text, and the SSN detector matching a ZIP+4 code
|
|
348
|
+
(`22156-7224` parses as area+group+serial if separators aren't required
|
|
349
|
+
to be consistent). See the git history for `fix: detector false
|
|
350
|
+
positives found via synthetic corpus evaluation` and the phone/SSN
|
|
351
|
+
over-masking fix that followed it.
|
|
352
|
+
|
|
353
|
+
## Accuracy and limitations
|
|
354
|
+
|
|
355
|
+
- **Detection is never perfect.** Every regex-and-validation detector here
|
|
356
|
+
has a documented failure mode in its own module docstring (e.g.
|
|
357
|
+
`veil/detectors/phone.py` explains exactly which phone formats it
|
|
358
|
+
deliberately won't match, and why). Read those before relying on this
|
|
359
|
+
for a compliance-sensitive workload.
|
|
360
|
+
- **Names need a backend.** Out of the box, veil does not detect person,
|
|
361
|
+
organization, or location names — those require
|
|
362
|
+
`KnownEntitiesBackend` (recommended: you almost always already know the
|
|
363
|
+
customer's identity) or an optional, *unbenchmarked* spaCy backend. See
|
|
364
|
+
`veil/backends/spacy_backend.py` for why we don't claim an NER accuracy
|
|
365
|
+
number we haven't measured.
|
|
366
|
+
- **US-centric.** Phone (NANP), SSN, and the address heuristic assume
|
|
367
|
+
US formats primarily; IBAN and E.164 phone cover international cases,
|
|
368
|
+
but there's no general international address, national-ID, or
|
|
369
|
+
VAT-number detector yet.
|
|
370
|
+
- **Realistic surrogates can leak structure.** A format-preserving fake
|
|
371
|
+
IBAN still reveals the real one's country; a fake card still reveals
|
|
372
|
+
its network. See the trade-off discussion in `veil/surrogates.py`.
|
|
373
|
+
- **The vault is the whole game.** Anyone who reads the vault can
|
|
374
|
+
de-anonymize the masked text — see the threat model in `veil/vault.py`
|
|
375
|
+
before deciding where (or whether) to persist one.
|
|
376
|
+
- **Synthetic benchmarks overstate real-world performance.** The corpus
|
|
377
|
+
above is clean, well-formatted, English-language, and generated by the
|
|
378
|
+
same kind of logic the detectors use to validate — it is a correctness
|
|
379
|
+
check, not a claim about messy real-world text (typos, non-US formats,
|
|
380
|
+
mixed languages, OCR noise).
|
|
381
|
+
|
|
382
|
+
## Development
|
|
383
|
+
|
|
384
|
+
```bash
|
|
385
|
+
git clone https://github.com/antonsoo/veil
|
|
386
|
+
cd veil
|
|
387
|
+
uv sync --group dev
|
|
388
|
+
uv run pytest # 194 tests, including Hypothesis property tests
|
|
389
|
+
uv run ruff check . # lint
|
|
390
|
+
uv run mypy # typecheck (strict)
|
|
391
|
+
```
|
|
392
|
+
|
|
393
|
+
Correctness is checked against independent oracles where one exists: the
|
|
394
|
+
Luhn checksum for cards, ISO 7064 mod-97 for IBAN (verified against
|
|
395
|
+
published Wikipedia example IBANs for GB/DE/FR), and the stdlib
|
|
396
|
+
`ipaddress` module for IP parsing — see the module docstrings in
|
|
397
|
+
`veil/detectors/` for details, and `tests/` for the corresponding tests.
|
|
398
|
+
|
|
399
|
+
## Contributing
|
|
400
|
+
|
|
401
|
+
See [CONTRIBUTING.md](https://github.com/antonsoo/veil/blob/main/CONTRIBUTING.md).
|
|
402
|
+
|
|
403
|
+
## License
|
|
404
|
+
|
|
405
|
+
[MIT](https://github.com/antonsoo/veil/blob/main/LICENSE) © 2026 Anton Soloviev
|
|
406
|
+
|
|
407
|
+
---
|
|
408
|
+
|
|
409
|
+
<sub>Part of [Officina](https://antonsoo.github.io/officina/), a set of small open-source tools by [Anton Soloviev](https://github.com/antonsoo).</sub>
|