localmask 0.9.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- localmask-0.9.0/PKG-INFO +356 -0
- localmask-0.9.0/README.md +330 -0
- localmask-0.9.0/cli.py +2437 -0
- localmask-0.9.0/licensing.py +486 -0
- localmask-0.9.0/localmask/__init__.py +14 -0
- localmask-0.9.0/localmask/_edition.py +108 -0
- localmask-0.9.0/localmask/ask_local.py +126 -0
- localmask-0.9.0/localmask/engine.py +1125 -0
- localmask-0.9.0/localmask/gitops.py +368 -0
- localmask-0.9.0/localmask/masking.py +122 -0
- localmask-0.9.0/localmask/ner_patterns.json +256 -0
- localmask-0.9.0/localmask/publish.py +57 -0
- localmask-0.9.0/localmask/regex_patterns.json +1882 -0
- localmask-0.9.0/localmask/state.py +270 -0
- localmask-0.9.0/localmask/vault.py +50 -0
- localmask-0.9.0/localmask/vault_store.py +485 -0
- localmask-0.9.0/localmask.egg-info/PKG-INFO +356 -0
- localmask-0.9.0/localmask.egg-info/SOURCES.txt +26 -0
- localmask-0.9.0/localmask.egg-info/dependency_links.txt +1 -0
- localmask-0.9.0/localmask.egg-info/entry_points.txt +3 -0
- localmask-0.9.0/localmask.egg-info/requires.txt +3 -0
- localmask-0.9.0/localmask.egg-info/top_level.txt +7 -0
- localmask-0.9.0/mcp_server.py +872 -0
- localmask-0.9.0/ner_scanner.py +257 -0
- localmask-0.9.0/pyproject.toml +46 -0
- localmask-0.9.0/regex_rules_safe.py +436 -0
- localmask-0.9.0/server_core.py +790 -0
- localmask-0.9.0/setup.cfg +4 -0
localmask-0.9.0/PKG-INFO
ADDED
|
@@ -0,0 +1,356 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: localmask
|
|
3
|
+
Version: 0.9.0
|
|
4
|
+
Summary: Privacy-first secret & PII masking for code — 100% local, no phone-home.
|
|
5
|
+
Author-email: Shai Gury <hello@localmaskpro.com>
|
|
6
|
+
License-Expression: MIT
|
|
7
|
+
Project-URL: Homepage, https://www.localmaskpro.com
|
|
8
|
+
Project-URL: Source, https://github.com/sgury/localmask
|
|
9
|
+
Project-URL: Issues, https://github.com/sgury/localmask/issues
|
|
10
|
+
Keywords: secrets,secret-scanning,pii,masking,security,credentials,detect-secrets,privacy,mcp,devsecops
|
|
11
|
+
Classifier: Development Status :: 4 - Beta
|
|
12
|
+
Classifier: Intended Audience :: Developers
|
|
13
|
+
Classifier: Topic :: Security
|
|
14
|
+
Classifier: Topic :: Software Development :: Quality Assurance
|
|
15
|
+
Classifier: Programming Language :: Python :: 3
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
19
|
+
Classifier: Environment :: Console
|
|
20
|
+
Classifier: Operating System :: OS Independent
|
|
21
|
+
Requires-Python: >=3.10
|
|
22
|
+
Description-Content-Type: text/markdown
|
|
23
|
+
Requires-Dist: pyyaml>=6.0.1
|
|
24
|
+
Requires-Dist: gitpython>=3.1.40
|
|
25
|
+
Requires-Dist: mcp>=1.28.0
|
|
26
|
+
|
|
27
|
+
# LocalMask
|
|
28
|
+
|
|
29
|
+
**Find and mask secrets in your code — 100% locally. Nothing ever leaves your machine.**
|
|
30
|
+
|
|
31
|
+
LocalMask scans a repository for credentials, keys, tokens, and PII, and replaces
|
|
32
|
+
each one with a stable placeholder like `~[DATABASE_PASSWORD_0]~`. You get a masked
|
|
33
|
+
copy you can safely paste into an AI tool, share in a ticket, or publish — while
|
|
34
|
+
keeping a local map back to the real values.
|
|
35
|
+
|
|
36
|
+
The free edition is open source and runs entirely offline: a 27+ pattern regex
|
|
37
|
+
engine, entropy detection, masking + rehydrate, a publishable masked repo, git
|
|
38
|
+
sync, and a CLI + MCP server. No AI model, no cloud, no account.
|
|
39
|
+
|
|
40
|
+
### New in this version
|
|
41
|
+
- **Persistent local vault** — tokens now stay stable across scans, syncs, and
|
|
42
|
+
process restarts (and rehydration works in a fresh process). The mapping is
|
|
43
|
+
stored in an encrypted local SQLite file (`~/.localmask/vault.sqlite`, 0600),
|
|
44
|
+
keyed by repo so re-scanning reuses the same tokens.
|
|
45
|
+
- **Editable detection rules (data-driven)** — patterns live in
|
|
46
|
+
`regex_patterns.json`, not hard-coded. Add or tweak rules with no code:
|
|
47
|
+
edit the file, or call `RegexRulesSafe.add_pattern(...)` / `save_patterns()`.
|
|
48
|
+
|
|
49
|
+
> Want an AI model that catches what patterns miss and learns from your
|
|
50
|
+
> corrections, a web dashboard, the [AI proxy](https://localmaskpro.com) that
|
|
51
|
+
> scrubs secrets out of your live AI traffic, and a **team-shared vault** so
|
|
52
|
+
> everyone gets consistent tokens? See [LocalMask Pro](https://localmaskpro.com).
|
|
53
|
+
|
|
54
|
+
---
|
|
55
|
+
|
|
56
|
+
## Install
|
|
57
|
+
|
|
58
|
+
```bash
|
|
59
|
+
pip install localmask # from PyPI
|
|
60
|
+
# or from source:
|
|
61
|
+
pip install .
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
Requires Python 3.10+. No ML dependencies.
|
|
65
|
+
|
|
66
|
+
## Scan a repo
|
|
67
|
+
|
|
68
|
+
```bash
|
|
69
|
+
localmask scan ./my-project
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
You'll see every detected secret, its type, and its placeholder. A masked copy is
|
|
73
|
+
kept in the session; publish it or read it back whenever you want.
|
|
74
|
+
|
|
75
|
+
Sensitivity levels:
|
|
76
|
+
|
|
77
|
+
```bash
|
|
78
|
+
localmask scan ./my-project --sensitivity minimal # only high-confidence secrets
|
|
79
|
+
localmask scan ./my-project --sensitivity standard # default
|
|
80
|
+
localmask scan ./my-project --sensitivity strict # also flags PII, hostnames, IPs
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
## Edit detections (you're in control)
|
|
84
|
+
|
|
85
|
+
The engine is a starting point, not the final word. You can correct it — and these
|
|
86
|
+
edits are **free**, no model required:
|
|
87
|
+
|
|
88
|
+
```bash
|
|
89
|
+
# Ignore a false positive (stop masking this value)
|
|
90
|
+
localmask review # interactive: mark detections keep / ignore
|
|
91
|
+
|
|
92
|
+
# Teach a secret the patterns MISSED (always mask this value)
|
|
93
|
+
localmask teach <scan_id> "the-exact-missed-value" --subtype API_KEY
|
|
94
|
+
localmask teach <scan_id> "a-false-positive" --allow # or: never mask it
|
|
95
|
+
|
|
96
|
+
# …or do it inside the review UI: press [T] to teach a missed value,
|
|
97
|
+
# and it re-scans in place so you see it masked immediately.
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
Ignoring and teaching update a **persistent local lexicon** (stored encrypted
|
|
101
|
+
next to the vault, keyed by repo), so they apply automatically on **every future
|
|
102
|
+
scan and sync** of that repo — even in a fresh process. On a Team/Enterprise
|
|
103
|
+
shared vault, taught values propagate to the whole team.
|
|
104
|
+
|
|
105
|
+
## Publish a masked copy
|
|
106
|
+
|
|
107
|
+
```bash
|
|
108
|
+
localmask publish <scan_id> https://github.com/you/my-project-masked.git
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
Only masked content is pushed. Real values never leave your machine.
|
|
112
|
+
|
|
113
|
+
### Approval gate (review before publishing)
|
|
114
|
+
|
|
115
|
+
By default LocalMask **won't publish until the scan is reviewed & approved** — so
|
|
116
|
+
a masked mirror never goes out (and `sync`/`hook` never auto-republish) with
|
|
117
|
+
unreviewed detections. Approve either way:
|
|
118
|
+
|
|
119
|
+
```bash
|
|
120
|
+
localmask review <scan_id> # decide each detection; approves when none are left pending
|
|
121
|
+
localmask approve-all <scan_id> # approve everything in one step
|
|
122
|
+
localmask publish <scan_id> <url> # now allowed
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
New secrets found on a later `sync` **un-approve** the scan and hold the mirror
|
|
126
|
+
until you review them again. Prefer no gate (auto-approve + auto-publish on every
|
|
127
|
+
change)? Switch the policy:
|
|
128
|
+
|
|
129
|
+
```bash
|
|
130
|
+
localmask config # show current settings
|
|
131
|
+
localmask config publish-policy auto # auto-approve + auto-republish
|
|
132
|
+
localmask config publish-policy review # back to manual gate (default)
|
|
133
|
+
localmask publish <scan> <url> --force # one-off override of the gate
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
## Let your AI read the masked code (two ways)
|
|
137
|
+
|
|
138
|
+
The masked copy has only `~[TOKEN]~` placeholders — no real secrets — so the AI
|
|
139
|
+
can read it safely. **LocalMask never hands the AI any git credentials.** Pick
|
|
140
|
+
whichever fits:
|
|
141
|
+
|
|
142
|
+
**A) The AI reads the published masked git mirror.** Keep the masked repo private
|
|
143
|
+
and give the AI *its own* read access — LocalMask never shares your git token.
|
|
144
|
+
The AI **clones/pulls that repo** (a copy on its side, separate from your real
|
|
145
|
+
code) and authenticates as itself. **To get the updated version after you change
|
|
146
|
+
code:** `localmask sync <scan>` re-masks and re-pushes the mirror (once approved),
|
|
147
|
+
and the AI runs `git pull`. (Because it's masked you *could* also make the mirror
|
|
148
|
+
public and skip auth — no secrets are in it.)
|
|
149
|
+
|
|
150
|
+
*Grant that access in one step — pick how much (if anything) is handed over:*
|
|
151
|
+
|
|
152
|
+
```bash
|
|
153
|
+
# Nothing transferred — the mirror is masked, so just make it public:
|
|
154
|
+
localmask grant-ai <scan_id> --public
|
|
155
|
+
# → any AI clones it with NO credential at all.
|
|
156
|
+
|
|
157
|
+
# Nothing transferred — the AI uses its OWN account:
|
|
158
|
+
localmask grant-ai <scan_id> --collaborator <the-ai-bot-username>
|
|
159
|
+
# → grants that account read-only on this repo; the AI signs in as itself.
|
|
160
|
+
|
|
161
|
+
# A dedicated, throwaway key IS handed to the AI (its own, not yours):
|
|
162
|
+
localmask grant-ai <scan_id>
|
|
163
|
+
# → creates a read-only, single-repo SSH deploy key and prints the AI's
|
|
164
|
+
# private key + clone command.
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
**What is and isn't transferred:** LocalMask never shares *your* git token, SSH
|
|
168
|
+
key, or account. `--public` and `--collaborator` transfer **nothing** to the AI.
|
|
169
|
+
The default deploy-key mode hands the AI a **new, dedicated** credential that is
|
|
170
|
+
read-only and scoped to **only** that one repo (a GitHub deploy key can't access
|
|
171
|
+
any other repo or your account) — revoke it anytime with `gh repo deploy-key
|
|
172
|
+
delete`. If the AI runs on your machine (Claude Code, Cursor), it can just use
|
|
173
|
+
the git you already have and you don't need `grant-ai` at all.
|
|
174
|
+
|
|
175
|
+
**B) The AI reads live from LocalMask — nothing published.** In your AI editor's
|
|
176
|
+
MCP config, the assistant calls the `get_detections` and `get_file_masked` tools.
|
|
177
|
+
No git repo, no push, no `git pull` — LocalMask serves the **current** masked
|
|
178
|
+
content on each call (run `localmask sync <scan>` after code changes so the next
|
|
179
|
+
read is fresh). Use this when you don't want a mirror at all.
|
|
180
|
+
|
|
181
|
+
**Which to use:** (A) the AI holds its own git copy and *pulls* to update — good
|
|
182
|
+
for agents/CI that clone a repo; (B) LocalMask streams the masked files live,
|
|
183
|
+
always current, no repo. Either way the AI only ever sees `~[TOKEN]~` placeholders
|
|
184
|
+
and signs in with its own identity — LocalMask stays out of its authentication.
|
|
185
|
+
|
|
186
|
+
## Keep the masked copy in sync
|
|
187
|
+
|
|
188
|
+
```bash
|
|
189
|
+
localmask sync <scan_id> # re-scan after code changes; tokens stay stable
|
|
190
|
+
localmask hook <scan_id> # install a git hook to auto-sync on commit
|
|
191
|
+
```
|
|
192
|
+
|
|
193
|
+
Unchanged secrets keep the same placeholder across syncs; new secrets get new ones.
|
|
194
|
+
|
|
195
|
+
## Git integrations — all the ways
|
|
196
|
+
|
|
197
|
+
| Integration | Command | What it does |
|
|
198
|
+
|---|---|---|
|
|
199
|
+
| Scan a local folder | `localmask scan ./repo` | mask secrets on disk |
|
|
200
|
+
| Scan a remote repo | `localmask scan https://github.com/org/repo.git` | clone → mask (never stored unmasked) |
|
|
201
|
+
| Publish a masked mirror | `localmask publish <scan> <remote-url>` | push a masked copy to any git remote |
|
|
202
|
+
| Keep it synced | `localmask sync <scan>` | re-scan on change, tokens stay stable |
|
|
203
|
+
| Auto-sync on commit | `localmask hook <scan>` | installs a git `post-commit` / `pre-push` hook |
|
|
204
|
+
| Drive it from your AI editor | MCP (below) | assistant calls scan/publish for you |
|
|
205
|
+
|
|
206
|
+
The remote can be **GitHub, GitLab, Bitbucket, a self-hosted git server, or
|
|
207
|
+
Google Secure Source Manager** — any `https://`, `ssh://`, `git@`, or `file://`
|
|
208
|
+
remote.
|
|
209
|
+
|
|
210
|
+
### Private repos (tokens)
|
|
211
|
+
For a private source or a private masked mirror, give LocalMask a token:
|
|
212
|
+
|
|
213
|
+
```bash
|
|
214
|
+
localmask store-token # prompts for the token HIDDEN,
|
|
215
|
+
# stores it encrypted, returns a credential_id
|
|
216
|
+
localmask scan https://github.com/org/private.git -c <credential_id>
|
|
217
|
+
localmask publish <scan> https://github.com/org/masked.git -c <credential_id>
|
|
218
|
+
```
|
|
219
|
+
|
|
220
|
+
The token is stored encrypted in a local 0600 SQLite file and only a random
|
|
221
|
+
`credential_id` is ever passed on the command line — so your token never lands
|
|
222
|
+
in your shell history or in `ps`. **Don't** pass the token as an argument
|
|
223
|
+
(`store-token ghp_…`): that *does* leak into shell history. You can also rely on
|
|
224
|
+
the git credentials already on your machine (e.g. `gh auth login`), or pass
|
|
225
|
+
`--token` for a throwaway one-off.
|
|
226
|
+
|
|
227
|
+
> **You don't need to create the masked repo yourself.** If it doesn't exist,
|
|
228
|
+
> `publish` offers to create it for you (private by default) — via a stored
|
|
229
|
+
> token or your `gh` CLI login — after asking. Use `--yes` to skip the prompt,
|
|
230
|
+
> `--public` to make it public. Use a PAT with **`repo`** scope (or `gh auth`).
|
|
231
|
+
|
|
232
|
+
## How the git integration stays secure
|
|
233
|
+
|
|
234
|
+
- **Tokens never touch the URL, process arguments, or `.git/config`.** LocalMask
|
|
235
|
+
authenticates via `GIT_ASKPASS`, so your token isn't visible in `ps`, shell
|
|
236
|
+
history, or the cloned repo's config.
|
|
237
|
+
- **Git URLs are validated against an allowlist** (`https/ssh/git@/file`), and a
|
|
238
|
+
`--` separator is placed before them — this blocks argument-injection tricks
|
|
239
|
+
like `--upload-pack=<cmd>` and the `ext::` transport that could run commands.
|
|
240
|
+
- **The git username is passed via an environment variable, never interpolated
|
|
241
|
+
into a shell script** — so a hostile username can't inject commands.
|
|
242
|
+
- **Only masked content is ever pushed.** The published mirror contains
|
|
243
|
+
`~[TOKEN]~` placeholders; the real values stay in your local vault.
|
|
244
|
+
- **Tokens can be stored short-lived and encrypted** (`store-token`), or not
|
|
245
|
+
stored at all (`--token` per command).
|
|
246
|
+
|
|
247
|
+
## Using AI with LocalMask (free)
|
|
248
|
+
|
|
249
|
+
Masking and **rehydration are 100% local and deterministic** — they're just a
|
|
250
|
+
vault lookup, so they need **no AI and no API key** and are always exact. That
|
|
251
|
+
means the free edition works with *any* AI.
|
|
252
|
+
|
|
253
|
+
### Ask any AI with your own key
|
|
254
|
+
```bash
|
|
255
|
+
# Save your key once (typed hidden, stored encrypted locally) — then just ask:
|
|
256
|
+
localmask set-key anthropic # prompts hidden; also openai/gemini/grok/groq/…
|
|
257
|
+
localmask ask <scan_id> "What are the top risks?" --provider anthropic
|
|
258
|
+
|
|
259
|
+
# …or pass the key per call:
|
|
260
|
+
localmask ask <scan_id> "What are the top risks?" --provider openai --api-key sk-...
|
|
261
|
+
localmask ask <scan_id> "..." --provider anthropic --api-key sk-ant-...
|
|
262
|
+
localmask ask <scan_id> "..." --provider gemini --api-key ...
|
|
263
|
+
localmask ask <scan_id> "..." --provider grok --api-key xai-...
|
|
264
|
+
localmask ask <scan_id> "..." --provider groq --api-key ... # Meta/Llama
|
|
265
|
+
localmask ask <scan_id> "..." --provider openrouter --base-url https://... --api-key ...
|
|
266
|
+
```
|
|
267
|
+
This default (`--source memory`) masks the repo + your question locally and
|
|
268
|
+
sends only `~[TOKEN]~` placeholders to the provider **you** chose with **your**
|
|
269
|
+
key, then rehydrates the answer locally. Keys can also come from env
|
|
270
|
+
(`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY`, `XAI_API_KEY`, or
|
|
271
|
+
`LOCALMASK_AI_KEY`). Works with OpenAI, Anthropic, Google Gemini, xAI/Grok,
|
|
272
|
+
Meta/Llama (via Groq/Together), OpenRouter, and any OpenAI-compatible endpoint.
|
|
273
|
+
|
|
274
|
+
### Let the AI read the masked git itself — don't ship the repo (`--source git`)
|
|
275
|
+
|
|
276
|
+
If the AI/agent already has its **own** read access to the published masked
|
|
277
|
+
mirror, you don't need to send it the code at all:
|
|
278
|
+
|
|
279
|
+
```bash
|
|
280
|
+
localmask ask <scan_id> "Why does <a secret in your question> fail?" --source git
|
|
281
|
+
# --git-url <url> (defaults to the scan's published mirror)
|
|
282
|
+
```
|
|
283
|
+
|
|
284
|
+
LocalMask **masks only your question** (by the found-secret vault — any real
|
|
285
|
+
secret you type becomes a token), sends just that masked question **plus the repo
|
|
286
|
+
URL** to your AI, and the AI **reads the private masked repo itself** with its own
|
|
287
|
+
grant. The answer is rehydrated locally. **No repo content and no git credentials
|
|
288
|
+
leave your machine** — only the masked question. (Best with agent/tool-capable
|
|
289
|
+
AIs that can clone a repo. For the MCP/agent flow, the same thing is exposed as
|
|
290
|
+
the `mask_prompt` and `rehydrate_answer` tools.)
|
|
291
|
+
|
|
292
|
+
### Or do it by hand — LocalMask never has to call anything
|
|
293
|
+
```bash
|
|
294
|
+
localmask export <scan_id> ./masked # write the masked repo to a folder…
|
|
295
|
+
# → point your AI tool / agent at ./masked. No keys, no repo permissions, no secrets.
|
|
296
|
+
echo "the AI's answer with ~[TOKEN]~s" | localmask rehydrate <scan_id> # local, exact
|
|
297
|
+
cat prompt.txt | localmask mask-text <scan_id> # mask before pasting
|
|
298
|
+
```
|
|
299
|
+
|
|
300
|
+
> The published/exported masked copy contains **no real secrets** — only tokens —
|
|
301
|
+
> so it's safe to make the masked mirror **public**, and any AI can read it with
|
|
302
|
+
> no credentials. Real values only ever exist in your local vault.
|
|
303
|
+
|
|
304
|
+
Pro adds the convenience layer: a built-in interactive Ask-AI, the automatic AI
|
|
305
|
+
**proxy** (scrub live prompts with zero manual steps), and a local model so you
|
|
306
|
+
need no external AI at all — see [localmaskpro.com](https://localmaskpro.com).
|
|
307
|
+
|
|
308
|
+
## Use it inside your AI editor (MCP)
|
|
309
|
+
|
|
310
|
+
LocalMask ships an MCP server so assistants (Claude, etc.) can scan and mask on
|
|
311
|
+
your behalf. Add to your `.mcp.json`:
|
|
312
|
+
|
|
313
|
+
```json
|
|
314
|
+
{
|
|
315
|
+
"mcpServers": {
|
|
316
|
+
"localmask": {
|
|
317
|
+
"command": "python3",
|
|
318
|
+
"args": ["-m", "mcp_server"]
|
|
319
|
+
}
|
|
320
|
+
}
|
|
321
|
+
}
|
|
322
|
+
```
|
|
323
|
+
|
|
324
|
+
Then your assistant can call `scan_repo`, `get_detections`, `review_detection`,
|
|
325
|
+
`teach_value`, `publish_masked_repo`, and more — all locally.
|
|
326
|
+
|
|
327
|
+
## How it works
|
|
328
|
+
|
|
329
|
+
```
|
|
330
|
+
your repo ──▶ regex + entropy detection ──▶ mask to ~[TOKEN]~ ──▶ masked copy
|
|
331
|
+
│
|
|
332
|
+
└── local vault maps tokens ⇄ real values
|
|
333
|
+
```
|
|
334
|
+
|
|
335
|
+
Everything is local. There is no telemetry and no network call in the free edition.
|
|
336
|
+
|
|
337
|
+
## What's in Free vs Pro
|
|
338
|
+
|
|
339
|
+
| | Free (this repo) | Pro | Team / Enterprise |
|
|
340
|
+
|---|---|---|---|
|
|
341
|
+
| Regex + entropy engine, 27+ types | ✓ | ✓ | ✓ |
|
|
342
|
+
| Editable pattern rules (`regex_patterns.json`) | ✓ | ✓ | ✓ |
|
|
343
|
+
| Mask / rehydrate | ✓ | ✓ | ✓ |
|
|
344
|
+
| Persistent local vault (stable tokens, encrypted) | ✓ | ✓ | ✓ |
|
|
345
|
+
| Edit detections (ignore / teach) | ✓ | ✓ | ✓ |
|
|
346
|
+
| Publish masked repo + git sync | ✓ | ✓ | ✓ |
|
|
347
|
+
| CLI + MCP server | ✓ | ✓ | ✓ |
|
|
348
|
+
| Local AI model that learns | — | ✓ | ✓ |
|
|
349
|
+
| Web dashboard | — | ✓ | ✓ |
|
|
350
|
+
| AI proxy (prompt firewall for your AI traffic) | — | ✓ | ✓ |
|
|
351
|
+
| Team-shared vault (consistent tokens across machines) | — | — | ✓ |
|
|
352
|
+
| Org shared rules · LDAP/AD · SSO | — | — | ✓ |
|
|
353
|
+
|
|
354
|
+
## License
|
|
355
|
+
|
|
356
|
+
Free edition released under the MIT license. See `LICENSE`.
|
|
@@ -0,0 +1,330 @@
|
|
|
1
|
+
# LocalMask
|
|
2
|
+
|
|
3
|
+
**Find and mask secrets in your code — 100% locally. Nothing ever leaves your machine.**
|
|
4
|
+
|
|
5
|
+
LocalMask scans a repository for credentials, keys, tokens, and PII, and replaces
|
|
6
|
+
each one with a stable placeholder like `~[DATABASE_PASSWORD_0]~`. You get a masked
|
|
7
|
+
copy you can safely paste into an AI tool, share in a ticket, or publish — while
|
|
8
|
+
keeping a local map back to the real values.
|
|
9
|
+
|
|
10
|
+
The free edition is open source and runs entirely offline: a 27+ pattern regex
|
|
11
|
+
engine, entropy detection, masking + rehydrate, a publishable masked repo, git
|
|
12
|
+
sync, and a CLI + MCP server. No AI model, no cloud, no account.
|
|
13
|
+
|
|
14
|
+
### New in this version
|
|
15
|
+
- **Persistent local vault** — tokens now stay stable across scans, syncs, and
|
|
16
|
+
process restarts (and rehydration works in a fresh process). The mapping is
|
|
17
|
+
stored in an encrypted local SQLite file (`~/.localmask/vault.sqlite`, 0600),
|
|
18
|
+
keyed by repo so re-scanning reuses the same tokens.
|
|
19
|
+
- **Editable detection rules (data-driven)** — patterns live in
|
|
20
|
+
`regex_patterns.json`, not hard-coded. Add or tweak rules with no code:
|
|
21
|
+
edit the file, or call `RegexRulesSafe.add_pattern(...)` / `save_patterns()`.
|
|
22
|
+
|
|
23
|
+
> Want an AI model that catches what patterns miss and learns from your
|
|
24
|
+
> corrections, a web dashboard, the [AI proxy](https://localmaskpro.com) that
|
|
25
|
+
> scrubs secrets out of your live AI traffic, and a **team-shared vault** so
|
|
26
|
+
> everyone gets consistent tokens? See [LocalMask Pro](https://localmaskpro.com).
|
|
27
|
+
|
|
28
|
+
---
|
|
29
|
+
|
|
30
|
+
## Install
|
|
31
|
+
|
|
32
|
+
```bash
|
|
33
|
+
pip install localmask # from PyPI
|
|
34
|
+
# or from source:
|
|
35
|
+
pip install .
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
Requires Python 3.10+. No ML dependencies.
|
|
39
|
+
|
|
40
|
+
## Scan a repo
|
|
41
|
+
|
|
42
|
+
```bash
|
|
43
|
+
localmask scan ./my-project
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
You'll see every detected secret, its type, and its placeholder. A masked copy is
|
|
47
|
+
kept in the session; publish it or read it back whenever you want.
|
|
48
|
+
|
|
49
|
+
Sensitivity levels:
|
|
50
|
+
|
|
51
|
+
```bash
|
|
52
|
+
localmask scan ./my-project --sensitivity minimal # only high-confidence secrets
|
|
53
|
+
localmask scan ./my-project --sensitivity standard # default
|
|
54
|
+
localmask scan ./my-project --sensitivity strict # also flags PII, hostnames, IPs
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
## Edit detections (you're in control)
|
|
58
|
+
|
|
59
|
+
The engine is a starting point, not the final word. You can correct it — and these
|
|
60
|
+
edits are **free**, no model required:
|
|
61
|
+
|
|
62
|
+
```bash
|
|
63
|
+
# Ignore a false positive (stop masking this value)
|
|
64
|
+
localmask review # interactive: mark detections keep / ignore
|
|
65
|
+
|
|
66
|
+
# Teach a secret the patterns MISSED (always mask this value)
|
|
67
|
+
localmask teach <scan_id> "the-exact-missed-value" --subtype API_KEY
|
|
68
|
+
localmask teach <scan_id> "a-false-positive" --allow # or: never mask it
|
|
69
|
+
|
|
70
|
+
# …or do it inside the review UI: press [T] to teach a missed value,
|
|
71
|
+
# and it re-scans in place so you see it masked immediately.
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
Ignoring and teaching update a **persistent local lexicon** (stored encrypted
|
|
75
|
+
next to the vault, keyed by repo), so they apply automatically on **every future
|
|
76
|
+
scan and sync** of that repo — even in a fresh process. On a Team/Enterprise
|
|
77
|
+
shared vault, taught values propagate to the whole team.
|
|
78
|
+
|
|
79
|
+
## Publish a masked copy
|
|
80
|
+
|
|
81
|
+
```bash
|
|
82
|
+
localmask publish <scan_id> https://github.com/you/my-project-masked.git
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
Only masked content is pushed. Real values never leave your machine.
|
|
86
|
+
|
|
87
|
+
### Approval gate (review before publishing)
|
|
88
|
+
|
|
89
|
+
By default LocalMask **won't publish until the scan is reviewed & approved** — so
|
|
90
|
+
a masked mirror never goes out (and `sync`/`hook` never auto-republish) with
|
|
91
|
+
unreviewed detections. Approve either way:
|
|
92
|
+
|
|
93
|
+
```bash
|
|
94
|
+
localmask review <scan_id> # decide each detection; approves when none are left pending
|
|
95
|
+
localmask approve-all <scan_id> # approve everything in one step
|
|
96
|
+
localmask publish <scan_id> <url> # now allowed
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
New secrets found on a later `sync` **un-approve** the scan and hold the mirror
|
|
100
|
+
until you review them again. Prefer no gate (auto-approve + auto-publish on every
|
|
101
|
+
change)? Switch the policy:
|
|
102
|
+
|
|
103
|
+
```bash
|
|
104
|
+
localmask config # show current settings
|
|
105
|
+
localmask config publish-policy auto # auto-approve + auto-republish
|
|
106
|
+
localmask config publish-policy review # back to manual gate (default)
|
|
107
|
+
localmask publish <scan> <url> --force # one-off override of the gate
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
## Let your AI read the masked code (two ways)
|
|
111
|
+
|
|
112
|
+
The masked copy has only `~[TOKEN]~` placeholders — no real secrets — so the AI
|
|
113
|
+
can read it safely. **LocalMask never hands the AI any git credentials.** Pick
|
|
114
|
+
whichever fits:
|
|
115
|
+
|
|
116
|
+
**A) The AI reads the published masked git mirror.** Keep the masked repo private
|
|
117
|
+
and give the AI *its own* read access — LocalMask never shares your git token.
|
|
118
|
+
The AI **clones/pulls that repo** (a copy on its side, separate from your real
|
|
119
|
+
code) and authenticates as itself. **To get the updated version after you change
|
|
120
|
+
code:** `localmask sync <scan>` re-masks and re-pushes the mirror (once approved),
|
|
121
|
+
and the AI runs `git pull`. (Because it's masked you *could* also make the mirror
|
|
122
|
+
public and skip auth — no secrets are in it.)
|
|
123
|
+
|
|
124
|
+
*Grant that access in one step — pick how much (if anything) is handed over:*
|
|
125
|
+
|
|
126
|
+
```bash
|
|
127
|
+
# Nothing transferred — the mirror is masked, so just make it public:
|
|
128
|
+
localmask grant-ai <scan_id> --public
|
|
129
|
+
# → any AI clones it with NO credential at all.
|
|
130
|
+
|
|
131
|
+
# Nothing transferred — the AI uses its OWN account:
|
|
132
|
+
localmask grant-ai <scan_id> --collaborator <the-ai-bot-username>
|
|
133
|
+
# → grants that account read-only on this repo; the AI signs in as itself.
|
|
134
|
+
|
|
135
|
+
# A dedicated, throwaway key IS handed to the AI (its own, not yours):
|
|
136
|
+
localmask grant-ai <scan_id>
|
|
137
|
+
# → creates a read-only, single-repo SSH deploy key and prints the AI's
|
|
138
|
+
# private key + clone command.
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
**What is and isn't transferred:** LocalMask never shares *your* git token, SSH
|
|
142
|
+
key, or account. `--public` and `--collaborator` transfer **nothing** to the AI.
|
|
143
|
+
The default deploy-key mode hands the AI a **new, dedicated** credential that is
|
|
144
|
+
read-only and scoped to **only** that one repo (a GitHub deploy key can't access
|
|
145
|
+
any other repo or your account) — revoke it anytime with `gh repo deploy-key
|
|
146
|
+
delete`. If the AI runs on your machine (Claude Code, Cursor), it can just use
|
|
147
|
+
the git you already have and you don't need `grant-ai` at all.
|
|
148
|
+
|
|
149
|
+
**B) The AI reads live from LocalMask — nothing published.** In your AI editor's
|
|
150
|
+
MCP config, the assistant calls the `get_detections` and `get_file_masked` tools.
|
|
151
|
+
No git repo, no push, no `git pull` — LocalMask serves the **current** masked
|
|
152
|
+
content on each call (run `localmask sync <scan>` after code changes so the next
|
|
153
|
+
read is fresh). Use this when you don't want a mirror at all.
|
|
154
|
+
|
|
155
|
+
**Which to use:** (A) the AI holds its own git copy and *pulls* to update — good
|
|
156
|
+
for agents/CI that clone a repo; (B) LocalMask streams the masked files live,
|
|
157
|
+
always current, no repo. Either way the AI only ever sees `~[TOKEN]~` placeholders
|
|
158
|
+
and signs in with its own identity — LocalMask stays out of its authentication.
|
|
159
|
+
|
|
160
|
+
## Keep the masked copy in sync
|
|
161
|
+
|
|
162
|
+
```bash
|
|
163
|
+
localmask sync <scan_id> # re-scan after code changes; tokens stay stable
|
|
164
|
+
localmask hook <scan_id> # install a git hook to auto-sync on commit
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
Unchanged secrets keep the same placeholder across syncs; new secrets get new ones.
|
|
168
|
+
|
|
169
|
+
## Git integrations — all the ways
|
|
170
|
+
|
|
171
|
+
| Integration | Command | What it does |
|
|
172
|
+
|---|---|---|
|
|
173
|
+
| Scan a local folder | `localmask scan ./repo` | mask secrets on disk |
|
|
174
|
+
| Scan a remote repo | `localmask scan https://github.com/org/repo.git` | clone → mask (never stored unmasked) |
|
|
175
|
+
| Publish a masked mirror | `localmask publish <scan> <remote-url>` | push a masked copy to any git remote |
|
|
176
|
+
| Keep it synced | `localmask sync <scan>` | re-scan on change, tokens stay stable |
|
|
177
|
+
| Auto-sync on commit | `localmask hook <scan>` | installs a git `post-commit` / `pre-push` hook |
|
|
178
|
+
| Drive it from your AI editor | MCP (below) | assistant calls scan/publish for you |
|
|
179
|
+
|
|
180
|
+
The remote can be **GitHub, GitLab, Bitbucket, a self-hosted git server, or
|
|
181
|
+
Google Secure Source Manager** — any `https://`, `ssh://`, `git@`, or `file://`
|
|
182
|
+
remote.
|
|
183
|
+
|
|
184
|
+
### Private repos (tokens)
|
|
185
|
+
For a private source or a private masked mirror, give LocalMask a token:
|
|
186
|
+
|
|
187
|
+
```bash
|
|
188
|
+
localmask store-token # prompts for the token HIDDEN,
|
|
189
|
+
# stores it encrypted, returns a credential_id
|
|
190
|
+
localmask scan https://github.com/org/private.git -c <credential_id>
|
|
191
|
+
localmask publish <scan> https://github.com/org/masked.git -c <credential_id>
|
|
192
|
+
```
|
|
193
|
+
|
|
194
|
+
The token is stored encrypted in a local 0600 SQLite file and only a random
|
|
195
|
+
`credential_id` is ever passed on the command line — so your token never lands
|
|
196
|
+
in your shell history or in `ps`. **Don't** pass the token as an argument
|
|
197
|
+
(`store-token ghp_…`): that *does* leak into shell history. You can also rely on
|
|
198
|
+
the git credentials already on your machine (e.g. `gh auth login`), or pass
|
|
199
|
+
`--token` for a throwaway one-off.
|
|
200
|
+
|
|
201
|
+
> **You don't need to create the masked repo yourself.** If it doesn't exist,
|
|
202
|
+
> `publish` offers to create it for you (private by default) — via a stored
|
|
203
|
+
> token or your `gh` CLI login — after asking. Use `--yes` to skip the prompt,
|
|
204
|
+
> `--public` to make it public. Use a PAT with **`repo`** scope (or `gh auth`).
|
|
205
|
+
|
|
206
|
+
## How the git integration stays secure
|
|
207
|
+
|
|
208
|
+
- **Tokens never touch the URL, process arguments, or `.git/config`.** LocalMask
|
|
209
|
+
authenticates via `GIT_ASKPASS`, so your token isn't visible in `ps`, shell
|
|
210
|
+
history, or the cloned repo's config.
|
|
211
|
+
- **Git URLs are validated against an allowlist** (`https/ssh/git@/file`), and a
|
|
212
|
+
`--` separator is placed before them — this blocks argument-injection tricks
|
|
213
|
+
like `--upload-pack=<cmd>` and the `ext::` transport that could run commands.
|
|
214
|
+
- **The git username is passed via an environment variable, never interpolated
|
|
215
|
+
into a shell script** — so a hostile username can't inject commands.
|
|
216
|
+
- **Only masked content is ever pushed.** The published mirror contains
|
|
217
|
+
`~[TOKEN]~` placeholders; the real values stay in your local vault.
|
|
218
|
+
- **Tokens can be stored short-lived and encrypted** (`store-token`), or not
|
|
219
|
+
stored at all (`--token` per command).
|
|
220
|
+
|
|
221
|
+
## Using AI with LocalMask (free)
|
|
222
|
+
|
|
223
|
+
Masking and **rehydration are 100% local and deterministic** — they're just a
|
|
224
|
+
vault lookup, so they need **no AI and no API key** and are always exact. That
|
|
225
|
+
means the free edition works with *any* AI.
|
|
226
|
+
|
|
227
|
+
### Ask any AI with your own key
|
|
228
|
+
```bash
|
|
229
|
+
# Save your key once (typed hidden, stored encrypted locally) — then just ask:
|
|
230
|
+
localmask set-key anthropic # prompts hidden; also openai/gemini/grok/groq/…
|
|
231
|
+
localmask ask <scan_id> "What are the top risks?" --provider anthropic
|
|
232
|
+
|
|
233
|
+
# …or pass the key per call:
|
|
234
|
+
localmask ask <scan_id> "What are the top risks?" --provider openai --api-key sk-...
|
|
235
|
+
localmask ask <scan_id> "..." --provider anthropic --api-key sk-ant-...
|
|
236
|
+
localmask ask <scan_id> "..." --provider gemini --api-key ...
|
|
237
|
+
localmask ask <scan_id> "..." --provider grok --api-key xai-...
|
|
238
|
+
localmask ask <scan_id> "..." --provider groq --api-key ... # Meta/Llama
|
|
239
|
+
localmask ask <scan_id> "..." --provider openrouter --base-url https://... --api-key ...
|
|
240
|
+
```
|
|
241
|
+
This default (`--source memory`) masks the repo + your question locally and
|
|
242
|
+
sends only `~[TOKEN]~` placeholders to the provider **you** chose with **your**
|
|
243
|
+
key, then rehydrates the answer locally. Keys can also come from env
|
|
244
|
+
(`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY`, `XAI_API_KEY`, or
|
|
245
|
+
`LOCALMASK_AI_KEY`). Works with OpenAI, Anthropic, Google Gemini, xAI/Grok,
|
|
246
|
+
Meta/Llama (via Groq/Together), OpenRouter, and any OpenAI-compatible endpoint.
|
|
247
|
+
|
|
248
|
+
### Let the AI read the masked git itself — don't ship the repo (`--source git`)
|
|
249
|
+
|
|
250
|
+
If the AI/agent already has its **own** read access to the published masked
|
|
251
|
+
mirror, you don't need to send it the code at all:
|
|
252
|
+
|
|
253
|
+
```bash
|
|
254
|
+
localmask ask <scan_id> "Why does <a secret in your question> fail?" --source git
|
|
255
|
+
# --git-url <url> (defaults to the scan's published mirror)
|
|
256
|
+
```
|
|
257
|
+
|
|
258
|
+
LocalMask **masks only your question** (by the found-secret vault — any real
|
|
259
|
+
secret you type becomes a token), sends just that masked question **plus the repo
|
|
260
|
+
URL** to your AI, and the AI **reads the private masked repo itself** with its own
|
|
261
|
+
grant. The answer is rehydrated locally. **No repo content and no git credentials
|
|
262
|
+
leave your machine** — only the masked question. (Best with agent/tool-capable
|
|
263
|
+
AIs that can clone a repo. For the MCP/agent flow, the same thing is exposed as
|
|
264
|
+
the `mask_prompt` and `rehydrate_answer` tools.)
|
|
265
|
+
|
|
266
|
+
### Or do it by hand — LocalMask never has to call anything
|
|
267
|
+
```bash
|
|
268
|
+
localmask export <scan_id> ./masked # write the masked repo to a folder…
|
|
269
|
+
# → point your AI tool / agent at ./masked. No keys, no repo permissions, no secrets.
|
|
270
|
+
echo "the AI's answer with ~[TOKEN]~s" | localmask rehydrate <scan_id> # local, exact
|
|
271
|
+
cat prompt.txt | localmask mask-text <scan_id> # mask before pasting
|
|
272
|
+
```
|
|
273
|
+
|
|
274
|
+
> The published/exported masked copy contains **no real secrets** — only tokens —
|
|
275
|
+
> so it's safe to make the masked mirror **public**, and any AI can read it with
|
|
276
|
+
> no credentials. Real values only ever exist in your local vault.
|
|
277
|
+
|
|
278
|
+
Pro adds the convenience layer: a built-in interactive Ask-AI, the automatic AI
|
|
279
|
+
**proxy** (scrub live prompts with zero manual steps), and a local model so you
|
|
280
|
+
need no external AI at all — see [localmaskpro.com](https://localmaskpro.com).
|
|
281
|
+
|
|
282
|
+
## Use it inside your AI editor (MCP)
|
|
283
|
+
|
|
284
|
+
LocalMask ships an MCP server so assistants (Claude, etc.) can scan and mask on
|
|
285
|
+
your behalf. Add to your `.mcp.json`:
|
|
286
|
+
|
|
287
|
+
```json
|
|
288
|
+
{
|
|
289
|
+
"mcpServers": {
|
|
290
|
+
"localmask": {
|
|
291
|
+
"command": "python3",
|
|
292
|
+
"args": ["-m", "mcp_server"]
|
|
293
|
+
}
|
|
294
|
+
}
|
|
295
|
+
}
|
|
296
|
+
```
|
|
297
|
+
|
|
298
|
+
Then your assistant can call `scan_repo`, `get_detections`, `review_detection`,
|
|
299
|
+
`teach_value`, `publish_masked_repo`, and more — all locally.
|
|
300
|
+
|
|
301
|
+
## How it works
|
|
302
|
+
|
|
303
|
+
```
|
|
304
|
+
your repo ──▶ regex + entropy detection ──▶ mask to ~[TOKEN]~ ──▶ masked copy
|
|
305
|
+
│
|
|
306
|
+
└── local vault maps tokens ⇄ real values
|
|
307
|
+
```
|
|
308
|
+
|
|
309
|
+
Everything is local. There is no telemetry and no network call in the free edition.
|
|
310
|
+
|
|
311
|
+
## What's in Free vs Pro
|
|
312
|
+
|
|
313
|
+
| | Free (this repo) | Pro | Team / Enterprise |
|
|
314
|
+
|---|---|---|---|
|
|
315
|
+
| Regex + entropy engine, 27+ types | ✓ | ✓ | ✓ |
|
|
316
|
+
| Editable pattern rules (`regex_patterns.json`) | ✓ | ✓ | ✓ |
|
|
317
|
+
| Mask / rehydrate | ✓ | ✓ | ✓ |
|
|
318
|
+
| Persistent local vault (stable tokens, encrypted) | ✓ | ✓ | ✓ |
|
|
319
|
+
| Edit detections (ignore / teach) | ✓ | ✓ | ✓ |
|
|
320
|
+
| Publish masked repo + git sync | ✓ | ✓ | ✓ |
|
|
321
|
+
| CLI + MCP server | ✓ | ✓ | ✓ |
|
|
322
|
+
| Local AI model that learns | — | ✓ | ✓ |
|
|
323
|
+
| Web dashboard | — | ✓ | ✓ |
|
|
324
|
+
| AI proxy (prompt firewall for your AI traffic) | — | ✓ | ✓ |
|
|
325
|
+
| Team-shared vault (consistent tokens across machines) | — | — | ✓ |
|
|
326
|
+
| Org shared rules · LDAP/AD · SSO | — | — | ✓ |
|
|
327
|
+
|
|
328
|
+
## License
|
|
329
|
+
|
|
330
|
+
Free edition released under the MIT license. See `LICENSE`.
|