code-airlock 1.0.0__tar.gz → 1.3.0__tar.gz

{code_airlock-1.0.0/src/code_airlock.egg-info → code_airlock-1.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: code-airlock
-Version: 1.0.0
+Version: 1.3.0
 Summary: Low-latency OpenAI-compatible PII redaction gateway and FastMCP server for coding agents
 Author: Sarthak Maurya
 License-Expression: MIT
@@ -45,8 +45,12 @@ Dynamic: license-file
 
 * **Dual Deployment Modes:** Run as a drop-in **OpenAI-compatible `/v1/chat/completions` reverse proxy** or as a **Model Context Protocol (MCP) server** (`stdio`) for OpenCode, Cursor, and Claude Code.
 * **2-Stage Hybrid Cascade:** Combines sub-millisecond deterministic regex + mathematical checksums (**Luhn, Verhoeff $D_5$, ISO 7064 MOD-97**) with contextual neural NER (`urchade/gliner_small-v2.1`) and a curated developer allowlist.
+* **Deterministic Secret Coverage:** Stage 1 also captures inline connection-string passwords and `PASSWORD` / `SECRET` / `ACCESS_TOKEN` / `API_KEY` assignments, excluding placeholders, booleans, function calls, and existing vault tokens.
+* **Git & CI Guardrails:** `code-airlock scan --staged` catches secrets in staged files at sub-millisecond Stage 1 speed, masks every value in output, and installs a `.git/hooks/pre-commit` gate via `init --pre-commit`.
+* **Safe Agent Execution:** `airlock_exec` injects real `.env` values into a guarded subprocess (`shell=False`) and scrubs both stdout and stderr before returning them to the model.
+* **Project-Aware Rules:** `.airlock.toml` adds team allowlist terms and custom Stage-1 regexes such as `EMPLOYEE_ID = 'EMP-[0-9]{6}'`.
 * **Zero-Delay Streaming Rehydration:** Stateful sliding-window SSE parser reassembles split tokens across arbitrary 2-byte network chunks without delaying code comparison or bitwise operators (`<`, `<<`, `template <typename T>`).
-* **Minimalist Systems Footprint:** ~500 lines of core runtime code (`src/code_airlock/engine.py`, `sanitizer.py`, `proxy.py`, `mcp_server.py`) plus a deterministic 2,000-row benchmark harness.
+* **Minimalist Systems Footprint:** ~800 lines of core runtime code (`engine.py`, `sanitizer.py`, `proxy.py`, `mcp_server.py`, `scanner.py`, `config.py`, `executor.py`) plus a deterministic 2,000-row benchmark harness.
 
 ---
 
@@ -126,7 +130,7 @@ Evaluated on **285 developer prompts** (sampled from `CodeAlpaca_20K` and `OpenC
 | `adversarial_sse` | 250 | 250 | 0 / 0 | 1,217 |
 | **Total** | **2,000** | **1,400** | **350 / 250** | **5,817** |
 
-Reference result (strict exact-span matching, `code-airlock` v0.1.0):
+Reference result (strict exact-span matching, `code-airlock` v1.3.0):
 
 | Split | Strict P / R / F1 | Hard-neg | Clean 0-FP | SSE 1/2/7-byte |
 | :--- | :---: | :---: | :---: | :---: |
@@ -136,7 +140,7 @@ Reference result (strict exact-span matching, `code-airlock` v0.1.0):
 | `adversarial_sse` | 100% / 100% / 100% | — | — | 250/250 |
 | **Combined** | **100% / 100% / 100%** | **250/250** | **350/350** | **2000/2000** |
 
-- Latency: **0.194 ms** p50 / 0.355 ms p95 Stage 1; **30.349 ms** p50 / 39.391 ms p95 Stage 2; **30.391 ms** p50 / 39.554 ms p95 total cascade.
+- Latency: **0.232 ms** p50 / 0.428 ms p95 Stage 1; **32.354 ms** p50 / 43.966 ms p95 Stage 2; **32.704 ms** p50 / 44.112 ms p95 total cascade.
 - Per-entity support: `API_KEY` 756, `AUTH_TOKEN` 467, `CREDIT_CARD` 440, `EMAIL` 754, `PHONE` 465, `IP_ADDRESS` 442, `GOVT_ID` 465, `IBAN_CRYPTO` 465, `PERSON` 729, `ORGANIZATION` 417, `LOCATION` 417. Every entity is at **100% P/R/F1**.
 - Invariants: MCP JSON parse **100/100**, JSON round-trip **100/100**, adversarial SSE **2,000/2,000**, hard negatives **250/250**, clean rows **350/350 with zero false positives**.
 
@@ -242,6 +246,77 @@ python -m code_airlock
 code-airlock-mcp
 ```
 
+### Zero-Friction CLI
+
+```bash
+python -m code_airlock doctor              # model warmup + round-trip + MCP config
+python -m code_airlock init                # add rules to .cursorrules / CLAUDE.md
+python -m code_airlock init --pre-commit   # also install .git/hooks/pre-commit
+python -m code_airlock init --config       # create a starter .airlock.toml
+python -m code_airlock scan --staged       # fast Stage 1 scan of staged files
+python -m code_airlock scan --deep src/    # add GLiNER detection for a path
+python -m code_airlock proxy               # start the HTTP proxy
+python -m code_airlock                     # start the MCP server (default)
+```
+
+Installed entrypoints: `code-airlock-mcp`, `code-airlock-proxy`, and the
+backward-compatible `code-airlock` alias.
+
+`scan` skips binaries, files over 1 MB, `tests/`, and benchmark fixtures. It
+prints `file:line -> [CATEGORY] masked-preview` and exits `1` when secrets are
+found, so it works unchanged in CI and Git hooks. `--deep` enables Stage 2 NER;
+the default Stage 1 path is sub-millisecond. `init` is idempotent, and
+`init --pre-commit` installs a hook that runs `python -m code_airlock scan --staged`.
+
+### 🧰 MCP Tools
+
+| Tool | Purpose |
+| :--- | :--- |
+| `airlock_read_file` | Read a bounded UTF-8 file through the vault; blocks `.ssh`, `.aws`, `.gnupg`, `.kube`, `/etc`, `System32`, `AIRLOCK_WORKSPACE` escapes, binaries, and files over 1 MB. |
+| `airlock_write_file` | Rehydrate locally and write; refuses to write if unresolved/hallucinated `<ENTITY_N>` tokens remain. |
+| `airlock_scrub_text` | Redact arbitrary in-memory text or terminal output; `use_ner=False` forces the fast Stage 1 path. |
+| `airlock_exec` | Run a guarded subprocess (`shell=False`), inject real `.env` values into its environment, and scrub stdout/stderr before returning. Blocks destructive executables and shells. |
+| `airlock_vault_status` | Return active token keys and per-category counts only; never raw values. |
+| `airlock_dehydrate` | Redact text and return session-stable tokens. |
+| `airlock_rehydrate` | Restore canonical, case-mangled, or whitespace-padded vault tokens. |
+
+All file tools resolve paths with `Path.expanduser().resolve()` and jail access
+to `AIRLOCK_WORKSPACE` when that variable is set. Rehydration is tolerant of
+`<api_key_1>` / `< API_KEY_1 >`, while the SSE `StreamRehydrator` remains
+zero-delay and exact.
+
+`airlock_exec` is designed for safe test runs:
+
+```python
+airlock_exec(
+    [sys.executable, "-c", "import os; print(os.environ['STRIPE_SECRET_KEY'])"],
+    ".env",
+)
+# stdout: "<API_KEY_1>"  ← the real key never reaches the model
+```
+
+### ⚙️ Project Config (`.airlock.toml`)
+
+Create a starter with `python -m code_airlock init --config`, or write one
+manually. `CascadeEngine` loads `.airlock.toml` from the current directory, or
+from `AIRLOCK_CONFIG` when set:
+
+```toml
+[allowlist]
+terms = ["MyInternalApp", "staging.corp.local"]
+
+[custom_patterns]
+EMPLOYEE_ID = 'EMP-[0-9]{6}'
+INTERNAL_TICKET = 'JIRA-[0-9]{4,6}'
+```
+
+- Allowlist terms are merged into the Stage 2 allowlist so they are not
+  misclassified as `PERSON`, `ORGANIZATION`, or `LOCATION`.
+- Custom patterns run in Stage 1 alongside built-in rules and produce
+  `<EMPLOYEE_ID_n>` / `<INTERNAL_TICKET_n>` vault tokens.
+- Invalid regexes and malformed labels are ignored rather than crashing the
+  engine.
+
 ---
 
 ### Option 3: OpenAI-Compatible Reverse Proxy (`/v1/chat/completions`)
@@ -277,15 +352,18 @@ To ensure your coding agent actually leverages the airlock tools instead of raw
 
 | File | Lines | Responsibility |
 | :--- | :---: | :--- |
-| `src/code_airlock/engine.py` | 199 | Stage 1 regex/checksum validators (incl. 20+ developer key formats) + Stage 2 GLiNER-small + `TECH_ALLOWLIST` + span merger |
-| `src/code_airlock/sanitizer.py` | 93 | Session-consistent `VaultSession` dehydrator + stateful `StreamRehydrator` SSE parser |
+| `src/code_airlock/engine.py` | 219 | Stage 1 regex/checksum validators + high-entropy URI/KV secrets + lazy Stage 2 GLiNER |
+| `src/code_airlock/sanitizer.py` | 108 | Vault hygiene, tolerant rehydration, unresolved-token detection, SSE parser |
 | `src/code_airlock/proxy.py` | 153 | FastAPI `/v1/chat/completions` gateway, SSE rehydration, telemetry headers, `main()` |
-| `src/code_airlock/mcp_server.py` | 71 | FastMCP `stdio` server exposing 5 zero-leak text and file tools |
+| `src/code_airlock/mcp_server.py` | 124 | FastMCP server with path jailing, file guardrails, `airlock_scrub_text`, `airlock_exec`, and write verification |
+| `src/code_airlock/config.py` | 39 | Optional `.airlock.toml` allowlist and custom Stage-1 pattern loader |
+| `src/code_airlock/executor.py` | 87 | Guarded subprocess execution, env injection, and output scrubbing |
+| `src/code_airlock/scanner.py` | 76 | Masked Stage-1/Stage-2 Git and path secret scanner |
 | `src/code_airlock/__init__.py` | 7 | Public API: `CascadeEngine`, `VaultSession`, `StreamRehydrator`, `__version__` |
-| `src/code_airlock/__main__.py` | 6 | `python -m code_airlock` → MCP server |
+| `src/code_airlock/__main__.py` | 127 | CLI: `mcp`, `proxy`, `doctor`, `init`, `scan` |
 | `tests/test_proxy.py` | 84 | End-to-end proxy tests (11-entity zero-leak check, 2-char SSE split, C++ syntax) |
-| `tests/test_mcp.py` | 66 | MCP tool verification (`.env` → `config.json` zero-leak migration) |
-| `benchmarks/benchmark.py` | 162 | Unified 4-suite evaluator → `benchmarks/BENCHMARK_RESULTS.md` |
+| `tests/test_mcp.py` | 181 | MCP tools, scanner, exec, config, pre-commit hook, file jail, tolerant rehydration |
+| `benchmarks/benchmark.py` | 163 | Unified 4-suite evaluator → `benchmarks/BENCHMARK_RESULTS.md` |
 | `benchmarks/build_benchmarks.py` | 239 | Deterministic 2,000-row generator (`seed=42`) |
 | `benchmarks/export_hf.py` | 88 | Publishes all 4 splits plus the auto-generated dataset card |
 | `benchmarks/BENCHMARK_RESULTS.md` | — | Generated metrics-only scorecard |
@@ -303,6 +381,15 @@ Detection is best-effort and local-only. It reduces exposure but is not a
 guarantee that every secret is removed. See [SECURITY.md](SECURITY.md) for the
 threat model, known gaps, and private disclosure instructions.
 
+The MCP file tools additionally block sensitive directories (`~/.ssh`, `~/.aws`,
+`~/.gnupg`, `~/.kube`, `/etc`, `System32`), files over 1 MB, binary files, and
+paths outside `AIRLOCK_WORKSPACE` when that variable is set. Writes containing
+unresolved or hallucinated `<ENTITY_N>` tokens are refused.
+
+`code-airlock scan --staged` adds a Git pre-commit gate for the five highest-risk
+secret categories (`API_KEY`, `AUTH_TOKEN`, `CREDIT_CARD`, `GOVT_ID`,
+`IBAN_CRYPTO`). It prints masked previews only and exits non-zero on findings.
+
 ## 📜 License
 
 MIT © 2026 Sarthak Maurya. See [LICENSE](LICENSE).

{code_airlock-1.0.0 → code_airlock-1.3.0}/README.md

@@ -6,8 +6,12 @@
 
 * **Dual Deployment Modes:** Run as a drop-in **OpenAI-compatible `/v1/chat/completions` reverse proxy** or as a **Model Context Protocol (MCP) server** (`stdio`) for OpenCode, Cursor, and Claude Code.
 * **2-Stage Hybrid Cascade:** Combines sub-millisecond deterministic regex + mathematical checksums (**Luhn, Verhoeff $D_5$, ISO 7064 MOD-97**) with contextual neural NER (`urchade/gliner_small-v2.1`) and a curated developer allowlist.
+* **Deterministic Secret Coverage:** Stage 1 also captures inline connection-string passwords and `PASSWORD` / `SECRET` / `ACCESS_TOKEN` / `API_KEY` assignments, excluding placeholders, booleans, function calls, and existing vault tokens.
+* **Git & CI Guardrails:** `code-airlock scan --staged` catches secrets in staged files at sub-millisecond Stage 1 speed, masks every value in output, and installs a `.git/hooks/pre-commit` gate via `init --pre-commit`.
+* **Safe Agent Execution:** `airlock_exec` injects real `.env` values into a guarded subprocess (`shell=False`) and scrubs both stdout and stderr before returning them to the model.
+* **Project-Aware Rules:** `.airlock.toml` adds team allowlist terms and custom Stage-1 regexes such as `EMPLOYEE_ID = 'EMP-[0-9]{6}'`.
 * **Zero-Delay Streaming Rehydration:** Stateful sliding-window SSE parser reassembles split tokens across arbitrary 2-byte network chunks without delaying code comparison or bitwise operators (`<`, `<<`, `template <typename T>`).
-* **Minimalist Systems Footprint:** ~500 lines of core runtime code (`src/code_airlock/engine.py`, `sanitizer.py`, `proxy.py`, `mcp_server.py`) plus a deterministic 2,000-row benchmark harness.
+* **Minimalist Systems Footprint:** ~800 lines of core runtime code (`engine.py`, `sanitizer.py`, `proxy.py`, `mcp_server.py`, `scanner.py`, `config.py`, `executor.py`) plus a deterministic 2,000-row benchmark harness.
 
 ---
 
@@ -87,7 +91,7 @@ Evaluated on **285 developer prompts** (sampled from `CodeAlpaca_20K` and `OpenC
 | `adversarial_sse` | 250 | 250 | 0 / 0 | 1,217 |
 | **Total** | **2,000** | **1,400** | **350 / 250** | **5,817** |
 
-Reference result (strict exact-span matching, `code-airlock` v0.1.0):
+Reference result (strict exact-span matching, `code-airlock` v1.3.0):
 
 | Split | Strict P / R / F1 | Hard-neg | Clean 0-FP | SSE 1/2/7-byte |
 | :--- | :---: | :---: | :---: | :---: |
@@ -97,7 +101,7 @@ Reference result (strict exact-span matching, `code-airlock` v0.1.0):
 | `adversarial_sse` | 100% / 100% / 100% | — | — | 250/250 |
 | **Combined** | **100% / 100% / 100%** | **250/250** | **350/350** | **2000/2000** |
 
-- Latency: **0.194 ms** p50 / 0.355 ms p95 Stage 1; **30.349 ms** p50 / 39.391 ms p95 Stage 2; **30.391 ms** p50 / 39.554 ms p95 total cascade.
+- Latency: **0.232 ms** p50 / 0.428 ms p95 Stage 1; **32.354 ms** p50 / 43.966 ms p95 Stage 2; **32.704 ms** p50 / 44.112 ms p95 total cascade.
 - Per-entity support: `API_KEY` 756, `AUTH_TOKEN` 467, `CREDIT_CARD` 440, `EMAIL` 754, `PHONE` 465, `IP_ADDRESS` 442, `GOVT_ID` 465, `IBAN_CRYPTO` 465, `PERSON` 729, `ORGANIZATION` 417, `LOCATION` 417. Every entity is at **100% P/R/F1**.
 - Invariants: MCP JSON parse **100/100**, JSON round-trip **100/100**, adversarial SSE **2,000/2,000**, hard negatives **250/250**, clean rows **350/350 with zero false positives**.
 
@@ -203,6 +207,77 @@ python -m code_airlock
 code-airlock-mcp
 ```
 
+### Zero-Friction CLI
+
+```bash
+python -m code_airlock doctor              # model warmup + round-trip + MCP config
+python -m code_airlock init                # add rules to .cursorrules / CLAUDE.md
+python -m code_airlock init --pre-commit   # also install .git/hooks/pre-commit
+python -m code_airlock init --config       # create a starter .airlock.toml
+python -m code_airlock scan --staged       # fast Stage 1 scan of staged files
+python -m code_airlock scan --deep src/    # add GLiNER detection for a path
+python -m code_airlock proxy               # start the HTTP proxy
+python -m code_airlock                     # start the MCP server (default)
+```
+
+Installed entrypoints: `code-airlock-mcp`, `code-airlock-proxy`, and the
+backward-compatible `code-airlock` alias.
+
+`scan` skips binaries, files over 1 MB, `tests/`, and benchmark fixtures. It
+prints `file:line -> [CATEGORY] masked-preview` and exits `1` when secrets are
+found, so it works unchanged in CI and Git hooks. `--deep` enables Stage 2 NER;
+the default Stage 1 path is sub-millisecond. `init` is idempotent, and
+`init --pre-commit` installs a hook that runs `python -m code_airlock scan --staged`.
+
+### 🧰 MCP Tools
+
+| Tool | Purpose |
+| :--- | :--- |
+| `airlock_read_file` | Read a bounded UTF-8 file through the vault; blocks `.ssh`, `.aws`, `.gnupg`, `.kube`, `/etc`, `System32`, `AIRLOCK_WORKSPACE` escapes, binaries, and files over 1 MB. |
+| `airlock_write_file` | Rehydrate locally and write; refuses to write if unresolved/hallucinated `<ENTITY_N>` tokens remain. |
+| `airlock_scrub_text` | Redact arbitrary in-memory text or terminal output; `use_ner=False` forces the fast Stage 1 path. |
+| `airlock_exec` | Run a guarded subprocess (`shell=False`), inject real `.env` values into its environment, and scrub stdout/stderr before returning. Blocks destructive executables and shells. |
+| `airlock_vault_status` | Return active token keys and per-category counts only; never raw values. |
+| `airlock_dehydrate` | Redact text and return session-stable tokens. |
+| `airlock_rehydrate` | Restore canonical, case-mangled, or whitespace-padded vault tokens. |
+
+All file tools resolve paths with `Path.expanduser().resolve()` and jail access
+to `AIRLOCK_WORKSPACE` when that variable is set. Rehydration is tolerant of
+`<api_key_1>` / `< API_KEY_1 >`, while the SSE `StreamRehydrator` remains
+zero-delay and exact.
+
+`airlock_exec` is designed for safe test runs:
+
+```python
+airlock_exec(
+    [sys.executable, "-c", "import os; print(os.environ['STRIPE_SECRET_KEY'])"],
+    ".env",
+)
+# stdout: "<API_KEY_1>"  ← the real key never reaches the model
+```
+
+### ⚙️ Project Config (`.airlock.toml`)
+
+Create a starter with `python -m code_airlock init --config`, or write one
+manually. `CascadeEngine` loads `.airlock.toml` from the current directory, or
+from `AIRLOCK_CONFIG` when set:
+
+```toml
+[allowlist]
+terms = ["MyInternalApp", "staging.corp.local"]
+
+[custom_patterns]
+EMPLOYEE_ID = 'EMP-[0-9]{6}'
+INTERNAL_TICKET = 'JIRA-[0-9]{4,6}'
+```
+
+- Allowlist terms are merged into the Stage 2 allowlist so they are not
+  misclassified as `PERSON`, `ORGANIZATION`, or `LOCATION`.
+- Custom patterns run in Stage 1 alongside built-in rules and produce
+  `<EMPLOYEE_ID_n>` / `<INTERNAL_TICKET_n>` vault tokens.
+- Invalid regexes and malformed labels are ignored rather than crashing the
+  engine.
+
 ---
 
 ### Option 3: OpenAI-Compatible Reverse Proxy (`/v1/chat/completions`)
@@ -238,15 +313,18 @@ To ensure your coding agent actually leverages the airlock tools instead of raw
 
 | File | Lines | Responsibility |
 | :--- | :---: | :--- |
-| `src/code_airlock/engine.py` | 199 | Stage 1 regex/checksum validators (incl. 20+ developer key formats) + Stage 2 GLiNER-small + `TECH_ALLOWLIST` + span merger |
-| `src/code_airlock/sanitizer.py` | 93 | Session-consistent `VaultSession` dehydrator + stateful `StreamRehydrator` SSE parser |
+| `src/code_airlock/engine.py` | 219 | Stage 1 regex/checksum validators + high-entropy URI/KV secrets + lazy Stage 2 GLiNER |
+| `src/code_airlock/sanitizer.py` | 108 | Vault hygiene, tolerant rehydration, unresolved-token detection, SSE parser |
 | `src/code_airlock/proxy.py` | 153 | FastAPI `/v1/chat/completions` gateway, SSE rehydration, telemetry headers, `main()` |
-| `src/code_airlock/mcp_server.py` | 71 | FastMCP `stdio` server exposing 5 zero-leak text and file tools |
+| `src/code_airlock/mcp_server.py` | 124 | FastMCP server with path jailing, file guardrails, `airlock_scrub_text`, `airlock_exec`, and write verification |
+| `src/code_airlock/config.py` | 39 | Optional `.airlock.toml` allowlist and custom Stage-1 pattern loader |
+| `src/code_airlock/executor.py` | 87 | Guarded subprocess execution, env injection, and output scrubbing |
+| `src/code_airlock/scanner.py` | 76 | Masked Stage-1/Stage-2 Git and path secret scanner |
 | `src/code_airlock/__init__.py` | 7 | Public API: `CascadeEngine`, `VaultSession`, `StreamRehydrator`, `__version__` |
-| `src/code_airlock/__main__.py` | 6 | `python -m code_airlock` → MCP server |
+| `src/code_airlock/__main__.py` | 127 | CLI: `mcp`, `proxy`, `doctor`, `init`, `scan` |
 | `tests/test_proxy.py` | 84 | End-to-end proxy tests (11-entity zero-leak check, 2-char SSE split, C++ syntax) |
-| `tests/test_mcp.py` | 66 | MCP tool verification (`.env` → `config.json` zero-leak migration) |
-| `benchmarks/benchmark.py` | 162 | Unified 4-suite evaluator → `benchmarks/BENCHMARK_RESULTS.md` |
+| `tests/test_mcp.py` | 181 | MCP tools, scanner, exec, config, pre-commit hook, file jail, tolerant rehydration |
+| `benchmarks/benchmark.py` | 163 | Unified 4-suite evaluator → `benchmarks/BENCHMARK_RESULTS.md` |
 | `benchmarks/build_benchmarks.py` | 239 | Deterministic 2,000-row generator (`seed=42`) |
 | `benchmarks/export_hf.py` | 88 | Publishes all 4 splits plus the auto-generated dataset card |
 | `benchmarks/BENCHMARK_RESULTS.md` | — | Generated metrics-only scorecard |
@@ -264,6 +342,15 @@ Detection is best-effort and local-only. It reduces exposure but is not a
 guarantee that every secret is removed. See [SECURITY.md](SECURITY.md) for the
 threat model, known gaps, and private disclosure instructions.
 
+The MCP file tools additionally block sensitive directories (`~/.ssh`, `~/.aws`,
+`~/.gnupg`, `~/.kube`, `/etc`, `System32`), files over 1 MB, binary files, and
+paths outside `AIRLOCK_WORKSPACE` when that variable is set. Writes containing
+unresolved or hallucinated `<ENTITY_N>` tokens are refused.
+
+`code-airlock scan --staged` adds a Git pre-commit gate for the five highest-risk
+secret categories (`API_KEY`, `AUTH_TOKEN`, `CREDIT_CARD`, `GOVT_ID`,
+`IBAN_CRYPTO`). It prints masked previews only and exits non-zero on findings.
+
 ## 📜 License
 
 MIT © 2026 Sarthak Maurya. See [LICENSE](LICENSE).

{code_airlock-1.0.0 → code_airlock-1.3.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "code-airlock"
-version = "1.0.0"
+version = "1.3.0"
 description = "Low-latency OpenAI-compatible PII redaction gateway and FastMCP server for coding agents"
 readme = "README.md"
 requires-python = ">=3.10"

{code_airlock-1.0.0 → code_airlock-1.3.0}/src/code_airlock/__init__.py

@@ -3,5 +3,5 @@
 from .engine import CascadeEngine
 from .sanitizer import StreamRehydrator, VaultSession
 
-__version__ = "1.0.0"
+__version__ = "1.3.0"
 __all__ = ["CascadeEngine", "StreamRehydrator", "VaultSession", "__version__"]

code_airlock-1.3.0/src/code_airlock/__main__.py

@@ -0,0 +1,127 @@
+"""Zero-friction CLI: MCP server, proxy, doctor, init, and secret scanning."""
+from __future__ import annotations
+import argparse, os, sys
+from pathlib import Path
+
+RULE = """# code-airlock security rules
+Always use `airlock_read_file` and `airlock_write_file` when reading or writing `.env`, `*secret*`, credentials, or deployment configs. Never inspect secret files using raw bash (`cat`, `grep`) or native file read tools.
+"""
+MARKER = "code-airlock security rules"
+HOOK = """#!/bin/sh
+# Installed by code-airlock
+python -m code_airlock scan --staged
+"""
+CONFIG_TEMPLATE = """# code-airlock project configuration
+[allowlist]
+terms = ["MyInternalApp", "staging.corp.local"]
+
+[custom_patterns]
+# EMPLOYEE_ID = 'EMP-[0-9]{6}'
+# INTERNAL_TICKET = 'JIRA-[0-9]{4,6}'
+"""
+CONFIG = """{
+  "mcpServers": {
+    "code-airlock": {
+      "command": "python",
+      "args": ["-m", "code_airlock.mcp_server"]
+    }
+  }
+}"""
+
+
+def doctor() -> int:
+    from .engine import CascadeEngine
+    from .sanitizer import VaultSession
+    sample = ("API key sk-" + "proj-abc123def456ghi789jkl012mno345; "
+              "postgres://user:" + "pass12345" + "@10.0.0.5:5432/db; "
+              "contact alice@acme.io and Alice Smith.")
+    engine = CascadeEngine(lazy=False)
+    spans = engine.detect(sample)
+    labels = {s.label for s in spans}
+    vault = VaultSession()
+    dehydrated, _ = vault.dehydrate(sample, spans)
+    restored = vault.rehydrate(dehydrated)
+    expected = {"API_KEY", "AUTH_TOKEN", "EMAIL", "PERSON"}
+    missing = expected - labels
+    ok = restored == sample and not missing
+    print(f"[{'ok' if not missing else 'fail'}] GLiNER ready on {engine.device}; labels: {', '.join(sorted(labels))}")
+    print(f"[{'ok' if restored == sample else 'fail'}] round-trip rehydration ({len(spans)} spans)")
+    if not ok:
+        if missing: print(f"[fail] missing detectors: {', '.join(sorted(missing))}")
+        return 1
+    print("\nOpenCode (~/.config/opencode/opencode.json):")
+    print('{"mcp": {"code-airlock": {"type": "local", "command": ["python", "-m", "code_airlock.mcp_server"]}}}')
+    print("\nCursor / Claude Code / Claude Desktop:")
+    print(CONFIG)
+    return 0
+
+
+def init(pre_commit: bool = False, config: bool = False) -> int:
+    if config:
+        config_path = Path.cwd() / ".airlock.toml"
+        if config_path.exists():
+            print("[ok] .airlock.toml already exists")
+        else:
+            config_path.write_text(CONFIG_TEMPLATE, encoding="utf-8")
+            print("[ok] created .airlock.toml")
+    for name in (".cursorrules", "CLAUDE.md"):
+        target = Path.cwd() / name
+        existing = target.read_text(encoding="utf-8") if target.exists() else ""
+        if MARKER in existing:
+            print(f"[ok] {name} already contains code-airlock rules")
+            continue
+        separator = "\n\n" if existing.strip() else ""
+        target.write_text(existing.rstrip() + separator + RULE, encoding="utf-8")
+        print(f"[ok] updated {name}")
+    if pre_commit:
+        hooks = Path.cwd() / ".git" / "hooks"
+        if not hooks.is_dir():
+            print("[skip] no .git/hooks directory")
+        else:
+            hook = hooks / "pre-commit"
+            existing = hook.read_text(encoding="utf-8") if hook.exists() else ""
+            if "code-airlock scan --staged" in existing:
+                print("[ok] .git/hooks/pre-commit already installed")
+            else:
+                separator = "\n\n" if existing.strip() else ""
+                hook.write_text(existing.rstrip() + separator + HOOK, encoding="utf-8")
+                os.chmod(hook, 0o755)
+                print("[ok] installed .git/hooks/pre-commit")
+    return 0
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(prog="code-airlock", description="Local PII/secret gate for coding agents")
+    sub = parser.add_subparsers(dest="command")
+    sub.add_parser("mcp", help="start the MCP stdio server (default)")
+    sub.add_parser("proxy", help="start the OpenAI-compatible HTTP proxy")
+    sub.add_parser("doctor", help="verify the model and print MCP configuration")
+    init_cmd = sub.add_parser("init", help="add code-airlock rules and optional pre-commit hook")
+    init_cmd.add_argument("--pre-commit", action="store_true", help="install .git/hooks/pre-commit")
+    init_cmd.add_argument("--config", action="store_true", help="create a starter .airlock.toml")
+    scan_cmd = sub.add_parser("scan", help="scan files or staged changes for secrets")
+    scan_cmd.add_argument("--staged", action="store_true", help="scan files staged in git")
+    scan_cmd.add_argument("--deep", action="store_true", help="enable Stage 2 GLiNER detection")
+    scan_cmd.add_argument("paths", nargs="*", default=["."], help="files or directories to scan")
+    args = parser.parse_args()
+    if args.command in (None, "mcp"):
+        from .mcp_server import main as run
+        run()
+        return 0
+    if args.command == "proxy":
+        from .proxy import main as run
+        run()
+        return 0
+    if args.command == "doctor":
+        return doctor()
+    if args.command == "init":
+        return init(args.pre_commit, args.config)
+    if args.command == "scan":
+        from .scanner import scan
+        return scan(args.paths, staged=args.staged, deep=args.deep)
+    parser.print_help()
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

code_airlock-1.3.0/src/code_airlock/config.py

@@ -0,0 +1,39 @@
+"""config.py - Optional .airlock.toml allowlist and custom Stage-1 patterns."""
+from __future__ import annotations
+import os, re
+from pathlib import Path
+
+try:
+    import tomllib
+except ImportError:  # Python 3.10 fallback; config is optional
+    try:
+        import tomli as tomllib
+    except ImportError:
+        tomllib = None
+
+
+def load_config(path: str | Path | None = None):
+    """Return (allowlist terms, compiled custom patterns) from .airlock.toml."""
+    if tomllib is None:
+        return frozenset(), {}
+    target = Path(path or os.getenv("AIRLOCK_CONFIG") or ".airlock.toml")
+    if not target.is_file():
+        return frozenset(), {}
+    try:
+        with target.open("rb") as handle:
+            data = tomllib.load(handle)
+    except (OSError, ValueError):
+        return frozenset(), {}
+    terms = data.get("allowlist", {})
+    terms = terms.get("terms", []) if isinstance(terms, dict) else []
+    allowlist = frozenset(str(term).lower() for term in terms) if isinstance(terms, list) else frozenset()
+    patterns = {}
+    patterns_data = data.get("custom_patterns", {})
+    for label, raw in (patterns_data.items() if isinstance(patterns_data, dict) else []):
+        if not re.fullmatch(r"[A-Z_]+", label) or not isinstance(raw, str):
+            continue
+        try:
+            patterns[label] = re.compile(raw)
+        except re.error:
+            continue
+    return allowlist, patterns

{code_airlock-1.0.0 → code_airlock-1.3.0}/src/code_airlock/engine.py

@@ -2,8 +2,7 @@
 from __future__ import annotations
 import re
 from dataclasses import dataclass
-import torch
-from gliner import GLiNER
+from .config import load_config
 
 @dataclass(frozen=True)
 class Span:
@@ -77,6 +76,11 @@ PAN_RE = re.compile(r"\b[A-Z]{5}[0-9]{4}[A-Z]\b")
 AADHAAR_RE = re.compile(r"\b[2-9]\d{3}[ -]?\d{4}[ -]?\d{4}\b")
 ETH_RE = re.compile(r"\b0x[a-fA-F0-9]{40}\b")
 IBAN_RE = re.compile(r"\b[A-Z]{2}\d{2}[A-Z0-9]{11,30}\b")
+URI_PASSWORD_RE = re.compile(r"://[^:\s/@]+:([^@\s/:]{8,})@")
+KV_SECRET_RE = re.compile(r"(?i)(?:PASSWORD|PASSWD|SECRET_KEY|SECRET|PRIVATE_KEY|ACCESS_TOKEN|AUTH_TOKEN|API_KEY)\s*[=:]\s*[\"']?([^\s\"'\r\n#;,]{8,})[\"']?")
+
+def _secret_ok(value: str) -> bool:
+    return value.lower() not in {"true", "false", "null", "none", "changeme", "localhost"} and not value.startswith(("<", "$", "{")) and not any(c in value for c in "(){}") and len(set(value)) > 2
 
 TECH_ALLOWLIST = frozenset({
     "docker", "redis", "fastapi", "pytorch", "react", "aws", "postgres", "postgresql",
@@ -102,20 +106,18 @@ PERSON_STOPWORDS = frozenset({
 })
 
 CODE_CHARS = frozenset("_()[]{}:;/\\=$@")
-SCHEMA_WORDS = frozenset({"production", "development", "staging", "prod", "dev", "test", "localhost",
-    "api", "backend", "frontend", "database", "server", "client", "service", "origin",
-    "environment", "region", "payments", "config", "metadata"})
+SCHEMA_WORDS = frozenset({"production", "development", "staging", "prod", "dev", "test", "localhost", "api", "backend", "frontend", "database", "server", "client", "service", "origin", "environment", "region", "payments", "config", "metadata"})
 
-def stage2_ok(text: str, label: str) -> bool:
+def stage2_ok(text: str, label: str, allowlist: frozenset = TECH_ALLOWLIST) -> bool:
     """Reject Stage 2 drift on code identifiers, tech names, and role words."""
     t = text.strip()
     if len(t) <= 2: return False
     low = t.lower()
-    if low in TECH_ALLOWLIST or low in {"location", "organization", "person", "address", "user"}: return False
+    if low in allowlist or low in {"location", "organization", "person", "address", "user"}: return False
     if label in ("LOCATION", "ORGANIZATION") and low in SCHEMA_WORDS: return False
     if label == "LOCATION" and (re.fullmatch(r"[a-z]{2}(?:-[a-z]+)+-\d+", low) or any(w in low.split() for w in ("cluster", "node", "proxy", "channel", "region"))): return False
     if label == "ORGANIZATION" and re.fullmatch(r"[a-z0-9]+(?:-[a-z0-9]+)+", t): return False
-    if (label == "ORGANIZATION" and (low.endswith(" service") or low.replace(".", "") in TECH_ALLOWLIST)) or re.fullmatch(r"[A-Z]{2}\d{2}[A-Z0-9]{11,30}", t): return False
+    if (label == "ORGANIZATION" and (low.endswith(" service") or low.replace(".", "") in allowlist)) or re.fullmatch(r"[A-Z]{2}\d{2}[A-Z0-9]{11,30}", t): return False
     if label == "PERSON" and low in PERSON_STOPWORDS: return False
     if label == "PERSON" and " " not in t and (len(t) > 25 or any(c.isdigit() for c in t) or sum(c.isupper() for c in t) / len(t) > 0.5): return False
     if any(ch in CODE_CHARS for ch in t) or (any(c.isdigit() for c in t) and "." in t) or (label in ("LOCATION", "PERSON", "ORGANIZATION") and t.isdigit()): return False
@@ -157,18 +159,35 @@ def merge_spans(stage1_spans: list[Span], stage2_spans: list[Span]) -> list[Span
     return sorted(s1 + s2, key=lambda s: s.start)
 
 class CascadeEngine:
-    def __init__(self, device: str | None = None, model_name: str = "urchade/gliner_small-v2.1", threshold: float = 0.5):
-        self.device = device or ("cuda" if torch.cuda.is_available() else "cpu")
+    def __init__(self, device: str | None = None, model_name: str = "urchade/gliner_small-v2.1",
+                 threshold: float = 0.5, lazy: bool = True):
+        self.device = device
+        self.model_name = model_name
         self.threshold = threshold
-        self.gliner = GLiNER.from_pretrained(model_name)
-        if hasattr(self.gliner, "to"): self.gliner.to(self.device)
+        extra, self.custom_patterns = load_config(); self.allowlist = TECH_ALLOWLIST | extra
         self.label_map = {"person": "PERSON", "organization": "ORGANIZATION", "location": "LOCATION"}
+        self._ner_model = None
+        if not lazy: self._get_ner()
+
+    def _get_ner(self):
+        if self._ner_model is None:
+            import torch
+            from gliner import GLiNER
+            self.device = self.device or ("cuda" if torch.cuda.is_available() else "cpu")
+            self._ner_model = GLiNER.from_pretrained(self.model_name)
+            if hasattr(self._ner_model, "to"): self._ner_model.to(self.device)
+        return self._ner_model
 
     def detect_stage1(self, text: str) -> list[Span]:
         spans: list[Span] = []
         for m in EMAIL_RE.finditer(text): spans.append(Span(m.start(), m.end(), "EMAIL", m.group(), 1.0, "stage1"))
         for m in API_KEY_RE.finditer(text): spans.append(Span(m.start(), m.end(), "API_KEY", m.group(), 1.0, "stage1"))
         for m in AUTH_TOKEN_RE.finditer(text): spans.append(Span(m.start(), m.end(), "AUTH_TOKEN", m.group(), 1.0, "stage1"))
+        for m in URI_PASSWORD_RE.finditer(text):
+            if _secret_ok(m.group(1)): spans.append(Span(m.start(1), m.end(1), "AUTH_TOKEN", m.group(1), 1.0, "stage1"))
+        for m in KV_SECRET_RE.finditer(text):
+            if _secret_ok(m.group(1)): spans.append(Span(m.start(1), m.end(1), "API_KEY", m.group(1), 1.0, "stage1"))
+        for label, pat in self.custom_patterns.items(): spans.extend(Span(m.start(), m.end(), label, m.group(), 1.0, "stage1") for m in pat.finditer(text) if m.group().lower() not in self.allowlist)
         for m in PHONE_RE.finditer(text):
             if is_valid_phone(m.group()): spans.append(Span(m.start(), m.end(), "PHONE", m.group(), 1.0, "stage1"))
         for m in IPV4_CAND_RE.finditer(text):
@@ -185,15 +204,16 @@ class CascadeEngine:
         return _resolve_same_stage(spans)
 
     def detect_stage2(self, text: str) -> list[Span]:
-        preds = self.gliner.predict_entities(text, ["person", "organization", "location"], threshold=self.threshold)
+        model = self._get_ner()
+        preds = model.predict_entities(text, ["person", "organization", "location"], threshold=self.threshold)
         raw_spans: list[Span] = []
         for p in preds:
             lbl = self.label_map.get(p["label"], p["label"].upper())
-            if not stage2_ok(p["text"], lbl): continue
+            if not stage2_ok(p["text"], lbl, self.allowlist): continue
             raw_spans.append(Span(p["start"], p["end"], lbl, p["text"], float(p.get("score", 0.0)), "stage2"))
         resolved = _resolve_same_stage(raw_spans)
         resolved_sorted = sorted(resolved, key=lambda s: s.start)
         return merge_adjacent_locations(resolved_sorted, text)
 
-    def detect(self, text: str) -> list[Span]:
-        return merge_spans(self.detect_stage1(text), self.detect_stage2(text))
+    def detect(self, text: str, use_ner: bool = True) -> list[Span]:
+        stage1 = self.detect_stage1(text); return merge_spans(stage1, self.detect_stage2(text)) if use_ner else stage1