dbveil 0.1.0__py3-none-any.whl

dbveil-0.1.0.dist-info/METADATA

@@ -0,0 +1,194 @@
+Metadata-Version: 2.4
+Name: dbveil
+Version: 0.1.0
+Summary: A local read-only, PII-redacting proxy that lets AI agents query your database safely.
+Project-URL: Homepage, https://github.com/mathu97/dbveil
+Project-URL: Repository, https://github.com/mathu97/dbveil
+Author: Mathusan Selvarajah
+License-Expression: MIT
+License-File: LICENSE
+Keywords: ai,claude,database,mcp,pii,postgres,proxy,read-only,redaction
+Requires-Python: >=3.10
+Requires-Dist: asyncpg>=0.29
+Requires-Dist: mcp>=1.2
+Requires-Dist: pglast>=6.0
+Requires-Dist: pydantic>=2.6
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: rich>=13.7
+Requires-Dist: typer>=0.12
+Provides-Extra: dev
+Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
+Requires-Dist: pytest>=8.0; extra == 'dev'
+Provides-Extra: llm
+Requires-Dist: httpx>=0.27; extra == 'llm'
+Provides-Extra: ner
+Requires-Dist: presidio-analyzer>=2.2; extra == 'ner'
+Requires-Dist: presidio-anonymizer>=2.2; extra == 'ner'
+Description-Content-Type: text/markdown
+
+# veil
+
+**A local read-only, PII-redacting proxy that lets AI agents query your database safely.**
+
+Point Claude Code (or any MCP client) at `veil` instead of your database. Every query is
+forced through three deterministic guarantees before a single row reaches the model:
+
+1. **Read-only guard** — the query is parsed with Postgres's real grammar (`libpg_query`).
+   Only `SELECT` / `SHOW` / `EXPLAIN` survive. Writes, DDL, multi-statements, data-modifying
+   CTEs, `SELECT INTO`, and row locks are rejected *before execution* — not by asking the model
+   nicely, by refusing to run them.
+2. **PII redaction** — results are scrubbed before they leave your machine: deterministic
+   column rules + always-on regex for structured PII (emails, phones, cards, SSNs), with an
+   optional NER/LLM backstop for free-text.
+3. **Audit** — every query and verdict is appended to a log you can tail live in a TUI.
+
+A guarded chokepoint in front of the DB, shrunk to a single open-source command with zero
+infrastructure to stand up.
+
+```
+Claude Code ──MCP──▶  veil  ──READ ONLY txn──▶  your database
+                       │
+                       ├─ guard:   parse → allow SELECT only
+                       ├─ redact:  column rules + regex + (optional) NER/LLM
+                       └─ audit:   veil-audit.jsonl
+```
+
+## Why
+
+You want an agent to act as a data analyst over real tables — "compare what we drafted vs what
+was actually sent" — without (a) risking a destructive query or (b) shipping customer PII to a
+model provider. Handing an agent raw DB credentials and hoping it only writes `SELECT` is not a
+control. `veil` makes the unsafe paths impossible at the layer the agent can't talk its way past.
+
+## Install
+
+```bash
+pip install dbveil          # or: uv pip install dbveil
+# optional extras:
+pip install 'dbveil[ner]'   # Presidio NER backstop for names/addresses
+pip install 'dbveil[llm]'   # local-LLM (Ollama) redaction
+```
+
+## Quickstart
+
+```bash
+veil init          # interactive: DB URL + auto-detect PII columns → writes veil.yaml
+veil doctor        # verify guard, connectivity, and that READ ONLY actually blocks writes
+veil test-query "SELECT email, created_at FROM users LIMIT 5"   # try it without an agent
+veil up            # run the MCP proxy on stdio (what Claude Code connects to)
+```
+
+Try a write to see the guard refuse it:
+
+```bash
+veil test-query "DELETE FROM users"
+# BLOCKED — write or DDL operation detected: DELETE
+```
+
+### Connect Claude Code
+
+```bash
+claude mcp add veil -- veil up
+```
+
+or commit a `.mcp.json` so your whole team gets it:
+
+```json
+{
+  "mcpServers": {
+    "veil": { "command": "veil", "args": ["up"], "env": { "VEIL_CONFIG": "veil.yaml" } }
+  }
+}
+```
+
+Now the agent has three tools — `query`, `list_tables`, `describe_table` — and physically
+cannot write or see raw PII.
+
+### Watch it live
+
+```bash
+veil monitor       # TUI tailing veil-audit.jsonl: allowed / blocked / redaction counts
+```
+
+## Configuration
+
+`veil init` writes a commented `veil.yaml`. Full reference in
+[`examples/veil.example.yaml`](examples/veil.example.yaml). The essentials:
+
+```yaml
+database:
+  url: ${DATABASE_URL}          # env refs kept out of the file
+
+guard:
+  allow_select_star: false      # block SELECT * on PII tables; force explicit columns
+  max_rows: 1000
+  statement_timeout_ms: 15000
+  pii_tables: [contacts, users]
+
+redact:
+  builtin_patterns: { email: true, phone: true, credit_card: true, ssn: true, ip: false }
+  columns:
+    - { column: email,     strategy: hash }      # sha256, still join-able
+    - { column: full_name, strategy: mask }      # -> [redacted]
+    - { column: ssn,       strategy: partial, keep: 4 }
+  ner: { enabled: false, engine: presidio }      # optional backstop
+```
+
+## How redaction is layered (and its honest limits)
+
+`veil` defends from the **deterministic** side first, because that's the only kind you can trust
+not to leak:
+
+| Layer | What it catches | Deterministic? |
+|---|---|---|
+| **Column rules** | Known PII columns (`email`, `ssn`, …) by name | ✅ yes |
+| **Built-in regex** | Emails, phones, Luhn-valid cards, SSNs, IPs — even aliased or in free-text | ✅ yes |
+| **NER (Presidio)** | Names / addresses in free-text the above miss | ⚠️ probabilistic |
+| **LLM (Ollama)** | Same, via a local model | ⚠️ probabilistic, experimental |
+
+**Use the probabilistic layers only as a backstop.** ML/NER *will* eventually miss a name or an
+oddly-formatted address — that's a leak. For columns you already know are sensitive, the column
+rules are the real control. The LLM redactor fails *closed*: if the model errors, the cell is
+masked, never passed through.
+
+## Security model
+
+- **Two independent read-only layers.** The parser rejects non-reads, *and* every query runs
+  inside a `SET TRANSACTION READ ONLY` transaction — so even a parser gap can't write.
+- **Give veil a least-privilege credential.** Best practice is a `GRANT SELECT`-only database
+  role (ideally on a read replica). Then "read-only" is enforced by the database itself, and the
+  credential `veil` holds is low-blast-radius: a leak exposes already-masked reads and can write
+  nothing. `veil doctor` confirms the READ ONLY transaction rejects writes against your DB.
+- **PII never leaves your machine unmasked.** Redaction happens in-process, before results are
+  serialized to the MCP client.
+
+## Secure connectivity
+
+`veil` connects to whatever DSN you give it, so the network path is yours to choose:
+
+- **Tailscale** — put your DB behind a tailnet and point `database.url` at the tailnet host. No
+  public DB port.
+- **Short-lived credentials** — `${DATABASE_URL}` is expanded at load, so you can inject an
+  ephemeral token (RDS IAM auth, Cloud SQL IAM, a Vault dynamic user) instead of a static
+  password.
+- **Railway / managed PaaS** — use the provided TLS endpoint with a dedicated read-only role.
+
+## Roadmap
+
+- **Postgres wire-protocol frontend** — so `psql`, BI tools, and any client (not just MCP) get
+  the same guard + redaction. The pipeline is already frontend-agnostic.
+- **More engines** — MySQL, SQLite (the guard's parser is the only Postgres-specific piece; it's
+  a pluggable backend).
+- **Schema-aware lineage** — resolve aliased PII columns back to their source table.
+
+## Development
+
+```bash
+uv venv && source .venv/bin/activate
+uv pip install -e '.[dev]'
+pytest
+```
+
+## License
+
+MIT

dbveil-0.1.0.dist-info/RECORD

@@ -0,0 +1,22 @@
+veil/__init__.py,sha256=2TDNw-XaEYh2swJ1VNHhXslRwR7YVNzLjq6jj-RBIwI,384
+veil/__main__.py,sha256=wcCrL4PjG51r5wVKqJhcoJPTLfHW0wNbD31DrUN0MWI,28
+veil/audit.py,sha256=tpwBYHpzvj-68bqmxvWW4XSEuQshjBp2tLwA-zgbZYc,932
+veil/cli.py,sha256=Qnjy8_XxWZ9zAJTL19vtxMe1LUZ69y_SyTQOMv9_3U0,9425
+veil/config.py,sha256=CBHQlarJ6By3ChbrY0v3vZXx7b6q6cJopA_j595xS1k,2643
+veil/executor.py,sha256=ryoWalRhEagXBxO7I66PeuCBh7OghqkOlz2QfY2E19g,2805
+veil/guard.py,sha256=-ffgmNmcEDLGCGE-07uNMA5ednSjzU5fR5uRheNG1-I,2967
+veil/mcp_server.py,sha256=McnkOwomGxVNfcjre2eda7abDcz7wthCK2yHH1eDlpw,1960
+veil/pipeline.py,sha256=8LPSYtHEe2bJnow6oM6s3DH2Ig95ky7eqFzk_7W-hP0,2218
+veil/result.py,sha256=i2VR_VRrrdzr1cakYYrrHDviVB04j48DADonzfdE0cA,202
+veil/serialize.py,sha256=oZoBRM93EZ4jOaLRsXFdrTFAnWyVSk63-pyjLiXC_3U,771
+veil/tui.py,sha256=elYFEoUzWRqXX8KYjhZ7hXQSIz5hDB8NGMXV85jdxn8,2207
+veil/redact/__init__.py,sha256=VsAYimZ1hSqNLzwuj1-_Sz5osYj8WuzeOZZI30Sf0HE,1373
+veil/redact/column_rules.py,sha256=Es1AzrOK1T3AJTt-CEc2M3DjTas7v64zaYJJ9DXG8ts,1255
+veil/redact/llm.py,sha256=mIcqaFvae42FMZHebKr3_NqH_Mc9YofSI43MIqicKO0,1726
+veil/redact/ner.py,sha256=MJWtHkAEUqVvQwmGb6ZWCGHxXsr3-JPDI6rxiDlPNPY,1238
+veil/redact/patterns.py,sha256=21DlmfhmFyhZCW03cNNg_ALtIOVzm9RNRwk7LTvunt4,1494
+dbveil-0.1.0.dist-info/METADATA,sha256=P_avXOBFoN_koHHwIH-8NFqEvClV4S9dcH1Vd77KZHM,7411
+dbveil-0.1.0.dist-info/WHEEL,sha256=mffPy8wBnZQn2VnJUU5jE99KsxaSfiyMHV9Yt0aLVxs,87
+dbveil-0.1.0.dist-info/entry_points.txt,sha256=YUbYmDRANeZqcDa4NUDoEspe3QBgQiAw3Z7aNijGRco,60
+dbveil-0.1.0.dist-info/licenses/LICENSE,sha256=zQYPO93UqEzYO4Sc_JjNqKUpWCqJooEcdbQkCbIelas,1076
+dbveil-0.1.0.dist-info/RECORD,,

dbveil-0.1.0.dist-info/WHEEL

@@ -0,0 +1,4 @@
+Wheel-Version: 1.0
+Generator: hatchling 1.30.1
+Root-Is-Purelib: true
+Tag: py3-none-any

dbveil-0.1.0.dist-info/entry_points.txt

@@ -0,0 +1,3 @@
+[console_scripts]
+dbveil = veil.cli:app
+veil = veil.cli:app

dbveil-0.1.0.dist-info/licenses/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Mathusan Selvarajah
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

veil/__init__.py

@@ -0,0 +1,14 @@
+from __future__ import annotations
+
+from importlib.metadata import PackageNotFoundError, version
+
+from .config import Config
+from .guard import check_query
+from .pipeline import Pipeline, QueryOutcome
+
+try:
+    __version__ = version("dbveil")
+except PackageNotFoundError:
+    __version__ = "0.0.0+local"
+
+__all__ = ["Config", "Pipeline", "QueryOutcome", "check_query", "__version__"]

veil/__main__.py

@@ -0,0 +1,3 @@
+from .cli import app
+
+app()

veil/audit.py

@@ -0,0 +1,31 @@
+from __future__ import annotations
+
+import json
+from datetime import datetime, timezone
+from pathlib import Path
+
+
+class AuditLog:
+    def __init__(self, path: str) -> None:
+        self.path = Path(path)
+
+    def record(self, sql: str, outcome) -> None:
+        if outcome.blocked_reason:
+            status = "blocked"
+        elif outcome.error:
+            status = "error"
+        else:
+            status = "allowed"
+        entry = {
+            "ts": datetime.now(timezone.utc).isoformat(),
+            "status": status,
+            "sql": sql.strip()[:2000],
+            "reason": outcome.blocked_reason,
+            "error": outcome.error,
+            "rows": outcome.row_count,
+            "redactions": outcome.redactions,
+            "truncated": outcome.truncated,
+            "duration_ms": round(outcome.duration_ms, 1),
+        }
+        with self.path.open("a") as f:
+            f.write(json.dumps(entry) + "\n")

veil/cli.py

@@ -0,0 +1,284 @@
+from __future__ import annotations
+
+import asyncio
+import os
+import re
+
+try:
+    import readline  # noqa: F401 — enables arrow-key line editing in prompts
+except ImportError:
+    pass
+
+import typer
+from rich.console import Console
+from rich.panel import Panel
+from rich.table import Table
+
+from . import __version__
+from .config import Config
+from .executor import Executor
+from .guard import check_query
+
+app = typer.Typer(
+    add_completion=False,
+    help="veil — a local read-only, PII-redacting proxy for safe AI database access.",
+)
+console = Console()
+err = Console(stderr=True)
+
+_PII_HINTS = (
+    "email", "e_mail", "phone", "mobile", "fax", "name", "first", "last",
+    "address", "street", "city", "zip", "postal", "ssn", "social",
+    "dob", "birth", "passport", "license", "ip_address",
+)
+
+
+def _resolve_env(url: str) -> str:
+    return re.sub(r"\$\{([A-Z0-9_]+)\}", lambda m: os.environ.get(m.group(1), m.group(0)), url)
+
+
+def _load(path):
+    try:
+        return Config.load(path)
+    except FileNotFoundError as exc:
+        err.print(f"[red]config error:[/] {exc}")
+        raise typer.Exit(1)
+    except Exception as exc:
+        err.print(f"[red]could not load config:[/] {exc}")
+        raise typer.Exit(1)
+
+
+@app.command()
+def version() -> None:
+    """Print the veil version."""
+    console.print(f"veil {__version__}")
+
+
+@app.command()
+def init(
+    force: bool = typer.Option(False, "--force", "-f", help="Overwrite an existing config."),
+) -> None:
+    """Create a veil.yaml config, optionally auto-detecting PII columns."""
+    path = Config.default_path()
+    if path.exists() and not force:
+        err.print(f"[yellow]{path} already exists. Use --force to overwrite.[/]")
+        raise typer.Exit(1)
+
+    console.print(Panel.fit("[bold]veil init[/] — let's set up safe database access", border_style="cyan"))
+    db_url = typer.prompt(
+        "Database URL (env refs like ${DATABASE_URL} are kept as-is in the file)",
+        default="${DATABASE_URL}",
+    )
+
+    rules: list[tuple[str, str, str]] = []
+    pii_tables: list[str] = []
+    if typer.confirm("Introspect the database now to auto-suggest PII columns?", default=False):
+        try:
+            rules, pii_tables = asyncio.run(_introspect(_resolve_env(db_url)))
+            console.print(f"[green]Found {len(rules)} likely PII column(s) across {len(pii_tables)} table(s).[/]")
+        except Exception as exc:
+            err.print(f"[yellow]Introspection failed ({exc}). Writing a template you can edit by hand.[/]")
+
+    path.write_text(_render_config(db_url, rules, pii_tables))
+    console.print(f"[bold green]Wrote {path}[/]")
+    console.print("Next: [cyan]veil doctor[/] to verify, then [cyan]veil up[/] to run the proxy.")
+
+
+@app.command()
+def doctor(
+    config: str = typer.Option(None, "--config", "-c", help="Path to veil.yaml."),
+) -> None:
+    """Verify the guard, database connectivity, and read-only enforcement."""
+    cfg = _load(config)
+
+    table = Table(title="veil doctor", show_header=True, header_style="bold")
+    table.add_column("check")
+    table.add_column("result")
+
+    guard_ok = (
+        check_query("SELECT 1").allowed
+        and not check_query("DROP TABLE users").allowed
+        and not check_query("UPDATE t SET x = 1").allowed
+        and not check_query("WITH w AS (DELETE FROM t RETURNING *) SELECT * FROM w").allowed
+        and not check_query("SELECT 1; DROP TABLE t").allowed
+    )
+    table.add_row("read-only guard (SELECT allowed, writes blocked)", _mark(guard_ok))
+
+    conn_ok = False
+    readonly_ok = False
+    detail = ""
+    try:
+        conn_ok, readonly_ok = asyncio.run(_probe(cfg))
+    except Exception as exc:
+        detail = str(exc)
+
+    table.add_row("database connection", _mark(conn_ok) + (f"  [dim]{detail}[/]" if detail else ""))
+    table.add_row("server-side READ ONLY transaction rejects writes", _mark(readonly_ok))
+
+    console.print(table)
+    if not (guard_ok and conn_ok and readonly_ok):
+        raise typer.Exit(1)
+
+
+@app.command(name="test-query")
+def test_query(
+    sql: str = typer.Argument(..., help="A read-only SQL query to run through veil."),
+    config: str = typer.Option(None, "--config", "-c"),
+) -> None:
+    """Run one query through the full guard + redact pipeline and print the result."""
+    cfg = _load(config)
+    outcome = asyncio.run(_run_one(cfg, sql))
+
+    if outcome.blocked_reason:
+        console.print(Panel(f"[bold red]BLOCKED[/]\n{outcome.blocked_reason}", border_style="red"))
+        raise typer.Exit(1)
+    if outcome.error:
+        console.print(Panel(f"[bold red]ERROR[/]\n{outcome.error}", border_style="red"))
+        raise typer.Exit(1)
+
+    from .serialize import to_jsonable
+
+    table = Table(show_header=True, header_style="bold")
+    for col in outcome.columns:
+        table.add_column(str(col))
+    from rich.markup import escape
+
+    for row in to_jsonable(outcome.rows):
+        table.add_row(*[("∅" if c is None else escape(str(c))) for c in row])
+    console.print(table)
+    console.print(
+        f"[dim]{outcome.row_count} row(s) · {outcome.redactions} redaction(s)"
+        f"{' · truncated' if outcome.truncated else ''} · {outcome.duration_ms:.0f} ms[/]"
+    )
+
+
+@app.command()
+def up(
+    config: str = typer.Option(None, "--config", "-c", help="Path to veil.yaml."),
+) -> None:
+    """Run the MCP proxy on stdio (this is what Claude Code connects to)."""
+    cfg = _load(config)
+    err.print(
+        f"[bold green]veil[/] up · stdio · guard=read-only · "
+        f"redact={'on' if cfg.redact.columns or cfg.redact.ner.enabled else 'patterns-only'} · "
+        f"audit→{cfg.audit_log}"
+    )
+    from .mcp_server import build_server
+
+    build_server(cfg).run()
+
+
+@app.command()
+def monitor(
+    config: str = typer.Option(None, "--config", "-c"),
+) -> None:
+    """Live view of the audit log (allowed / blocked / redactions). Ctrl-C to quit."""
+    cfg = _load(config)
+    from .tui import run_monitor
+
+    run_monitor(cfg.audit_log)
+
+
+async def _introspect(dsn: str) -> tuple[list[tuple[str, str, str]], list[str]]:
+    ex = Executor(dsn)
+    rs = await ex._fetch_meta(
+        "SELECT table_schema, table_name, column_name "
+        "FROM information_schema.columns "
+        "WHERE table_schema NOT IN ('pg_catalog', 'information_schema') "
+        "ORDER BY table_schema, table_name, ordinal_position"
+    )
+    await ex.close()
+
+    rules: list[tuple[str, str, str]] = []
+    pii_tables: set[str] = set()
+    seen: set[str] = set()
+    for schema, tname, col in rs.rows:
+        low = col.lower()
+        if any(h in low for h in _PII_HINTS):
+            if col not in seen:
+                strategy = "hash" if ("email" in low or "id" in low) else "mask"
+                rules.append((col, strategy, ""))
+                seen.add(col)
+            pii_tables.add(tname)
+    return rules, sorted(pii_tables)
+
+
+async def _probe(cfg: Config) -> tuple[bool, bool]:
+    ex = Executor(cfg.database.url)
+    try:
+        await ex.run("SELECT 1")
+        conn_ok = True
+        readonly_ok = False
+        try:
+            await ex.run("CREATE TEMP TABLE _veil_probe (x int)")
+        except Exception as exc:
+            readonly_ok = "read-only" in str(exc).lower() or "cannot execute" in str(exc).lower()
+        return conn_ok, readonly_ok
+    finally:
+        await ex.close()
+
+
+async def _run_one(cfg: Config, sql: str):
+    from .pipeline import Pipeline
+
+    pipeline = Pipeline(cfg)
+    try:
+        return await pipeline.query(sql)
+    finally:
+        await pipeline.close()
+
+
+def _mark(ok: bool) -> str:
+    return "[green]PASS[/]" if ok else "[red]FAIL[/]"
+
+
+def _render_config(db_url: str, rules: list[tuple[str, str, str]], pii_tables: list[str]) -> str:
+    lines = [
+        "# veil configuration — https://github.com/mathu97/dbveil",
+        "database:",
+        f"  url: {db_url}",
+        "",
+        "guard:",
+        "  allow_select_star: false   # block SELECT * on PII tables; force explicit columns",
+        "  max_rows: 1000",
+        "  statement_timeout_ms: 15000",
+        "  pii_tables:",
+    ]
+    if pii_tables:
+        lines += [f"    - {t}" for t in pii_tables]
+    else:
+        lines.append("    []   # tables where SELECT * is always rejected")
+
+    lines += [
+        "",
+        "redact:",
+        "  # Deterministic, always-on regex redaction for structured PII.",
+        "  builtin_patterns:",
+        "    email: true",
+        "    phone: true",
+        "    credit_card: true",
+        "    ssn: true",
+        "    ip: false",
+        "  hash_salt: \"\"   # set a stable secret to keep hashed values join-able across runs",
+        "  # Column-level rules. strategy: mask | null | hash | partial",
+        "  columns:",
+    ]
+    if rules:
+        for col, strategy, _ in rules:
+            lines.append(f"    - {{ column: {col}, strategy: {strategy} }}")
+    else:
+        lines.append("    []   # e.g. - { column: email, strategy: hash }")
+
+    lines += [
+        "  # Optional probabilistic NER for free-text PII (names/addresses).",
+        "  # Backstop only — never the sole control. Needs: pip install 'dbveil[ner]'",
+        "  ner:",
+        "    enabled: false",
+        "    engine: presidio   # presidio | llm",
+        "    entities: [PERSON, LOCATION, EMAIL_ADDRESS, PHONE_NUMBER]",
+        "    score_threshold: 0.5",
+        "",
+        "audit_log: veil-audit.jsonl",
+        "",
+    ]
+    return "\n".join(lines)

veil/config.py

@@ -0,0 +1,101 @@
+from __future__ import annotations
+
+import os
+import re
+from enum import Enum
+from pathlib import Path
+
+import yaml
+from pydantic import BaseModel, Field
+
+_ENV_PATTERN = re.compile(r"\$\{([A-Z0-9_]+)\}")
+
+
+class RedactStrategy(str, Enum):
+    MASK = "mask"
+    NULL = "null"
+    HASH = "hash"
+    PARTIAL = "partial"
+
+
+class ColumnRule(BaseModel):
+    column: str
+    strategy: RedactStrategy = RedactStrategy.MASK
+    keep: int = 4
+
+
+class BuiltinPatterns(BaseModel):
+    email: bool = True
+    phone: bool = True
+    credit_card: bool = True
+    ssn: bool = True
+    ip: bool = False
+
+
+class NerConfig(BaseModel):
+    enabled: bool = False
+    engine: str = "presidio"
+    entities: list[str] = Field(
+        default_factory=lambda: ["PERSON", "LOCATION", "EMAIL_ADDRESS", "PHONE_NUMBER"]
+    )
+    score_threshold: float = 0.5
+    ollama_url: str = "http://localhost:11434"
+    ollama_model: str = "llama3.2"
+
+
+class RedactConfig(BaseModel):
+    columns: list[ColumnRule] = Field(default_factory=list)
+    builtin_patterns: BuiltinPatterns = Field(default_factory=BuiltinPatterns)
+    ner: NerConfig = Field(default_factory=NerConfig)
+    hash_salt: str = ""
+
+
+class GuardConfig(BaseModel):
+    allow_select_star: bool = False
+    pii_tables: list[str] = Field(default_factory=list)
+    max_rows: int = 1000
+    statement_timeout_ms: int = 15000
+
+
+class DatabaseConfig(BaseModel):
+    url: str
+
+
+class Config(BaseModel):
+    database: DatabaseConfig
+    guard: GuardConfig = Field(default_factory=GuardConfig)
+    redact: RedactConfig = Field(default_factory=RedactConfig)
+    audit_log: str = "veil-audit.jsonl"
+
+    @classmethod
+    def default_path(cls) -> Path:
+        return Path(os.environ.get("VEIL_CONFIG", "veil.yaml"))
+
+    @classmethod
+    def load(cls, path: str | Path | None = None) -> "Config":
+        path = Path(path) if path else cls.default_path()
+        if not path.exists():
+            raise FileNotFoundError(
+                f"config not found at {path}. Run `veil init` to create one."
+            )
+        data = yaml.safe_load(path.read_text()) or {}
+        _expand_env(data)
+        return cls(**data)
+
+    def dump_yaml(self) -> str:
+        return yaml.safe_dump(self.model_dump(mode="json"), sort_keys=False)
+
+
+def _expand_env(node):
+    if isinstance(node, dict):
+        for k, v in node.items():
+            node[k] = _expand_env(v)
+        return node
+    if isinstance(node, list):
+        return [_expand_env(v) for v in node]
+    if isinstance(node, str):
+        def repl(m):
+            return os.environ.get(m.group(1), m.group(0))
+
+        return _ENV_PATTERN.sub(repl, node)
+    return node