priveil 0.1.0__tar.gz

priveil-0.1.0/.cursor/skills/conduit-core.md

@@ -0,0 +1,54 @@
+---
+name: conduit-core
+description: >
+  Forces the laziest clean pipeline that actually works. Question whether the
+  transform needs to exist at all, reach for standard and internal libraries
+  before writing anything new, validate at every trust boundary, write pure
+  functions that compose, document contracts, and treat security as
+  load-bearing. This is the language-agnostic core. Pair it with conduit-py
+  for concrete tooling.
+argument-hint: "[lite|full|ultra]"
+---
+
+# Conduit — Core
+
+Lazy means efficient, not careless. The best code is the code never written.
+Data flows in from hostile territory, gets validated, passes through the
+minimum pure transforms required, and exits clean. Security is load-bearing,
+not decorative. Types are documentation. The schema is the gate.
+
+## The Ladder
+
+Stop at the first rung that holds:
+
+1. **Does this need to exist at all?** Speculative transform = skip it. (YAGNI)
+2. **Does an internal library already do it?** Use it.
+3. **Does the standard library do it?** Reach for it before any custom logic.
+4. **Does this data cross a serialization boundary?** Give it a schema.
+5. **Is this a transformation?** Pure function. Input → output, no side effects.
+6. **Can it be a pipeline?** Compose. One function per concern.
+7. **Does it cross a trust boundary?** Validate in, sanitize out, log the action (never the secret).
+8. **Is the contract documented?** Docstring — one-line summary, args, return, errors.
+9. **Only then:** write the minimum implementation that works.
+
+## Rules
+
+**Laziness**
+- No unrequested abstractions.
+- Deletion over addition. Shortest working diff wins.
+- `conduit:` comments mark deliberate simplifications — name the ceiling and the upgrade path.
+
+**Security Champion**
+- All external data is hostile until a parsed schema says otherwise.
+- No secrets in logs, no secrets in code. Secrets come from the environment.
+- Parameterised queries only — never build SQL or shell commands by string interpolation.
+- Least privilege: functions receive only the data they need.
+- On bad input: reject loudly with a typed error, never silently coerce.
+
+## Intensity
+
+| Level | What changes |
+|-------|-------------|
+| **lite** | Build what's asked with type hints and a docstring. |
+| **full** | Ladder enforced. YAGNI first, schemas at boundaries, pure transforms, security at every entry point. Default. |
+| **ultra** | YAGNI extremist. Delete before adding. Challenge the requirement before writing a line. |

priveil-0.1.0/.cursor/skills/conduit-py.md

@@ -0,0 +1,50 @@
+---
+name: conduit-py
+description: >
+  Python tooling for the conduit discipline — Pydantic at every serialization
+  boundary, pure composable functions, stdlib-first functional style,
+  Google-style docstrings, and a pytest + hypothesis test stack.
+  Read conduit-core for the ladder, philosophy, and security principles.
+argument-hint: "[lite|full|ultra]"
+---
+
+> Read `conduit-core` before this file. This skill adds Python-specific tooling.
+
+# Conduit — Python
+
+## Data & Types
+
+- Pydantic models at every serialization boundary: API inputs, env vars (`BaseSettings`), external service responses.
+- Never reach into deserialized JSON with `.get()` chains when a model exists. Use `Model.model_validate(data)`.
+- Type hints on every function signature, return type included.
+- Immutable by default: frozen Pydantic models, tuples over lists where mutation adds nothing.
+- Never use `Any` without `# conduit: Any here because [reason]`.
+
+## Functional Style
+
+- Pure functions are the default. Same input → same output, always.
+- Small, composable units. One responsibility each.
+- Generator pipelines for large data: never load what you can stream.
+- `functools` first: `partial`, `reduce`, `lru_cache`.
+- No mutable default arguments. Ever.
+
+## Security Champion (Python surface)
+
+- No secrets in logs, no secrets in code. Env vars via `BaseSettings`.
+- Secret variables wrapped in Pydantic's `SecretStr` or `Secret[T]`.
+- Parameterised queries only. No f-strings into SQL or shell commands.
+- On bad input: reject loudly with a clear `ValueError` or `ValidationError`, never silently coerce.
+
+## Documentation
+
+- Google-style docstrings on every non-trivial function.
+- One-line imperative summary (`Validate and parse...`, `Transform...`).
+- Args / Returns / Raises — one short line each, only what isn't obvious from the type.
+- Never document what the type signature already says.
+
+## Test stack
+
+- `pytest` for all test running. No `unittest`.
+- `hypothesis` for data edge cases: null values, empty collections, out-of-range values, schema surprises.
+- `asyncio_mode = "auto"` in pytest config; use `async def test_` directly.
+- No mocks of internal logic — test real behaviour with real (small) data.

priveil-0.1.0/.cursor/skills/domain-doc.md

@@ -0,0 +1,70 @@
+# Domain Doc
+
+You are a Domain-Driven Design facilitator helping a data engineering or data platform team build out their domain documentation. Your goal is to extract tacit knowledge from the team and turn it into structured, durable documentation that raises the context of every squad member — present and future.
+
+## When to use this skill
+
+- A new domain or subdomain is being defined
+- A team is starting a new service, pipeline, or platform capability
+- An existing area lacks documentation and context is siloed in people's heads
+- Onboarding new engineers who need to understand the domain fast
+
+## How to run a domain doc session
+
+### Step 1 — Establish scope
+
+Ask the user: "What domain or subdomain are we documenting? Give me one sentence on what it does and who it serves."
+
+Wait for their answer before proceeding.
+
+### Step 2 — Extract the ubiquitous language
+
+Work through these questions one at a time (don't dump them all at once):
+
+1. "What are the core nouns in this domain — the things you work with every day? List them out, don't worry about definitions yet."
+2. For each noun: "How would you define [term] to someone joining the team tomorrow? Be precise — what is it, what isn't it, and does it mean something different here than in common usage?"
+3. "Are there any terms that sound similar but mean different things in this context? Or terms outsiders use differently to how you use them?"
+
+Document each term as you go in this format:
+
+```
+**[Term]**
+Definition: [precise definition]
+Alias / external term: [if different outside this team]
+Not to be confused with: [if there's a common mix-up]
+```
+
+### Step 3 — Map bounded contexts
+
+Ask:
+- "Does this domain have clear subdomains — areas that could almost stand alone? What are they?"
+- "Where are the edges? What does this domain own vs. depend on from elsewhere?"
+- "What are the integration points — where does data or control flow in or out?"
+
+### Step 4 — Capture domain events
+
+Ask:
+- "What are the key things that *happen* in this domain? Think in past tense — 'DatasetPublished', 'PipelineRun completed', 'AccessRequest approved'."
+- "Which of these events trigger something else downstream?"
+
+### Step 5 — Decisions and constraints
+
+Document each as an Architecture Decision Record (ADR) stub:
+
+```
+**Decision: [title]**
+Context: [why this came up]
+Decision: [what was decided]
+Consequences: [what this means for the domain]
+Constraints: [privacy/security/compliance if relevant]
+```
+
+### Step 6 — Goals and success
+
+Ask:
+- "What does 'good' look like for this domain? What are you optimising for?"
+- "How do you know when this domain is working well vs. struggling?"
+
+### Step 7 — Produce the output
+
+Assemble everything into a structured markdown document saved at `docs/domain/[domain-name].md`.

priveil-0.1.0/.cursor/skills/ship-and-watch.md

@@ -0,0 +1,86 @@
+---
+name: ship-and-watch
+description: >
+  Opens a pull request with the GitHub CLI, then watches it to completion.
+  Polls CI checks, review approval, and review-thread resolution on a loop
+  until every gate is green. Gates on unresolved review threads — not just
+  CHANGES_REQUESTED. Merges only when all three gates are green, then writes
+  a summary. Use when asked to "ship and watch", "ship it", "raise the PR and
+  merge when green", or to create, monitor, and merge a pull request.
+argument-hint: "[base-branch]"
+---
+
+# Ship and Watch
+
+A PR is mergeable only when **all three gates** are satisfied:
+
+1. **Checks** — every required CI run has passed.
+2. **Approval** — at least one approving review.
+3. **Comments resolved** — every review thread is resolved (use GraphQL, not `reviewDecision`).
+
+## Step 0 — Pre-flight
+
+```bash
+gh auth status
+git rev-parse --abbrev-ref HEAD   # must be a feature branch, not base
+git status --porcelain            # must be clean
+```
+
+Push if needed: `git push -u origin HEAD`
+
+## Step 1 — Open the PR
+
+```bash
+git log --oneline "$(gh repo view --json defaultBranchRef -q .defaultBranchRef.name)..HEAD"
+gh pr create --base "<base>" --head "$(git rev-parse --abbrev-ref HEAD)" \
+  --title "<imperative title>" --body "<what changed and why>"
+PR=$(gh pr view --json number -q .number)
+OPEN_SHA=$(git rev-parse HEAD)
+```
+
+## Step 2 — Watch loop (poll all three gates each round)
+
+**Gate A — Checks:** `gh pr checks "$PR"` — exit 0 = pass, exit 8 = pending, exit 1 = STOP.
+
+**Gate B — Approval:** read `reviewDecision` from `gh pr view "$PR" --json reviewDecision`.
+- `APPROVED` → green. `null` → ask user before merging. `REVIEW_REQUIRED` / `CHANGES_REQUESTED` → keep polling.
+
+**Gate C — Unresolved threads (GraphQL):**
+```bash
+gh api graphql -F owner='<owner>' -F repo='<repo>' -F pr="$PR" -f query='
+  query($owner:String!, $repo:String!, $pr:Int!) {
+    repository(owner:$owner, name:$repo) {
+      pullRequest(number:$pr) {
+        reviewThreads(first:100) {
+          nodes { isResolved isOutdated path comments(first:1) { nodes { author { login } body } } }
+        }
+      }
+    }
+  }' | jq '[.data.repository.pullRequest.reviewThreads.nodes[] | select(.isResolved==false)] | length'
+```
+
+Merge only when count == 0. Never assume `isOutdated == true` means resolved — confirm the fix, then resolve.
+
+## Step 3 — Merge
+
+When A + C green and `mergeable != CONFLICTING`:
+- `APPROVED` → `gh pr merge "$PR" --squash --delete-branch`
+- `null` → prompt user first; wait for explicit yes.
+
+## Step 4 — Ship report
+
+```bash
+git log --oneline "$OPEN_SHA"..HEAD
+git diff --stat "$OPEN_SHA"..HEAD
+gh pr view "$PR" --json reviews,comments
+gh api "repos/<owner>/<repo>/pulls/$PR/comments"
+```
+
+Report: what changed after the PR opened, each piece of feedback and its resolution, final gate status.
+
+## Hard stops
+
+- Gate A exit 1 (check failed).
+- Gate C count > 0 (unresolved threads).
+- `mergeable == CONFLICTING`.
+- 20 polling rounds elapsed — report and hand back.

priveil-0.1.0/.cursor/skills/vertical-slices.md

@@ -0,0 +1,65 @@
+---
+name: vertical-slices
+description: >
+  Guidance for breaking features into tracer bullet vertical slices. Use when
+  designing new features, epics, or initiative plans for this service. Each
+  slice cuts through ALL integration layers end-to-end.
+---
+
+# Vertical Slices
+
+A vertical slice is a thin end-to-end cut through every layer of the system —
+schema, engine, API, tests. It is NOT a horizontal layer (e.g. "add all domain
+models first").
+
+## Rules
+
+- **A completed slice is demo-able or verifiable on its own** — no sibling slice needed.
+- **Each slice delivers a narrow but COMPLETE path** — no half-implemented schemas, no skipped tests.
+- **Prefactoring ships first** as Slice 0: scaffold, shared models, engine wiring.
+- **A new slice NEVER modifies a prior slice's public contract** — extend, don't break.
+- **LLM / AI paths are always additive** — core deterministic paths never depend on them.
+
+## Shape of a good slice
+
+| Layer  | Must include                                    |
+|--------|-------------------------------------------------|
+| Schema | Pydantic request + response models              |
+| Engine | Business logic / service layer                  |
+| API    | FastAPI route wired end-to-end                  |
+| Tests  | ≥1 unit test + ≥1 integration test via ASGI client |
+
+## Anti-patterns
+
+- "Add all the domain models" — horizontal slice, not vertical.
+- "Wire up the engine layer" — same problem.
+- A slice that cannot be verified without a later slice being complete.
+- An API route without a test.
+
+## Process
+
+1. Identify the narrowest path that delivers value.
+2. Name it as an imperative user capability: "Detect entities in text".
+3. List the layers it touches (schema → engine → API → test).
+4. Write the acceptance criteria as observable outputs, not internal state.
+5. Implement each layer in order; run the test before calling the slice done.
+
+## Build order for this service
+
+```
+Slice 0: Scaffold (prerequisite for everything)
+  ↓
+Slice 1: Text entity detection  →  POST /detect
+  ↓
+Slice 2: AU financial recognisers  (extends Slice 1 entities)
+  ↓
+Slice 3: Anonymisation  →  POST /anonymise
+  ↓
+Slice 5: LLM judge (non-streaming)  →  POST /judge
+  ↓
+Slice 6: Streaming judge  →  POST /judge/stream
+
+(Slice 4 — image detection — deferred; see backlog)
+```
+
+Each arrow = hard dependency. Slices at the same level run in parallel.

priveil-0.1.0/.dockerignore

@@ -0,0 +1,17 @@
+.venv/
+__pycache__/
+*.py[cod]
+*.egg-info/
+.git/
+.github/
+.mypy_cache/
+.ruff_cache/
+.pytest_cache/
+.hypothesis/
+dist/
+build/
+htmlcov/
+*.coverage
+.env
+.env.*
+.cursor/

priveil-0.1.0/.env.example

@@ -0,0 +1,43 @@
+# Copy this file to .env and fill in real values.
+# All variables are prefixed with PRIVEIL_ except provider API keys.
+
+# ── LLM judge (optional) ─────────────────────────────────────────────────────
+# Enables two things:
+#   1. Judge mode on POST /detect and POST /anonymise (mode='judge', default)
+#   2. POST /assess — content risk and sensitivity assessment
+#
+# When unset: mode='judge' degrades silently to mode='fast'; /assess returns 503.
+#
+# Built-in providers — use provider:model format:
+#   PRIVEIL_JUDGE_MODEL=anthropic:claude-sonnet-4-6
+#   PRIVEIL_JUDGE_MODEL=openai:gpt-4o
+#
+# Custom OpenAI-compatible endpoint (Databricks, Azure AI, Ollama, etc.):
+#   PRIVEIL_JUDGE_MODEL=<deployment-name>   # model/deployment name on the endpoint
+#   PRIVEIL_JUDGE_BASE_URL=https://<workspace>.azuredatabricks.net/serving-endpoints
+#   PRIVEIL_JUDGE_API_KEY=<personal-access-token>
+PRIVEIL_JUDGE_MODEL=anthropic:claude-sonnet-4-6
+# PRIVEIL_JUDGE_BASE_URL=
+# PRIVEIL_JUDGE_API_KEY=
+
+# Sampling temperature for the judge (0.0 = deterministic).
+PRIVEIL_JUDGE_TEMPERATURE=0.0
+
+# ── Provider API keys ────────────────────────────────────────────────────────
+# Required when PRIVEIL_JUDGE_MODEL uses the anthropic provider.
+ANTHROPIC_API_KEY=your-anthropic-api-key-here
+
+# Required when PRIVEIL_JUDGE_MODEL uses the openai provider.
+# OPENAI_API_KEY=your-openai-api-key-here
+
+# ── spaCy model ──────────────────────────────────────────────────────────────
+# en_core_web_sm  — small, fast, used in CI and tests (default)
+# en_core_web_lg  — larger, more accurate NER, recommended for production
+# Download: uv run python -m spacy download en_core_web_lg
+PRIVEIL_SPACY_MODEL=en_core_web_sm
+
+# ── Server ───────────────────────────────────────────────────────────────────
+PRIVEIL_DEBUG=false
+
+# Thread-pool workers for presidio (CPU-bound). Rule of thumb: number of cores.
+PRIVEIL_EXECUTOR_MAX_WORKERS=4

priveil-0.1.0/.github/dependabot.yml

@@ -0,0 +1,18 @@
+version: 2
+updates:
+  - package-ecosystem: "pip"
+    directory: "/"
+    schedule:
+      interval: "weekly"
+    groups:
+      python-packages:
+        patterns:
+          - "*"
+  - package-ecosystem: "github-actions"
+    directory: "/"
+    schedule:
+      interval: "weekly"
+    groups:
+      github-actions:
+        patterns:
+          - "*"

priveil-0.1.0/.github/workflows/ci.yml

@@ -0,0 +1,51 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+
+jobs:
+  lint:
+    name: Lint & type-check
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v7
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v7
+        with:
+          enable-cache: true
+          python-version-file: ".python-version"
+
+      - name: Install dependencies
+        run: uv sync
+
+      - name: Ruff
+        run: uv run ruff check src/ tests/
+
+      - name: Mypy
+        run: uv run mypy src/
+
+  test:
+    name: Tests (Python ${{ matrix.python-version }})
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.11", "3.12", "3.13"]
+
+    steps:
+      - uses: actions/checkout@v7
+
+      - name: Build test image
+        run: |
+          docker build \
+            --target test \
+            --build-arg PYTHON_VERSION=${{ matrix.python-version }} \
+            --tag priveil-test:${{ matrix.python-version }} \
+            .
+
+      - name: Run tests
+        run: docker run --rm priveil-test:${{ matrix.python-version }}

priveil-0.1.0/.github/workflows/publish.yml

@@ -0,0 +1,35 @@
+name: Publish to PyPi
+
+on:
+  release:
+    types: [published]
+
+jobs:
+  publish-to-pypi:
+    runs-on: ubuntu-latest
+    permissions:
+      id-token: write  # required for OIDC trusted publishing
+    steps:
+    - uses: actions/checkout@v4
+    - name: Set up Python
+      uses: actions/setup-python@v5
+      with:
+        python-version: '3.x'
+    - name: Set up uv
+      uses: astral-sh/setup-uv@v5
+    - name: Build and publish
+      run: |
+        uv build
+        uv publish --trusted-publishing always
+  deploy-docs:
+    runs-on: ubuntu-latest
+    needs:
+      - publish-to-pypi
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: 3.x
+      - uses: astral-sh/setup-uv@v5
+      - run: make install && make install-docs
+      - run: uv run --group docs mkdocs gh-deploy --force

priveil-0.1.0/.gitignore

@@ -0,0 +1,15 @@
+.venv/
+__pycache__/
+*.py[cod]
+*.egg-info/
+dist/
+build/
+.mypy_cache/
+.ruff_cache/
+.pytest_cache/
+.hypothesis/
+*.coverage
+htmlcov/
+.env
+.env.*
+!.env.example

priveil-0.1.0/.python-version

@@ -0,0 +1 @@
+3.12

priveil-0.1.0/Dockerfile

@@ -0,0 +1,38 @@
+ARG PYTHON_VERSION=3.12
+
+# ── base: production dependency install ───────────────────────────────────────
+FROM python:${PYTHON_VERSION}-slim AS base
+
+COPY --from=ghcr.io/astral-sh/uv:0.11.24 /uv /usr/local/bin/uv
+
+ENV UV_SYSTEM_PYTHON=1
+
+WORKDIR /app
+
+# Dependencies before source for layer caching
+COPY pyproject.toml uv.lock ./
+# uv only includes the dev group by default; models must be explicit.
+RUN uv sync --frozen --no-dev --group models --no-cache
+
+COPY src/ ./src/
+RUN uv pip install --no-deps . --no-cache-dir
+
+# ── runtime ───────────────────────────────────────────────────────────────────
+FROM base AS runtime
+
+# Production deployments use en_core_web_lg; download at deploy time via:
+#   PRIVEIL_SPACY_MODEL=en_core_web_lg python -m spacy download en_core_web_lg
+# or bake into a derived image.
+EXPOSE 8000
+
+CMD ["uv", "run", "uvicorn", "priveil.app:app", "--host", "0.0.0.0", "--port", "8000"]
+
+# ── test ──────────────────────────────────────────────────────────────────────
+FROM base AS test
+
+# --all-groups includes dev + models (en-core-web-sm) in one step.
+RUN uv sync --frozen --all-groups --no-cache
+
+COPY tests/ ./tests/
+
+CMD ["uv", "run", "pytest", "tests/", "-v"]

priveil-0.1.0/Makefile

@@ -0,0 +1,35 @@
+.DEFAULT_GOAL := help
+
+.PHONY: help install install-docs serve test lint format docker-build docker-serve docker-test
+
+help:
+	@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-20s\033[0m %s\n", $$1, $$2}'
+
+install: ## Install all dependencies
+	uv sync --all-groups
+
+install-docs: ## Install docs dependencies
+	uv sync --group docs
+
+serve: ## Run the server locally with hot-reload
+	uv run uvicorn priveil.app:app --reload --host 0.0.0.0 --port 8000
+
+test: ## Run tests locally
+	uv run pytest tests/ -v
+
+lint: ## Lint and type-check
+	uv run ruff check src/ tests/
+	uv run mypy src/
+
+format: ## Auto-fix lint issues
+	uv run ruff format src/ tests/
+	uv run ruff check --fix src/ tests/
+
+docker-build: ## Build all Docker images
+	docker compose build
+
+docker-serve: ## Run the service via Docker
+	docker compose up api
+
+docker-test: ## Run tests via Docker
+	docker compose run --rm test

priveil-0.1.0/PKG-INFO

@@ -0,0 +1,14 @@
+Metadata-Version: 2.4
+Name: priveil
+Version: 0.1.0
+Summary: Pseudonymisation service for Australian financial services
+Author-email: Mitchell Lisle <m.lisle90@gmail.com>
+Requires-Python: <4.0,>=3.11
+Requires-Dist: fastapi>=0.138.1
+Requires-Dist: presidio-analyzer>=2.2.362
+Requires-Dist: presidio-anonymizer>=2.2.362
+Requires-Dist: pydantic-ai>=2.0.0
+Requires-Dist: pydantic-settings<3.0,>=2.14.2
+Requires-Dist: pydantic<3.0,>=2.13.4
+Requires-Dist: spacy>=3.8.14
+Requires-Dist: uvicorn[standard]>=0.49.0