shroud-privacy 2.0.20 → 2.2.0

package/README.md

@@ -5,7 +5,7 @@
 <h1 align="center">Shroud — Community Edition</h1>
 
 <p align="center">
-  Privacy obfuscation plugin for <a href="https://openclaw.ai">OpenClaw</a>. Detects sensitive data (PII, network infrastructure, credentials) and replaces it with deterministic fake values before anything reaches the LLM. Tool calls still work because Shroud deobfuscates on the way back.
+  Privacy obfuscation for AI agents. Detects sensitive data (PII, network infrastructure, credentials) and replaces it with deterministic fake values before anything reaches the LLM. Tool calls still work because Shroud deobfuscates on the way back. Works with <a href="https://openclaw.ai">OpenClaw</a> (plugin) or any agent via the Agent Privacy Protocol (APP).
 </p>
 
 > **Open-source Community Edition** — free to use under Apache 2.0 license. [Enterprise Edition](#enterprise-edition) available with additional features for teams.
@@ -26,57 +26,94 @@
 | `before_tool_call` | LLM → Tool | Deobfuscate tool parameters + track tool chain depth |
 | `tool_result_persist` | Tool → History | Obfuscate tool results before storing |
 | `message_sending` | Agent → User | Deobfuscate outbound messages (all channels) |
+| `globalThis.__shroudDeobfuscate` | Agent → Channel | Global deobfuscation hook — called by OpenClaw before ANY channel send |
 
-> **Streaming deobfuscation:** On first load, Shroud patches pi-ai's EventStream to deobfuscate ALL LLM responses at the stream level — every provider (Anthropic, OpenAI, Google), every channel (Slack, WhatsApp, Telegram, etc.). A single gateway restart activates the patch.
+> **Privacy guarantee:** Shroud intercepts ALL outbound LLM API calls (Anthropic, OpenAI, Google, any provider) at the `fetch` level and obfuscates PII in every message — including assistant history and Slack `<mailto:>` markup — before it leaves the process. No PII reaches the LLM. On the channel delivery side, Shroud registers `globalThis.__shroudDeobfuscate` — a single function that OpenClaw calls before sending to ANY channel (Slack, WhatsApp, Signal, web, etc.). One hook, all channels, transparent no-op if Shroud isn't loaded.
+
+> **Requires OpenClaw 2026.3.24 or later** with the channel delivery patch (see [OpenClaw patch](#openclaw-channel-delivery-patch) below).
 
 ## Install
 
-### OpenClaw
+### OpenClaw (2026.3.24+)
 
 ```bash
+# Ensure you're on OpenClaw 2026.3.24 or later
+openclaw --version
+
+# Install Shroud
 openclaw plugins install shroud-privacy
 ```
 
-That's it. Configure in `~/.openclaw/openclaw.json` under `plugins.entries."shroud-privacy".config`.
+Configure in `~/.openclaw/openclaw.json` under `plugins.entries."shroud-privacy".config`. No OpenClaw file modifications needed — Shroud uses runtime prototype patches only.
 
-### From source (development)
+### Any agent (via APP)
+
+The **Agent Privacy Protocol** (APP) lets any AI agent add privacy obfuscation — no OpenClaw required. Shroud ships with an APP server and a Python client.
 
 ```bash
-git clone https://github.com/walterkeating-stack/shroud.git
-cd shroud
-npm install && npm run build
-bash deploy-local.sh     # → OpenClaw (~/.openclaw/extensions/)
+npm install shroud-privacy
 ```
 
-## Updating
+**Python:**
 
-OpenClaw doesn't have a `plugins update` command yet, so updating requires removing the old install first. A helper script is included:
+```python
+from shroud_client import ShroudClient
 
-```bash
-# Update to latest version (preserves your config)
-bash scripts/update-openclaw-plugin.sh
+with ShroudClient() as shroud:
+    # Before sending to LLM
+    result = shroud.obfuscate("Contact admin@acme.com about 10.1.0.1")
+    send_to_llm(result.text)  # "Contact user@example.net about 100.64.0.12"
 
-# Update to a specific version
-bash scripts/update-openclaw-plugin.sh <version>
+    # After receiving from LLM
+    restored = shroud.deobfuscate(llm_response)
+    show_to_user(restored.text)  # original values restored
 ```
 
-The script saves your plugin config from `openclaw.json`, removes the old extension, reinstalls from npm, restores your config, and restarts the gateway.
+Copy `clients/python/shroud_client.py` into your project, or import it directly from the npm install path. Requires Node.js on the PATH.
 
-### Manual update
+**Any language:**
 
-If you prefer to do it manually:
+Spawn the APP server and talk JSON-RPC over stdin/stdout:
 
 ```bash
-# 1. Remove old plugin files
-rm -rf ~/.openclaw/extensions/shroud-privacy
+node node_modules/shroud-privacy/app-server.mjs node_modules/shroud-privacy/dist
+```
 
-# 2. Reinstall (this resets your plugin config to defaults)
-openclaw plugins install shroud-privacy
+Handshake (server writes on startup):
+```json
+{"app":"1.0","engine":"shroud","version":"2.1.0","capabilities":["obfuscate","deobfuscate","batch","stats","health","configure","audit","partitions"]}
+```
 
-# 3. Re-apply your config in ~/.openclaw/openclaw.json
-#    (under plugins.entries."shroud-privacy".config)
+Obfuscate:
+```json
+→ {"id":1,"method":"obfuscate","params":{"text":"Contact admin@acme.com"}}
+← {"id":1,"result":{"text":"Contact user@example.net","entityCount":1,"categories":{"email":1},"modified":true}}
+```
+
+Deobfuscate:
+```json
+→ {"id":2,"method":"deobfuscate","params":{"text":"Contact user@example.net"}}
+← {"id":2,"result":{"text":"Contact admin@acme.com","replacementCount":1,"modified":true}}
+```
 
-# 4. Restart
+Other methods: `reset`, `stats`, `health`, `configure`, `shutdown`.
+
+### From source (development)
+
+```bash
+git clone https://github.com/walterkeating-stack/shroud.git
+cd shroud
+npm install && npm run build
+openclaw plugins install --path .
+openclaw gateway restart
+```
+
+## Updating
+
+```bash
+# Remove old plugin, reinstall from npm, restart
+openclaw plugins remove shroud-privacy
+openclaw plugins install shroud-privacy
 openclaw gateway restart
 ```
 
@@ -195,16 +232,35 @@ alias shroud-stats="node ~/.openclaw/extensions/shroud-privacy/scripts/shroud-st
 
 The CLI reads live stats from `/tmp/shroud-stats.json` (override with `SHROUD_STATS_FILE` env var). The stats file is updated by the running gateway on every obfuscation event.
 
-### Auto-patching on first install
+### How privacy works
+
+Shroud uses runtime prototype patches — **no OpenClaw files are modified**:
+
+**Outbound (PII → LLM):** Patches `globalThis.fetch` to intercept all POST requests to LLM API endpoints (`/messages`, `/chat/completions`). Obfuscates every message in the request body — user, assistant, system, tool results — before the request leaves the process. Strips Slack `<mailto:>` markup to prevent PII leaking through chat formatting. Re-obfuscates deobfuscated assistant messages in conversation history to prevent multi-turn PII leaks. Works for every LLM provider.
+
+**Inbound (LLM → User):** Patches `EventStream.prototype.push()` to deobfuscate streaming responses in real-time. Fake values are replaced with real values as they stream.
+
+**Channel delivery:** Shroud registers `globalThis.__shroudDeobfuscate(text)` — a single global function that converts fake values back to real values. OpenClaw calls this once in its generic message delivery function, before sending to any channel. If Shroud isn't loaded, the function doesn't exist — transparent no-op. The `message_sending` hook provides a backup deobfuscation path.
 
-On first load, Shroud automatically patches pi-ai's `EventStream.push()` to enable streaming deobfuscation across all LLM providers and delivery channels. The patch:
+All patches are applied once at plugin load and are idempotent — subsequent loads detect and skip them.
+
+### OpenClaw channel delivery patch
+
+Shroud requires a one-line addition to OpenClaw's message delivery function — the single point where all outbound channel messages pass through. This covers ALL channels (Slack, WhatsApp, Signal, Telegram, web, etc.) with one change:
+
+```js
+// In OpenClaw's generic message delivery function:
+const deob = globalThis.__shroudDeobfuscate;
+if (deob && typeof text === 'string') text = deob(text);
+```
 
-1. Backs up the original file (`.shroud-backup`)
-2. Injects a 4-line hook that calls `globalThis.__shroudStreamDeobfuscate`
-3. Clears the Node.js V8 compile cache
-4. Triggers a gateway restart via SIGUSR1
+**Why this works:**
+- **One change, all channels** — no per-channel hooks needed
+- **Transparent** — if Shroud isn't loaded, `globalThis.__shroudDeobfuscate` is `undefined`, the `if` is false, zero overhead
+- **All releases** — works on any OpenClaw version that sends channel messages through a common delivery path
+- **Last-moment deobfuscation** — fakes are replaced with real values at the latest possible point, just before the HTTP API call
 
-On subsequent loads, the patch is detected and skipped. To revert: restore the `.shroud-backup` file and restart.
+The `deploy-local.sh` script includes a post-install verification step that tests the full chain.
 
 ### Rule hit counters
 
@@ -303,6 +359,109 @@ With proof hashes enabled:
 
 OpenClaw logs each plugin message twice (once under the plugin subsystem logger, once under the parent `openclaw` logger). This is normal OpenClaw behavior. Filter to `"name":"openclaw"` to get one line per event, as shown in the verify command above.
 
+## Agent Privacy Protocol (APP)
+
+APP is an open protocol for adding privacy obfuscation to any AI agent. Shroud is the reference implementation.
+
+### Overview
+
+```
+┌─────────────────┐     stdin/stdout     ┌──────────────────┐
+│   Your Agent    │ ◄──── JSON-RPC ────► │  APP Server      │
+│  (any language) │                      │  (app-server.mjs)│
+└─────────────────┘                      └──────────────────┘
+        │                                        │
+        │ 1. obfuscate(user_input)               │ detects PII,
+        │ 2. send to LLM ──────────────►         │ returns fakes
+        │ 3. deobfuscate(llm_response)           │ restores reals
+        │ 4. show to user                        │
+```
+
+### Protocol specification
+
+- **Transport**: Newline-delimited JSON-RPC 2.0 over stdin/stdout
+- **Encoding**: UTF-8
+- **Process model**: Agent spawns APP server as subprocess, one per agent instance
+
+### Handshake
+
+On startup, the server writes a single JSON line to stdout:
+
+```json
+{"app":"1.0","engine":"shroud","version":"2.1.0","capabilities":["obfuscate","deobfuscate","batch","stats","health","configure","audit","partitions"]}
+```
+
+The agent must read this line before sending requests. Fields:
+- `app` — protocol version (always `"1.0"`)
+- `engine` — implementation name
+- `version` — implementation version
+- `capabilities` — supported methods
+
+### Methods
+
+| Method | Params | Returns | Description |
+|--------|--------|---------|-------------|
+| `obfuscate` | `{text}` | `{text, entityCount, categories, modified, audit}` | Replace real values with fakes |
+| `deobfuscate` | `{text}` | `{text, replacementCount, modified, audit}` | Restore fakes to real values |
+| `reset` | `{}` | `{ok, summary}` | Clear all mappings |
+| `stats` | `{}` | `{storeMappings, ruleHits, ...}` | Engine statistics |
+| `health` | `{}` | `{uptime, requests, avgLatencyMs}` | Liveness check |
+| `configure` | `{config}` | `{ok}` | Hot-reload configuration |
+| `batch` | `{operations: [{direction, text}]}` | `{results: [...]}` | Batch obfuscate/deobfuscate |
+| `shutdown` | `{}` | `{ok}` | Graceful shutdown (flushes stats) |
+
+### Request/response format
+
+```
+→ {"id":1,"method":"obfuscate","params":{"text":"Server 10.1.0.1 is down"}}
+← {"id":1,"result":{"text":"Server 100.64.0.12 is down","entityCount":1,"categories":{"ip_address":1},"modified":true,"audit":{"requestId":"a1b2c3","proofIn":"8a3c1f","proofOut":"f7d2a1"}}}
+```
+
+Errors:
+```
+← {"id":1,"error":{"code":-32602,"message":"Missing required param: text"}}
+```
+
+### Heartbeat
+
+The server writes JSON heartbeats to stderr every 30 seconds:
+```json
+{"heartbeat":true,"pid":12345,"uptime":120,"requests":42,"avgLatencyMs":1.2,"storeSize":15,"memoryMB":28}
+```
+
+### Integration checklist
+
+1. `npm install shroud-privacy`
+2. Spawn: `node node_modules/shroud-privacy/app-server.mjs node_modules/shroud-privacy/dist`
+3. Read handshake line from stdout
+4. Before LLM: send `obfuscate`, use returned `text`
+5. After LLM: send `deobfuscate`, show returned `text` to user
+6. On agent shutdown: send `shutdown`
+
+### Python client
+
+A ready-made Python client is included at `clients/python/shroud_client.py`:
+
+```python
+from shroud_client import ShroudClient
+
+client = ShroudClient()
+client.start()
+
+safe = client.obfuscate("Contact admin@acme.com about 10.1.0.1")
+print(safe.text)          # fakes
+print(safe.entity_count)  # 2
+print(safe.categories)    # {"email": 1, "ip_address": 1}
+
+real = client.deobfuscate(llm_response)
+print(real.text)           # originals restored
+print(real.residual_fakes) # any CGNAT/ULA IPs that survived
+
+client.stop()
+```
+
+Supports context manager, auto-restart on crash, residual fake detection, and hot-reload via `configure()`.
+
 ## Development
 
 ```bash
@@ -316,7 +475,7 @@ npm run lint      # type-check without emitting
 
 ```bash
 npm run build
-bash deploy-local.sh   # → OpenClaw (~/.openclaw/extensions/shroud-privacy/)
+openclaw plugins install --path .
 openclaw gateway restart
 ```
 
@@ -329,8 +488,8 @@ openclaw gateway restart
 # 2. Update CHANGELOG.md
 # 3. Commit and tag
 git add -A
-git commit -m "Release v1.x.y"
-git tag v1.x.y
+git commit -m "release: vX.Y.Z"
+git tag vX.Y.Z
 git push && git push --tags
 ```