shroud-privacy 2.0.19 → 2.1.0

package/README.md

@@ -1,6 +1,12 @@
-# Shroud — Community Edition
+<p align="center">
+  <img src="logo.png" alt="Shroud" width="160" height="160">
+</p>
 
-Privacy obfuscation plugin for [OpenClaw](https://openclaw.ai). Detects sensitive data (PII, network infrastructure, credentials) and replaces it with deterministic fake values before anything reaches the LLM. Tool calls still work because Shroud deobfuscates on the way back.
+<h1 align="center">Shroud — Community Edition</h1>
+
+<p align="center">
+  Privacy obfuscation for AI agents. Detects sensitive data (PII, network infrastructure, credentials) and replaces it with deterministic fake values before anything reaches the LLM. Tool calls still work because Shroud deobfuscates on the way back. Works with <a href="https://openclaw.ai">OpenClaw</a> (plugin) or any agent via the Agent Privacy Protocol (APP).
+</p>
 
 > **Open-source Community Edition** — free to use under Apache 2.0 license. [Enterprise Edition](#enterprise-edition) available with additional features for teams.
 
@@ -19,19 +25,77 @@ Privacy obfuscation plugin for [OpenClaw](https://openclaw.ai). Detects sensitiv
 | `before_message_write` | Any → History | Obfuscate non-assistant messages; deobfuscate assistant messages |
 | `before_tool_call` | LLM → Tool | Deobfuscate tool parameters + track tool chain depth |
 | `tool_result_persist` | Tool → History | Obfuscate tool results before storing |
-| `message_sending` | Agent → User | Deobfuscate outbound messages (Telegram only) |
+| `message_sending` | Agent → User | Deobfuscate outbound messages (all channels) |
+
+> **Privacy guarantee:** Shroud intercepts ALL outbound LLM API calls (Anthropic, OpenAI, Google, any provider) at the `fetch` level and obfuscates PII in every message — including assistant history and Slack `<mailto:>` markup — before it leaves the process. No PII reaches the LLM. On the inbound side, streaming responses are deobfuscated in real-time via `EventStream.prototype.push()` — no OpenClaw file modifications needed.
 
-> **Streaming deobfuscation:** On first load, Shroud patches pi-ai's EventStream to deobfuscate ALL LLM responses at the stream level — every provider (Anthropic, OpenAI, Google), every channel (Slack, WhatsApp, Telegram, etc.). A single gateway restart activates the patch.
+> **Requires OpenClaw 2026.3.24 or later.** Older versions do not call `message_sending` for Slack/WhatsApp channels, causing duplicate messages with fake tokens. Shroud 2.1+ is tested exclusively against OpenClaw 2026.3.24.
 
 ## Install
 
-### OpenClaw
+### OpenClaw (2026.3.24+)
 
 ```bash
+# Ensure you're on OpenClaw 2026.3.24 or later
+openclaw --version
+
+# Install Shroud
 openclaw plugins install shroud-privacy
 ```
 
-That's it. Configure in `~/.openclaw/openclaw.json` under `plugins.entries."shroud-privacy".config`.
+Configure in `~/.openclaw/openclaw.json` under `plugins.entries."shroud-privacy".config`. No OpenClaw file modifications needed — Shroud uses runtime prototype patches only.
+
+### Any agent (via APP)
+
+The **Agent Privacy Protocol** (APP) lets any AI agent add privacy obfuscation — no OpenClaw required. Shroud ships with an APP server and a Python client.
+
+```bash
+npm install shroud-privacy
+```
+
+**Python:**
+
+```python
+from shroud_client import ShroudClient
+
+with ShroudClient() as shroud:
+    # Before sending to LLM
+    result = shroud.obfuscate("Contact admin@acme.com about 10.1.0.1")
+    send_to_llm(result.text)  # "Contact user@example.net about 100.64.0.12"
+
+    # After receiving from LLM
+    restored = shroud.deobfuscate(llm_response)
+    show_to_user(restored.text)  # original values restored
+```
+
+Copy `clients/python/shroud_client.py` into your project, or import it directly from the npm install path. Requires Node.js on the PATH.
+
+**Any language:**
+
+Spawn the APP server and talk JSON-RPC over stdin/stdout:
+
+```bash
+node node_modules/shroud-privacy/app-server.mjs node_modules/shroud-privacy/dist
+```
+
+Handshake (server writes on startup):
+```json
+{"app":"1.0","engine":"shroud","version":"2.1.0","capabilities":["obfuscate","deobfuscate","batch","stats","health","configure","audit","partitions"]}
+```
+
+Obfuscate:
+```json
+→ {"id":1,"method":"obfuscate","params":{"text":"Contact admin@acme.com"}}
+← {"id":1,"result":{"text":"Contact user@example.net","entityCount":1,"categories":{"email":1},"modified":true}}
+```
+
+Deobfuscate:
+```json
+→ {"id":2,"method":"deobfuscate","params":{"text":"Contact user@example.net"}}
+← {"id":2,"result":{"text":"Contact admin@acme.com","replacementCount":1,"modified":true}}
+```
+
+Other methods: `reset`, `stats`, `health`, `configure`, `shutdown`.
 
 ### From source (development)
 
@@ -189,16 +253,17 @@ alias shroud-stats="node ~/.openclaw/extensions/shroud-privacy/scripts/shroud-st
 
 The CLI reads live stats from `/tmp/shroud-stats.json` (override with `SHROUD_STATS_FILE` env var). The stats file is updated by the running gateway on every obfuscation event.
 
-### Auto-patching on first install
+### How privacy works
+
+Shroud uses runtime prototype patches — **no OpenClaw files are modified**:
+
+**Outbound (PII → LLM):** Patches `globalThis.fetch` to intercept all POST requests to LLM API endpoints (`/messages`, `/chat/completions`). Obfuscates every message in the request body — user, assistant, system, tool results — before the request leaves the process. Strips Slack `<mailto:>` markup to prevent PII leaking through chat formatting. Re-obfuscates deobfuscated assistant messages in conversation history to prevent multi-turn PII leaks. Works for every LLM provider.
 
-On first load, Shroud automatically patches pi-ai's `EventStream.push()` to enable streaming deobfuscation across all LLM providers and delivery channels. The patch:
+**Inbound (LLM → User):** Patches `EventStream.prototype.push()` to deobfuscate streaming responses in real-time. Fake values are replaced with real values as they stream.
 
-1. Backs up the original file (`.shroud-backup`)
-2. Injects a 4-line hook that calls `globalThis.__shroudStreamDeobfuscate`
-3. Clears the Node.js V8 compile cache
-4. Triggers a gateway restart via SIGUSR1
+**Channel delivery:** On OpenClaw 2026.3.24+, the `message_sending` hook fires for ALL channels (Slack, WhatsApp, Telegram, etc.) and deobfuscates outbound messages. Older OpenClaw versions skip this hook for some channels, causing fake tokens in channel output.
 
-On subsequent loads, the patch is detected and skipped. To revert: restore the `.shroud-backup` file and restart.
+All patches are applied once at plugin load and are idempotent — subsequent loads detect and skip them.
 
 ### Rule hit counters
 
@@ -297,11 +362,114 @@ With proof hashes enabled:
 
 OpenClaw logs each plugin message twice (once under the plugin subsystem logger, once under the parent `openclaw` logger). This is normal OpenClaw behavior. Filter to `"name":"openclaw"` to get one line per event, as shown in the verify command above.
 
+## Agent Privacy Protocol (APP)
+
+APP is an open protocol for adding privacy obfuscation to any AI agent. Shroud is the reference implementation.
+
+### Overview
+
+```
+┌─────────────────┐     stdin/stdout     ┌──────────────────┐
+│   Your Agent    │ ◄──── JSON-RPC ────► │  APP Server      │
+│  (any language) │                      │  (app-server.mjs)│
+└─────────────────┘                      └──────────────────┘
+        │                                        │
+        │ 1. obfuscate(user_input)               │ detects PII,
+        │ 2. send to LLM ──────────────►         │ returns fakes
+        │ 3. deobfuscate(llm_response)           │ restores reals
+        │ 4. show to user                        │
+```
+
+### Protocol specification
+
+- **Transport**: Newline-delimited JSON-RPC 2.0 over stdin/stdout
+- **Encoding**: UTF-8
+- **Process model**: Agent spawns APP server as subprocess, one per agent instance
+
+### Handshake
+
+On startup, the server writes a single JSON line to stdout:
+
+```json
+{"app":"1.0","engine":"shroud","version":"2.1.0","capabilities":["obfuscate","deobfuscate","batch","stats","health","configure","audit","partitions"]}
+```
+
+The agent must read this line before sending requests. Fields:
+- `app` — protocol version (always `"1.0"`)
+- `engine` — implementation name
+- `version` — implementation version
+- `capabilities` — supported methods
+
+### Methods
+
+| Method | Params | Returns | Description |
+|--------|--------|---------|-------------|
+| `obfuscate` | `{text}` | `{text, entityCount, categories, modified, audit}` | Replace real values with fakes |
+| `deobfuscate` | `{text}` | `{text, replacementCount, modified, audit}` | Restore fakes to real values |
+| `reset` | `{}` | `{ok, summary}` | Clear all mappings |
+| `stats` | `{}` | `{storeMappings, ruleHits, ...}` | Engine statistics |
+| `health` | `{}` | `{uptime, requests, avgLatencyMs}` | Liveness check |
+| `configure` | `{config}` | `{ok}` | Hot-reload configuration |
+| `batch` | `{operations: [{direction, text}]}` | `{results: [...]}` | Batch obfuscate/deobfuscate |
+| `shutdown` | `{}` | `{ok}` | Graceful shutdown (flushes stats) |
+
+### Request/response format
+
+```
+→ {"id":1,"method":"obfuscate","params":{"text":"Server 10.1.0.1 is down"}}
+← {"id":1,"result":{"text":"Server 100.64.0.12 is down","entityCount":1,"categories":{"ip_address":1},"modified":true,"audit":{"requestId":"a1b2c3","proofIn":"8a3c1f","proofOut":"f7d2a1"}}}
+```
+
+Errors:
+```
+← {"id":1,"error":{"code":-32602,"message":"Missing required param: text"}}
+```
+
+### Heartbeat
+
+The server writes JSON heartbeats to stderr every 30 seconds:
+```json
+{"heartbeat":true,"pid":12345,"uptime":120,"requests":42,"avgLatencyMs":1.2,"storeSize":15,"memoryMB":28}
+```
+
+### Integration checklist
+
+1. `npm install shroud-privacy`
+2. Spawn: `node node_modules/shroud-privacy/app-server.mjs node_modules/shroud-privacy/dist`
+3. Read handshake line from stdout
+4. Before LLM: send `obfuscate`, use returned `text`
+5. After LLM: send `deobfuscate`, show returned `text` to user
+6. On agent shutdown: send `shutdown`
+
+### Python client
+
+A ready-made Python client is included at `clients/python/shroud_client.py`:
+
+```python
+from shroud_client import ShroudClient
+
+client = ShroudClient()
+client.start()
+
+safe = client.obfuscate("Contact admin@acme.com about 10.1.0.1")
+print(safe.text)          # fakes
+print(safe.entity_count)  # 2
+print(safe.categories)    # {"email": 1, "ip_address": 1}
+
+real = client.deobfuscate(llm_response)
+print(real.text)           # originals restored
+print(real.residual_fakes) # any CGNAT/ULA IPs that survived
+
+client.stop()
+```
+
+Supports context manager, auto-restart on crash, residual fake detection, and hot-reload via `configure()`.
+
 ## Development
 
 ```bash
 npm install
-npm test          # run vitest (210 tests)
+npm test          # run vitest (718 tests)
 npm run build     # compile TypeScript
 npm run lint      # type-check without emitting
 ```