@datafog/fogclaw 0.1.5 → 0.2.0

package/docs/specs/2026-02-17-feat-tool-result-pii-scanning-spec.md

@@ -0,0 +1,122 @@
+---
+slug: 2026-02-17-feat-tool-result-pii-scanning
+status: intake-complete
+date: 2026-02-17T00:00:00Z
+owner: sidmohan
+plan_mode: lightweight
+spike_recommended: no
+priority: high
+---
+
+# feat: Add PII scanning to tool results via tool_result_persist hook
+
+## Purpose / Big Picture
+
+FogClaw currently only scans the user prompt text (`before_agent_start`). The majority of PII entering an agent's context comes from **tool results** — file reads, web fetches, API calls, database queries. This content bypasses FogClaw entirely today.
+
+By hooking into OpenClaw's `tool_result_persist` lifecycle, FogClaw can scan and redact PII in tool results **before they are persisted to the session transcript**, closing the largest gap in FogClaw's coverage.
+
+## Scope
+
+### In Scope
+
+- Register a `tool_result_persist` hook handler in FogClaw's plugin registration
+- Extract text content from `AgentMessage` tool result payloads
+- Scan extracted text using the **regex engine only** (synchronous constraint)
+- Apply the existing `guardrail_mode`, `entityActions`, `redactStrategy`, and `allowlist` config to detected entities
+- Redact PII spans in tool result text content (all modes — redact, block, warn — produce span-level redaction in tool results)
+- Emit audit log entries for tool result detections when `auditEnabled: true`
+- Add unit tests for the new hook handler
+- Add integration test confirming the hook registers and transforms tool results
+
+### Boundaries
+
+- **No GLiNER on this path.** The `tool_result_persist` hook is synchronous-only; async handlers are rejected by OpenClaw. Regex covers structured PII (SSN, email, phone, credit card, IP, date, zip). Unstructured entity detection (person names, organizations) is out of scope for this hook.
+- **No `before_tool_call` hook.** This hook exists in OpenClaw's type system but has zero active invocation sites upstream. Will be addressed in a future initiative once OpenClaw wires it in.
+- **No `message_sending` hook.** Outbound message scanning is a separate priority.
+- **No scanning of `event.messages` history.** Historical message scanning is a separate priority.
+- **No new config surface.** Reuse existing FogClaw config — no `toolResultScanning` sub-object.
+- **No changes to OpenClaw upstream.** This initiative is FogClaw-only.
+
+## Non-Goals
+
+- Blocking tool execution (requires `before_tool_call`, which is not wired upstream)
+- Modifying files on disk
+- Scanning binary/image content in tool results
+- Real-time GLiNER inference on tool results
+
+## Risks
+
+- **Performance on hot path.** `tool_result_persist` runs synchronously on every tool result. Regex scanning is sub-millisecond for typical payloads, but very large tool results (e.g., reading a 10K-line file) could add measurable latency. Mitigation: benchmark and consider a size cap with configurable threshold.
+- **AgentMessage structure varies.** Tool results are typed as `AgentMessage`, whose internal structure depends on the tool and provider. Text extraction must handle multiple content formats without crashing on unexpected shapes. Mitigation: defensive extraction with fallback to no-op.
+- **Redaction alters tool output semantics.** Replacing `123-45-6789` with `[SSN_1]` in a tool result changes what the model sees. This is the intended behavior, but could cause unexpected downstream effects if the model tries to use the redacted value literally. Mitigation: this is inherent to the feature and matches existing `before_agent_start` behavior.
+
+## Rollout
+
+- Ship as part of next FogClaw patch release (0.1.7 or 0.2.0)
+- Enabled by default when FogClaw is enabled (no separate toggle)
+- Audit logging captures tool result scans for observability
+
+## Validation and Acceptance Signals
+
+- Unit tests pass for text extraction from various `AgentMessage` shapes
+- Unit tests pass for regex scanning + redaction of tool result content
+- Integration test confirms `tool_result_persist` hook registers via `api.on()`
+- Integration test confirms a tool result containing PII is transformed before persistence
+- Audit log entries are emitted for tool result detections
+- Existing `before_agent_start` tests continue to pass (no regression)
+- Manual verification: install FogClaw in OpenClaw, have agent read a file with PII, confirm session transcript shows redacted content
+
+## Requirements
+
+| ID | Priority | Requirement |
+|---|---|---|
+| R1 | critical | Register a `tool_result_persist` hook handler that scans tool result text for PII using the regex engine |
+| R2 | critical | Redact detected PII spans in tool result messages using the configured `redactStrategy` (token/mask/hash) |
+| R3 | critical | Handler must be synchronous (no Promises returned) — OpenClaw rejects async `tool_result_persist` handlers |
+| R4 | high | Apply existing `entityActions` and `guardrail_mode` config to determine per-entity action; all actions produce span-level redaction in tool results |
+| R5 | high | Respect existing `allowlist` config (global values, patterns, per-entity lists) |
+| R6 | high | Extract text content defensively from `AgentMessage` payloads — handle string content, array-of-content-blocks, and unexpected shapes without throwing |
+| R7 | medium | Emit audit log entry per tool result scan when `auditEnabled: true`, including tool name, entity count, and labels (no raw PII values in logs) |
+| R8 | medium | Skip scanning for tool results with no extractable text content (binary, empty, non-string) |
+| R9 | low | Include `source: "tool_result"` in audit log entries to distinguish from prompt-level scans |
+
+## Key Decisions
+
+- **Regex-only on hot path**: GLiNER is async and cannot run in a synchronous hook. Regex covers the 7 structured PII types (SSN, email, phone, credit card, IP, date, zip) at sub-millisecond latency. This is a deliberate tradeoff — unstructured entities (person names, orgs) are not scanned in tool results.
+- **Reuse existing config**: No separate config section for tool result scanning. The same `guardrail_mode`, `entityActions`, `redactStrategy`, and `allowlist` apply everywhere. Simpler mental model for users.
+- **Span-level redaction for all modes**: Even when `entityActions` says `block` for an entity type, the tool result is redacted at the span level (not replaced entirely). This preserves non-PII context for the agent while removing sensitive values.
+
+## Success Criteria
+
+- PII in tool results (file reads, web fetches, etc.) is redacted before entering the session transcript
+- Regex engine detects SSN, email, phone, credit card, IP, date, and zip in tool result content
+- No measurable latency impact for typical tool results (<1KB text)
+- Audit log captures tool result scan events with entity counts and labels
+- All existing tests pass; new tests cover the hook handler, text extraction, and edge cases
+
+## Constraints
+
+- `tool_result_persist` handler MUST be synchronous (OpenClaw constraint)
+- Must not introduce new dependencies
+- Must not change the existing `FogClawConfig` type (reuse existing fields)
+- Regex engine only — no ONNX/GLiNER on this path
+
+## Priority
+
+- priority: high
+- rationale: This closes the single largest gap in FogClaw's PII coverage. Tool results are the primary vector for PII entering agent context, and this hook is the only active interception point OpenClaw provides for that data flow.
+
+## Initial Milestone Candidates
+
+- M1: Text extraction utility — defensively extract text from `AgentMessage` tool result payloads, handling string content, content block arrays, and edge cases. Likely files: `src/extract.ts`, `tests/extract.test.ts`.
+- M2: `tool_result_persist` hook handler — register the hook, wire in regex scanning + redaction + audit logging, return transformed message. Likely files: `src/index.ts`, `tests/tool-result-hook.test.ts`.
+- M3: Integration smoke test — end-to-end test confirming a registered FogClaw plugin transforms a tool result containing PII. Likely files: `tests/plugin-smoke.test.ts` (extend existing).
+
+## Handoff
+
+After spec approval, proceed to `he-plan` for implementation breakdown. No spike needed — the OpenClaw hook contract is well-documented and the regex engine + redactor already exist in FogClaw.
+
+## Revision Notes
+
+- 2026-02-17T00:00:00Z: Initialized spec from template. Reason: establish intake baseline for tool result PII scanning via `tool_result_persist` hook.

package/fogclaw.config.example.json

@@ -4,6 +4,10 @@
   "redactStrategy": "token",
   "model": "onnx-community/gliner_large-v2.1",
   "confidence_threshold": 0.5,
+  "entityConfidenceThresholds": {
+    "PERSON": 0.6,
+    "ORGANIZATION": 0.7
+  },
   "custom_entities": ["project codename", "internal tool name"],
   "entityActions": {
     "SSN": "block",
@@ -11,5 +15,19 @@
     "EMAIL": "redact",
     "PHONE": "redact",
     "PERSON": "warn"
-  }
+  },
+  "allowlist": {
+    "values": [
+      "noreply@example.com"
+    ],
+    "patterns": [
+      "^internal-"
+    ],
+    "entities": {
+      "PERSON": [
+        "john doe"
+      ]
+    }
+  },
+  "auditEnabled": true
 }

package/openclaw.plugin.json

@@ -1,8 +1,8 @@
 {
   "id": "fogclaw",
   "name": "FogClaw",
-  "version": "0.1.4",
-  "description": "PII detection & custom entity redaction powered by DataFog",
+  "version": "0.2.0",
+  "description": "PII detection & custom entity redaction plugin powered by DataFog",
   "configSchema": {
     "type": "object",
     "properties": {
@@ -30,6 +30,15 @@
         "minimum": 0,
         "maximum": 1
       },
+      "entityConfidenceThresholds": {
+        "type": "object",
+        "additionalProperties": {
+          "type": "number",
+          "minimum": 0,
+          "maximum": 1
+        },
+        "default": {}
+      },
       "custom_entities": {
         "type": "array",
         "items": {
@@ -44,6 +53,43 @@
           "enum": ["redact", "block", "warn"]
         },
         "default": {}
+      },
+      "allowlist": {
+        "type": "object",
+        "properties": {
+          "values": {
+            "type": "array",
+            "items": {
+              "type": "string"
+            },
+            "default": []
+          },
+          "patterns": {
+            "type": "array",
+            "items": {
+              "type": "string"
+            },
+            "default": []
+          },
+          "entities": {
+            "type": "object",
+            "additionalProperties": {
+              "type": "array",
+              "items": {
+                "type": "string"
+              }
+            }
+          }
+        },
+        "default": {
+          "values": [],
+          "patterns": [],
+          "entities": {}
+        }
+      },
+      "auditEnabled": {
+        "type": "boolean",
+        "default": true
       }
     }
   },
@@ -70,6 +116,11 @@
       "help": "Minimum GLiNER score (0-1) required before an entity is treated as a detection.",
       "advanced": true
     },
+    "entityConfidenceThresholds": {
+      "label": "Per-Entity Confidence Thresholds",
+      "help": "Override confidence thresholds by entity label (for example: {\"PERSON\": 0.95, \"ORGANIZATION\": 0.7}).",
+      "advanced": true
+    },
     "custom_entities": {
       "label": "Custom Entity Labels",
       "help": "Extra labels to detect as sensitive entities (for example: `project code`, `competitor name`)."
@@ -78,6 +129,16 @@
       "label": "Entity Actions",
       "help": "Map specific entity labels to per-entity behavior (for example: {\"EMAIL\": \"block\", \"PHONE\": \"redact\"}).",
       "advanced": true
+    },
+    "allowlist": {
+      "label": "Allowlist / Exemptions",
+      "help": "Global and per-entity allowlist entries to exclude from enforcement. Supports exact values and regex patterns.",
+      "advanced": true
+    },
+    "auditEnabled": {
+      "label": "Audit Logging",
+      "help": "Emit structured audit summaries for guardrail decisions.",
+      "advanced": true
     }
   }
 }

package/package.json

@@ -1,10 +1,17 @@
 {
   "name": "@datafog/fogclaw",
-  "version": "0.1.5",
+  "version": "0.2.0",
   "description": "OpenClaw plugin for PII detection & custom entity redaction powered by DataFog",
   "type": "module",
   "main": "dist/index.js",
   "types": "dist/index.d.ts",
+  "scripts": {
+    "build": "tsc",
+    "test": "vitest run",
+    "test:watch": "vitest",
+    "test:plugin-smoke": "vitest run tests/plugin-smoke.test.ts",
+    "lint": "tsc --noEmit"
+  },
   "dependencies": {
     "gliner": "^0.0.19",
     "onnxruntime-node": "1.19.2",
@@ -32,12 +39,5 @@
   "overrides": {
     "onnxruntime-web": "1.21.0",
     "sharp": "0.34.5"
-  },
-  "scripts": {
-    "build": "tsc",
-    "test": "vitest run",
-    "test:watch": "vitest",
-    "test:plugin-smoke": "vitest run tests/plugin-smoke.test.ts",
-    "lint": "tsc --noEmit"
   }
-}
+}

package/src/config.ts

@@ -1,8 +1,109 @@
-import type { FogClawConfig, GuardrailAction, RedactStrategy } from "./types.js";
+import {
+  canonicalType,
+  type EntityAllowlist,
+  type FogClawConfig,
+  type GuardrailAction,
+  type RedactStrategy,
+} from "./types.js";
 
 const VALID_GUARDRAIL_MODES: GuardrailAction[] = ["redact", "block", "warn"];
 const VALID_REDACT_STRATEGIES: RedactStrategy[] = ["token", "mask", "hash"];
 
+function ensureStringList(value: unknown, path: string): string[] {
+  if (!Array.isArray(value)) {
+    throw new Error(`${path} must be an array of strings`);
+  }
+
+  const entries = value.filter((entry): entry is string => {
+    if (typeof entry !== "string") {
+      throw new Error(`${path} must contain only strings`);
+    }
+
+    return true;
+  });
+
+  return entries.map((entry) => entry.trim()).filter((entry) => entry.length > 0);
+}
+
+function ensureEntityAllowlist(value: unknown): EntityAllowlist {
+  if (value == null) {
+    return { values: [], patterns: [], entities: {} };
+  }
+
+  if (typeof value !== "object" || Array.isArray(value)) {
+    throw new Error("allowlist must be an object");
+  }
+
+  const raw = value as Record<string, unknown>;
+  const values = ensureStringList(raw.values ?? [], "allowlist.values");
+  const patterns = ensureStringList(raw.patterns ?? [], "allowlist.patterns");
+
+  for (const pattern of patterns) {
+    try {
+      new RegExp(pattern);
+    } catch {
+      throw new Error(`allowlist.patterns contains invalid regex pattern: "${pattern}"`);
+    }
+  }
+
+  const entitiesValue = raw.entities ?? {};
+  if (
+    typeof entitiesValue !== "object" ||
+    Array.isArray(entitiesValue) ||
+    entitiesValue === null
+  ) {
+    throw new Error("allowlist.entities must be an object mapping entity labels to string arrays");
+  }
+
+  const entities: Record<string, string[]> = {};
+  for (const [entityType, entryValue] of Object.entries(entitiesValue)) {
+    const normalizedType = canonicalType(entityType);
+    entities[normalizedType] = ensureStringList(entryValue, `allowlist.entities.${entityType}`);
+  }
+
+  return {
+    values: [...new Set(values)],
+    patterns: [...new Set(patterns)],
+    entities,
+  };
+}
+
+function ensureEntityConfidenceThresholds(
+  value: unknown,
+): Record<string, number> {
+  if (!value) {
+    return {};
+  }
+
+  if (typeof value !== "object" || Array.isArray(value) || value === null) {
+    throw new Error("entityConfidenceThresholds must be an object");
+  }
+
+  const raw = value as Record<string, unknown>;
+  const normalized: Record<string, number> = {};
+
+  for (const [entityType, rawThreshold] of Object.entries(raw)) {
+    if (typeof rawThreshold !== "number" || Number.isNaN(rawThreshold)) {
+      throw new Error(
+        `entityConfidenceThresholds["${entityType}"] must be a number between 0 and 1, got ${String(
+          rawThreshold,
+        )}`,
+      );
+    }
+
+    if (rawThreshold < 0 || rawThreshold > 1) {
+      throw new Error(
+        `entityConfidenceThresholds["${entityType}"] must be between 0 and 1, got ${rawThreshold}`,
+      );
+    }
+
+    const canonicalTypeKey = canonicalType(entityType);
+    normalized[canonicalTypeKey] = rawThreshold;
+  }
+
+  return normalized;
+}
+
 export const DEFAULT_CONFIG: FogClawConfig = {
   enabled: true,
   guardrail_mode: "redact",
@@ -11,10 +112,37 @@ export const DEFAULT_CONFIG: FogClawConfig = {
   confidence_threshold: 0.5,
   custom_entities: [],
   entityActions: {},
+  entityConfidenceThresholds: {},
+  allowlist: {
+    values: [],
+    patterns: [],
+    entities: {},
+  },
+  auditEnabled: true,
 };
 
 export function loadConfig(overrides: Partial<FogClawConfig>): FogClawConfig {
-  const config: FogClawConfig = { ...DEFAULT_CONFIG, ...overrides };
+  const config: FogClawConfig = {
+    ...DEFAULT_CONFIG,
+    ...overrides,
+    entityActions: {
+      ...DEFAULT_CONFIG.entityActions,
+      ...(overrides.entityActions ?? {}),
+    },
+    entityConfidenceThresholds: {
+      ...DEFAULT_CONFIG.entityConfidenceThresholds,
+      ...(overrides.entityConfidenceThresholds ?? {}),
+    },
+  };
+
+  config.allowlist = ensureEntityAllowlist(overrides.allowlist ?? DEFAULT_CONFIG.allowlist);
+  config.entityConfidenceThresholds = ensureEntityConfidenceThresholds(
+    config.entityConfidenceThresholds,
+  );
+
+  if (typeof config.enabled !== "boolean") {
+    throw new Error(`enabled must be true or false`);
+  }
 
   if (!VALID_GUARDRAIL_MODES.includes(config.guardrail_mode)) {
     throw new Error(
@@ -34,13 +162,22 @@ export function loadConfig(overrides: Partial<FogClawConfig>): FogClawConfig {
     );
   }
 
+  if (typeof config.auditEnabled !== "boolean") {
+    throw new Error(`auditEnabled must be true or false`);
+  }
+
+  const normalizedActions: Record<string, GuardrailAction> = {};
   for (const [entityType, action] of Object.entries(config.entityActions)) {
     if (!VALID_GUARDRAIL_MODES.includes(action)) {
       throw new Error(
         `Invalid action "${action}" for entity type "${entityType}". Must be one of: ${VALID_GUARDRAIL_MODES.join(", ")}`,
       );
     }
+
+    const normalizedType = canonicalType(entityType);
+    normalizedActions[normalizedType] = action;
   }
+  config.entityActions = normalizedActions;
 
   return config;
 }

package/src/extract.ts

@@ -0,0 +1,98 @@
+/**
+ * Utilities for extracting text from AgentMessage tool result payloads
+ * and replacing text content after redaction.
+ *
+ * AgentMessage shapes handled:
+ * - Plain string
+ * - Object with `content: string`
+ * - Object with `content: [{ type: "text", text: "..." }, ...]`
+ *
+ * When multiple text blocks exist in a content array, they are joined
+ * with a null byte separator (\0) so entity offsets stay valid across
+ * the concatenated string. replaceText splits on the same separator
+ * to map redacted text back to individual blocks.
+ */
+
+// Separator between text segments from content block arrays.
+// Null byte won't appear in regex PII patterns or normal text content.
+const SEGMENT_SEP = "\0";
+
+/**
+ * Extract all text content from an AgentMessage tool result payload.
+ * Returns an empty string if no text content is found.
+ */
+export function extractText(message: unknown): string {
+  if (message == null) return "";
+  if (typeof message === "string") return message;
+  if (typeof message !== "object") return "";
+
+  const msg = message as Record<string, unknown>;
+  const content = msg.content;
+
+  if (content == null) return "";
+  if (typeof content === "string") return content;
+
+  if (Array.isArray(content)) {
+    const textParts: string[] = [];
+    for (const block of content) {
+      if (
+        block != null &&
+        typeof block === "object" &&
+        (block as Record<string, unknown>).type === "text" &&
+        typeof (block as Record<string, unknown>).text === "string"
+      ) {
+        textParts.push((block as Record<string, unknown>).text as string);
+      }
+    }
+    if (textParts.length === 0) return "";
+    return textParts.join(SEGMENT_SEP);
+  }
+
+  return "";
+}
+
+/**
+ * Replace text content in an AgentMessage tool result payload with
+ * the redacted version. Returns a shallow copy; does not mutate.
+ *
+ * If the message shape is not recognized or has no text, returns
+ * the original message unchanged.
+ */
+export function replaceText(message: unknown, redactedText: string): unknown {
+  if (message == null) return message;
+  if (typeof message === "string") return redactedText;
+  if (typeof message !== "object") return message;
+
+  const msg = message as Record<string, unknown>;
+  const content = msg.content;
+
+  if (content == null) return message;
+
+  if (typeof content === "string") {
+    return { ...msg, content: redactedText };
+  }
+
+  if (Array.isArray(content)) {
+    const segments = redactedText.split(SEGMENT_SEP);
+    let segmentIndex = 0;
+
+    const newContent = content.map((block) => {
+      if (
+        block != null &&
+        typeof block === "object" &&
+        (block as Record<string, unknown>).type === "text" &&
+        typeof (block as Record<string, unknown>).text === "string" &&
+        segmentIndex < segments.length
+      ) {
+        const replaced = { ...(block as Record<string, unknown>), text: segments[segmentIndex] };
+        segmentIndex++;
+        return replaced;
+      }
+      return block;
+    });
+
+    return { ...msg, content: newContent };
+  }
+
+  return message;
+}