openredaction 1.0.9 → 1.1.0

package/README.md

@@ -1,6 +1,6 @@
 # OpenRedaction
 
-Production-ready PII detection and redaction library with 571+ built-in patterns, multiple redaction modes, compliance presets, enterprise SaaS features, and zero dependencies.
+Production-ready PII detection and redaction library with 571+ built-in patterns, multiple redaction modes, compliance presets, and optional enterprise-style modules. The published package lists **no required runtime dependencies**; optional peers (e.g. React, PDF) apply only when you use those integrations.
 
 ## Installation
 
@@ -36,59 +36,12 @@ import { useOpenRedaction, usePIIDetector } from 'openredaction/react';
 
 `react` is an optional peer dependency; only install it if you use the React entry.
 
-## Optional AI Assist
+## Node HTTP API & Prometheus (optional)
 
-OpenRedaction supports an optional AI-assisted detection mode that enhances regex-based detection by calling a hosted AI endpoint. This feature is **OFF by default** and requires explicit configuration.
-
-### Configuration
-
-```typescript
-import { OpenRedaction } from 'openredaction';
-
-const detector = new OpenRedaction({
-  // ... other options ...
-  ai: {
-    enabled: true,
-    endpoint: 'https://your-api.example.com' // Optional: defaults to OPENREDACTION_AI_ENDPOINT env var
-  }
-});
-
-const result = await detector.detect('Contact John Doe at john@example.com');
-```
-
-### How It Works
-
-1. **Regex Detection First**: The library always runs regex detection first (existing behavior)
-2. **AI Enhancement**: If `ai.enabled === true` and an endpoint is configured, the library calls the `/ai-detect` endpoint
-3. **Smart Merging**: AI entities are merged with regex detections, with regex taking precedence on conflicts
-4. **Graceful Fallback**: If the AI endpoint fails or is unavailable, the library silently falls back to regex-only detection
-
-### Environment Variables
-
-In Node.js environments, you can set the endpoint via environment variable:
-
-```bash
-export OPENREDACTION_AI_ENDPOINT=https://your-api.example.com
-```
-
-### Important Notes
-
-- **AI is optional**: The library works exactly as before when `ai.enabled` is `false` or omitted
-- **Regex is primary**: AI only adds additional entities; regex detections always take precedence
-- **No breaking changes**: When AI is disabled, detection is still regex-only; `detect()` always returns a `Promise`
-- **Browser support**: In browsers, you must provide an explicit `ai.endpoint` (env vars not available)
-- **Network dependency**: AI mode requires network access to the endpoint
-
-### For Sensitive Workloads
-
-For maximum security and privacy, keep AI disabled and rely purely on regex detection:
+`APIServer`, `createAPIServer`, `PrometheusServer`, and `createPrometheusServer` use Node’s built-in `http` module. They are **not** re-exported from the main entry (`openredaction`) so the default bundle stays free of `node:http` for clearer static analysis.
 
 ```typescript
-const detector = new OpenRedaction({
-  // AI not configured = pure regex detection
-  includeNames: true,
-  includeEmails: true
-});
+import { APIServer, createPrometheusServer } from 'openredaction/server';
 ```
 
 ## Documentation
@@ -101,7 +54,7 @@ const detector = new OpenRedaction({
 - 🚀 **Fast & Accurate** - 10-20ms for 2-3KB text
 - 🎯 **571+ PII Patterns** - Comprehensive coverage across multiple categories
 - 🔐 **Enterprise SaaS Ready** - Multi-tenancy, persistent audit logging, webhooks, REST API
-- 📊 **Production Monitoring** - Prometheus metrics, Grafana dashboards, health checks
+- 📊 **Production Monitoring** - In-memory metrics collector; optional Prometheus HTTP server via `openredaction/server`
 - 🧠 **Semantic Detection** - Hybrid NER + regex with 40+ contextual rules
 - 🎨 **Multiple Redaction Modes** - Placeholder, mask-middle, mask-all, format-preserving, token-replace
 - ✅ **Built-in Validators** - Luhn, IBAN, NHS, National ID checksums
@@ -109,9 +62,9 @@ const detector = new OpenRedaction({
 - 🎭 **Deterministic Placeholders** - Consistent redaction for same values
 - 🌍 **Global Coverage** - 50+ countries
 - 📄 **Structured Data Support** - JSON, CSV, XLSX with path/cell tracking
-- 🌳 **Zero Dependencies** - No external packages required (core)
+- 🌳 **No required runtime deps** - Core redaction does not pull mandatory npm packages
 - 📝 **TypeScript Native** - Full type safety and IntelliSense
-- 🧪 **Battle Tested** - 276+ passing tests
+- 🧪 **Battle Tested** - Large automated test suite
 
 ## Pattern Categories
 
@@ -137,9 +90,9 @@ Retail, Legal, Real Estate, Logistics, Insurance, Healthcare, Emergency Response
 
 - **Persistent Audit Logging** - SQLite/PostgreSQL with cryptographic hashing
 - **Multi-Tenancy** - Tenant isolation, quotas, usage tracking
-- **Prometheus Metrics** - HTTP server with Grafana dashboards
+- **Prometheus Metrics** - Optional scrape endpoint (`openredaction/server`)
 - **Webhook System** - Event-driven alerts with retry logic
-- **REST API** - Production-ready HTTP API with authentication
+- **REST API** - Optional HTTP API (`openredaction/server`)
 
 ## License
 

package/dist/index.cli.cjs

@@ -11081,6 +11081,38 @@ const transportLogisticsPreset = {
 	]
 };
 /**
+* PCI-DSS oriented preset — cardholder data and common payment identifiers
+*/
+const pciDssPreset = {
+	includeNames: true,
+	includeEmails: true,
+	includePhones: true,
+	includeAddresses: true,
+	categories: [
+		"personal",
+		"contact",
+		"financial",
+		"network"
+	]
+};
+/**
+* SOC 2 oriented preset — broad PII and credentials for trust services contexts
+*/
+const soc2Preset = {
+	includeNames: true,
+	includeEmails: true,
+	includePhones: true,
+	includeAddresses: true,
+	categories: [
+		"personal",
+		"contact",
+		"financial",
+		"government",
+		"network",
+		"digital-identity"
+	]
+};
+/**
 * Get preset configuration by name
 */
 function getPreset(name) {
@@ -11097,6 +11129,10 @@ function getPreset(name) {
 		case "transport-logistics":
 		case "transportation":
 		case "logistics": return transportLogisticsPreset;
+		case "pci-dss":
+		case "pci_dss": return pciDssPreset;
+		case "soc2":
+		case "soc-2": return soc2Preset;
 		default: return {};
 	}
 }
@@ -11615,7 +11651,7 @@ var ConfigLoader = class {
 	static createDefaultConfig(outputPath = ".openredaction.config.js") {
 		fs.writeFileSync(outputPath, `/**
  * OpenRedaction Configuration
- * @see https://github.com/openredact/openredact
+ * @see https://github.com/sam247/openredaction
  */
 export default {
   // Extend built-in presets
@@ -14282,133 +14318,6 @@ function validatePattern(pattern) {
 	}
 }
 
-//#endregion
-//#region src/utils/ai-assist.ts
-/**
-* Get the AI endpoint URL from options or environment
-*/
-function getAIEndpoint(aiOptions) {
-	if (!aiOptions?.enabled) return null;
-	if (aiOptions.endpoint) return aiOptions.endpoint;
-	if (typeof process !== "undefined" && process.env) {
-		const envEndpoint = process.env.OPENREDACTION_AI_ENDPOINT;
-		if (envEndpoint) return envEndpoint;
-	}
-	return null;
-}
-/**
-* Check if fetch is available in the current environment
-*/
-function isFetchAvailable() {
-	return typeof fetch !== "undefined";
-}
-/**
-* Call the AI endpoint to get additional PII entities
-* Returns null if AI is disabled, endpoint unavailable, or on error
-*/
-async function callAIDetect(text, endpoint, debug) {
-	if (!isFetchAvailable()) {
-		if (debug) console.warn("[OpenRedaction] AI assist requires fetch API. Not available in this environment.");
-		return null;
-	}
-	try {
-		const url = endpoint.endsWith("/ai-detect") ? endpoint : `${endpoint}/ai-detect`;
-		if (debug) console.log(`[OpenRedaction] Calling AI endpoint: ${url}`);
-		const response = await fetch(url, {
-			method: "POST",
-			headers: { "Content-Type": "application/json" },
-			body: JSON.stringify({ text })
-		});
-		if (!response.ok) {
-			if (debug) {
-				const statusText = response.status === 429 ? "Rate limit exceeded (429)" : `${response.status}: ${response.statusText}`;
-				console.warn(`[OpenRedaction] AI endpoint returned ${statusText}`);
-			}
-			return null;
-		}
-		const data = await response.json();
-		if (!data.entities || !Array.isArray(data.entities)) {
-			if (debug) console.warn("[OpenRedaction] Invalid AI response format: missing entities array");
-			return null;
-		}
-		return data.entities;
-	} catch (error) {
-		if (debug) console.warn(`[OpenRedaction] AI endpoint error: ${error instanceof Error ? error.message : "Unknown error"}`);
-		return null;
-	}
-}
-/**
-* Validate an AI entity
-*/
-function validateAIEntity(entity, textLength) {
-	if (!entity.type || !entity.value || typeof entity.start !== "number" || typeof entity.end !== "number") return false;
-	if (entity.start < 0 || entity.end < 0 || entity.start >= entity.end) return false;
-	if (entity.start >= textLength || entity.end > textLength) return false;
-	if (entity.value.length !== entity.end - entity.start) return false;
-	return true;
-}
-/**
-* Check if two detections overlap significantly
-* Returns true if they overlap by more than 50% of the shorter detection
-*/
-function detectionsOverlap(det1, det2) {
-	const [start1, end1] = det1.position;
-	const [start2, end2] = det2.position;
-	const overlapStart = Math.max(start1, start2);
-	const overlapEnd = Math.min(end1, end2);
-	if (overlapStart >= overlapEnd) return false;
-	const overlapLength = overlapEnd - overlapStart;
-	const length1 = end1 - start1;
-	const length2 = end2 - start2;
-	return overlapLength > Math.min(length1, length2) * .5;
-}
-/**
-* Convert AI entity to PIIDetection format
-*/
-function convertAIEntityToDetection(entity, text) {
-	if (!validateAIEntity(entity, text.length)) return null;
-	const actualValue = text.substring(entity.start, entity.end);
-	let type = entity.type.toUpperCase();
-	if (type.includes("EMAIL") || type === "EMAIL_ADDRESS") type = "EMAIL";
-	else if (type.includes("PHONE") || type === "PHONE_NUMBER") type = "PHONE_US";
-	else if (type.includes("NAME") || type === "PERSON") type = "NAME";
-	else if (type.includes("SSN") || type === "SOCIAL_SECURITY_NUMBER") type = "SSN";
-	else if (type.includes("ADDRESS")) type = "ADDRESS_STREET";
-	let severity = "medium";
-	if (type === "SSN" || type === "CREDIT_CARD") severity = "critical";
-	else if (type === "EMAIL" || type === "PHONE_US" || type === "NAME") severity = "high";
-	return {
-		type,
-		value: actualValue,
-		placeholder: `[${type}_${Math.random().toString(36).substring(2, 9)}]`,
-		position: [entity.start, entity.end],
-		severity,
-		confidence: entity.confidence ?? .7
-	};
-}
-/**
-* Merge AI entities with regex detections
-* Prefers regex detections on conflicts
-*/
-function mergeAIEntities(regexDetections, aiEntities, text) {
-	const merged = [...regexDetections];
-	const processedRanges = regexDetections.map((d) => d.position);
-	for (const aiEntity of aiEntities) {
-		const detection = convertAIEntityToDetection(aiEntity, text);
-		if (!detection) continue;
-		let hasOverlap = false;
-		for (const regexDet of regexDetections) if (detectionsOverlap(regexDet, detection)) {
-			hasOverlap = true;
-			break;
-		}
-		if (!hasOverlap) {
-			merged.push(detection);
-			processedRanges.push(detection.position);
-		}
-	}
-	return merged;
-}
-
 //#endregion
 //#region src/config/ConfigExporter.ts
 var ConfigExporter_exports = /* @__PURE__ */ __exportAll({
@@ -16553,7 +16462,7 @@ var OpenRedaction = class OpenRedaction {
 			redactionMode: "placeholder",
 			enableContextAnalysis: true,
 			confidenceThreshold: .5,
-			enableFalsePositiveFilter: false,
+			enableFalsePositiveFilter: true,
 			falsePositiveThreshold: .7,
 			enableMultiPass: false,
 			multiPassCount: 3,
@@ -16782,8 +16691,9 @@ var OpenRedaction = class OpenRedaction {
 				throw error;
 			}
 		}
-		if (this.nerDetector && detections.length > 0) {
-			const piiMatches = detections.map((det) => ({
+		if (this.nerDetector && this.nerDetector.isAvailable()) {
+			const nerMatches = this.nerDetector.detect(text);
+			let piiMatches = detections.map((det) => ({
 				type: det.type,
 				value: det.value,
 				start: det.position[0],
@@ -16794,11 +16704,43 @@ var OpenRedaction = class OpenRedaction {
 					after: text.substring(det.position[1], Math.min(text.length, det.position[1] + 50))
 				}
 			}));
-			const hybridMatches = this.nerDetector.hybridDetection(piiMatches, text);
-			detections = detections.map((det, index) => ({
-				...det,
-				confidence: hybridMatches[index].confidence
-			}));
+			if (detections.length > 0) {
+				const hybridMatches = this.nerDetector.hybridDetection(piiMatches, text);
+				detections = detections.map((det, index) => ({
+					...det,
+					confidence: hybridMatches[index].confidence
+				}));
+				piiMatches = detections.map((det) => ({
+					type: det.type,
+					value: det.value,
+					start: det.position[0],
+					end: det.position[1],
+					confidence: det.confidence || 1,
+					context: {
+						before: text.substring(Math.max(0, det.position[0] - 50), det.position[0]),
+						after: text.substring(det.position[1], Math.min(text.length, det.position[1] + 50))
+					}
+				}));
+			}
+			const nerOnly = this.nerDetector.extractNEROnly(nerMatches, piiMatches);
+			for (const ner of nerOnly) {
+				const syntheticPattern = {
+					type: `NER_${ner.type}`,
+					regex: /.^/,
+					priority: 1,
+					placeholder: `[NER_${ner.type}_{n}]`,
+					severity: "medium"
+				};
+				const placeholder = this.generatePlaceholder(ner.text, syntheticPattern);
+				detections.push({
+					type: syntheticPattern.type,
+					value: ner.text,
+					placeholder,
+					position: [ner.start, ner.end],
+					severity: "medium",
+					confidence: ner.confidence
+				});
+			}
 		}
 		if (this.contextRulesEngine && detections.length > 0) {
 			const piiMatches = detections.map((det) => ({
@@ -16822,7 +16764,7 @@ var OpenRedaction = class OpenRedaction {
 	}
 	/**
 	* Detect PII in text
-	* Now async to support optional AI assist
+	* Async API for detection pipeline (NER, multi-pass, etc.)
 	*/
 	async detect(text) {
 		if (this.rbacManager && !this.rbacManager.hasPermission("detection:detect")) throw new Error("[OpenRedaction] Permission denied: detection:detect required");
@@ -16862,21 +16804,6 @@ var OpenRedaction = class OpenRedaction {
 			}
 			detections = mergePassDetections(passDetections, this.multiPassConfig);
 		} else detections = this.processPatterns(text, this.patterns, processedRanges);
-		if (this.options.ai?.enabled) {
-			const aiEndpoint = getAIEndpoint(this.options.ai);
-			if (aiEndpoint) try {
-				if (this.options.debug) console.log("[OpenRedaction] AI assist enabled, calling AI endpoint...");
-				const aiEntities = await callAIDetect(text, aiEndpoint, this.options.debug);
-				if (aiEntities && aiEntities.length > 0) {
-					if (this.options.debug) console.log(`[OpenRedaction] AI returned ${aiEntities.length} additional entities`);
-					detections = mergeAIEntities(detections, aiEntities, text);
-					if (this.options.debug) console.log(`[OpenRedaction] After AI merge: ${detections.length} total detections`);
-				} else if (this.options.debug) console.log("[OpenRedaction] AI endpoint returned no additional entities");
-			} catch (error) {
-				if (this.options.debug) console.warn(`[OpenRedaction] AI assist failed, using regex-only: ${error instanceof Error ? error.message : "Unknown error"}`);
-			}
-			else if (this.options.debug) console.warn("[OpenRedaction] AI assist enabled but no endpoint configured. Set ai.endpoint or OPENREDACTION_AI_ENDPOINT env var.");
-		}
 		detections.sort((a, b) => b.position[0] - a.position[0]);
 		let redacted = text;
 		const redactionMap = {};