payload-guard-filter 1.2.0 → 1.3.0

package/README.md

@@ -13,6 +13,21 @@
 
 ---
 
+## 🛡️ Workflow
+
+```mermaid
+graph LR
+    A[Request] --> B(Gatekeeper)
+    B --> C{Shape Check}
+    C -- Valid --> D[Redact & Clean]
+    C -- Invalid --> E[Strict Error / Fail Safe]
+    D --> F[Secure Response]
+    E --> F
+    F --> G((Metrics))
+```
+
+---
+
 ## ✨ Features
 
 - **Shape-based filtering** — Define what you want, auto-remove everything else
@@ -260,58 +275,107 @@ const postShape = guard.shape({
 
 ---
 
-## 🏢 Enterprise Features
-
-These features are designed for high-throughput, production systems where bandwidth, predictability and observability matter.
+## 🏢 Enterprise Features (v1.2.0+)
 
-- **Payload Metrics (payload reduction):** enable measurements to show before/after sizes and percentage saved. This makes cost and bandwidth impact visible to engineers.
+Designed for high-performance production systems, Payload Guard includes enterprise-grade features for security, observability, and performance.
 
+### 🛡️ 1. Header Sanitization
+Automatically remove sensitive headers like `Authorization` or `Cookie` before your business logic handles the request.
 ```ts
-// enable metrics globally
-import { guard } from 'payload-guard';
+app.use(guardMiddleware({
+  sanitizeHeaders: true,
+  sensitiveHeaders: ['x-api-key', 'session-id'] // optional extras
+}));
+```
 
-guard.config({
-  payloadMetrics: true,
-  logger: (msg, level = 'info') => console.log(`[payload-guard:${level}]`, msg),
-});
+### 🧠 2. Memory Safety (`maxArrayLength`)
+Prevent DoS attacks via extremely large arrays by automatically truncating them to a safe limit.
+```ts
+const shape = guard.shape({ items: guard.array('string') }, { maxArrayLength: 1000 });
 ```
 
-When enabled middleware or `res.guardJson()` will log reductions such as:
+### ⚡ 3. Async Safety (`maxPayloadSize`)
+Avoid processing massive JSON payloads that could block the event loop.
+```ts
+app.use(guardMiddleware({
+  maxPayloadSize: 1024 * 512, // 512KB
+  skipLargePayload: true       // Skips filtering if too large
+}));
+```
 
+### ⏱️ 4. Middleware Timing Stats
+Track exactly how much time Payload Guard adds to your request cycle.
 ```
-payload reduced: 18KB → 4KB (78%)
+[payload-guard] [POST] /api/data: reduced 15KB -> 4KB (73%) in 0.12ms
 ```
 
-- **Strict Mode:** in strict mode the filter will throw on invalid or missing required fields instead of silently ignoring them. Use when you want validation failures to fail-fast in production.
-
-```ts
-// per-shape strict mode
-const userStrict = guard.shape({ id: 'number', email: 'string' }, { strict: true });
+### 📊 5. Human-Readable Metrics
+Visibility into bandwidth savings with formatted byte sizes and percentages.
 
-// or global
-guard.config({ strict: true });
+### 🛠️ 6. CLI Type Sync
+Generate frontend TypeScript types from your backend shapes with one command.
+```bash
+npx payload-guard sync
 ```
 
-- **Compile Mode:** pre-compile shapes once and reuse the compiled function for maximum throughput. This avoids repeated runtime parsing and makes performance predictable.
+### ⚠️ 7. Route-Aware Dev Warnings
+Dev mode warnings now include the full method, path, and nested field locations (e.g., `user.profile.password`) for faster debugging.
 
+### 🛣️ 8. Ignore Routes
+Skip processing for high-volume or incompatible routes like file uploads or health checks.
 ```ts
-// compile once at startup
-const compiledUser = guard.compile({ id: 'number', name: 'string' });
+app.use(guardMiddleware({ ignoreRoutes: ['/health', '/upload'] }));
+```
+
+### 🏗️ 9. Shared Schemas & Examples
+Built-in support for reusable schemas across your monorepo and a `examples/real-world` project for reference.
+
+### 🚀 10. Performance Benchmarks
+A dedicated benchmark suite to verify sub-millisecond overhead. `npm run benchmark`.
+
+---
 
-// then use per-request
-app.get('/user/:id', (req, res) => {
-const raw = getUserFromDb();
-  res.json(compiledUser(raw));
+## 🏗️ Production Hardening
+
+### 🛡️ Fail-Safe Mode (`failOpen`)
+In production, we prioritize availability. If filtering fails for any reason, `failOpen: true` (default) ensures the original data is sent instead of breaking the request.
+
+### 🚫 Non-Serializable Protection
+Payload Guard automatically detects and skips `Buffer`, `Stream`, `File`, and `Blob` objects to prevent crashes and performance bottlenecks.
+
+### 🔍 Detailed Debug Logs
+Enable `logRemovedFields: true` to see exactly which fields were stripped from your payloads:
+`[payload-guard] /api/user Removed fields: password, token, internal_id`
+
+---
+
+## 🌐 Multi-Framework Support
+
+The core is zero-dependency and framework-agnostic. Use it anywhere:
+
+### Hono / Cloudflare Workers
+```ts
+app.post('/user', async (c) => {
+  const body = await c.req.json();
+  return c.json(userShape(body));
 });
 ```
 
-- **Ignore Routes:** skip middleware processing for low-value routes (health checks, metrics, webhooks) to avoid adding overhead.
-
+### Fastify
 ```ts
-app.use(guardMiddleware({ ignoreRoutes: ['/health', '/metrics', '/webhook'] }));
+fastify.post('/user', (req, reply) => {
+  reply.send(userShape(req.body));
+});
 ```
 
-- **Middleware non-blocking guarantee:** the middleware is defensive — any internal error, oversized payload, or stream-like body will be skipped and the response path will continue unblocked. Use `logger` or `devMode` to surface warnings.
+---
+
+## 🛑 When NOT to use
+
+Payload Guard is optimized for JSON APIs. It is **not** suitable for:
+1. **Binary Data**: PDFs, Images, or raw Buffers.
+2. **File Streams**: Use `multer` or similar for multipart data.
+3. **Heavy Computation**: Don't use it on payloads > 10MB without adjusting `maxPayloadSize`.
 
 ---
 

package/dist/core/filter.d.ts

@@ -12,13 +12,7 @@ export declare function getConfig(): GuardConfig;
  * Build a shape filter function from a shape descriptor
  * Returns a callable function that filters objects to match the shape
  */
-export declare function buildShape<S extends ShapeDescriptor>(shape: S, opts?: {
-    sensitive?: string[];
-    dev?: boolean;
-    strict?: boolean;
-    redact?: (k: string, v: unknown) => unknown;
-    logger?: (msg: string, level?: 'warn' | 'error' | 'info') => void;
-}): ShapeFunction<S>;
+export declare function buildShape<S extends ShapeDescriptor>(shape: S, opts?: GuardConfig): ShapeFunction<S>;
 /**
  * Main guard API object
  */
@@ -29,18 +23,9 @@ export declare const guard: {
      * const userShape = guard.shape({ id: 'number', name: 'string' });
      * const filtered = userShape(userData);
      */
-    shape: <S extends ShapeDescriptor>(descriptor: S, opts?: {
-        sensitive?: string[];
-        dev?: boolean;
-    }) => ShapeFunction<S>;
+    shape: <S extends ShapeDescriptor>(descriptor: S, opts?: GuardConfig) => ShapeFunction<S>;
     /** Compile a descriptor into a fast filter function directly */
-    compile: <S extends ShapeDescriptor>(descriptor: S, opts?: {
-        sensitive?: string[];
-        dev?: boolean;
-        strict?: boolean;
-        redact?: (k: string, v: unknown) => unknown;
-        logger?: (msg: string) => void;
-    }) => import("./types").CompiledFilter;
+    compile: <S extends ShapeDescriptor>(descriptor: S, opts?: GuardConfig) => import("./types").CompiledFilter;
     /**
      * Create an array shape descriptor
      * @example

package/dist/core/filter.js

@@ -17,6 +17,12 @@ let globalConfig = {
     maxPayloadSize: undefined,
     ignoreRoutes: undefined,
     payloadMetrics: false,
+    maxArrayLength: undefined,
+    sanitizeHeaders: false,
+    sensitiveHeaders: ['authorization', 'cookie', 'x-api-key'],
+    failOpen: true,
+    skipNonSerializable: true,
+    logRemovedFields: false,
 };
 /**
  * Configure global guard settings
@@ -36,11 +42,15 @@ function getConfig() {
  */
 function buildShape(shape, opts) {
     const mergedOpts = {
-        sensitive: [...(opts?.sensitive || []), ...(globalConfig.sensitiveFields || [])],
-        dev: opts?.dev ?? globalConfig.devMode,
+        sensitive: [...(opts?.sensitiveFields || []), ...(globalConfig.sensitiveFields || [])],
+        dev: opts?.devMode ?? globalConfig.devMode,
         strict: opts?.strict ?? globalConfig.strict,
         redact: opts?.redact ?? globalConfig.redact,
         logger: opts?.logger ?? globalConfig.logger,
+        maxArrayLength: opts?.maxArrayLength ?? globalConfig.maxArrayLength,
+        failOpen: opts?.failOpen ?? globalConfig.failOpen,
+        skipNonSerializable: opts?.skipNonSerializable ?? globalConfig.skipNonSerializable,
+        logRemovedFields: opts?.logRemovedFields ?? globalConfig.logRemovedFields,
     };
     const compiledFn = (0, compiler_1.compile)(shape, mergedOpts);
     // Create callable wrapper with compile method attached
@@ -70,7 +80,15 @@ exports.guard = {
      */
     shape: (descriptor, opts) => buildShape(descriptor, opts),
     /** Compile a descriptor into a fast filter function directly */
-    compile: (descriptor, opts) => (0, compiler_1.compile)(descriptor, { sensitive: opts?.sensitive, dev: opts?.dev, strict: opts?.strict, redact: opts?.redact, logger: opts?.logger }),
+    compile: (descriptor, opts) => (0, compiler_1.compile)(descriptor, {
+        ...opts,
+        sensitive: [...(opts?.sensitiveFields || []), ...(globalConfig.sensitiveFields || [])],
+        dev: opts?.devMode ?? globalConfig.devMode,
+        strict: opts?.strict ?? globalConfig.strict,
+        redact: opts?.redact ?? globalConfig.redact,
+        logger: opts?.logger ?? globalConfig.logger,
+        maxArrayLength: opts?.maxArrayLength ?? globalConfig.maxArrayLength,
+    }),
     /**
      * Create an array shape descriptor
      * @example

package/dist/core/types.d.ts

@@ -51,6 +51,12 @@ export interface GuardConfig {
     sanitizeHeaders?: boolean;
     /** Sensitive header names to filter (default: authorization, cookie, x-api-key) */
     sensitiveHeaders?: string[];
+    /** Fail-safe mode: if filtering fails, return original data instead of error (default: true) */
+    failOpen?: boolean;
+    /** Skip filtering for non-serializable objects (Buffer, Stream, File) (default: true) */
+    skipNonSerializable?: boolean;
+    /** Log names of fields removed during filtering (debug mode) */
+    logRemovedFields?: boolean;
 }
 export interface GuardMiddlewareOptions extends GuardConfig {
     /** Sanitize request body */

package/dist/middleware/express.js

@@ -4,6 +4,7 @@ exports.guardMiddleware = guardMiddleware;
 exports.routeGuard = routeGuard;
 const compiler_1 = require("../core/compiler");
 const security_1 = require("../core/security");
+const perf_1 = require("../core/perf");
 /**
  * Express middleware for payload sanitization and response filtering
  *
@@ -49,47 +50,41 @@ function guardMiddleware(opts = {}) {
                 }
             }
             // Sanitize request body
-            if (sanitizeBody && req.body && typeof req.body === 'object') {
-                // max payload size check
-                if (typeof maxPayloadSize === 'number') {
+            if (sanitizeBody && req.body) {
+                // streaming/non-serializable safety
+                const isNonSerializable = req.body && (Buffer.isBuffer(req.body) ||
+                    typeof req.body.pipe === 'function' ||
+                    (typeof File !== 'undefined' && req.body instanceof File) ||
+                    (typeof Blob !== 'undefined' && req.body instanceof Blob));
+                if (!(isNonSerializable && (opts.skipNonSerializable !== false))) {
+                    // execution with fail-open safety
                     try {
-                        const size = Buffer.byteLength(JSON.stringify(req.body || ''), 'utf8');
-                        if (size > maxPayloadSize) {
-                            if (devMode) {
-                                (opts.logger ?? console.warn)(`[payload-guard] ${routeName}: payload exceeds maxPayloadSize (${size} > ${maxPayloadSize} bytes)`, 'warn');
-                            }
-                            // Skip processing if skipLargePayload is true (default), else still process
-                            if (!skipLargePayload) {
-                                if (compiledRequestShape) {
-                                    req.body = compiledRequestShape(req.body);
-                                }
-                                else {
-                                    req.body = (0, security_1.removeSensitive)(req.body, sensitiveFields);
+                        // max payload size check
+                        if (typeof maxPayloadSize === 'number') {
+                            const size = Buffer.byteLength(JSON.stringify(req.body || ''), 'utf8');
+                            if (size > maxPayloadSize && skipLargePayload) {
+                                if (devMode) {
+                                    (opts.logger ?? console.warn)(`[payload-guard] ${routeName}: payload too large, skipping`, 'info');
                                 }
                             }
-                        }
-                        else {
-                            if (compiledRequestShape) {
-                                req.body = compiledRequestShape(req.body);
-                            }
                             else {
-                                req.body = (0, security_1.removeSensitive)(req.body, sensitiveFields);
+                                req.body = compiledRequestShape ? compiledRequestShape(req.body) : (0, security_1.removeSensitive)(req.body, sensitiveFields);
                             }
                         }
+                        else {
+                            req.body = compiledRequestShape ? compiledRequestShape(req.body) : (0, security_1.removeSensitive)(req.body, sensitiveFields);
+                        }
                     }
                     catch (e) {
-                        // if serialization fails, skip and continue
                         if (devMode)
-                            (opts.logger ?? console.warn)('[payload-guard] error checking payload size: ' + String(e), 'warn');
+                            (opts.logger ?? console.warn)('[payload-guard] request filtering error: ' + String(e), 'warn');
+                        if (opts.failOpen === false)
+                            throw e;
+                        // fail-open: keep original req.body
                     }
                 }
-                else {
-                    if (compiledRequestShape) {
-                        req.body = compiledRequestShape(req.body);
-                    }
-                    else {
-                        req.body = (0, security_1.removeSensitive)(req.body, sensitiveFields);
-                    }
+                else if (isNonSerializable && devMode) {
+                    (opts.logger ?? console.info)('[payload-guard] skip: non-serializable request body', 'info');
                 }
             }
             // Dev mode: warn about sensitive fields in request with full path
@@ -136,24 +131,54 @@ function guardMiddleware(opts = {}) {
                         }
                         catch { }
                     }
-                    // streaming safety: if body looks like a stream, skip
-                    const isStreamLike = data && typeof data.pipe === 'function';
-                    if (isStreamLike)
+                    // streaming/non-serializable safety
+                    const isNonSerializable = data && (Buffer.isBuffer(data) ||
+                        typeof data.pipe === 'function' ||
+                        (typeof File !== 'undefined' && data instanceof File) ||
+                        (typeof Blob !== 'undefined' && data instanceof Blob));
+                    if (isNonSerializable && (opts.skipNonSerializable !== false)) {
+                        if (devMode)
+                            opts.logger?.('[payload-guard] skip: non-serializable payload detected', 'info');
                         return res.json(data);
-                    // timing start
+                    }
+                    // execution with fail-open safety
+                    let filtered;
                     const start = performance.now();
-                    const filtered = filterFn(data);
+                    try {
+                        filtered = filterFn(data);
+                    }
+                    catch (e) {
+                        if (opts.failOpen !== false) {
+                            if (devMode)
+                                opts.logger?.('[payload-guard] fail-open: filtering failed, sending original', 'warn');
+                            return res.json(data);
+                        }
+                        throw e;
+                    }
                     const end = performance.now();
                     const duration = (end - start).toFixed(2);
+                    // log removed fields for debugging
+                    if (opts.logRemovedFields && data && typeof data === 'object' && filtered && typeof filtered === 'object') {
+                        const originalKeys = Object.keys(data);
+                        const filteredKeys = Object.keys(filtered);
+                        const removed = originalKeys.filter(k => !filteredKeys.includes(k));
+                        if (removed.length > 0) {
+                            const logger = opts.logger || console.info;
+                            logger(`[payload-guard] ${routeName} Removed fields: ${removed.join(', ')}`, 'info');
+                        }
+                    }
                     // metrics
                     if (payloadMetrics) {
                         try {
-                            const { metric } = require('../core/perf');
-                            const m = metric(data, filtered);
+                            const m = (0, perf_1.metric)(data, filtered);
                             const reduction = `reduced ${m.formatBefore} -> ${m.formatAfter} (${m.percent}%)`;
-                            (opts.logger ?? console.info)(`[payload-guard] ${routeName}: ${reduction} in ${duration}ms`, 'info');
+                            const logger = opts.logger || console.info;
+                            logger(`[payload-guard] ${routeName}: ${reduction} in ${duration}ms`, 'info');
+                        }
+                        catch (e) {
+                            if (devMode)
+                                console.warn('[payload-guard] metrics error:', e);
                         }
-                        catch { }
                     }
                     return res.json(filtered);
                 }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "payload-guard-filter",
-  "version": "1.2.0",
+  "version": "1.3.0",
   "description": "Lightweight, zero-dependency shape-based payload filtering and sanitization for Node.js and browser",
   "main": "dist/index.js",
   "module": "dist/index.js",