@vpdeva/blackwall-llm-shield-js 0.1.6 → 0.1.8

package/README.md

@@ -81,6 +81,10 @@ Use `shadowMode` with `shadowPolicyPacks` or `comparePolicyPacks` to record what
 
 Use `createOpenAIAdapter()`, `createAnthropicAdapter()`, `createGeminiAdapter()`, or `createOpenRouterAdapter()` with `protectWithAdapter()` when you want Blackwall to wrap the provider call end to end.
 
+### Controlled-pilot rollout
+
+The current recommendation for enterprise teams is a controlled pilot first: start in shadow mode, aggregate route-level telemetry, tune suppressions explicitly, then promote the cleanest routes to enforcement.
+
 ### Observability and control-plane support
 
 Use `summarizeOperationalTelemetry()` with emitted telemetry events when you want route-level summaries, blocked-event counts, and rollout visibility for operators.
@@ -105,7 +109,7 @@ Use `require('@vpdeva/blackwall-llm-shield-js/providers')` for provider adapter
 
 Use it to sanitize inbound messages, mask sensitive data, score prompt-injection risk, and decide whether the request should continue to the model provider.
 
-It also exposes `protectModelCall()`, `protectWithAdapter()`, and `reviewModelResponse()` so you can enforce request checks before provider calls and review outputs before they go back to the user.
+It also exposes `protectModelCall()`, `protectJsonModelCall()`, `protectWithAdapter()`, and `reviewModelResponse()` so you can enforce request checks before provider calls and review outputs before they go back to the user.
 
 ### `OutputFirewall`
 
@@ -129,6 +133,10 @@ Recommended presets:
 - `strict` for high-sensitivity routes
 - `ragSafe` for retrieval-heavy flows
 - `agentTools` for tool-calling and approval-gated agent actions
+- `agentPlanner` for JSON-heavy planner and internal ops routes
+- `documentReview` for classification and document-review pipelines
+- `ragSearch` for search-heavy retrieval endpoints
+- `toolCalling` for routes that broker external actions
 
 ### `AuditTrail`
 
@@ -177,6 +185,35 @@ const result = await shield.protectWithAdapter({
 console.log(result.stage, result.allowed);
 ```
 
+### Wrap Blackwall behind your own app adapter
+
+```js
+function createModelShield(shield) {
+  return {
+    async run({ messages, metadata, callProvider }) {
+      return shield.protectModelCall({
+        messages,
+        metadata,
+        callModel: callProvider,
+      });
+    },
+  };
+}
+```
+
+### Protect a strict JSON workflow
+
+```js
+const result = await shield.protectJsonModelCall({
+  messages: [{ role: 'user', content: 'Return the shipment triage plan as JSON.' }],
+  metadata: { route: '/api/planner', feature: 'planner' },
+  requiredSchema: { steps: 'object' },
+  callModel: async () => JSON.stringify({ steps: ['triage', 'notify-ops'] }),
+});
+
+console.log(result.json.parsed);
+```
+
 ### Use presets and route-level policy overrides
 
 ```js
@@ -201,6 +238,18 @@ const shield = new BlackwallShield({
 });
 ```
 
+### Next.js App Router plus Gemini pattern
+
+For App Router route handlers, the cleanest production shape is:
+
+- parse the request in `app/api/.../route.ts`
+- use `preset: 'shadowFirst'` or a route-specific preset like `agentPlanner` or `documentReview`
+- attach `route`, `feature`, and `tenantId` metadata
+- wrap the Gemini SDK call with `createGeminiAdapter()` plus `protectWithAdapter()`
+- ship `report.telemetry` and `onTelemetry` into a route-level log sink
+
+That keeps request guarding, output review, and operator reporting in one path without scattering policy logic across the route.
+
 ### Route and domain examples
 
 For RAG:
@@ -231,12 +280,47 @@ const toolFirewall = new ToolPermissionFirewall({
 });
 ```
 
+For document review and verification:
+
+```js
+const shield = new BlackwallShield({
+  preset: 'documentReview',
+  routePolicies: [
+    {
+      route: '/api/verify',
+      options: {
+        shadowMode: true,
+        outputFirewallDefaults: { requiredSchema: { verdict: 'string' } },
+      },
+    },
+  ],
+});
+```
+
+### Choose your integration path
+
+- Request-only guard: `guardModelRequest()`
+- Request + output review: `protectModelCall()`
+- Strict JSON planner/document workflows: `protectJsonModelCall()`
+- Full provider wrapper: `protectWithAdapter()`
+- Tool firewall + RAG sanitizer: `ToolPermissionFirewall` + `RetrievalSanitizer`
+
+### False-positive tuning
+
+- Start with route-level `shadowMode: true`
+- Add `suppressPromptRules` only per route, not globally, so the reason for each suppression stays obvious
+- Log `report.promptInjection.matches` and `report.telemetry.promptInjectionRuleHits` to explain why a request was flagged
+- Review `summary.noisiestRoutes`, `summary.byFeature`, and `summary.weeklyBlockEstimate` before raising enforcement
+
 ### Operational telemetry summaries
 
 ```js
 const { summarizeOperationalTelemetry } = require('@vpdeva/blackwall-llm-shield-js');
 const summary = summarizeOperationalTelemetry(events);
 console.log(summary.byRoute);
+console.log(summary.byFeature);
+console.log(summary.noisiestRoutes);
+console.log(summary.weeklyBlockEstimate);
 console.log(summary.highestSeverity);
 ```
 
@@ -281,6 +365,14 @@ console.log(tools.inspectCall({ tool: 'lookupCustomer', args: { id: 'cus_123' }
 
 For Next.js, the most production-real patterns are App Router route handlers, server actions for trusted internal mutations, and streaming endpoints that apply output review to assembled or final chunks instead of raw intermediate tokens.
 
+For Gemini-heavy apps, the bundled adapter now preserves system instructions plus mixed text/image/file parts so App Router handlers can wrap direct `@google/generative-ai` calls with less translation glue.
+
+## Enterprise Adoption Notes
+
+- A controlled pilot is a good fit today when you want shadow-mode prompt and output protection without forcing hard blocking on every route immediately.
+- If you prefer not to depend on Blackwall directly everywhere, wrap it behind your own internal model-security abstraction and expose only the contract your app teams need.
+- For broader approval, focus rollout reviews on false-positive rates, noisiest routes, and latency budgets alongside jailbreak coverage.
+
 ## Release Commands
 
 - `npm run release:check` runs the JS test suite before release

package/index.d.ts

@@ -32,6 +32,16 @@ export interface ReviewResult {
   [key: string]: unknown;
 }
 
+export interface JsonProtectionResult extends Record<string, unknown> {
+  allowed: boolean;
+  blocked: boolean;
+  json?: {
+    parsed: unknown;
+    schemaValid: boolean;
+    parseError?: string;
+  };
+}
+
 export interface ProviderAdapter {
   provider: string;
   invoke(payload: { messages: ShieldMessage[]; metadata?: Record<string, unknown>; guard?: GuardResult }): Promise<unknown> | unknown;
@@ -54,6 +64,7 @@ export class BlackwallShield {
   guardModelRequest(input?: { messages?: ShieldMessage[]; metadata?: Record<string, unknown>; allowSystemMessages?: boolean; comparePolicyPacks?: string[] }): Promise<GuardResult>;
   reviewModelResponse(input?: { output: unknown; metadata?: Record<string, unknown>; outputFirewall?: OutputFirewall | null; firewallOptions?: Record<string, unknown> }): Promise<ReviewResult>;
   protectModelCall(input: Record<string, unknown>): Promise<Record<string, unknown>>;
+  protectJsonModelCall(input: Record<string, unknown>): Promise<JsonProtectionResult>;
   protectWithAdapter(input: { adapter: ProviderAdapter; messages?: ShieldMessage[]; metadata?: Record<string, unknown>; allowSystemMessages?: boolean; comparePolicyPacks?: string[]; outputFirewall?: OutputFirewall | null; firewallOptions?: Record<string, unknown> }): Promise<Record<string, unknown>>;
 }
 
@@ -85,6 +96,7 @@ export const POLICY_PACKS: Record<string, Record<string, unknown>>;
 
 export function buildShieldOptions(options?: Record<string, unknown>): Record<string, unknown>;
 export function summarizeOperationalTelemetry(events?: Array<Record<string, unknown>>): Record<string, unknown>;
+export function parseJsonOutput(output: unknown): unknown;
 
 export function createOpenAIAdapter(input: Record<string, unknown>): ProviderAdapter;
 export function createAnthropicAdapter(input: Record<string, unknown>): ProviderAdapter;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@vpdeva/blackwall-llm-shield-js",
-  "version": "0.1.6",
+  "version": "0.1.8",
   "description": "Open-source JavaScript enterprise LLM protection toolkit for Node.js and Next.js",
   "license": "Apache-2.0",
   "author": "Vish <hello@vish.au> (https://vish.au)",

package/src/index.js

@@ -145,6 +145,34 @@ const SHIELD_PRESETS = {
     notifyOnRiskLevel: 'medium',
     shadowMode: false,
   },
+  agentPlanner: {
+    blockOnPromptInjection: true,
+    promptInjectionThreshold: 'medium',
+    notifyOnRiskLevel: 'medium',
+    shadowMode: true,
+    shadowPolicyPacks: ['government'],
+  },
+  documentReview: {
+    blockOnPromptInjection: true,
+    promptInjectionThreshold: 'high',
+    notifyOnRiskLevel: 'medium',
+    shadowMode: true,
+    policyPack: 'healthcare',
+  },
+  ragSearch: {
+    blockOnPromptInjection: true,
+    promptInjectionThreshold: 'medium',
+    notifyOnRiskLevel: 'medium',
+    shadowMode: true,
+    shadowPolicyPacks: ['government'],
+  },
+  toolCalling: {
+    blockOnPromptInjection: true,
+    promptInjectionThreshold: 'medium',
+    notifyOnRiskLevel: 'medium',
+    shadowMode: false,
+    policyPack: 'finance',
+  },
 };
 
 const CORE_INTERFACE_VERSION = '1.0';
@@ -152,6 +180,7 @@ const CORE_INTERFACES = Object.freeze({
   guardModelRequest: CORE_INTERFACE_VERSION,
   reviewModelResponse: CORE_INTERFACE_VERSION,
   protectModelCall: CORE_INTERFACE_VERSION,
+  protectJsonModelCall: CORE_INTERFACE_VERSION,
   toolPermissionFirewall: CORE_INTERFACE_VERSION,
   retrievalSanitizer: CORE_INTERFACE_VERSION,
 });
@@ -350,6 +379,7 @@ function summarizeOperationalTelemetry(events = []) {
     shadowModeEvents: 0,
     byType: {},
     byRoute: {},
+    byFeature: {},
     byTenant: {},
     byModel: {},
     byPolicyOutcome: {
@@ -359,11 +389,14 @@ function summarizeOperationalTelemetry(events = []) {
     },
     topRules: {},
     highestSeverity: 'low',
+    noisiestRoutes: [],
+    weeklyBlockEstimate: 0,
   };
   for (const event of Array.isArray(events) ? events : []) {
     const type = event && event.type ? event.type : 'unknown';
     const metadata = event && event.metadata ? event.metadata : {};
     const route = metadata.route || metadata.path || 'unknown';
+    const feature = metadata.feature || metadata.capability || route;
     const tenant = metadata.tenantId || metadata.tenant_id || 'unknown';
     const model = metadata.model || metadata.modelName || 'unknown';
     const severity = event && event.report && event.report.outputReview
@@ -372,6 +405,7 @@ function summarizeOperationalTelemetry(events = []) {
     summary.totalEvents += 1;
     summary.byType[type] = (summary.byType[type] || 0) + 1;
     summary.byRoute[route] = (summary.byRoute[route] || 0) + 1;
+    summary.byFeature[feature] = (summary.byFeature[feature] || 0) + 1;
     summary.byTenant[tenant] = (summary.byTenant[tenant] || 0) + 1;
     summary.byModel[model] = (summary.byModel[model] || 0) + 1;
     if (event && event.blocked) summary.blockedEvents += 1;
@@ -390,9 +424,19 @@ function summarizeOperationalTelemetry(events = []) {
   summary.topRules = Object.fromEntries(
     Object.entries(summary.topRules).sort((a, b) => b[1] - a[1]).slice(0, 10)
   );
+  summary.noisiestRoutes = Object.entries(summary.byRoute)
+    .sort((a, b) => b[1] - a[1])
+    .slice(0, 5)
+    .map(([route, count]) => ({ route, count }));
+  summary.weeklyBlockEstimate = summary.byPolicyOutcome.blocked + summary.byPolicyOutcome.shadowBlocked;
   return summary;
 }
 
+function parseJsonOutput(output) {
+  if (typeof output === 'string') return JSON.parse(output);
+  return output;
+}
+
 function resolveShieldPreset(name) {
   if (!name) return {};
   return SHIELD_PRESETS[name] ? { ...SHIELD_PRESETS[name] } : {};
@@ -1352,6 +1396,69 @@ class BlackwallShield {
       },
     });
   }
+
+  async protectJsonModelCall({
+    messages = [],
+    metadata = {},
+    allowSystemMessages = this.options.allowSystemMessages,
+    comparePolicyPacks = [],
+    callModel,
+    mapMessages = null,
+    mapOutput = null,
+    outputFirewall = null,
+    firewallOptions = {},
+    requiredSchema = null,
+  } = {}) {
+    const result = await this.protectModelCall({
+      messages,
+      metadata,
+      allowSystemMessages,
+      comparePolicyPacks,
+      callModel,
+      mapMessages,
+      mapOutput,
+      outputFirewall,
+      firewallOptions,
+    });
+    if (result.blocked) return result;
+    try {
+      const parsed = parseJsonOutput(result.review.maskedOutput != null ? result.review.maskedOutput : result.response);
+      const schemaValid = validateRequiredSchema(parsed, requiredSchema);
+      if (!schemaValid) {
+        return {
+          ...result,
+          allowed: false,
+          blocked: true,
+          stage: 'output',
+          reason: 'Model output failed JSON schema validation',
+          json: {
+            parsed,
+            schemaValid: false,
+          },
+        };
+      }
+      return {
+        ...result,
+        json: {
+          parsed,
+          schemaValid: true,
+        },
+      };
+    } catch (error) {
+      return {
+        ...result,
+        allowed: false,
+        blocked: true,
+        stage: 'output',
+        reason: 'Model output is not valid JSON',
+        json: {
+          parsed: null,
+          schemaValid: false,
+          parseError: error.message,
+        },
+      };
+    }
+  }
 }
 
 function validateGrounding(text, documents = [], options = {}) {
@@ -2060,6 +2167,7 @@ module.exports = {
   runRedTeamSuite,
   buildShieldOptions,
   summarizeOperationalTelemetry,
+  parseJsonOutput,
   createOpenAIAdapter,
   createAnthropicAdapter,
   createGeminiAdapter,

package/src/providers.js

@@ -12,6 +12,38 @@ function stringifyContent(content) {
   return String(content || '');
 }
 
+function toGeminiPart(item) {
+  if (typeof item === 'string') return { text: item };
+  if (!item || typeof item !== 'object') return null;
+  if ((item.type === 'text' || item.type === 'input_text') && typeof item.text === 'string') {
+    return { text: item.text };
+  }
+  if (item.type === 'image_url' && typeof item.image_url === 'string') {
+    return { fileData: { fileUri: item.image_url } };
+  }
+  if (item.type === 'file') {
+    if (item.file_data && typeof item.file_data === 'object') return { inlineData: item.file_data };
+    if (typeof item.file_uri === 'string') return { fileData: { fileUri: item.file_uri } };
+    if (typeof item.file_id === 'string') return { fileData: { fileUri: item.file_id } };
+  }
+  if (item.type === 'json' && typeof item.value === 'string') {
+    return { text: item.value };
+  }
+  if (typeof item.text === 'string') return { text: item.text };
+  return null;
+}
+
+function toGeminiParts(content) {
+  if (typeof content === 'string') return [{ text: content }];
+  if (Array.isArray(content)) return content.map((item) => toGeminiPart(item)).filter(Boolean);
+  if (content && typeof content === 'object') {
+    if (Array.isArray(content.parts)) return toGeminiParts(content.parts);
+    const part = toGeminiPart(content);
+    return part ? [part] : [{ text: stringifyContent(content) }];
+  }
+  return [{ text: String(content || '') }];
+}
+
 function toOpenAIInput(messages = []) {
   return messages.map((message) => ({
     role: message.role,
@@ -101,19 +133,30 @@ function createGeminiAdapter({ client, model, request = {}, extractOutput = null
   return {
     provider: 'gemini',
     async invoke({ messages }) {
+      const systemInstruction = extractSystemPrompt(messages);
       const response = await client.models.generateContent({
         model,
-        contents: messages.map((message) => ({
-          role: message.role === 'assistant' ? 'model' : 'user',
-          parts: [{ text: stringifyContent(message.content) }],
-        })),
+        contents: messages
+          .filter((message) => message.role !== 'system')
+          .map((message) => ({
+            role: message.role === 'assistant' ? 'model' : 'user',
+            parts: toGeminiParts(message.content),
+          })),
+        ...(systemInstruction ? { systemInstruction: { parts: [{ text: systemInstruction }] } } : {}),
         ...request,
       });
-      return defaultAdapterResult(response, response && typeof response.text === 'string' ? response.text : '');
+      return defaultAdapterResult(response, this.extractOutput(response));
     },
     extractOutput(response) {
       if (typeof extractOutput === 'function') return extractOutput(response);
       if (response && typeof response.text === 'string') return response.text;
+      if (response && Array.isArray(response.candidates)) {
+        return response.candidates
+          .flatMap((candidate) => (((candidate || {}).content || {}).parts || []))
+          .map((part) => (part && typeof part.text === 'string' ? part.text : ''))
+          .filter(Boolean)
+          .join('\n');
+      }
       if (typeof response === 'string') return response;
       return '';
     },