agent-security-scanner-mcp 3.7.0 → 3.8.0

package/README.md

@@ -8,7 +8,11 @@ Security scanner for AI coding agents and autonomous assistants. Scans code for
 [![Benchmark: 97.7% precision](https://img.shields.io/badge/precision-97.7%25-brightgreen.svg)](benchmarks/RESULTS.md)
 [![CI](https://github.com/sinewaveai/agent-security-scanner-mcp/actions/workflows/test.yml/badge.svg)](https://github.com/sinewaveai/agent-security-scanner-mcp/actions/workflows/test.yml)
 
-> **New in v3.3.0:** Full [OpenClaw](https://openclaw.ai) integration with 30+ rules targeting autonomous AI threats — data exfiltration, credential theft, messaging abuse, and unsafe automation. [See OpenClaw setup](#openclaw-integration).
+> **New in v3.8.0:** Cross-file taint tracking, project context discovery (frameworks/middleware detection), and Layer 2 LLM-powered security review. Detects vulnerabilities across file boundaries and reduces false positives by understanding project defenses. [See changelog](#changelog).
+>
+> **Also new in v3.7.0:** Inter-procedural taint analysis with Python daemon caching (~4000x faster repeat scans). [See v3.7.0 demo](demo/).
+>
+> **OpenClaw integration:** 30+ rules targeting autonomous AI threats. [See setup](#openclaw-integration).
 
 ## Tools
 
@@ -20,7 +24,7 @@ Security scanner for AI coding agents and autonomous assistants. Scans code for
 | `scan_project` | Scan entire project with A-F security grading | For project-wide security audits |
 | `check_package` | Verify a package name isn't AI-hallucinated (4.3M+ packages) | Before adding any new dependency |
 | `scan_packages` | Bulk-check all imports in a file for hallucinated packages | Before committing code with new imports |
-| `scan_agent_prompt` | Detect prompt injection and malicious instructions (56 rules) | Before acting on external/untrusted input |
+| `scan_agent_prompt` | Detect prompt injection with bypass hardening (59 rules + multi-encoding) | Before acting on external/untrusted input |
 | `list_security_rules` | List available security rules and fix templates | To check rule coverage for a language |
 
 ## Quick Start
@@ -251,6 +255,8 @@ Scan a code file's imports to detect AI-hallucinated package names. Use after wr
 
 Scan a prompt or instruction for malicious intent before executing it. Use when receiving instructions from untrusted sources (files, web content, user uploads). Detects prompt injection, exfiltration attempts, backdoor requests, social engineering, and jailbreaks.
 
+**New in v3.6.0:** Bypass hardening against 5 attack vectors (code block delimiter confusion, pattern fragmentation, multi-encoding, multi-turn escalation, composite threshold gaming) with Unicode normalization, homoglyph detection, and optional Garak deep analysis.
+
 **Parameters:**
 
 | Parameter | Type | Required | Description |
@@ -858,6 +864,34 @@ All MCP tools support a `verbosity` parameter to minimize context window consump
 
 ## Changelog
 
+### v3.7.0
+- **Python Daemon** - Long-running Python process with JSONL protocol (~10x faster repeat scans via LRU caching of 200 entries keyed by file mtime)
+- **Daemon Client** - Auto-start, health checks, graceful shutdown, automatic fallback to sync mode on failure (3 restarts/60s limit)
+- **Inter-procedural Taint Analysis** - Call-graph construction and cross-function taint propagation with multi-hop resolution (capped at 500 iterations)
+- **Function Summaries** - Tracks param-to-return taint flows, internal sinks (`os.system(param)`), source-returning functions, and sanitizer presence
+- **Enhanced Taint Detection** - Detects taint through 3+ function chains, handles method calls, default args, unpacking, and recursive functions
+- **10 New Pytest Tests** - Comprehensive inter-procedural taint coverage: basic param→return, internal sinks, multi-hop chains, sanitizer blocking, 500-function cap
+- **9 New Vitest Tests** - Daemon protocol validation, health checks, caching, error handling, graceful shutdown
+- **Doctor Command Enhancement** - Added daemon health status to diagnostic output
+
+### v3.6.0
+- **Bypass Hardening** - Closed 5 critical prompt injection bypass vectors: code block delimiter confusion (`~~~`, `<code>`, `<!---->`), pattern fragmentation (string concat, C-style comments), multi-encoding (base64/hex/URL/ROT13 cascade), multi-turn escalation (cross-turn boundary scanning, Crescendo frame-setting), and composite threshold gaming (co-occurrence matrix, orthogonal dimension scoring)
+- **Unicode Normalization Pipeline** - NFKC normalization, Cyrillic/Greek homoglyph canonicalization (40+ mappings), zero-width character stripping, Zalgo diacritics removal, invisible Unicode detection as obfuscation indicator
+- **Multi-Encoding Decode Cascade** - Replaced base64-only decoder with comprehensive cascade supporting nested base64, hex, URL encoding, and indicator-gated ROT13
+- **Enhanced Composite Scoring** - Category co-occurrence boost matrix (12 suspicious pairs, +40% cap), orthogonal dimension scoring (7 attack dimensions, +40 flat bonus), low-signal accumulation for multiple LOW-confidence findings
+- **Garak Integration** - Optional NVIDIA Garak LLM vulnerability scanner integration via `deep_scan` parameter for advanced encoding probes and latent injection detection
+- **PromptFoo Red-Team Suite** - 13 automated test cases with custom MCP provider for continuous bypass detection validation (`npm run test:redteam`)
+- **3 New YAML Rules** - Whitespace fragmentation, Crescendo escalation setup, leetspeak/character substitution obfuscation
+- **Test Coverage Expansion** - 28 new prompt scanner tests covering all bypass vectors and false positive regression
+
+### v3.5.2
+- **Prompt Injection Fixes** - Closed 5 bypass vectors: tilde code fences (~~~), string fragmentation, base64 encoding, multi-turn escalation, and composite indicators
+- **Advanced Decoding** - Added Morse code, Braille Unicode, and Zalgo diacritics decoding to detect obfuscated prompt attacks
+- **Garak Red-Team Validation** - Improved detection rates to 100% across all categories (encoding, promptinject, jailbreak)
+- **npm Bloom Filter** - Ships npm-bloom.json (7.9 MB) in base package — all 7 ecosystems now work out of the box (npm, PyPI, RubyGems, crates.io, pub.dev, CPAN, raku.land)
+- **Expanded Benchmarks** - Benchmark corpus increased to 424 annotations across 17 files (was 335/13)
+- **CI Improvements** - Added pytest to requirements.txt, expanded test matrix with AST mode on Node 22
+
 ### v3.4.0
 - **Severity Calibration** - 207-rule severity map with HIGH/MEDIUM/LOW confidence scores for more accurate prioritization
 - **Cross-Engine Deduplication** - ~30-50% noise reduction by deduplicating findings across AST, taint, and regex engines
@@ -894,20 +928,20 @@ All MCP tools support a `verbosity` parameter to minimize context window consump
 
 ## Installation Options
 
-### Default Package (Lightweight - 2.7 MB)
+### Default Package (10.6 MB)
 
 ```bash
 npm install -g agent-security-scanner-mcp
 ```
 
-Includes hallucination detection for: **PyPI, RubyGems, crates.io, pub.dev, CPAN, raku.land** (1M+ packages)
+**New in v3.5.2:** Now includes **all 7 ecosystems** out of the box — npm, PyPI, RubyGems, crates.io, pub.dev, CPAN, raku.land (4.3M+ packages total)
 
-### Full Package (With npm - 10.3 MB)
+### Legacy Lightweight Package (2.7 MB)
 
-If you need **npm/JavaScript hallucination detection** (3.3M packages):
+For environments with strict size constraints (excludes npm bloom filter):
 
 ```bash
-npm install -g agent-security-scanner-mcp-full
+npm install -g agent-security-scanner-mcp@3.4.1
 ```
 
 ---
@@ -919,4 +953,4 @@ npm install -g agent-security-scanner-mcp-full
 
 ## License
 
-MIT
+MIT

package/analyzer.py

@@ -11,6 +11,7 @@ import sys
 import json
 import os
 import re
+import argparse
 from typing import List, Dict, Any
 
 # Add the directory containing this script to the path
@@ -91,6 +92,7 @@ def analyze_file_regex(file_path):
                                 'column': match.start() + col_offset,
                                 'length': match.end() - match.start(),
                                 'severity': rule['severity'],
+                                'confidence': rule.get('metadata', {}).get('confidence', 'MEDIUM'),
                                 'metadata': rule.get('metadata', {}),
                                 'engine': 'regex'
                             })
@@ -191,6 +193,7 @@ def analyze_file_ast(file_path):
                 'column': f.column,
                 'length': length,
                 'severity': f.severity,
+                'confidence': f.metadata.get('confidence', getattr(f, 'confidence', 'MEDIUM')),
                 'metadata': f.metadata,
                 'engine': 'taint' if is_taint else 'ast',
             })
@@ -229,16 +232,30 @@ def analyze_file(file_path):
 
 
 def main():
-    if len(sys.argv) < 2:
-        print(json.dumps({'error': 'No file path provided'}))
-        sys.exit(1)
+    parser = argparse.ArgumentParser(description='Security Analyzer - AST-based with regex fallback')
+    parser.add_argument('file_path', help='Path to the file to analyze')
+    parser.add_argument('--engine', choices=['auto', 'ast', 'regex'], default='auto',
+                        help='Analysis engine: auto (default), ast (tree-sitter only), regex (regex only)')
+    args = parser.parse_args()
 
-    file_path = sys.argv[1]
+    file_path = args.file_path
     if not os.path.exists(file_path):
         print(json.dumps({'error': f'File not found: {file_path}'}))
         sys.exit(1)
 
-    results = analyze_file(file_path)
+    engine = args.engine
+
+    if engine == 'regex':
+        results = analyze_file_regex(file_path)
+    elif engine == 'ast':
+        if not HAS_AST_ENGINE:
+            print(json.dumps({'error': 'AST engine requested but tree-sitter is not available. Install dependencies: python3 -m pip install -r requirements.txt'}))
+            sys.exit(1)
+        results = analyze_file_ast(file_path)
+    else:
+        # auto: use AST if available, otherwise regex
+        results = analyze_file(file_path)
+
     print(json.dumps(results))
 
 

package/cross_file_analyzer.py

@@ -0,0 +1,216 @@
+#!/usr/bin/env python3
+"""Cross-file taint analysis for security scanning.
+
+Builds an import graph across local files, runs per-file analysis,
+and propagates taint warnings when a file imports from another file
+that has ERROR-severity findings.
+"""
+
+import json
+import os
+import re
+import sys
+
+# Import the per-file analyzer
+from analyzer import analyze_file
+
+
+def extract_js_imports(source):
+    """Extract import/require statements from JavaScript/TypeScript."""
+    imports = []
+    # require('...')
+    for m in re.finditer(r'''require\s*\(\s*['"]([^'"]+)['"]\s*\)''', source):
+        imports.append(m.group(1))
+    # import ... from '...'
+    for m in re.finditer(r'''from\s+['"]([^'"]+)['"]''', source):
+        imports.append(m.group(1))
+    # import '...'
+    for m in re.finditer(r'''import\s+['"]([^'"]+)['"]''', source):
+        imports.append(m.group(1))
+    return imports
+
+
+def extract_py_imports(source):
+    """Extract import statements from Python."""
+    imports = []
+    # import module
+    for m in re.finditer(r'^import\s+(\S+)', source, re.MULTILINE):
+        imports.append(m.group(1).split('.')[0])
+    # from module import ...
+    for m in re.finditer(r'^from\s+(\S+)\s+import', source, re.MULTILINE):
+        imports.append(m.group(1).split('.')[0])
+    return imports
+
+
+def detect_language(file_path):
+    """Detect language from file extension."""
+    ext = os.path.splitext(file_path)[1].lower()
+    lang_map = {
+        '.py': 'python', '.js': 'javascript', '.ts': 'typescript',
+        '.tsx': 'typescript', '.jsx': 'javascript',
+    }
+    return lang_map.get(ext, 'unknown')
+
+
+def resolve_local_import(module, base_dir, lang):
+    """Resolve a relative/local import to an actual file path."""
+    if lang in ('javascript', 'typescript'):
+        # Only resolve relative imports
+        if not module.startswith('.'):
+            return None
+        # Try common extensions
+        candidates = [
+            module,
+            module + '.js', module + '.ts', module + '.tsx', module + '.jsx',
+            os.path.join(module, 'index.js'), os.path.join(module, 'index.ts'),
+        ]
+        for candidate in candidates:
+            full = os.path.normpath(os.path.join(base_dir, candidate))
+            if os.path.isfile(full):
+                return full
+    elif lang == 'python':
+        # Only resolve relative imports (starting with .)
+        if module.startswith('.'):
+            rel = module.lstrip('.')
+            candidates = [
+                os.path.join(base_dir, rel.replace('.', os.sep) + '.py'),
+                os.path.join(base_dir, rel.replace('.', os.sep), '__init__.py'),
+            ]
+            for candidate in candidates:
+                if os.path.isfile(candidate):
+                    return candidate
+        # Also check if the module name matches a sibling file
+        sibling = os.path.join(base_dir, module + '.py')
+        if os.path.isfile(sibling):
+            return sibling
+    return None
+
+
+def extract_exports(source, lang):
+    """Extract exported function/class names."""
+    exports = []
+    if lang in ('javascript', 'typescript'):
+        for m in re.finditer(r'export\s+(?:function|class|const|let|var)\s+(\w+)', source):
+            exports.append(m.group(1))
+        for m in re.finditer(r'module\.exports\s*=', source):
+            exports.append('default')
+    elif lang == 'python':
+        for m in re.finditer(r'^(?:def|class)\s+(\w+)', source, re.MULTILINE):
+            exports.append(m.group(1))
+    return exports
+
+
+def build_import_graph(file_paths):
+    """Build import graph: {file -> [{module, resolved_path, line}]}."""
+    graph = {}
+    file_set = set(os.path.abspath(f) for f in file_paths)
+
+    for file_path in file_paths:
+        abs_path = os.path.abspath(file_path)
+        lang = detect_language(file_path)
+        if lang == 'unknown':
+            continue
+
+        try:
+            source = open(file_path, 'r', encoding='utf-8', errors='ignore').read()
+        except (OSError, IOError):
+            continue
+
+        if lang in ('javascript', 'typescript'):
+            modules = extract_js_imports(source)
+        elif lang == 'python':
+            modules = extract_py_imports(source)
+        else:
+            continue
+
+        base_dir = os.path.dirname(abs_path)
+        edges = []
+        for mod in modules:
+            resolved = resolve_local_import(mod, base_dir, lang)
+            if resolved:
+                resolved_abs = os.path.abspath(resolved)
+                if resolved_abs in file_set and resolved_abs != abs_path:
+                    edges.append({
+                        'module': mod,
+                        'resolved_path': resolved_abs,
+                    })
+
+        graph[abs_path] = edges
+
+    return graph
+
+
+def cross_file_analyze(file_paths):
+    """Run cross-file taint analysis.
+
+    1. Analyze each file independently
+    2. Build import graph
+    3. For each file importing from another file with ERROR-severity findings,
+       add a cross-file-taint-warning
+    """
+    # Analyze each file
+    file_findings = {}
+    all_findings = []
+
+    for file_path in file_paths:
+        try:
+            results = analyze_file(file_path)
+            if isinstance(results, list):
+                file_findings[os.path.abspath(file_path)] = results
+                for finding in results:
+                    finding['file'] = file_path
+                all_findings.extend(results)
+        except Exception:
+            continue
+
+    # Build import graph
+    graph = build_import_graph(file_paths)
+
+    # Propagate taint warnings
+    cross_file_warnings = []
+    for file_path, edges in graph.items():
+        for edge in edges:
+            imported_path = edge['resolved_path']
+            imported_findings = file_findings.get(imported_path, [])
+
+            # Check for ERROR-severity findings in imported file
+            error_findings = [f for f in imported_findings if f.get('severity') == 'error']
+            if error_findings:
+                warning = {
+                    'ruleId': 'cross-file-taint-warning',
+                    'severity': 'warning',
+                    'message': f"Imports from '{os.path.basename(imported_path)}' which has {len(error_findings)} critical finding(s): {', '.join(set(f.get('ruleId', 'unknown') for f in error_findings))}",
+                    'file': file_path,
+                    'line': 0,
+                    'metadata': {
+                        'imported_file': imported_path,
+                        'imported_findings_count': len(error_findings),
+                    }
+                }
+                cross_file_warnings.append(warning)
+
+    # Combine: per-file findings + cross-file warnings
+    combined = all_findings + cross_file_warnings
+    return combined
+
+
+def main():
+    """CLI entry point. Accepts file paths as arguments, outputs JSON."""
+    if len(sys.argv) < 2:
+        print(json.dumps({'error': 'Usage: cross_file_analyzer.py file1 file2 ...'}))
+        sys.exit(1)
+
+    file_paths = sys.argv[1:]
+    # Filter to existing files
+    file_paths = [f for f in file_paths if os.path.isfile(f)]
+
+    if not file_paths:
+        print(json.dumps({'error': 'No valid files provided'}))
+        sys.exit(1)
+
+    results = cross_file_analyze(file_paths)
+    print(json.dumps(results))
+
+
+if __name__ == '__main__':
+    main()

package/daemon.py

@@ -0,0 +1,179 @@
+#!/usr/bin/env python3
+"""JSONL daemon wrapping analyzer.py for persistent process reuse.
+
+Protocol: One JSON object per line over stdin/stdout. stderr for debug logs only.
+Startup: sends {"id":"__ready__","success":true,"result":{"status":"ready"}}
+Actions: analyze, cross_file_analyze, health, shutdown
+"""
+
+import sys
+import os
+
+# CRITICAL: Redirect stdout to stderr BEFORE any imports can print to stdout.
+# This prevents any imported library from corrupting the JSONL protocol channel.
+_protocol_stdout = sys.stdout
+sys.stdout = sys.stderr
+
+# Now safe to import everything
+import json
+import time
+from collections import OrderedDict
+
+# Add script directory to path so analyzer imports work
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+
+from analyzer import analyze_file, analyze_file_ast, analyze_file_regex
+
+try:
+    from cross_file_analyzer import cross_file_analyze
+    HAS_CROSS_FILE = True
+except ImportError:
+    HAS_CROSS_FILE = False
+
+
+class LRUCache:
+    """Simple LRU cache keyed by (file_path, mtime), capped at max_size entries."""
+
+    def __init__(self, max_size=200):
+        self._cache = OrderedDict()
+        self._max_size = max_size
+
+    def get(self, file_path):
+        try:
+            mtime = os.path.getmtime(file_path)
+        except OSError:
+            return None
+        key = (file_path, mtime)
+        if key in self._cache:
+            self._cache.move_to_end(key)
+            return self._cache[key]
+        return None
+
+    def put(self, file_path, result):
+        try:
+            mtime = os.path.getmtime(file_path)
+        except OSError:
+            return
+        key = (file_path, mtime)
+        self._cache[key] = result
+        self._cache.move_to_end(key)
+        while len(self._cache) > self._max_size:
+            self._cache.popitem(last=False)
+
+    @property
+    def size(self):
+        return len(self._cache)
+
+
+_cache = LRUCache(max_size=200)
+
+
+def send_response(obj):
+    """Write a JSON line to the protocol channel (original stdout)."""
+    line = json.dumps(obj, separators=(',', ':'))
+    _protocol_stdout.write(line + '\n')
+    _protocol_stdout.flush()
+
+
+def handle_analyze(req):
+    file_path = req.get('file_path')
+    engine = req.get('engine', 'auto')
+
+    if not file_path or not os.path.exists(file_path):
+        return {'success': False, 'error': f'File not found: {file_path}'}
+
+    # Check cache (only for engine=auto)
+    if engine == 'auto':
+        cached = _cache.get(file_path)
+        if cached is not None:
+            return {'success': True, 'result': cached, 'cached': True, 'cache_size': _cache.size}
+
+    try:
+        if engine == 'regex':
+            result = analyze_file_regex(file_path)
+        elif engine == 'ast':
+            result = analyze_file_ast(file_path)
+        else:
+            result = analyze_file(file_path)
+    except Exception as e:
+        return {'success': False, 'error': str(e)}
+
+    if isinstance(result, dict) and 'error' in result:
+        return {'success': False, 'error': result['error']}
+
+    # Cache result (only for engine=auto)
+    if engine == 'auto':
+        _cache.put(file_path, result)
+
+    return {'success': True, 'result': result, 'cached': False, 'cache_size': _cache.size}
+
+
+def handle_cross_file_analyze(req):
+    file_paths = req.get('file_paths', [])
+    if not file_paths:
+        return {'success': False, 'error': 'No file_paths provided'}
+    if not HAS_CROSS_FILE:
+        return {'success': False, 'error': 'cross_file_analyzer not available'}
+
+    try:
+        result = cross_file_analyze(file_paths)
+        return {'success': True, 'result': result}
+    except Exception as e:
+        return {'success': False, 'error': str(e)}
+
+
+def handle_health():
+    return {
+        'success': True,
+        'result': {
+            'status': 'healthy',
+            'cache_size': _cache.size,
+            'pid': os.getpid(),
+            'uptime': time.time() - _start_time,
+        }
+    }
+
+
+def main():
+    global _start_time
+    _start_time = time.time()
+
+    # Signal readiness
+    send_response({
+        'id': '__ready__',
+        'success': True,
+        'result': {'status': 'ready'}
+    })
+
+    for line in sys.stdin:
+        line = line.strip()
+        if not line:
+            continue
+
+        try:
+            req = json.loads(line)
+        except json.JSONDecodeError as e:
+            send_response({'id': None, 'success': False, 'error': f'Invalid JSON: {e}'})
+            continue
+
+        req_id = req.get('id')
+        action = req.get('action')
+
+        if action == 'shutdown':
+            send_response({'id': req_id, 'success': True, 'result': {'status': 'shutdown'}})
+            break
+        elif action == 'health':
+            resp = handle_health()
+        elif action == 'analyze':
+            resp = handle_analyze(req)
+        elif action == 'cross_file_analyze':
+            resp = handle_cross_file_analyze(req)
+        else:
+            resp = {'success': False, 'error': f'Unknown action: {action}'}
+
+        resp['id'] = req_id
+        send_response(resp)
+
+
+if __name__ == '__main__':
+    main()