vektor-scan 0.2.0__tar.gz

vektor_scan-0.2.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 LLMGuard-Lite Contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

vektor_scan-0.2.0/PKG-INFO

@@ -0,0 +1,341 @@
+Metadata-Version: 2.4
+Name: vektor-scan
+Version: 0.2.0
+Summary: AI security testing framework — pytest for LLM applications
+Author-email: Swapnil <swapnil.wankhede23@spit.ac.in>
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/Swapnil565/Vektor-
+Project-URL: Issues, https://github.com/Swapnil565/Vektor-/issues
+Keywords: llm,security,testing,prompt-injection,ai-safety
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Topic :: Security
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: openai>=1.12.0
+Requires-Dist: click>=8.1.7
+Requires-Dist: rich>=13.7.0
+Requires-Dist: python-docx>=1.1.0
+Requires-Dist: PyYAML>=6.0.1
+Provides-Extra: dev
+Requires-Dist: pytest>=7.4.3; extra == "dev"
+Requires-Dist: pytest-cov>=4.1.0; extra == "dev"
+Requires-Dist: pytest-mock>=3.12.0; extra == "dev"
+Requires-Dist: black>=23.12.0; extra == "dev"
+Requires-Dist: ruff>=0.1.9; extra == "dev"
+Dynamic: license-file
+
+# Vektor 🛡️
+
+**pytest for AI security — scan LLM apps for vulnerabilities in 30 seconds**
+
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
+
+## 🎯 What is Vektor?
+
+An automated security testing framework that scans LLM applications for vulnerabilities:
+- ✅ **27 validated attack vectors** across 6 categories
+- ✅ **$0.50 average scan cost** with built-in budget controls
+- ✅ **30-second results** - Docker run, immediate feedback
+- ✅ **CI/CD ready** - Integrate into your deployment pipeline
+- ✅ **Novel research** - First tool to systematically test document-based instruction hijacking
+
+## 🚀 Quick Start
+
+```bash
+# Step 1: Install
+pip install vektor
+
+# Step 2: Zero-setup demo (no API key)
+vektor demo
+
+# Step 3: Real scan, $0 cost, always works
+vektor scan --target vulnerable --output my-first-report.html
+
+# Step 4: Open the report
+# Windows: start my-first-report.html
+# Mac:     open my-first-report.html
+# Linux:   xdg-open my-first-report.html
+
+# Step 5 (optional): Scan your own LLM app
+export OPENAI_API_KEY=sk-your-key
+vektor scan --target openai --budget 1.0
+```
+
+## 🌐 Scan Any AI API — No SDK Needed
+
+Point Vektor at any HTTP endpoint:
+```bash
+# Auto-detects OpenAI/Anthropic/custom shapes
+vektor scan --url http://localhost:8000/chat
+
+# With auth header
+vektor scan --url https://my-app.com/api \
+  --header "Authorization: Bearer YOUR_TOKEN"
+
+# Custom request/response field names
+vektor scan --url http://localhost:8000/predict \
+  --request-field prompt --response-field answer
+
+# Query-parameter mode (e.g. /api/parse?text=PAYLOAD)
+vektor scan --url http://localhost:8000/api/parse \
+  --param-field text
+
+# Rate-limited API — add delay between requests
+vektor scan --url http://localhost:8000/chat \
+  --request-delay 12.0
+```
+
+## 💡 Why Vektor?
+
+| Feature | Vektor | Garak | Promptfoo | PyRIT |
+| :--- | :---: | :---: | :---: | :---: |
+| **Primary Focus** | **Actionable Security** | Vulnerability Scanning | General Eval / Testing | Red Teaming Framework |
+| **Setup Time** | **< 30s** | ~10 mins | ~5 mins | ~30 mins |
+| **Scan Speed** | **Fast (Targeted)** | Slow (Exhaustive) | Fast | Slow (Agentic) |
+| **Cost Control** | **✅ Built-in Budget** | ❌ | ❌ | ❌ |
+| **CI/CD Ready** | **✅ Native** | ⚠️ Heavy | ✅ | ⚠️ Complex |
+| **RAG/Doc Attacks** | **✅ Specialized** | ⚠️ Limited | ✅ | ✅ |
+
+| Before | After |
+|--------|-------|
+| Hire pentester ($5K) | Run: `vektor scan` |
+| Wait 2 weeks | Get results in 1 minute |
+| Get 50-page report | Actionable JSON/HTML reports |
+| Still don't know if fixes work | Re-run to validate fixes |
+
+## 🔬 Attack Categories
+
+### 1. Prompt Injection (6 attacks)
+- Direct instruction injection
+- System prompt override
+- Delimiter confusion
+- Role manipulation
+- Multi-turn context poisoning
+- Encoding-based bypass
+
+### 2. Data Extraction (4 attacks)
+- Training data leak attempts
+- System prompt disclosure
+- Context window extraction
+- PII leakage testing
+
+### 3. Instruction Hijacking (5 attacks) — **NOVEL**
+- Simple document injection
+- DOCX hidden text injection
+- DOCX footnote injection
+- Markdown comment injection
+- Multi-document context poisoning
+
+### 4. RAG Attacks (5 attacks)
+- Context poisoning via retrieved docs
+- RAG prompt leakage
+- Source fabrication / hallucination injection
+- Indirect injection via document store
+- Chunking boundary exploitation
+
+### 5. Agent Attacks (4 attacks)
+- Tool call injection
+- Goal hijacking
+- Memory poisoning
+- Agent scope escape
+
+### 6. Structured Output Injection (3 attacks)
+- JSON schema bypass
+- Output format injection
+- Type confusion attack
+
+## 📦 Installation
+
+### Docker
+```bash
+docker build -t vektor .
+docker run -e OPENAI_API_KEY=$OPENAI_API_KEY vektor scan --target openai
+```
+
+### pip
+```bash
+pip install vektor
+```
+
+### From Source
+```bash
+git clone https://github.com/swapnilwankhede23/vektor.git
+cd vektor
+pip install -e .
+```
+
+## 📖 Usage
+
+### Basic Scan
+```bash
+vektor scan --target openai --budget 1.0
+```
+
+### Quick Mode (High-confidence attacks only)
+```bash
+vektor scan --target openai --quick
+```
+
+### CI/CD Integration
+```bash
+vektor scan --target openai --ci --output report.json
+```
+
+### Specific Attacks
+```bash
+vektor scan --target openai --attacks direct_injection,system_override
+```
+
+### Demo Mode (No API calls)
+```bash
+vektor demo
+```
+
+## 📊 Sample Output
+
+```
+╔══════════════════════════════════╗
+║  V  E  K  T  O  R               ║
+║  AI Security Testing Framework   ║
+╚══════════════════════════════════╝
+
+⠋ Testing attacks... ━━━━━━━━━━━━━━━━━━━━━━ 100% (15/15)
+
+┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━┓
+┃ Attack                  ┃ Severity  ┃ Success    ┃
+┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━┩
+│ Direct Injection        │ HIGH      │ 83%        │
+│ DOCX Hidden Text        │ CRITICAL  │ 67%        │
+│ System Prompt Reveal    │ HIGH      │ 50%        │
+└────────────────────────┴───────────┴────────────┘
+
+╭─────────────────────────────────╮
+│ Summary                         │
+├─────────────────────────────────┤
+│ Risk Score: 72/100              │
+│ Total Vulnerabilities: 3        │
+│ Cost: $0.47                     │
+│                                 │
+│ HIGH RISK: Address all critical │
+│ vulnerabilities before deploy   │
+╰─────────────────────────────────╯
+
+✓ Report saved to: report.json
+```
+
+## 🔍 Novel Research: Instruction Hijacking
+
+Vektor is the first tool to systematically test **document-based instruction hijacking** - a new class of vulnerabilities where attackers embed malicious instructions in document formats (DOCX, PDF, Markdown) that get processed by RAG systems.
+
+Our research found:
+- **60% of RAG systems** vulnerable to hidden text injection
+- **40% vulnerable** to footnote/comment injection
+- Standard sanitization **doesn't catch** format-specific exploits
+
+[Read the full research paper →](docs/INSTRUCTION_HIJACKING.md)
+
+## 🛠️ CI/CD Integration
+
+### GitHub Actions
+```yaml
+name: LLM Security Scan
+on: [push, pull_request]
+jobs:
+  security:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - uses: actions/setup-python@v4
+        with:
+          python-version: '3.11'
+      - name: Install Vektor
+        run: pip install vektor
+      - name: Scan (no API key needed)
+        run: vektor scan --target vulnerable --ci --output report.json
+      - name: Upload Report
+        uses: actions/upload-artifact@v3
+        with:
+          name: security-report
+          path: report.json
+      # Optional: scan your real LLM endpoint
+      # - name: Scan real endpoint
+      #   env:
+      #     OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
+      #   run: vektor scan --target openai --ci --output report.json
+```
+
+## 🤝 Contributing
+
+We welcome contributions! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
+
+### Adding Custom Attacks
+```python
+from vektor.attacks.base import BaseAttack, Vulnerability
+
+class MyCustomAttack(BaseAttack):
+    def __init__(self):
+        super().__init__(name="my_attack", category="Custom")
+
+    def execute(self, target):
+        # Your attack logic
+        pass
+```
+
+## 📚 Documentation
+
+- [Demo Walkthrough](docs/DEMO.md)
+- [Installation Guide](docs/INSTALL.md)
+- [Usage Reference](docs/USAGE.md)
+- [Research: Instruction Hijacking](docs/INSTRUCTION_HIJACKING.md)
+
+## 🗺️ Roadmap
+
+### v0.2 (Current)
+- ✅ 27 attack vectors across 6 categories
+- ✅ HTTP endpoint target (`vektor scan --url http://localhost:8000/chat`)
+- ✅ RAG pipeline targets (LangChain, LlamaIndex)
+- ✅ Agent targets (LangGraph, CrewAI, AutoGen)
+- ✅ Regression diff system for CI gating
+- ✅ Python scan() API
+- ✅ Docker deployment + CI/CD integration
+
+### v0.3 (Next)
+- ⏳ Web dashboard
+- ⏳ PDF document testing
+- ⏳ Multi-model comparison
+- ⏳ Compliance reporting (OWASP LLM Top 10 mapping)
+
+## 📄 License
+
+MIT License - see [LICENSE](LICENSE) for details
+
+## 🙏 Acknowledgments
+
+Built on research from:
+- Simon Willison ([@simonw](https://twitter.com/simonw)) - Prompt injection taxonomy
+- Greshake et al. - Indirect prompt injection
+- OWASP LLM Top 10 Project
+
+## ⭐ Support
+
+If you find this useful, please star the repository and share with your network!
+
+- GitHub: [vektor](https://github.com/swapnilwankhede23/vektor)
+- Issues: [Bug reports & feature requests](https://github.com/swapnilwankhede23/vektor/issues)
+
+## 📧 Contact
+
+- Email: swapnil.wankhede23@spit.ac.in
+- Author: Swapnil
+
+---
+
+**⚠️ Disclaimer:** This tool is for security testing purposes only. Use responsibly and only on systems you have permission to test.

vektor_scan-0.2.0/README.md

@@ -0,0 +1,308 @@
+# Vektor 🛡️
+
+**pytest for AI security — scan LLM apps for vulnerabilities in 30 seconds**
+
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
+
+## 🎯 What is Vektor?
+
+An automated security testing framework that scans LLM applications for vulnerabilities:
+- ✅ **27 validated attack vectors** across 6 categories
+- ✅ **$0.50 average scan cost** with built-in budget controls
+- ✅ **30-second results** - Docker run, immediate feedback
+- ✅ **CI/CD ready** - Integrate into your deployment pipeline
+- ✅ **Novel research** - First tool to systematically test document-based instruction hijacking
+
+## 🚀 Quick Start
+
+```bash
+# Step 1: Install
+pip install vektor
+
+# Step 2: Zero-setup demo (no API key)
+vektor demo
+
+# Step 3: Real scan, $0 cost, always works
+vektor scan --target vulnerable --output my-first-report.html
+
+# Step 4: Open the report
+# Windows: start my-first-report.html
+# Mac:     open my-first-report.html
+# Linux:   xdg-open my-first-report.html
+
+# Step 5 (optional): Scan your own LLM app
+export OPENAI_API_KEY=sk-your-key
+vektor scan --target openai --budget 1.0
+```
+
+## 🌐 Scan Any AI API — No SDK Needed
+
+Point Vektor at any HTTP endpoint:
+```bash
+# Auto-detects OpenAI/Anthropic/custom shapes
+vektor scan --url http://localhost:8000/chat
+
+# With auth header
+vektor scan --url https://my-app.com/api \
+  --header "Authorization: Bearer YOUR_TOKEN"
+
+# Custom request/response field names
+vektor scan --url http://localhost:8000/predict \
+  --request-field prompt --response-field answer
+
+# Query-parameter mode (e.g. /api/parse?text=PAYLOAD)
+vektor scan --url http://localhost:8000/api/parse \
+  --param-field text
+
+# Rate-limited API — add delay between requests
+vektor scan --url http://localhost:8000/chat \
+  --request-delay 12.0
+```
+
+## 💡 Why Vektor?
+
+| Feature | Vektor | Garak | Promptfoo | PyRIT |
+| :--- | :---: | :---: | :---: | :---: |
+| **Primary Focus** | **Actionable Security** | Vulnerability Scanning | General Eval / Testing | Red Teaming Framework |
+| **Setup Time** | **< 30s** | ~10 mins | ~5 mins | ~30 mins |
+| **Scan Speed** | **Fast (Targeted)** | Slow (Exhaustive) | Fast | Slow (Agentic) |
+| **Cost Control** | **✅ Built-in Budget** | ❌ | ❌ | ❌ |
+| **CI/CD Ready** | **✅ Native** | ⚠️ Heavy | ✅ | ⚠️ Complex |
+| **RAG/Doc Attacks** | **✅ Specialized** | ⚠️ Limited | ✅ | ✅ |
+
+| Before | After |
+|--------|-------|
+| Hire pentester ($5K) | Run: `vektor scan` |
+| Wait 2 weeks | Get results in 1 minute |
+| Get 50-page report | Actionable JSON/HTML reports |
+| Still don't know if fixes work | Re-run to validate fixes |
+
+## 🔬 Attack Categories
+
+### 1. Prompt Injection (6 attacks)
+- Direct instruction injection
+- System prompt override
+- Delimiter confusion
+- Role manipulation
+- Multi-turn context poisoning
+- Encoding-based bypass
+
+### 2. Data Extraction (4 attacks)
+- Training data leak attempts
+- System prompt disclosure
+- Context window extraction
+- PII leakage testing
+
+### 3. Instruction Hijacking (5 attacks) — **NOVEL**
+- Simple document injection
+- DOCX hidden text injection
+- DOCX footnote injection
+- Markdown comment injection
+- Multi-document context poisoning
+
+### 4. RAG Attacks (5 attacks)
+- Context poisoning via retrieved docs
+- RAG prompt leakage
+- Source fabrication / hallucination injection
+- Indirect injection via document store
+- Chunking boundary exploitation
+
+### 5. Agent Attacks (4 attacks)
+- Tool call injection
+- Goal hijacking
+- Memory poisoning
+- Agent scope escape
+
+### 6. Structured Output Injection (3 attacks)
+- JSON schema bypass
+- Output format injection
+- Type confusion attack
+
+## 📦 Installation
+
+### Docker
+```bash
+docker build -t vektor .
+docker run -e OPENAI_API_KEY=$OPENAI_API_KEY vektor scan --target openai
+```
+
+### pip
+```bash
+pip install vektor
+```
+
+### From Source
+```bash
+git clone https://github.com/swapnilwankhede23/vektor.git
+cd vektor
+pip install -e .
+```
+
+## 📖 Usage
+
+### Basic Scan
+```bash
+vektor scan --target openai --budget 1.0
+```
+
+### Quick Mode (High-confidence attacks only)
+```bash
+vektor scan --target openai --quick
+```
+
+### CI/CD Integration
+```bash
+vektor scan --target openai --ci --output report.json
+```
+
+### Specific Attacks
+```bash
+vektor scan --target openai --attacks direct_injection,system_override
+```
+
+### Demo Mode (No API calls)
+```bash
+vektor demo
+```
+
+## 📊 Sample Output
+
+```
+╔══════════════════════════════════╗
+║  V  E  K  T  O  R               ║
+║  AI Security Testing Framework   ║
+╚══════════════════════════════════╝
+
+⠋ Testing attacks... ━━━━━━━━━━━━━━━━━━━━━━ 100% (15/15)
+
+┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━┓
+┃ Attack                  ┃ Severity  ┃ Success    ┃
+┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━┩
+│ Direct Injection        │ HIGH      │ 83%        │
+│ DOCX Hidden Text        │ CRITICAL  │ 67%        │
+│ System Prompt Reveal    │ HIGH      │ 50%        │
+└────────────────────────┴───────────┴────────────┘
+
+╭─────────────────────────────────╮
+│ Summary                         │
+├─────────────────────────────────┤
+│ Risk Score: 72/100              │
+│ Total Vulnerabilities: 3        │
+│ Cost: $0.47                     │
+│                                 │
+│ HIGH RISK: Address all critical │
+│ vulnerabilities before deploy   │
+╰─────────────────────────────────╯
+
+✓ Report saved to: report.json
+```
+
+## 🔍 Novel Research: Instruction Hijacking
+
+Vektor is the first tool to systematically test **document-based instruction hijacking** - a new class of vulnerabilities where attackers embed malicious instructions in document formats (DOCX, PDF, Markdown) that get processed by RAG systems.
+
+Our research found:
+- **60% of RAG systems** vulnerable to hidden text injection
+- **40% vulnerable** to footnote/comment injection
+- Standard sanitization **doesn't catch** format-specific exploits
+
+[Read the full research paper →](docs/INSTRUCTION_HIJACKING.md)
+
+## 🛠️ CI/CD Integration
+
+### GitHub Actions
+```yaml
+name: LLM Security Scan
+on: [push, pull_request]
+jobs:
+  security:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - uses: actions/setup-python@v4
+        with:
+          python-version: '3.11'
+      - name: Install Vektor
+        run: pip install vektor
+      - name: Scan (no API key needed)
+        run: vektor scan --target vulnerable --ci --output report.json
+      - name: Upload Report
+        uses: actions/upload-artifact@v3
+        with:
+          name: security-report
+          path: report.json
+      # Optional: scan your real LLM endpoint
+      # - name: Scan real endpoint
+      #   env:
+      #     OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
+      #   run: vektor scan --target openai --ci --output report.json
+```
+
+## 🤝 Contributing
+
+We welcome contributions! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
+
+### Adding Custom Attacks
+```python
+from vektor.attacks.base import BaseAttack, Vulnerability
+
+class MyCustomAttack(BaseAttack):
+    def __init__(self):
+        super().__init__(name="my_attack", category="Custom")
+
+    def execute(self, target):
+        # Your attack logic
+        pass
+```
+
+## 📚 Documentation
+
+- [Demo Walkthrough](docs/DEMO.md)
+- [Installation Guide](docs/INSTALL.md)
+- [Usage Reference](docs/USAGE.md)
+- [Research: Instruction Hijacking](docs/INSTRUCTION_HIJACKING.md)
+
+## 🗺️ Roadmap
+
+### v0.2 (Current)
+- ✅ 27 attack vectors across 6 categories
+- ✅ HTTP endpoint target (`vektor scan --url http://localhost:8000/chat`)
+- ✅ RAG pipeline targets (LangChain, LlamaIndex)
+- ✅ Agent targets (LangGraph, CrewAI, AutoGen)
+- ✅ Regression diff system for CI gating
+- ✅ Python scan() API
+- ✅ Docker deployment + CI/CD integration
+
+### v0.3 (Next)
+- ⏳ Web dashboard
+- ⏳ PDF document testing
+- ⏳ Multi-model comparison
+- ⏳ Compliance reporting (OWASP LLM Top 10 mapping)
+
+## 📄 License
+
+MIT License - see [LICENSE](LICENSE) for details
+
+## 🙏 Acknowledgments
+
+Built on research from:
+- Simon Willison ([@simonw](https://twitter.com/simonw)) - Prompt injection taxonomy
+- Greshake et al. - Indirect prompt injection
+- OWASP LLM Top 10 Project
+
+## ⭐ Support
+
+If you find this useful, please star the repository and share with your network!
+
+- GitHub: [vektor](https://github.com/swapnilwankhede23/vektor)
+- Issues: [Bug reports & feature requests](https://github.com/swapnilwankhede23/vektor/issues)
+
+## 📧 Contact
+
+- Email: swapnil.wankhede23@spit.ac.in
+- Author: Swapnil
+
+---
+
+**⚠️ Disclaimer:** This tool is for security testing purposes only. Use responsibly and only on systems you have permission to test.