humanbound-cli 0.1.0__tar.gz

humanbound_cli-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Humanbound
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

humanbound_cli-0.1.0/PKG-INFO

@@ -0,0 +1,498 @@
+Metadata-Version: 2.1
+Name: humanbound-cli
+Version: 0.1.0
+Summary: Humanbound CLI - command line interface for AI agent security testing.
+Author-email: Kostas Siabanis <hello@humanbound.io>, Demetris Gerogiannis <hello@humanbound.io>
+License: Apache-2.0
+Project-URL: Homepage, https://github.com/Humanbound/humanbound-cli
+Project-URL: Documentation, https://docs.humanbound.io/cli
+Project-URL: Issues, https://github.com/Humanbound/humanbound-cli/issues
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Operating System :: OS Independent
+Classifier: Environment :: Console
+Classifier: Framework :: Pytest
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: click>=8.1.0
+Requires-Dist: rich>=13.0.0
+Requires-Dist: requests>=2.32.0
+Requires-Dist: pyyaml>=6.0.0
+Provides-Extra: pytest
+Requires-Dist: pytest>=7.0.0; extra == "pytest"
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0.0; extra == "dev"
+Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
+
+# Humanbound CLI (Beta)
+
+> CLI-first security testing for AI agents and chatbots. Adversarial attacks, behavioral QA, posture scoring, and guardrails export — from your terminal to your CI/CD pipeline.
+
+[![PyPI](https://img.shields.io/pypi/v/humanbound-cli)](https://pypi.org/project/humanbound-cli/)
+[![License](https://img.shields.io/badge/license-proprietary-blue)]()
+
+```
+pip install humanbound-cli
+```
+
+---
+
+## Overview
+
+Humanbound runs automated adversarial attacks against your bot's live endpoint, evaluates responses using LLM-as-a-judge, and produces structured findings aligned with the [OWASP Top 10 for LLM Applications](https://owasp.org/www-project-top-10-for-large-language-model-applications/) and the [OWASP Agentic AI Threats](https://genai.owasp.org/resource/agentic-ai-threats-and-mitigations/).
+
+### Platform Services
+
+| Service | Description |
+|---------|-------------|
+| **CLI Tool** | Full-featured command line interface. Initialize projects, run tests, check posture, export guardrails. |
+| **pytest Plugin** | Native pytest integration with markers, fixtures, and baseline comparison. Run security tests alongside unit tests. |
+| **Adversarial Testing** | OWASP-aligned attack scenarios: single-turn, multi-turn, adaptive, and agentic. |
+| **Behavioral Testing** | Validate intent boundaries, response quality, and functional correctness. |
+| **Posture Scoring** | Quantified 0-100 security score with breakdown by findings, coverage, and resilience. Track over time. |
+| **Guardrails Export** | Generate protection rules from test findings. Export to OpenAI, Azure AI Content Safety, AWS Bedrock, or Humanbound format. |
+
+### Why Humanbound?
+
+Manual red-teaming doesn't scale. Static analysis can't catch runtime behavior. Generic pentesting tools don't understand LLM-specific attack vectors like prompt injection, jailbreaks, or tool abuse.
+
+Humanbound is built for this. Point it at your bot's endpoint, define the scope (or let it extract one from your system prompt), and get a structured security report with actionable findings — all mapped to OWASP LLM and Agentic AI categories.
+
+Testing feeds into hardening: export guardrails, track posture across releases, and catch regressions before they reach production. Works with any chatbot or agent, cloud or on-prem.
+
+---
+
+## Get Started
+
+### 1. Install & authenticate
+
+```bash
+pip install humanbound-cli
+hb login
+```
+
+### 2. Scan your bot & create a project
+
+`hb init` scans your bot, extracts its scope and risk profile, and creates a project — all in one step. Point it at one or more sources:
+
+```bash
+# From a system prompt file
+hb init -n "My Bot" --prompt ./system_prompt.txt
+
+# From a live bot endpoint (API probing)
+hb init -n "My Bot" -e ./bot-config.json
+
+# From a live URL (browser discovery)
+hb init -n "My Bot" -u https://my-bot.example.com
+
+# Combine sources for better analysis
+hb init -n "My Bot" --prompt ./system.txt -e ./bot-config.json
+```
+
+The `--endpoint/-e` flag accepts a JSON config (file or inline string) matching the experiment integration shape:
+
+```json
+{
+  "streaming": false,
+  "thread_auth": {"endpoint": "", "headers": {}, "payload": {}},
+  "thread_init": {"endpoint": "https://bot.com/threads", "headers": {}, "payload": {}},
+  "chat_completion": {"endpoint": "https://bot.com/chat", "headers": {"Authorization": "Bearer token"}, "payload": {"content": "$PROMPT"}}
+}
+```
+
+After scanning, you'll see the extracted scope, policies (permitted/restricted intents), and a risk dashboard with threat profile. Confirm to create the project.
+
+### 3. Run a security test
+
+```bash
+# Run against your bot (uses project's default integration if configured during init)
+hb test
+
+# Or specify an endpoint directly
+hb test -e ./bot-config.json
+
+# Choose test category and depth
+hb test -t owasp_multi_turn -l system
+```
+
+### 4. Review results
+
+```bash
+# Watch experiment progress
+hb status --watch
+
+# View logs
+hb logs
+
+# Check posture score
+hb posture
+
+# Export guardrails
+hb guardrails --vendor openai -o guardrails.json
+```
+
+---
+
+## Test Categories
+
+| Category | Mode | Description |
+|----------|------|-------------|
+| `owasp_single_turn` | Adversarial | Single-prompt attacks: prompt injection, jailbreaks, data exfiltration. Fast coverage of basic vulnerabilities. |
+| `owasp_multi_turn` | Adversarial | Conversational attacks that build context over multiple turns. Tests context manipulation and gradual escalation. |
+| `owasp_agentic_multi_turn` | Adversarial | Targets tool-using agents. Tests goal hijacking, tool misuse, and privilege escalation. |
+| `behavioral` | QA | Intent boundary validation and response quality testing. Ensures agent behaves within defined scope. |
+
+**Adaptive mode:** Both `owasp_multi_turn` and `owasp_agentic_multi_turn` support an adaptive flag that enables evolutionary search — the attack strategy adapts based on bot responses instead of following scripted prompts.
+
+### Testing Levels
+
+| Level | Description |
+|-------|-------------|
+| `unit` | Standard coverage (~20 min) — default |
+| `system` | Deep testing (~45 min) |
+| `acceptance` | Full coverage (~90 min) |
+
+---
+
+## pytest Integration
+
+Run security tests alongside your existing test suite with native pytest markers and fixtures.
+
+```python
+# test_security.py
+import pytest
+
+@pytest.mark.hb
+def test_prompt_injection(hb):
+    """Test prompt injection defenses."""
+    result = hb.test("llm001")
+    assert result.passed, f"Failed: {result.findings}"
+
+@pytest.mark.hb
+def test_posture_threshold(hb_posture):
+    """Ensure posture meets minimum."""
+    assert hb_posture["score"] >= 70
+
+@pytest.mark.hb
+def test_no_regressions(hb, hb_baseline):
+    """Compare against baseline."""
+    result = hb.test("llm001")
+    if hb_baseline:
+        regressions = result.compare(hb_baseline)
+        assert not regressions
+```
+
+```bash
+# Run with Humanbound enabled
+pytest --hb tests/
+
+# Filter by category
+pytest --hb --hb-category=adversarial
+
+# Set failure threshold
+pytest --hb --hb-fail-on=high
+
+# Compare to baseline
+pytest --hb --hb-baseline=baseline.json
+
+# Save new baseline
+pytest --hb --hb-save-baseline=baseline.json
+```
+
+---
+
+## CI/CD Integration
+
+Block insecure deployments automatically with exit codes.
+
+```
+Build -> Unit Tests -> AI Security (hb test) -> Deploy
+```
+
+```yaml
+# .github/workflows/security.yml
+name: AI Security Tests
+on: [push, pull_request]
+
+jobs:
+  security:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - run: pip install humanbound-cli
+      - name: Run Security Tests
+        env:
+          HUMANBOUND_API_KEY: ${{ secrets.HUMANBOUND_API_KEY }}
+        run: |
+          hb test --wait --fail-on=high
+```
+
+---
+
+## Usage
+
+```
+hb [--base-url URL] COMMAND [OPTIONS] [ARGS]
+```
+
+### Authentication
+
+| Command | Description |
+|---------|-------------|
+| `login` | Authenticate via browser (OAuth PKCE) |
+| `logout` | Clear stored credentials |
+| `whoami` | Show current authentication status |
+
+### Organisation Management
+
+| Command | Description |
+|---------|-------------|
+| `orgs list` | List available organisations |
+| `orgs current` | Show current organisation |
+| `switch <id>` | Switch to organisation |
+
+### Provider Management
+
+Providers are LLM configurations used for running security tests.
+
+| Command | Description |
+|---------|-------------|
+| `providers list` | List configured providers |
+| `providers add` | Add new provider |
+| `providers update <id>` | Update provider config |
+| `providers remove <id>` | Remove provider |
+
+<details>
+<summary><code>providers add</code> options</summary>
+
+```
+--name, -n        Provider name: openai, claude, azureopenai, gemini, grok, custom
+--api-key, -k     API key
+--endpoint, -e    Endpoint URL (required for azureopenai, custom)
+--model, -m       Model name (optional)
+--default         Set as default provider
+--interactive     Interactive configuration mode
+```
+
+</details>
+
+### Project Management
+
+| Command | Description |
+|---------|-------------|
+| `projects list` | List projects |
+| `projects use <id>` | Select project |
+| `projects current` | Show current project |
+| `projects show [id]` | Show project details |
+
+<details>
+<summary><code>init</code> — scan bot & create project</summary>
+
+```
+hb init --name NAME [OPTIONS]
+
+Sources (at least one required):
+  --prompt, -p PATH       System prompt file (text source)
+  --url, -u URL           Live bot URL for browser discovery (url source)
+  --endpoint, -e CONFIG   Bot integration config — JSON string or file path (endpoint source)
+  --repo, -r PATH         Repository path to scan (agentic or text source)
+  --openapi, -o PATH      OpenAPI spec file (text source)
+
+Options:
+  --description, -d       Project description
+  --timeout, -t SECONDS   Scan timeout (default: 180)
+  --yes, -y               Auto-confirm project creation (no interactive prompts)
+```
+
+</details>
+
+### Test Execution
+
+<details>
+<summary><code>test</code> — run security tests on current project</summary>
+
+```
+hb test [OPTIONS]
+
+Test Category:
+  --test-category, -t   Test to run (default: owasp_multi_turn)
+                        Values: owasp_single_turn, owasp_multi_turn,
+                                owasp_agentic_multi_turn, behavioral
+
+Testing Level:
+  --testing-level, -l   Depth of testing (default: unit)
+                        unit | system | acceptance
+
+Chat Endpoint (required):
+  --chat-endpoint       Chat completion URL of the bot to test
+  --chat-header         Header for chat endpoint (repeatable)
+  --chat-payload        JSON payload template for chat
+
+Init Endpoint (optional):
+  --init-endpoint       Thread initialization URL
+  --init-header         Header for init endpoint (repeatable)
+  --init-payload        JSON payload for init
+
+Auth Endpoint (optional):
+  --auth-endpoint       Auth/token endpoint URL
+  --auth-header         Header for auth endpoint (repeatable)
+  --auth-payload        JSON payload for auth
+
+Other:
+  --provider-id         Provider to use (default: first available)
+  --name, -n            Experiment name (auto-generated if omitted)
+  --lang                Language (default: english). Accepts codes: en, de, es...
+  --adaptive            Enable adaptive mode (evolutionary attack strategy)
+  --streaming           Enable streaming mode (requires wss:// endpoint)
+  --no-auto-start       Create without starting (manual mode)
+  --wait, -w            Wait for completion
+  --fail-on SEVERITY    Exit non-zero if findings >= severity
+                        Values: critical, high, medium, low, any
+```
+
+</details>
+
+### Experiment Management
+
+| Command | Description |
+|---------|-------------|
+| `experiments list` | List experiments |
+| `experiments show <id>` | Show experiment details |
+| `experiments status <id>` | Check status |
+| `experiments status <id> --watch` | Watch until completion |
+| `experiments wait <id>` | Wait with progressive backoff (30s -> 60s -> 120s -> 300s) |
+| `experiments logs <id>` | List experiment logs |
+| `experiments report <id>` | Download HTML report |
+
+`status` is also available as a top-level alias — without an ID it shows the most recent experiment:
+
+```bash
+hb status [experiment_id] [--watch]
+```
+
+### Results & Export
+
+```bash
+# View experiment results (table, json, or csv)
+hb logs [experiment_id] [--format table] [--verdict pass|fail] [--page N] [--size N]
+
+# Security posture score
+hb posture [--json]
+
+# Export guardrails configuration
+hb guardrails [--vendor humanbound|openai] [--format json|yaml] [-o FILE]
+```
+
+### Documentation
+
+```bash
+hb docs
+```
+
+Opens documentation in browser.
+
+---
+
+## Examples
+
+### End-to-end: scan, create project, test, review
+
+```bash
+hb login
+hb switch abc123
+
+# Scan bot & create project (uses endpoint config file)
+hb init -n "Support Bot" -e ./bot-config.json
+
+# Run adversarial test (uses project's default integration)
+hb test -t owasp_multi_turn -l unit
+
+# Watch and review
+hb status --watch
+hb logs
+hb posture
+```
+
+### Multi-source project init
+
+```bash
+# Combine system prompt + live endpoint for best scope extraction
+hb init \
+  --name "Support Bot" \
+  --prompt ./prompts/system.txt \
+  --endpoint ./bot-config.json
+
+# From repository + OpenAPI spec
+hb init \
+  --name "API Agent" \
+  --repo ./my-agent \
+  --openapi ./openapi.yaml
+```
+
+### Bot config with auth + thread init
+
+```json
+{
+  "streaming": false,
+  "thread_auth": {
+    "endpoint": "https://bot.com/oauth/token",
+    "headers": {},
+    "payload": {"client_id": "x", "client_secret": "y"}
+  },
+  "thread_init": {
+    "endpoint": "https://bot.com/threads",
+    "headers": {"Content-Type": "application/json"},
+    "payload": {}
+  },
+  "chat_completion": {
+    "endpoint": "https://bot.com/chat",
+    "headers": {"Content-Type": "application/json"},
+    "payload": {"messages": [{"role": "user", "content": "$PROMPT"}]}
+  }
+}
+```
+
+```bash
+# Use with init or test
+hb init -n "My Bot" -e ./bot-config.json
+hb test -e ./bot-config.json
+```
+
+### Export guardrails
+
+```bash
+hb guardrails --vendor openai --format json -o guardrails.json
+```
+
+---
+
+### On-Premises
+
+```bash
+export HUMANBOUND_BASE_URL=https://api.your-domain.com
+hb login
+```
+
+### Files
+
+| Path | Description |
+|------|-------------|
+| `~/.humanbound/` | Configuration directory |
+| `~/.humanbound/credentials.json` | Auth tokens (mode `600`) |
+
+---
+
+## Exit Codes
+
+| Code | Meaning |
+|------|---------|
+| `0` | Success |
+| `1` | Error or test failure (with `--fail-on`) |
+
+---
+
+## Links
+
+- [Documentation](https://docs.humanbound.io)
+- [GitHub](https://github.com/Humanbound/humanbound-cli)