aegis-pentest 0.1.0__tar.gz

aegis_pentest-0.1.0/.gitignore

@@ -0,0 +1,17 @@
+# Python-generated files
+__pycache__/
+*.py[oc]
+build/
+dist/
+wheels/
+*.egg-info
+
+# Virtual environments
+.venv
+
+# Claude / AI tooling
+.claude/
+AEGIS_MVP_PROMPT.md
+
+# Engagement data (client-specific, never commit)
+engagements/*/

aegis_pentest-0.1.0/.python-version

@@ -0,0 +1 @@
+3.12

aegis_pentest-0.1.0/PKG-INFO

@@ -0,0 +1,347 @@
+Metadata-Version: 2.4
+Name: aegis-pentest
+Version: 0.1.0
+Summary: AEGIS — Adaptive Engagement & Generic Inspection Scanner
+Author-email: Majd Bnat <hey@majdb.com>
+Requires-Python: >=3.12
+Requires-Dist: aiosqlite>=0.20
+Requires-Dist: anthropic>=0.40
+Requires-Dist: anyio>=4.0
+Requires-Dist: dnspython>=2.6
+Requires-Dist: fastmcp>=0.4
+Requires-Dist: httpx>=0.27
+Requires-Dist: jinja2>=3.1
+Requires-Dist: packaging>=24.0
+Requires-Dist: pydantic>=2.0
+Requires-Dist: python-dateutil>=2.9
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: rich>=13.0
+Requires-Dist: sqlmodel>=0.0.21
+Requires-Dist: structlog>=24.0
+Requires-Dist: typer[all]>=0.12
+Description-Content-Type: text/markdown
+
+# aegis
+
+**Adaptive Engagement & Generic Inspection Scanner**
+
+An autonomous, AI-orchestrated web penetration testing agent. Aegis handles reconnaissance, fingerprinting, vulnerability discovery, and verification so the human pentester focuses on what actually requires human judgment.
+
+---
+
+## What it is
+
+Aegis runs structured, methodology-driven engagements against web targets. It profiles the host environment, fingerprints the target stack, selects relevant tools and tests from the PTES + OWASP WSTG v4.2 playbook, executes them concurrently, and produces a verified findings report with remediation guidance.
+
+It is not a Burp Suite replacement. It is the autonomous recon and vuln-discovery layer that feeds the human pentester, eliminating roughly 70% of routine busywork.
+
+**Hard constraints built into the runtime:**
+
+- Every engagement requires a signed `scope.yaml` before any network egress.
+- Every outbound request is matched against in-scope and out-of-scope rules before the socket opens.
+- Scope violations abort the current task and are written to the audit log.
+- There is no `--force` flag that bypasses scope.
+
+---
+
+## Installation
+
+**Via pipx (recommended):**
+
+```bash
+pipx install aegis-pentest
+aegis init
+```
+
+**Via AUR (Arch Linux):**
+
+```bash
+yay -S aegis-pentest
+aegis init
+```
+
+**Requirements:**
+
+- Python 3.12+
+- `ANTHROPIC_API_KEY` in environment
+- Pentest toolchain (see `aegis env tools` after init)
+
+---
+
+## Quickstart
+
+```bash
+# First run: profiles host, lists missing tools
+aegis init
+
+# Sync the knowledge base (NVD, GHSA, nuclei-templates)
+aegis kb sync
+
+# Create a new engagement
+aegis engagement new --client "Acme Corp" --domain "acme.com"
+
+# Fill in the authorization details
+vim engagements/2026-05-acme/scope.yaml
+
+# Run
+aegis run engagements/2026-05-acme
+
+# Generate report
+aegis report engagements/2026-05-acme --format html
+```
+
+---
+
+## scope.yaml
+
+Aegis refuses to run without this file. No exceptions.
+
+```yaml
+engagement_id: "BL-2026-007"
+client: "Acme Corp"
+operator: "Majd Bnat <hey@majdb.com>"
+authorization:
+  document_ref: "SOW-2026-007.pdf"
+  signed_date: "2026-05-12"
+  expiry: "2026-06-12"
+in_scope:
+  domains:
+    - "*.acme.com"
+    - "api-staging.acme.io"
+  ips:
+    - "203.0.113.0/24"
+out_of_scope:
+  - "admin.acme.com"
+  - "*.internal.acme.com"
+rules_of_engagement:
+  rate_limit_rps: 10
+  business_hours_only: false
+  destructive_tests: false
+  no_credential_stuffing: true
+  no_dos_tests: true
+```
+
+---
+
+## Architecture
+
+```
+                          aegis CLI
+                              |
+                   Engagement Manager
+              (scope validation, lifecycle, audit log)
+                              |
+          +-----------------+-----------------+
+          |                 |                 |
+   Environment          Target            Methodology
+   Profiler             Profiler          Engine
+   (host info)          (fingerprint)     (PTES phases)
+          |                 |                 |
+          +--------+--------+--------+--------+
+                            |
+                   LLM Orchestrator
+                  (Haiku / Sonnet / Opus)
+                            |
+               +------------+------------+
+               |            |            |
+           Tool          Knowledge     Findings
+           Registry      Base          DB
+           (35+ tools)   (NVD/GHSA)    (SQLite)
+               |                            |
+         Tool Executor                  Reporter
+         (async, sandboxed)             (md/html/json)
+```
+
+The orchestrator runs a bounded loop per phase:
+
+```
+plan(phase_context) -> execute(action) -> observe(result) -> update(state)
+  ^                                                               |
+  +---------------------------------------------------------------+
+                   until phase complete OR budget exceeded
+```
+
+Three independent budgets bound each phase: token budget, wall-clock time, and action count. Whichever trips first ends the phase and triggers finalize mode.
+
+---
+
+## Token model
+
+Aegis is designed to complete a full medium-scope engagement for under $2 in LLM tokens. This is achieved through several layered tactics:
+
+| Tactic | Impact |
+|---|---|
+| Tiered model routing (Haiku handles ~70% of calls) | -60% cost |
+| Prompt caching on system prompt + engagement context | -40% on input tokens |
+| Parsed tool output, never raw stdout to the LLM | -80% on tool-heavy phases |
+| Structured tool-use schema, no prose planning | -30% output tokens |
+| Methodology-driven action space pruning | -50% wasted calls |
+| SQLite-cached recon reused across phases | variable |
+| Finding deduplication before LLM sees results | -10-30% |
+
+Model tiers:
+
+| Tier | Model | Used for |
+|---|---|---|
+| NANO | claude-haiku-4-5 | Parsing, classification, summarization |
+| MAIN | claude-sonnet-4-6 | Planning, hypothesis generation, verification probes |
+| DEEP | claude-opus-4-7 | Attack chain analysis, hard reasoning |
+| LOCAL | ollama (optional) | Offline pre-classification |
+
+The live cost meter runs in the terminal throughout each phase:
+
+```
+Phase: VULN_ANALYSIS  [>>>>>>>>--] 80%
+Budget: $0.74 / $5.00   Tokens: 41.2k / 200k   Time: 12m / 60m
+Tier breakdown: NANO 24% . MAIN 71% . DEEP 5%   Cache hit: 82%
+```
+
+---
+
+## Tool catalog
+
+Aegis wraps 35+ tools. Raw output is never passed to the LLM. Each tool has a typed parser that produces structured `Finding` or `Observation` models. A nmap scan returning 47 open ports becomes 47 `OpenPort` observations of ~80 bytes each, not 200 KB of XML.
+
+| Category | Tools |
+|---|---|
+| Subdomain enumeration | subfinder, amass, assetfinder, dnsx |
+| Live host detection | httpx, httprobe |
+| Port scanning | nmap, naabu, rustscan |
+| Web crawling | katana, gospider, hakrawler |
+| Content discovery | ffuf, feroxbuster, gobuster, dirsearch |
+| Tech fingerprinting | whatweb, wappalyzer-cli, httpx -tech-detect |
+| Vulnerability scanning | nuclei, nikto, wpscan, droopescan, joomscan |
+| TLS auditing | testssl.sh, sslscan, sslyze |
+| Parameter discovery | arjun, paramspider, x8 |
+| Secrets | trufflehog, gitleaks |
+| Visual recon | gowitness, aquatone |
+| SQLi verification | sqlmap (safe flags only: --batch --crawl=0 --level=1 --risk=1) |
+
+---
+
+## Environment profiling
+
+On first run, `aegis init` profiles the host and derives auto-tuned concurrency settings:
+
+```
+Host: arch-workstation
+  OS        Arch Linux (rolling, kernel 6.9.3-arch1-1)
+  CPU       AMD Ryzen 7 5800X . 8 cores / 16 threads . 4.7 GHz
+  Memory    32 GB total . 24 GB available
+  Repos     core, extra, multilib, blackarch
+
+Pentest toolchain: 28/35 detected
+  nmap 7.95       nuclei 3.2.9      httpx 1.6.6
+  ffuf 2.1.0      subfinder 2.6.6   katana 1.1.0
+  sqlmap 1.8.5    wpscan 3.8.27     nikto 2.5.0
+  gobuster 3.6.0  amass 4.2.0       gowitness 3.0.3
+
+Missing: testssl.sh  feroxbuster  dnsrecon  arjun  paramspider  trufflehog
+  -> Run: aegis env install --missing
+
+Auto-tuned concurrency:
+  nmap_parallelism=16   nuclei_concurrency=32   ffuf_threads=64
+  httpx_concurrency=80  max_parallel_tools=4
+```
+
+---
+
+## CLI reference
+
+```
+aegis init                                    First-run setup and env profile
+aegis env show                                Display host profile
+aegis env tools                               Tool inventory
+aegis env install --missing                   Generate install commands for missing tools
+aegis env refresh                             Re-detect host profile
+
+aegis kb sync [--source nvd|ghsa|nuclei]      Sync knowledge base
+aegis kb stats                                Knowledge base summary
+aegis kb query --product nginx --min-cvss 7   Query CVEs
+
+aegis engagement new --client X --domain Y   Scaffold engagement dir and scope.yaml
+aegis engagement list                         List engagements
+
+aegis run <dir> [--phase PHASE]              Run engagement
+aegis run <dir> --dry-run                    Preview planned actions
+aegis run <dir> --budget-usd 2.00           Cap spend
+
+aegis report <dir> [--format md|html|json]   Generate report
+aegis findings list <dir> [--severity high]  List findings
+aegis findings verify <finding-id>           Re-run verification probe
+aegis findings suppress <finding-id> --reason "..."
+
+aegis cost <dir>                             Detailed cost breakdown
+aegis audit <dir>                            Full audit log
+```
+
+All commands support `--json` for scripting, `-v/-vv/-vvv` for verbosity, `--quiet` for CI.
+
+---
+
+## Configuration
+
+Global config lives at `~/.config/aegis/config.toml`. Any key can be overridden per engagement in `engagement_dir/config.toml`.
+
+```toml
+[api]
+anthropic_api_key_env = "ANTHROPIC_API_KEY"
+
+[models]
+nano  = "claude-haiku-4-5-20251001"
+main  = "claude-sonnet-4-6"
+deep  = "claude-opus-4-7"
+
+[models.local]
+enabled  = false
+endpoint = "http://localhost:11434"
+model    = "qwen2.5:7b"
+
+[budgets]
+tokens_per_phase       = 30000
+tokens_per_engagement  = 200000
+usd_per_engagement     = 5.00
+wall_time_per_phase_sec = 1800
+
+[caching]
+prompt_cache          = true
+kb_cache_dir          = "~/.cache/aegis/kb"
+fingerprint_cache_ttl_hours = 168
+
+[tooling]
+docker_isolate         = false
+default_rate_limit_rps = 10
+respect_robots_txt     = false
+
+[reporting]
+default_format   = "html"
+include_audit_log = true
+```
+
+---
+
+## Tech stack
+
+| Layer | Choice |
+|---|---|
+| Language | Python 3.12+ |
+| CLI | Typer + Rich |
+| Async | asyncio + anyio |
+| HTTP | httpx (async, HTTP/2) |
+| Models | Pydantic v2 |
+| Storage | SQLite + SQLModel |
+| LLM | Anthropic SDK (Claude) |
+| Templating | Jinja2 |
+| Logging | structlog + rich |
+| Packaging | uv (dev), hatch (build) |
+| Testing | pytest + pytest-asyncio + respx |
+
+---
+
+## License
+
+MIT. Use responsibly and only against systems you are authorized to test.
+
+---
+
+Built by [Majd Bnat](mailto:hey@majdb.com)

aegis_pentest-0.1.0/README.md

@@ -0,0 +1,324 @@
+# aegis
+
+**Adaptive Engagement & Generic Inspection Scanner**
+
+An autonomous, AI-orchestrated web penetration testing agent. Aegis handles reconnaissance, fingerprinting, vulnerability discovery, and verification so the human pentester focuses on what actually requires human judgment.
+
+---
+
+## What it is
+
+Aegis runs structured, methodology-driven engagements against web targets. It profiles the host environment, fingerprints the target stack, selects relevant tools and tests from the PTES + OWASP WSTG v4.2 playbook, executes them concurrently, and produces a verified findings report with remediation guidance.
+
+It is not a Burp Suite replacement. It is the autonomous recon and vuln-discovery layer that feeds the human pentester, eliminating roughly 70% of routine busywork.
+
+**Hard constraints built into the runtime:**
+
+- Every engagement requires a signed `scope.yaml` before any network egress.
+- Every outbound request is matched against in-scope and out-of-scope rules before the socket opens.
+- Scope violations abort the current task and are written to the audit log.
+- There is no `--force` flag that bypasses scope.
+
+---
+
+## Installation
+
+**Via pipx (recommended):**
+
+```bash
+pipx install aegis-pentest
+aegis init
+```
+
+**Via AUR (Arch Linux):**
+
+```bash
+yay -S aegis-pentest
+aegis init
+```
+
+**Requirements:**
+
+- Python 3.12+
+- `ANTHROPIC_API_KEY` in environment
+- Pentest toolchain (see `aegis env tools` after init)
+
+---
+
+## Quickstart
+
+```bash
+# First run: profiles host, lists missing tools
+aegis init
+
+# Sync the knowledge base (NVD, GHSA, nuclei-templates)
+aegis kb sync
+
+# Create a new engagement
+aegis engagement new --client "Acme Corp" --domain "acme.com"
+
+# Fill in the authorization details
+vim engagements/2026-05-acme/scope.yaml
+
+# Run
+aegis run engagements/2026-05-acme
+
+# Generate report
+aegis report engagements/2026-05-acme --format html
+```
+
+---
+
+## scope.yaml
+
+Aegis refuses to run without this file. No exceptions.
+
+```yaml
+engagement_id: "BL-2026-007"
+client: "Acme Corp"
+operator: "Majd Bnat <hey@majdb.com>"
+authorization:
+  document_ref: "SOW-2026-007.pdf"
+  signed_date: "2026-05-12"
+  expiry: "2026-06-12"
+in_scope:
+  domains:
+    - "*.acme.com"
+    - "api-staging.acme.io"
+  ips:
+    - "203.0.113.0/24"
+out_of_scope:
+  - "admin.acme.com"
+  - "*.internal.acme.com"
+rules_of_engagement:
+  rate_limit_rps: 10
+  business_hours_only: false
+  destructive_tests: false
+  no_credential_stuffing: true
+  no_dos_tests: true
+```
+
+---
+
+## Architecture
+
+```
+                          aegis CLI
+                              |
+                   Engagement Manager
+              (scope validation, lifecycle, audit log)
+                              |
+          +-----------------+-----------------+
+          |                 |                 |
+   Environment          Target            Methodology
+   Profiler             Profiler          Engine
+   (host info)          (fingerprint)     (PTES phases)
+          |                 |                 |
+          +--------+--------+--------+--------+
+                            |
+                   LLM Orchestrator
+                  (Haiku / Sonnet / Opus)
+                            |
+               +------------+------------+
+               |            |            |
+           Tool          Knowledge     Findings
+           Registry      Base          DB
+           (35+ tools)   (NVD/GHSA)    (SQLite)
+               |                            |
+         Tool Executor                  Reporter
+         (async, sandboxed)             (md/html/json)
+```
+
+The orchestrator runs a bounded loop per phase:
+
+```
+plan(phase_context) -> execute(action) -> observe(result) -> update(state)
+  ^                                                               |
+  +---------------------------------------------------------------+
+                   until phase complete OR budget exceeded
+```
+
+Three independent budgets bound each phase: token budget, wall-clock time, and action count. Whichever trips first ends the phase and triggers finalize mode.
+
+---
+
+## Token model
+
+Aegis is designed to complete a full medium-scope engagement for under $2 in LLM tokens. This is achieved through several layered tactics:
+
+| Tactic | Impact |
+|---|---|
+| Tiered model routing (Haiku handles ~70% of calls) | -60% cost |
+| Prompt caching on system prompt + engagement context | -40% on input tokens |
+| Parsed tool output, never raw stdout to the LLM | -80% on tool-heavy phases |
+| Structured tool-use schema, no prose planning | -30% output tokens |
+| Methodology-driven action space pruning | -50% wasted calls |
+| SQLite-cached recon reused across phases | variable |
+| Finding deduplication before LLM sees results | -10-30% |
+
+Model tiers:
+
+| Tier | Model | Used for |
+|---|---|---|
+| NANO | claude-haiku-4-5 | Parsing, classification, summarization |
+| MAIN | claude-sonnet-4-6 | Planning, hypothesis generation, verification probes |
+| DEEP | claude-opus-4-7 | Attack chain analysis, hard reasoning |
+| LOCAL | ollama (optional) | Offline pre-classification |
+
+The live cost meter runs in the terminal throughout each phase:
+
+```
+Phase: VULN_ANALYSIS  [>>>>>>>>--] 80%
+Budget: $0.74 / $5.00   Tokens: 41.2k / 200k   Time: 12m / 60m
+Tier breakdown: NANO 24% . MAIN 71% . DEEP 5%   Cache hit: 82%
+```
+
+---
+
+## Tool catalog
+
+Aegis wraps 35+ tools. Raw output is never passed to the LLM. Each tool has a typed parser that produces structured `Finding` or `Observation` models. A nmap scan returning 47 open ports becomes 47 `OpenPort` observations of ~80 bytes each, not 200 KB of XML.
+
+| Category | Tools |
+|---|---|
+| Subdomain enumeration | subfinder, amass, assetfinder, dnsx |
+| Live host detection | httpx, httprobe |
+| Port scanning | nmap, naabu, rustscan |
+| Web crawling | katana, gospider, hakrawler |
+| Content discovery | ffuf, feroxbuster, gobuster, dirsearch |
+| Tech fingerprinting | whatweb, wappalyzer-cli, httpx -tech-detect |
+| Vulnerability scanning | nuclei, nikto, wpscan, droopescan, joomscan |
+| TLS auditing | testssl.sh, sslscan, sslyze |
+| Parameter discovery | arjun, paramspider, x8 |
+| Secrets | trufflehog, gitleaks |
+| Visual recon | gowitness, aquatone |
+| SQLi verification | sqlmap (safe flags only: --batch --crawl=0 --level=1 --risk=1) |
+
+---
+
+## Environment profiling
+
+On first run, `aegis init` profiles the host and derives auto-tuned concurrency settings:
+
+```
+Host: arch-workstation
+  OS        Arch Linux (rolling, kernel 6.9.3-arch1-1)
+  CPU       AMD Ryzen 7 5800X . 8 cores / 16 threads . 4.7 GHz
+  Memory    32 GB total . 24 GB available
+  Repos     core, extra, multilib, blackarch
+
+Pentest toolchain: 28/35 detected
+  nmap 7.95       nuclei 3.2.9      httpx 1.6.6
+  ffuf 2.1.0      subfinder 2.6.6   katana 1.1.0
+  sqlmap 1.8.5    wpscan 3.8.27     nikto 2.5.0
+  gobuster 3.6.0  amass 4.2.0       gowitness 3.0.3
+
+Missing: testssl.sh  feroxbuster  dnsrecon  arjun  paramspider  trufflehog
+  -> Run: aegis env install --missing
+
+Auto-tuned concurrency:
+  nmap_parallelism=16   nuclei_concurrency=32   ffuf_threads=64
+  httpx_concurrency=80  max_parallel_tools=4
+```
+
+---
+
+## CLI reference
+
+```
+aegis init                                    First-run setup and env profile
+aegis env show                                Display host profile
+aegis env tools                               Tool inventory
+aegis env install --missing                   Generate install commands for missing tools
+aegis env refresh                             Re-detect host profile
+
+aegis kb sync [--source nvd|ghsa|nuclei]      Sync knowledge base
+aegis kb stats                                Knowledge base summary
+aegis kb query --product nginx --min-cvss 7   Query CVEs
+
+aegis engagement new --client X --domain Y   Scaffold engagement dir and scope.yaml
+aegis engagement list                         List engagements
+
+aegis run <dir> [--phase PHASE]              Run engagement
+aegis run <dir> --dry-run                    Preview planned actions
+aegis run <dir> --budget-usd 2.00           Cap spend
+
+aegis report <dir> [--format md|html|json]   Generate report
+aegis findings list <dir> [--severity high]  List findings
+aegis findings verify <finding-id>           Re-run verification probe
+aegis findings suppress <finding-id> --reason "..."
+
+aegis cost <dir>                             Detailed cost breakdown
+aegis audit <dir>                            Full audit log
+```
+
+All commands support `--json` for scripting, `-v/-vv/-vvv` for verbosity, `--quiet` for CI.
+
+---
+
+## Configuration
+
+Global config lives at `~/.config/aegis/config.toml`. Any key can be overridden per engagement in `engagement_dir/config.toml`.
+
+```toml
+[api]
+anthropic_api_key_env = "ANTHROPIC_API_KEY"
+
+[models]
+nano  = "claude-haiku-4-5-20251001"
+main  = "claude-sonnet-4-6"
+deep  = "claude-opus-4-7"
+
+[models.local]
+enabled  = false
+endpoint = "http://localhost:11434"
+model    = "qwen2.5:7b"
+
+[budgets]
+tokens_per_phase       = 30000
+tokens_per_engagement  = 200000
+usd_per_engagement     = 5.00
+wall_time_per_phase_sec = 1800
+
+[caching]
+prompt_cache          = true
+kb_cache_dir          = "~/.cache/aegis/kb"
+fingerprint_cache_ttl_hours = 168
+
+[tooling]
+docker_isolate         = false
+default_rate_limit_rps = 10
+respect_robots_txt     = false
+
+[reporting]
+default_format   = "html"
+include_audit_log = true
+```
+
+---
+
+## Tech stack
+
+| Layer | Choice |
+|---|---|
+| Language | Python 3.12+ |
+| CLI | Typer + Rich |
+| Async | asyncio + anyio |
+| HTTP | httpx (async, HTTP/2) |
+| Models | Pydantic v2 |
+| Storage | SQLite + SQLModel |
+| LLM | Anthropic SDK (Claude) |
+| Templating | Jinja2 |
+| Logging | structlog + rich |
+| Packaging | uv (dev), hatch (build) |
+| Testing | pytest + pytest-asyncio + respx |
+
+---
+
+## License
+
+MIT. Use responsibly and only against systems you are authorized to test.
+
+---
+
+Built by [Majd Bnat](mailto:hey@majdb.com)