pwn 0.5.628 → 0.5.629

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 44fdb9e4897d83ad9ab54d65bc774eed6be14c379eea42b1a7984abab23c03bd
-  data.tar.gz: ba801dabf738c43b3fe7ce9cf40bdfb38d1cf62f7a0b98b05861626c33d2d008
+  metadata.gz: 2f0c536ea0f5b7dbd2c88621241adfa866f7ed726ee4e2109abe83aef13b0385
+  data.tar.gz: 7e3bc1436b9ce27c63da25dc609eab317a009b1339a843a6cdfe03469386b195
 SHA512:
-  metadata.gz: 205805e3088246b711249ba3b778e9418fb1616ad576dde6c22c0c632409de88b2545fda88ee30316fe7a203a2ff9a46236136e94ba291a7dc27c800d7b8c02c
-  data.tar.gz: 4e439c01661f7707c9bf79e8e4be656e76f9d57d5bb17cf9778536bd1a8ebbc63aebd4effbcce38d4ed89f4cfdbe188072ab710a7374d0a951fd8a0c7bd7d0fd
+  metadata.gz: 572496cd5d2754022fdb35c360db8e5fec65cd91693ea196f8e44666c28141f198b3353146cd658aa07b93d0add52e3808448fe0836b38830c2fb1b553d8dad2
+  data.tar.gz: 7dc0f6160402b01d8acb694e72d86daef36743fd9b9f0b84a2f87da0d77367798e8ef429463e25f36747f359c5b621615e61e26df29033de9561a24d926af4a6

data/README.md

@@ -33,7 +33,7 @@ hardware / radio exploitation, to reporting and disclosure — and puts a
 **self-improving, tool-calling, multi-agent AI** on top of it.
 
 **In numbers:** 66 `PWN::Plugins` · 48 `PWN::SAST` rules · 90 `PWN::AWS`
-service wrappers · 21 `PWN::WWW` site drivers · 52 `bin/pwn_*` CLI drivers ·
+service wrappers · 21 `PWN::WWW` site drivers · 53 `bin/pwn_*` CLI drivers ·
 5 LLM engines · 10 agent toolsets · 45+ LLM-callable tools.
 
 Full page: [What is PWN](documentation/What-is-PWN.md)
@@ -86,7 +86,7 @@ The complete wiki lives in this repo at **[`documentation/Home.md`](documentatio
 |---|---|---|---|
 | [What is PWN](documentation/What-is-PWN.md) | [`pwn` REPL](documentation/pwn-REPL.md) | [AI / LLM Integration](documentation/AI-Integration.md) | [Plugins (66)](documentation/Plugins.md) |
 | [Why PWN](documentation/Why-PWN.md) | [`pwn-ai` Agent](documentation/pwn-ai-Agent.md) | [Agent Tool Registry](documentation/Agent-Tool-Registry.md) | [SAST (48)](documentation/SAST.md) |
-| [How PWN Works](documentation/How-PWN-Works.md) | [CLI Drivers (52)](documentation/CLI-Drivers.md) | [Memory · Skills · Learning](documentation/Skills-Memory-Learning.md) | [AWS (90)](documentation/AWS.md) |
+| [How PWN Works](documentation/How-PWN-Works.md) | [CLI Drivers (53)](documentation/CLI-Drivers.md) | [Memory · Skills · Learning](documentation/Skills-Memory-Learning.md) | [AWS (90)](documentation/AWS.md) |
 | [Installation](documentation/Installation.md) | [Build a Driver](documentation/Drivers.md) | [Mistakes (neg-feedback)](documentation/Mistakes.md) | [WWW (21)](documentation/WWW.md) |
 | [General Usage](documentation/General-PWN-Usage.md) | | [Extrospection](documentation/Extrospection.md) | [SDR / Radio](documentation/SDR.md) |
 | [Configuration](documentation/Configuration.md) | | [Swarm (multi-agent)](documentation/Swarm.md) | [Hardware](documentation/Hardware.md) |
@@ -103,18 +103,31 @@ Rebuild every SVG from its Graphviz source:
 
 ### **Installation** ###
 
-Tested on Debian-based Linux & macOS, Ruby via RVM.
+PWN is a **single gem** with a built-in post-install doctor/provisioner —
+`pwn setup` — that detects your package manager (`apt` · `dnf` · `pacman` ·
+`brew` · `port`) and installs exactly the OS headers + external tools each
+`PWN::` capability needs. Tested on Kali/Debian/Ubuntu, Fedora, Arch, macOS.
 
 ```
-$ cd /opt
-$ sudo git clone https://github.com/0dayinc/pwn
-$ cd /opt/pwn
-$ ./install.sh
-$ ./install.sh ruby-gem
+$ gem install pwn
+$ pwn setup                        # read-only doctor: which capabilities are usable?
+$ pwn setup --profile full --yes   # provision everything (or: web | net | sdr | vision | …)
 $ pwn
-pwn[v0.5.628]:001 >>> PWN.help
+pwn[v0.5.629]:001 >>> PWN.help
 ```
 
+Only need a subset?
+
+```
+$ pwn setup --list-profiles
+$ pwn setup --profile web          # TransparentBrowser · Burp · ZAP · Tor · sqlmap
+$ pwn setup --profile sdr --yes    # GQRX · rtl-sdr · hackrf · SoapySDR · FFI DSP
+$ pwn setup --profile net --dry-run
+```
+
+Also available as `pwn_setup` (standalone driver) and `pwn --setup[=PROFILE]`.
+The doctor exits non-zero when capabilities are degraded, so CI can gate on it.
+
 [![Installing the pwn Security Automation Framework](https://raw.githubusercontent.com/0dayInc/pwn/master/documentation/pwn_install.png)](https://youtu.be/G7iLUY4FzsI)
 
 Full page: [Installation](documentation/Installation.md) ·
@@ -131,20 +144,16 @@ Update PWN frequently — new plugins, agent tools, skills and zero-day tooling
 land regularly:
 
 ```
-$ rvm list gemsets
-$ rvm use ruby-4.0.5@pwn
-$ gem uninstall --all --executables pwn
-$ gem install --verbose pwn
+$ gem update pwn
+$ pwn setup            # re-doctor — new versions may add capabilities
 $ pwn
-pwn[v0.5.628]:001 >>> PWN.help
+pwn[v0.5.629]:001 >>> PWN.help
 ```
 
-If using a multi-user RVM install:
+From a git checkout:
 
 ```
-$ rvm use ruby-4.0.5@pwn
-$ rvmsudo gem uninstall --all --executables pwn
-$ rvmsudo gem install --verbose pwn
+$ cd /opt/pwn && git pull && rake install && pwn setup
 ```
 
 **Inside the `pwn` REPL:**
@@ -160,11 +169,10 @@ $ echo "$LONG_PROMPT" | pwn --ai -
 $ pwn -Y ./ci/pwn.yaml --ai 'Run pwn_sast against ./src and summarise HIGH findings' > findings.txt
 ```
 
-PWN periodically upgrades to the latest Ruby (`/opt/pwn/.ruby-version`).
-Easiest upgrade of Ruby + pwn from a previous install:
+**Provision a CI runner / Docker image:**
 
 ```
-$ /opt/pwn/vagrant/provisioners/pwn.sh
+$ pwn setup --profile web --yes && pwn setup --check   # exits 1 if degraded
 ```
 
 ---

data/bin/pwn

@@ -3,6 +3,19 @@
 
 require 'pwn'
 
+# ----------------------------------------------------------------------
+# Bare-word subcommand: `pwn setup [...]` → delegate to bin/pwn_setup so
+# `gem install pwn && pwn setup` is the canonical two-step install path.
+# Must be handled BEFORE OptionParser (which would reject `setup`).
+# ----------------------------------------------------------------------
+if ARGV.first == 'setup'
+  ARGV.shift
+  setup_bin = File.expand_path('pwn_setup', __dir__)
+  setup_bin = Gem.bin_path('pwn', 'pwn_setup') unless File.file?(setup_bin)
+  load setup_bin
+  exit
+end
+
 opts = PWN::Env[:driver_opts]
 pwn_driver = PWN::Driver::Parser.new do |options|
   options.on(
@@ -14,6 +27,16 @@ pwn_driver = PWN::Driver::Parser.new do |options|
   ) do |a|
     opts[:ai] = a
   end
+
+  options.on(
+    '-S[PROFILE]',
+    '--setup[=PROFILE]',
+    '<Optional - Post-install doctor / capability provisioner. With no ' \
+    "PROFILE runs a read-only check; with PROFILE (#{PWN::Setup::PROFILES.keys.join('|')}) " \
+    'installs OS deps for that capability set. Same as `pwn setup` / `pwn_setup`>'
+  ) do |p|
+    opts[:setup] = p || true
+  end
 end
 pwn_driver.auto_opts_help = false
 pwn_driver.parse!
@@ -21,7 +44,18 @@ pwn_driver.parse!
 begin
   pwn_pid = Process.pid
 
-  if opts[:ai]
+  if opts[:setup]
+    # ------------------------------------------------------------------
+    # `pwn --setup`            → PWN::Setup.check   (read-only doctor)
+    # `pwn --setup=web`        → PWN::Setup.deps(profile: :web)
+    # ------------------------------------------------------------------
+    if opts[:setup] == true
+      r = PWN::Setup.check
+      exit(r[:ok] ? 0 : 1)
+    else
+      PWN::Setup.deps(profile: opts[:setup])
+    end
+  elsif opts[:ai]
     # ------------------------------------------------------------------
     # Headless pwn-ai: behave as if the user typed PROMPT at the pwn-ai
     # TUI, but non-interactively — a single agent-loop turn with native

data/bin/pwn_setup

@@ -0,0 +1,64 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require 'pwn'
+
+opts = PWN::Env[:driver_opts]
+pwn_driver = PWN::Driver::Parser.new do |options|
+  options.banner = "USAGE: #{File.basename($PROGRAM_NAME)} [opts]\n       " \
+                   "Post-install doctor & capability provisioner for the PWN gem.\n       " \
+                   "With no options, runs a read-only capability check.\n"
+
+  options.on('-c', '--check',
+             '<Optional - Read-only doctor: which PWN capabilities are usable on this host? (default action)>') do |o|
+    opts[:check] = o
+  end
+
+  options.on('-d', '--deps',
+             '<Optional - Install OS headers + external tools + rebuild native gems for --profile (default: full)>') do |o|
+    opts[:deps] = o
+  end
+
+  options.on('-pNAME', '--profile=NAME',
+             "<Optional - Capability profile: #{PWN::Setup::PROFILES.keys.join(' | ')} (default: full)>") do |o|
+    opts[:profile] = o
+  end
+
+  options.on('-l', '--list-profiles',
+             '<Optional - List capability profiles and exit>') do |o|
+    opts[:list] = o
+  end
+
+  options.on('-y', '--yes',
+             '<Optional - Assume yes; non-interactive (CI-friendly)>') do |o|
+    opts[:yes] = o
+  end
+
+  options.on('-n', '--dry-run',
+             '<Optional - Print the commands that would run, do not execute>') do |o|
+    opts[:dry_run] = o
+  end
+end
+pwn_driver.auto_opts_help = false
+pwn_driver.parse!
+
+begin
+  if opts[:list]
+    PWN::Setup.list_profiles
+  elsif opts[:deps] || opts[:profile]
+    PWN::Setup.deps(
+      profile: opts[:profile],
+      yes: opts[:yes],
+      dry_run: opts[:dry_run]
+    )
+  else
+    r = PWN::Setup.check
+    exit 1 unless r[:ok]
+  end
+rescue Interrupt
+  puts "\naborted."
+  exit 130
+rescue StandardError => e
+  warn "pwn_setup: #{e.class}: #{e.message}"
+  exit 1
+end

data/documentation/AI-Integration.md

@@ -1,7 +1,7 @@
 # AI / LLM Integration — `PWN::AI`
 
 One agent loop, five interchangeable engines. Swap providers by changing one
-line in `~/.pwn/config.yml`; the tool-calling contract is normalised so the
+line in `~/.pwn/pwn.yaml`; the tool-calling contract is normalised so the
 agent code never cares which model is behind it.
 
 ![Multi-provider integration](diagrams/ai-integration-tool-calling.svg)
@@ -14,23 +14,58 @@ agent code never cares which model is behind it.
 | `anthropic` | `PWN::AI::Anthropic` | `key:` | tool-use native |
 | `grok` | `PWN::AI::Grok` | `key:` **or** `oauth: true` | OAuth = RFC-8628 device-code flow using xAI's public Grok-CLI client id (no secret) — see skill `xai_grok_oauth_device_flow` |
 | `gemini` | `PWN::AI::Gemini` | `key:` | function-calling native |
-| `ollama` | `PWN::AI::Ollama` | none | local, `base_url:` + `model:` |
+| `ollama` | `PWN::AI::Ollama` | none | local — native `/api/chat` (`num_ctx`, `keep_alive`, low-temp + `format:'json'` on tool turns) and `/api/embed` for `PWN::MemoryIndex` |
+
+> PWN is **model-agnostic**. `ai.<engine>.model` is passed straight through to
+> the provider — the codebase and docs deliberately never name a specific
+> model id so you can point each engine at whatever the vendor currently
+> ships (or whatever `ollama list` shows locally).
 
 ## Selecting an engine
 
 ```yaml
-# ~/.pwn/config.yml
+# ~/.pwn/pwn.yaml
 ai:
-  engine: grok
+  active: grok
   grok:
-    oauth: true   # first run opens https://accounts.x.ai/… device page
+    oauth:
+      enroll: true    # first run opens https://accounts.x.ai/… device page
 ```
 
 ```ruby
 # at runtime
-PWN::Env[:ai][:engine] = :ollama
+PWN::Env[:ai][:active] = :ollama
 ```
 
+## Engine-aware behaviour
+
+The harness adapts to the *class* of engine, not the model name:
+
+| Concern | Frontier (`openai` · `anthropic` · `grok` · `gemini`) | Local (`ollama`) |
+|---|---|---|
+| **PromptBuilder.budget** | full MEMORY / METRICS / MISTAKES / LEARNING / EXTRO blocks | tightened via `ai.ollama.prompt_budget` (extro off by default) |
+| **MEMORY ranking** | relevance-ranked when a local Ollama `embed_model` is reachable, else newest-first | relevance-ranked via `PWN::MemoryIndex` (`~/.pwn/memory.idx`) |
+| **Tool schemas shipped** | all toolsets | `CORE_TOOLS` + top-K keyword matches when `ai.agent.tool_router` is on |
+| **Pre-pass** | none | `plan_first` numbered tool plan before first dispatch |
+| **Few-shot** | none | `Learning.exemplars_for(request)` splices a prior successful trace |
+| **Dispatch parsing** | strict | tolerant — Levenshtein tool-name repair + JSON5-ish arg cleanup, each repair fingerprinted into `Mistakes` |
+| **Post-answer** | `auto_introspect` | `auto_introspect` **+** `fact_check_local_final` (auto `extro_verify` on CVE/version-shaped claims) |
+| **Metrics bucket** | `metrics.json[:tools][name][:engines][:<engine>]` | same — the `TOOL EFFECTIVENESS` block is per-engine so local telemetry never blends with frontier |
+
+## Teacher-student reflection
+
+`ai.reflect_engine` decouples *doing* from *learning-about-doing*:
+
+```yaml
+ai:
+  active: ollama            # the local model executes tools and answers
+  reflect_engine: anthropic # a frontier model writes the durable Memory :lesson
+```
+
+`PWN::AI::Agent::Reflect.on` temporarily flips the active engine for the
+introspection call only, so the local model reads back distilled reasoning it
+could not have produced itself.
+
 ## Direct client use (no agent)
 
 ```ruby
@@ -43,10 +78,14 @@ puts resp[:content]
 ## Model diversity in Swarm
 
 Because each persona in [`agents.yml`](Swarm.md) can override `engine:`, an
-`agent_debate` can literally pit Claude against Grok against a local Llama —
-real antagonism, not one model role-playing three voices.
+`agent_debate` can pit five *different providers* against each other — real
+antagonism, not one model role-playing three voices. The same mechanism backs
+`ai.agent.escalation_persona`: when a local model is stuck, `Loop.run` asks a
+frontier persona for a 3-line corrective hint and injects it as a synthetic
+tool result.
 
 **See also:** [pwn-ai Agent](pwn-ai-Agent.md) ·
-[Agent Tool Registry](Agent-Tool-Registry.md) · [Swarm](Swarm.md)
+[Agent Tool Registry](Agent-Tool-Registry.md) · [Swarm](Swarm.md) ·
+[Configuration](Configuration.md)
 
 [← Home](Home.md)

data/documentation/AWS.md

@@ -10,7 +10,7 @@ with the PWN `opts = {}` convention so they compose in the REPL and in
 
 Standard AWS SDK chain: `~/.aws/credentials` profile, env vars
 (`AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` / `AWS_SESSION_TOKEN`), or
-instance profile. Optionally pin in `~/.pwn/config.yml` under `aws:`.
+instance profile. Optionally pin in `~/.pwn/pwn.yaml` under `aws:`.
 
 ## Quick enumeration
 

data/documentation/Agent-Tool-Registry.md

@@ -21,6 +21,30 @@ toolsets; the JSON-Schema for each tool is what the model actually sees.
 | `cron` | `cron_list` · `cron_create` · `cron_run` · `cron_enable` · `cron_disable` · `cron_remove` | `PWN::Cron` → `~/.pwn/cron/jobs.yml` |
 | `swarm` | `agent_list` · `agent_spawn` · `agent_ask` · `agent_debate` · `agent_broadcast` · `swarm_bus` · `swarm_list` | `PWN::AI::Agent::Swarm` → `~/.pwn/agents.yml` + `~/.pwn/swarm/` |
 
+
+## Dynamic tool-set slimming (`ai.agent.tool_router`)
+
+Shipping every schema on every turn overwhelms a small local model — the
+choice space is huge and it mis-routes (e.g. picks an RF tool for a git
+question). When `ai.agent.tool_router: true` **and** `Loop.run` passes the
+user request through as `relevance:`, `Registry.definitions` shrinks the
+pool to:
+
+```text
+CORE_TOOLS  = shell · pwn_eval · memory_remember · memory_recall
+              mistakes_record · mistakes_resolve · learning_note_outcome
+            + top-K keyword-ranked matches for THIS request
+              (ties break on Metrics per-engine success_rate → the router
+               itself is a learned component)
+```
+
+```ruby
+PWN::AI::Agent::Registry.definitions(relevance: 'nmap sweep 10.0.0.0/8', top_k: 10)
+PWN::AI::Agent::Registry.rank(query: 'run a shell command')   # inspect ranking
+```
+
+Frontier engines leave `tool_router` off and receive the full set.
+
 ## Adding a tool
 
 ```ruby

data/documentation/BurpSuite.md

@@ -6,10 +6,13 @@ Professional's headless mode + REST API.
 
 ![Web application testing](diagrams/web-application-testing.svg)
 
+> **Install:** `pwn setup --profile web` — burpsuite · zaproxy · chromium ·
+> geckodriver · sqlmap · tor. See [Installation](Installation.md).
+
 ## Configure
 
 ```yaml
-# ~/.pwn/config.yml
+# ~/.pwn/pwn.yaml  (edit via `pwn-vault`)
 burp:
   jar: /opt/burpsuite_pro/burpsuite_pro.jar
   api_key: <REST-API-KEY>

data/documentation/CLI-Drivers.md

@@ -1,6 +1,6 @@
 # CLI Drivers — `bin/pwn_*`
 
-52 headless executables, each a thin `OptionParser` wrapper over one plugin
+53 headless executables, each a thin `OptionParser` wrapper over one plugin
 (or one workflow). They exist so CI/CD can call PWN without a REPL or an LLM.
 
 ![Driver anatomy](diagrams/driver-framework.svg)
@@ -26,19 +26,34 @@ pwn_crt_sh                       pwn_serial_check_voicemail
 pwn_defectdojo_engagement_create pwn_serial_msr206
 pwn_defectdojo_importscan        pwn_serial_qualcomm_commands
 pwn_defectdojo_reimportscan      pwn_serial_son_micro_sm132_rfid
-pwn_diff_csv_files_w_column_exclude  pwn_shodan_graphql_introspection
-pwn_domain_reversewhois          pwn_shodan_search
-pwn_fuzz_net_app_proto           pwn_simple_http_server
-pwn_gqrx_scanner                 pwn_web_cache_deception
-pwn_jenkins_create_job           pwn_www_checkip
-pwn_jenkins_create_view          pwn_www_uri_buster
-pwn_jenkins_install_plugin       pwn_xss_dom_vectors
-                                 pwn_zaproxy_active_rest_api_scan
-                                 pwn_zaproxy_active_scan
+pwn_diff_csv_files_w_column_exclude  pwn_setup
+pwn_domain_reversewhois          pwn_shodan_graphql_introspection
+pwn_fuzz_net_app_proto           pwn_shodan_search
+pwn_gqrx_scanner                 pwn_simple_http_server
+pwn_jenkins_create_job           pwn_web_cache_deception
+pwn_jenkins_create_view          pwn_www_checkip
+pwn_jenkins_install_plugin       pwn_www_uri_buster
 ```
 
 Run any with `--help` for its flags.
 
+## `pwn_setup` — post-install doctor & capability provisioner
+
+The one driver that isn't a plugin wrapper. It grows a bare `gem install pwn`
+into a fully-armed host by installing OS headers / external tools for whatever
+capability profile you ask for. Also reachable as `pwn setup` and
+`pwn --setup[=PROFILE]`.
+
+```bash
+pwn_setup                        # read-only doctor; exit 1 if degraded
+pwn_setup --list-profiles
+pwn_setup --profile web --yes    # CI-friendly, non-interactive
+pwn_setup --deps --dry-run       # print the apt/dnf/pacman/brew/port commands only
+```
+
+See [Installation](Installation.md) for the full profile table and
+`PWN::Setup` API.
+
 ## Typical CI usage
 
 ```yaml
@@ -46,6 +61,8 @@ Run any with `--help` for its flags.
 sast:
   image: 0dayinc/pwn:latest
   script:
+    - pwn_setup --profile net --yes
+    - pwn_setup --check                                   # gate: exit 1 if degraded
     - pwn_sast -d "$CI_PROJECT_DIR" -o sast_out/
     - pwn_defectdojo_importscan -f sast_out/report.json -e "$DD_ENGAGEMENT"
 ```

data/documentation/Configuration.md

@@ -16,6 +16,11 @@ writes a fully-commented template with every key below (values set to
 `'optional - …'` / `'required - …'` placeholder strings), encrypts it, and
 generates the decryptor.
 
+> **Before configuring**, run `pwn setup` — the
+> [doctor](Installation.md#pwn-setup--the-post-install-doctor--provisioner)
+> reports whether `~/.pwn/`, `pwn.yaml`, its decryptor, and an AI-engine key
+> are present, and exits non-zero for CI if any are missing.
+
 ---
 
 ## Full annotated example
@@ -30,7 +35,7 @@ ai:
   grok:
     base_uri: https://api.x.ai/v1  # xAI API base URL. Override for a self-hosted proxy / private endpoint.
     key: xai-…                     # xAI API key. If blank AND no oauth.* below, PWN prompts interactively at load.
-    model: grok-4.5                # Model id sent on every chat / tool-loop request.
+    model: <model-id>              # Model id sent on every chat / tool-loop request. See provider docs / API for currently-supported ids.
     system_role_content: 'You are an ethically hacking xAI Grok agent.'   # Base system prompt (MEMORY/SKILLS/LEARNING/EXTROSPECTION blocks are appended to this).
     temp: 1.0                      # Sampling temperature passed to the chat endpoint.
     max_prompt_length: 256000      # Soft input-context ceiling (chars) used for prompt truncation / chunking.
@@ -46,7 +51,7 @@ ai:
   openai:
     base_uri: https://api.openai.com/v1   # OpenAI API base URL. Override for Azure OpenAI / VPC gateway / local proxy.
     key: sk-…                             # OpenAI API key (`sk-…`). Prompted interactively if blank.
-    model: gpt-4o                         # Model id sent on every chat / tool-loop request.
+    model: <model-id>                     # Model id sent on every chat / tool-loop request. See provider docs / API for currently-supported ids.
     system_role_content: 'You are an ethically hacking OpenAI agent.'   # Base system prompt for this engine.
     temp: 1.0                             # Sampling temperature.
     max_prompt_length: 128000             # Soft input-context ceiling (chars) for truncation / chunking.
@@ -55,7 +60,7 @@ ai:
   anthropic:
     base_uri: https://api.anthropic.com/v1   # Anthropic API base URL. Override for Bedrock / private gateway.
     key: sk-ant-…                            # Anthropic API key (`sk-ant-…`). Prompted interactively if blank.
-    model: claude-3-5-sonnet-20240620        # Model id sent on every chat / tool-loop request.
+    model: <model-id>                        # Model id sent on every chat / tool-loop request. See provider docs / API for currently-supported ids.
     system_role_content: 'You are an ethically hacking Anthropic agent.'   # Base system prompt for this engine.
     temp: 1.0                                # Sampling temperature.
     max_tokens: 8192                         # Max OUTPUT tokens per response. Raise if tool-call JSON truncates.
@@ -64,7 +69,7 @@ ai:
   gemini:
     base_uri: https://generativelanguage.googleapis.com/v1beta   # Google Generative Language API base URL.
     key: AIza…                               # Google AI Studio API key (`AIza…`). Prompted interactively if blank.
-    model: gemini-2.5-pro                    # Model id sent on every chat / tool-loop request.
+    model: <model-id>                        # Model id sent on every chat / tool-loop request. See provider docs / API for currently-supported ids.
     system_role_content: 'You are an ethically hacking Gemini agent.'   # Base system prompt for this engine.
     temp: 1.0                                # Sampling temperature.
     max_prompt_length: 1000000               # Soft input-context ceiling (chars) — Gemini supports very large contexts.
@@ -72,17 +77,31 @@ ai:
   ollama:
     base_uri: https://ollama.local           # REQUIRED for ollama — Open WebUI / ollama-serve base URL (no vendor default).
     key: eyJ…                                # Open WebUI JWT (Settings → Account → API Key). Prompted if blank.
-    model: llama3.1:70b                      # Local model tag as `ollama list` shows it.
+    model: <local-model-tag>                 # Local model tag exactly as `ollama list` shows it.
+    embed_model: <embed-model-tag>           # Embedding model for PWN::MemoryIndex (relevance-ranked MEMORY). Must be pulled locally.
     system_role_content: 'You are an ethically hacking Ollama agent.'   # Base system prompt for this engine.
-    temp: 1.0                                # Sampling temperature.
+    temp: 1.0                                # Sampling temperature (used on the FINAL text-only turn — tool-bearing turns are pinned low for deterministic routing).
+    num_ctx: 32768                           # Context window passed to /api/chat options.num_ctx. Ollama's default (2048) is too small for the pwn-ai system prompt.
+    keep_alive: 30m                          # How long ollama keeps the model resident between iterations (avoids reload latency mid-turn).
+    prompt_budget:                           # Per-block caps applied by PromptBuilder.budget so a small model spends attention on the request, not the harness.
+      memory: 6                              # Max MEMORY entries injected (relevance-ranked via PWN::MemoryIndex when available).
+      metrics: 3                             # Max TOOL EFFECTIVENESS rows.
+      mistakes: 3                            # Max KNOWN MISTAKES / FIXES rows.
+      learning: 2                            # Max recent LEARNING outcomes shown.
+      extro: false                           # Gate the (heaviest) EXTROSPECTION block entirely for local models.
     max_prompt_length: 32000                 # Soft input-context ceiling (chars) — tune per local model's real context window.
 
+  reflect_engine: ~                # Teacher-student reflection: EXECUTE on ai.active, but write durable lessons via THIS engine (nil = same as active). Lets a local model act while a frontier model authors the Memory :lesson entries it reads back.
+
   agent:
     native_tools: true             # Use provider-native tool_calls / function-calling. false → legacy text-parsed tool protocol.
     max_iters: 25                  # Hard cap on tool-call rounds per user turn before a forced final answer.
     max_depth: 3                   # Recursion guard: how many levels deep agent_ask/agent_debate sub-agents may spawn sub-agents.
     auto_introspect: true          # Run Learning.auto_introspect (outcome logging + lesson mining) after every final answer.
     auto_extrospect: false         # Optional ambient baseline (host/repo/env ONLY — never launches burpsuite/zaproxy/msf/gqrx). Sense tools (intel/verify/watch/rf_tune/observe) stay on-demand.
+    plan_first: ~                  # Plan-then-act pre-pass: force the model to externalise a numbered tool plan BEFORE its first dispatch. nil = auto (true when ai.active == ollama).
+    tool_router: false             # Dynamic tool-set slimming: ship only Registry::CORE_TOOLS + top-K keyword-relevant schemas per turn (helps small models route correctly).
+    escalation_persona: ~          # Swarm persona name to ask for a 3-line corrective hint once a local model burns ≥ Loop::ESCALATE_AFTER_FAILS in-turn failures. nil = disabled.
     toolsets: ~                    # Allow-list of toolsets exposed to the agent. nil = all. Valid: cron, extrospection, learning, memory, metrics, pwn, sessions, skills, swarm, terminal.
     extrospection:
       web:
@@ -193,12 +212,17 @@ PWN::Config.refresh_env
 | `ai.module_reflection` | Boolean | `false` | `PWN::AI::Agent::Reflect`, `PWN::SAST::*`, `PWN::Plugins::BurpSuite` | Master gate for LLM-driven self-analysis (SAST triage, Burp finding enrichment, `Learning.llm_reflect`). |
 | `ai.<engine>.base_uri` | String | provider default | `PWN::AI::<Engine>.rest_call` | Override the API base URL (self-hosted proxy, private endpoint, Azure/VPC gateway). **Required** for `ollama`. |
 | `ai.<engine>.key` | String | — | `PWN::AI::<Engine>` | API key / bearer token. If blank AND no OAuth is configured, PWN prompts interactively at load. |
-| `ai.<engine>.model` | String | provider default | `PWN::AI::<Engine>.chat` / `.chat_tool_loop` | Model id sent on every request (e.g. `grok-4.5`, `gpt-4o`, `claude-3-5-sonnet-20240620`, `gemini-2.5-pro`, `llama3.1:70b`). |
+| `ai.<engine>.model` | String | provider default | `PWN::AI::<Engine>.chat` / `.chat_tool_loop` | Model id sent on every request. Use whatever id the provider / `ollama list` currently exposes — PWN never hard-codes a specific model. |
 | `ai.<engine>.system_role_content` | String | ethical-hacker persona | `PWN::AI::Agent::PromptBuilder`, `PWN::Plugins::REPL` | Base system prompt prepended to MEMORY / SKILLS / LEARNING / EXTROSPECTION blocks. |
 | `ai.<engine>.temp` | Float | `1.0` | `PWN::AI::<Engine>.chat` | Sampling temperature. |
 | `ai.<engine>.max_prompt_length` | Integer | per-engine | `PWN::AI::<Engine>`, `PWN::Plugins::REPL` | Soft input-context ceiling used for prompt truncation / chunking. |
 | `ai.anthropic.max_tokens` | Integer | `8192` | `PWN::AI::Anthropic.chat_tool_loop` | Max **output** tokens per response. Raise if tool-call JSON truncates. |
 | `ai.openai.max_tokens` | Integer | `16384` | `PWN::AI::OpenAI.chat` | Max **output** tokens per response. Mapped to OpenAI's wire param `max_completion_tokens` (legacy env key `max_completion_tokens` still accepted). |
+| `ai.ollama.embed_model` | String | provider default | `PWN::MemoryIndex` | Local embedding model tag used to build `~/.pwn/memory.idx` for **relevance-ranked** MEMORY injection. Falls back to substring recall when unset / unreachable. |
+| `ai.ollama.num_ctx` | Integer | `32768` | `PWN::AI::Ollama.chat_with_tools` | Context window sent as `options.num_ctx` on the native `/api/chat` call. Ollama's own default (2048) truncates the pwn-ai system prompt. |
+| `ai.ollama.keep_alive` | String | `30m` | `PWN::AI::Ollama.chat_with_tools` | How long the model stays resident in ollama between iterations of a single turn. |
+| `ai.ollama.prompt_budget` | Hash | `{memory:6, metrics:3, mistakes:3, learning:2, extro:false}` | `PWN::AI::Agent::PromptBuilder.budget` | Per-block caps on injected context so a small local model spends its attention on the request, not the harness. Any engine may set this. |
+| `ai.reflect_engine` | Symbol \| `nil` | `nil` (= `ai.active`) | `PWN::AI::Agent::Reflect.on`, `Learning.reflect` | **Teacher-student** override: run the task on `ai.active`, but generate durable lessons via *this* engine. Lets a local model execute while a frontier model writes the Memory it reads back. |
 | `ai.grok.oauth.refresh_token` | String | — | `PWN::AI::Grok.resolve_auth` | Durable OAuth refresh token (from `PWN::AI::Grok.obtain_oauth_bearer_token` device flow). Enables silent re-auth without an API key. |
 | `ai.grok.oauth.bearer_token` | String | — | `PWN::AI::Grok.resolve_auth` | Short-lived OAuth access JWT. Auto-refreshed each run when `refresh_token` is present; live-cached back into this hash. |
 | `ai.grok.oauth.client_id` | String | Grok-CLI public id | `PWN::AI::Grok` | Override the public OAuth client id used for device-flow / refresh. |
@@ -217,6 +241,9 @@ PWN::Config.refresh_env
 | `ai.agent.auto_introspect` | Boolean | `true` | `PWN::AI::Agent::Learning.auto_introspect` | Run outcome logging + lesson mining after every final answer. Toggle live via `learning_auto_introspect_toggle`. |
 | `ai.agent.auto_extrospect` | Boolean | `false` | `PWN::AI::Agent::Extrospection.auto_extrospect` | Optional ambient baseline after every final answer (`AUTO_SECTIONS` = host/repo/env only; never spawns GUI/JVM tools). Prefer on-demand sense tools (`intel`/`verify`/`watch`/`rf_tune`/`observe`). Toggle live via `extro_auto_toggle`. |
 | `ai.agent.toolsets` | Array\<String\> \| `nil` | `nil` (all) | `bin/pwn`, `PWN::Plugins::REPL`, `PWN::AI::Agent::Registry` | Allow-list of toolsets exposed to the agent. Valid: `cron`, `extrospection`, `learning`, `memory`, `metrics`, `pwn`, `sessions`, `skills`, `swarm`, `terminal`. |
+| `ai.agent.plan_first` | Boolean \| `nil` | `nil` (auto: `true` when `ai.active == ollama`) | `PWN::AI::Agent::Loop.plan_first` | Plan-then-act pre-pass: the model must emit a numbered tool plan (as an assistant message) *before* it may dispatch anything. Cheap chain-of-thought scaffolding for local models. |
+| `ai.agent.tool_router` | Boolean | `false` | `PWN::AI::Agent::Registry.definitions` | Dynamic tool-set slimming: expose only `Registry::CORE_TOOLS` + the top-K keyword-relevant schemas for *this* request. Ties break on historical `Metrics` success rate so the router itself is a learned component. |
+| `ai.agent.escalation_persona` | String \| `nil` | `nil` | `PWN::AI::Agent::Loop.escalate` → `Swarm.ask` | Circuit-breaker: once a local model accumulates ≥ `Loop::ESCALATE_AFTER_FAILS` in-turn failures, ask this Swarm persona for a 3-line corrective hint (injected as a synthetic tool result). The local model still authors the final answer so Learning/Metrics stay attributed. |
 | `ai.agent.extrospection.web.anchors` | Array\<String\> | `DEFAULT_WEB_ANCHORS` | `PWN::AI::Agent::Extrospection.probe_web` | URLs the headless browser fingerprints on `extro_snapshot(sections:[:web])`. Alias: `web_anchors`. |
 | `ai.agent.extrospection.web.proxy` | String | — | `Extrospection.probe_web` / `.verify` / `.watch` | Upstream proxy for `PWN::Plugins::TransparentBrowser` (e.g. `tor`, `http://127.0.0.1:8080`). |
 | `ai.agent.extrospection.web.max_anchors` | Integer | `8` | `Extrospection.probe_web` | Cap on anchors rendered per snapshot. |
@@ -309,9 +336,9 @@ agent system prompt.
 
 `pwn.yaml` is the only file you edit; everything else is machine-written
 state. See **[Persistence](Persistence.md)** for the full map
-(`memory.json`, `learning.jsonl`, `mistakes.json`, `metrics.json`,
-`extrospection.json`, `sessions/`, `skills/`, `cron/`, `agents.yml`,
-`swarm/`).
+(`memory.json`, `memory.idx`, `learning.jsonl`, `mistakes.json`,
+`metrics.json`, `extrospection.json`, `sessions/`, `skills/`,
+`finetune/`, `cron/`, `agents.yml`, `swarm/`).
 
 Multi-agent personas are **not** configured here — they live in
 `~/.pwn/agents.yml` and are managed with `agent_spawn` / `agent_list`

data/documentation/Contributing.md

@@ -4,10 +4,11 @@
 
 ```text
 lib/pwn/                # all namespaces
+lib/pwn/setup.rb        # PWN::Setup — doctor/provisioner data tables
 lib/pwn/plugins/        # 66 plugin modules
 lib/pwn/ai/agent/       # agent core
 lib/pwn/ai/agent/tools/ # LLM tool registrations
-bin/                    # 52 pwn_* drivers
+bin/                    # 53 pwn_* drivers (incl. pwn_setup)
 spec/                   # RSpec (incl. conventions_spec)
 documentation/          # this wiki + diagrams
 ```
@@ -23,9 +24,11 @@ documentation/          # this wiki + diagrams
 ## Quality gates
 
 ```bash
-rvmsudo rake            # rubocop + rspec — must be zero offenses
+rake            # rubocop + rspec — must be zero offenses
 ```
 
+(`rvmsudo rake` on multi-user RVM installs.)
+
 ## Adding a plugin
 
 1. `lib/pwn/plugins/my_thing.rb` following the conventions above.
@@ -34,7 +37,14 @@ rvmsudo rake            # rubocop + rspec — must be zero offenses
 4. Optional `bin/pwn_my_thing` driver (see [Drivers](Drivers.md)).
 5. Optional agent tool in `lib/pwn/ai/agent/tools/` (see
    [Agent Tool Registry](Agent-Tool-Registry.md)).
-6. Update [Plugins.md](Plugins.md) and, if it changes a data flow, the
+6. **If it needs a native gem or external binary**, add **one row** to
+   `PWN::Setup::NATIVE_GEMS` or `PWN::Setup::TOOLCHAIN` in
+   `lib/pwn/setup.rb` (with `apt:`/`dnf:`/`pacman:`/`brew:`/`port:` package
+   names + the `plugins:` it unlocks) and, if it belongs in a capability
+   set, reference it from `PWN::Setup::PROFILES`. That single edit makes
+   `pwn setup` install it on every OS and every install path (gem, git,
+   Docker, Packer, Vagrant, CI). **Do not** add a new bash provisioner.
+7. Update [Plugins.md](Plugins.md) and, if it changes a data flow, the
    relevant `.dot` in `documentation/diagrams/dot/` → `./build.sh`.
 
 ## Commit / release