pwn 0.5.668 → 0.5.669

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 21fe6bb04741fbc78392d434588c1c86d0f605db0988bfa2192f68e9baa870ec
-  data.tar.gz: cf66359fb95bb12f08705d11dd304625878115902133d9e1ad0f8becfa8fb83a
+  metadata.gz: 746a770eb9c9d4d410cd3fff4deb631ac67476a8d4f01ba7cf17c8414d4930d9
+  data.tar.gz: 0aa9628ff2b8f80ed5b06fe682d8c252d4609c3545c96b68dd3890897de3d402
 SHA512:
-  metadata.gz: 35f079f6d7c765c6395f490f07c06ce3494b81487bba31c978367fc2b8a23a421a6b1d9a68cda02592fd94868ca343a13c441323b6e1dee0656f46ade8c88aef
-  data.tar.gz: b9a45751eaf30849ac1c98bd8d603df89d8522fb950dfc830f188769b0fc8e7b69848ed1f71053f7053da24a85e0c2f56f0f9a2c26b95de2040835a7e9871c47
+  metadata.gz: adee5114bbd4a83820b12f591d334aeec378f5bf47d7b51d10e3ee9d14497e08207ee532eb2eab7881cb8bb2a1f8910ef6a83712b8b3cb1f7ed7d25cc14f0312
+  data.tar.gz: 5343e693c473baf4112b47b1c8a932a22b6ec46ddf1998c34e828aa79483d7c350bce500ed502f015944960b8e8ec867028f4261f11ddd57360853286778a829

data/CONTRIBUTING.md

@@ -11,3 +11,18 @@ PWN Contribution Rules:
 -   'Submit Pull Requests'
 -   'Pass Upstream Tests'
 - end
+
+## Documentation after code changes
+
+Whenever code under this repo changes:
+
+1. Clear RuboCop on the paths you touched (`bundle exec rubocop -a <paths>`).
+2. Run the full suite (`bundle exec rake`) and fix failures.
+3. Only then refresh user docs: `README.md`, `documentation/*`, and
+   `documentation/diagrams/dot/*.dot` (rebuild with
+   `documentation/diagrams/build.sh`).
+4. Keep user-facing markdown in plain US English (ASCII hyphens, no marketing
+   filler, no internal agent backlog codes).
+
+Do not commit documentation while lint or tests are still red.
+See also `documentation/Contributing.md`.

data/README.md

@@ -35,7 +35,7 @@ and automate instead of gluing together a pile of separate CLIs.
 
 **In numbers:** 66 `PWN::Plugins` · 48 `PWN::SAST` rules · 90 `PWN::AWS`
 service wrappers · 21 `PWN::WWW` site drivers · 53 `bin/pwn_*` CLI drivers ·
-5 LLM engines · 12 agent toolsets · 78 LLM-callable tools.
+6 LLM engines · 12 agent toolsets · 82 LLM-callable tools.
 
 Full page: [What is PWN](documentation/What-is-PWN.md)
 
@@ -46,7 +46,7 @@ Here's what `grok-4.5` has to say about pwn (more specifically `pwn-ai`):
 pwn.ai:grok/grok-4.5/0:500K[CURRENT_VERSION]:002 >>> Briefly describe your take on the reinforced learning feedback loop implemented within `pwn-ai` in clear, concise English so that a human understands your response.
 
 [ <TIMESTAMP> → pwn-ai → task ]   Goal: Briefly describe your take on the reinforced learning feedback loop implemented within `pwn-ai` in clear, concise English so that a human understands your response.
-  Tangible tasks (6) — each task may leverage one or more tools to complete its objective(s):
+  Tangible tasks (6) - each task may use one or more tools to complete its objective(s):
     task 1/6: Review available knowledge of pwn-ai architecture and any reinforced learning elements
     task 2/6: Identify the key stages of the feedback loop used for reinforcement
     task 3/6: Summarize how actions, rewards, and model updates connect in that loop
@@ -58,7 +58,7 @@ pwn.ai:grok/grok-4.5/0:500K[CURRENT_VERSION]:002 >>> Briefly describe your take
 
 **My take, in plain English:**
 
-pwn-ai’s reinforced learning loop is less a classic offline RL trainer and more a living, multi-store control system wrapped around an agent turn. My take is that it is one of the more complete in-context self-improvement stacks I’ve seen in an agent harness, and also that its real power and its main failure modes come from the same place: it learns loudly in the prompt long before it learns quietly in weights.
+pwn-ai's reinforced learning loop is less a classic offline RL trainer and more a living, multi-store control system wrapped around an agent turn. My take is that it is one of the more complete in-context self-improvement stacks I've seen in an agent harness, and also that its real power and its main failure modes come from the same place: it learns loudly in the prompt long before it learns quietly in weights.
 
 ## Bottom line
 
@@ -71,7 +71,7 @@ My take: pwn-ai implements a **closed-loop, dual-horizon, self-and-world-aware r
 - and an external reality check (Extrospection),
 - with a slower supervised/DPO hatch only when the diet and gates look sane.
 
-It feels less like “fine-tune the model forever” and more like giving the agent a nervous system: pain, habit, memory, practice, and a rudimentary sense of whether the world changed. That is why it can improve overnight on a host with no trainer. It is also why health has to be measured by judge gap, repeating-mistake trend, trajectory fraction, and resolved scars rather than by tool success_rate alone.
+It feels less like "fine-tune the model forever" and more like giving the agent a nervous system: pain, habit, memory, practice, and a rudimentary sense of whether the world changed. That is why it can improve overnight on a host with no trainer. It is also why health has to be measured by judge gap, repeating-mistake trend, trajectory fraction, and resolved scars rather than by tool success_rate alone.
 ```
 
 Offensive work is hard because the *tools* do not fit together. PWN's fix is

data/documentation/AI-Integration.md

@@ -1,6 +1,6 @@
 # AI / LLM Integration - `PWN::AI`
 
-One agent loop, five interchangeable engines. Swap providers by changing one
+One agent loop, six interchangeable engines. Swap providers by changing one
 line in `~/.pwn/pwn.yaml`; the tool-calling contract is normalized so the
 agent code never cares which model is behind it.
 
@@ -15,6 +15,7 @@ agent code never cares which model is behind it.
 | `grok` | `PWN::AI::Grok` | `key:` **or** `oauth: true` | OAuth = RFC-8628 device-code flow using xAI's public Grok-CLI client id (no secret) - see skill `xai_grok_oauth_device_flow` |
 | `gemini` | `PWN::AI::Gemini` | `key:` | function-calling native |
 | `ollama` | `PWN::AI::Ollama` | none | local - native `/api/chat` (`num_ctx`, `keep_alive`, low-temp + `format:'json'` on tool turns) and `/api/embed` for `PWN::MemoryIndex` |
+| `openwebui` | `PWN::AI::OpenWebUI` | `key:` (JWT / API token) + `base_uri:` | Open WebUI gateway - OpenAI-compatible `/api/v1/chat/completions` plus proxied Ollama `/ollama/api/*` (including embed) |
 
 > PWN is **model-agnostic**. `ai.<engine>.model` is passed straight through to
 > the provider - the codebase and docs deliberately never name a specific
@@ -41,12 +42,13 @@ PWN::Env[:ai][:active] = :ollama
 
 The harness adapts to the *class* of engine, not the model name:
 
-| Concern | Frontier (`openai` · `anthropic` · `grok` · `gemini`) | Local (`ollama`) |
+| Concern | Frontier (`openai` · `anthropic` · `grok` · `gemini`) | Local / gateway (`ollama` · `openwebui`) |
 |---|---|---|
 | **PromptBuilder.budget** | full MEMORY / METRICS / MISTAKES / LEARNING / EXTRO blocks | tightened via `ai.ollama.prompt_budget` (extro off by default) |
 | **MEMORY ranking** | relevance-ranked when a local Ollama `embed_model` is reachable, else newest-first | relevance-ranked via `PWN::MemoryIndex` (`~/.pwn/memory.idx`) |
 | **Tool schemas shipped** | all toolsets | `CORE_TOOLS` + top-K keyword matches when `ai.agent.tool_router` is on |
 | **Pre-pass** | none | `plan_first` numbered tool plan before first dispatch |
+| **Intent route** | always | `request_intent` short-circuits how-to (text only) and unauthorized recon (refuse) on all engines; critical for ollama/openwebui |
 | **Few-shot** | none | `Learning.exemplars_for(request)` splices a prior successful trace |
 | **Dispatch parsing** | strict | tolerant - Levenshtein tool-name repair + JSON5-ish arg cleanup, each repair fingerprinted into `Mistakes` |
 | **Post-answer** | `auto_introspect` | `auto_introspect` **+** `fact_check_local_final` (auto `extro_verify` on CVE/version-shaped claims) |
@@ -78,7 +80,7 @@ puts resp[:content]
 ## Model diversity in Swarm
 
 Because each persona in [`agents.yml`](Swarm.md) can override `engine:`, an
-`agent_debate` can pit five *different providers* against each other - real
+`agent_debate` can pit several *different providers* against each other - real
 antagonism, not one model role-playing three voices. The same mechanism backs
 `ai.agent.escalation_persona`: when a local model is stuck, `Loop.run` asks a
 frontier persona for a 3-line corrective hint and injects it as a synthetic

data/documentation/Agent-Tool-Registry.md

@@ -6,16 +6,16 @@ toolsets; the JSON-Schema for each tool is what the model actually sees.
 
 ![Tool registry](diagrams/agent-tool-registry.svg)
 
-## Toolsets → Tools  (12 toolsets · 78 tools)
+## Toolsets to tools  (12 toolsets · 82 tools)
 
 | Toolset | Tools | Backed by |
 |---|---|---|
 | `terminal` | `shell` | `Open3.capture3` on the host |
 | `pwn` | `pwn_eval` | `TOPLEVEL_BINDING.eval` in the live REPL process |
-| `memory` | `memory_remember` · `memory_recall` · `memory_forget` · `memory_clear` | `PWN::Memory` → `~/.pwn/memory.json` |
+| `memory` | `memory_remember` · `memory_recall` · `memory_forget` · `memory_clear` · **`memory_lean`** | `PWN::Memory` → `~/.pwn/memory.json` |
 | `skills` | `skill_list` · `skill_view` · `skill_create` · `skill_add_reference` · `skill_delete` · `skill_migrate_legacy` | `~/.pwn/skills/<name>/SKILL.md` (**[agentskills.io](https://agentskills.io) spec**; legacy flat `*.md` auto-migrated) |
-| `sessions` | `sessions_list` · `sessions_view` · `sessions_current` · `sessions_delete` · `sessions_stats` | `PWN::Sessions` → `~/.pwn/sessions/` |
-| `learning` | `learning_note_outcome` · `learning_reflect` · `learning_distill_skill` · `learning_stats` · `learning_outcomes` · `learning_consolidate` · `learning_reset` · `learning_auto_introspect_toggle` · **`mistakes_list`** · **`mistakes_record`** · **`mistakes_resolve`** · **`mistakes_reset`** · **`reward_judge`** · **`reward_prm`** · **`reward_sentinel`** · **`reward_preferences`** · **`reward_export_dpo`** · **`reward_warm_sentinel`** · **`reward_scrub_preferences`** · **`reward_preference_balance`** · **`curriculum_practice`** · **`curriculum_train`** · **`curriculum_hindsight`** · **`curriculum_offline_judge`** · **`curriculum_preference_balance`** · **`learning_purge_noise`** | `PWN::AI::Agent::Learning` + `Mistakes` + `Reward` + `Curriculum` → `~/.pwn/learning.jsonl` + `~/.pwn/mistakes.json` + `~/.pwn/preferences.jsonl` + `~/.pwn/curriculum/` + `~/.pwn/finetune/` |
+| `sessions` | `sessions_list` · `sessions_view` · `sessions_current` · `sessions_delete` · `sessions_stats` · **`sessions_lean`** | `PWN::Sessions` → `~/.pwn/sessions/` |
+| `learning` | `learning_note_outcome` · `learning_reflect` · `learning_distill_skill` · `learning_stats` · `learning_outcomes` · `learning_consolidate` · `learning_reset` · `learning_auto_introspect_toggle` · **`learning_gc_stores`** · **`learning_purge_noise`** · **`mistakes_list`** · **`mistakes_record`** · **`mistakes_resolve`** · **`mistakes_reset`** · **`mistakes_lean`** · **`reward_judge`** · **`reward_prm`** · **`reward_sentinel`** · **`reward_preferences`** · **`reward_export_dpo`** · **`reward_warm_sentinel`** · **`reward_scrub_preferences`** · **`reward_preference_balance`** · **`curriculum_practice`** · **`curriculum_train`** · **`curriculum_hindsight`** · **`curriculum_offline_judge`** · **`curriculum_preference_balance`** | `PWN::AI::Agent::Learning` + `Mistakes` + `Reward` + `Curriculum` → `~/.pwn/learning.jsonl` + `~/.pwn/mistakes.json` + `~/.pwn/preferences.jsonl` + `~/.pwn/curriculum/` + `~/.pwn/finetune/` |
 | `reward` | **`reward_generator_mix`** | `PWN::AI::Agent::Reward.generator_mix` → online preference source-mix controller (`preferences.jsonl`) |
 | `curriculum` | **`curriculum_practice_kpi`** | `PWN::AI::Agent::Curriculum.practice_kpi` → `~/.pwn/curriculum_kpi.jsonl` |
 | `metrics` | `metrics_summary` · `metrics_reset` | `PWN::AI::Agent::Metrics` → `~/.pwn/metrics.json` |
@@ -23,18 +23,22 @@ toolsets; the JSON-Schema for each tool is what the model actually sees.
 | `cron` | `cron_list` · `cron_create` · `cron_run` · `cron_enable` · `cron_disable` · `cron_remove` | `PWN::Cron` → `~/.pwn/cron/jobs.yml` |
 | `swarm` | `agent_list` · `agent_spawn` · `agent_ask` · `agent_debate` · `agent_broadcast` · `swarm_bus` · `swarm_list` | `PWN::AI::Agent::Swarm` → `~/.pwn/agents.yml` + `~/.pwn/swarm/` |
 
-The `learning` toolset is deliberately fat: **Mistakes** (negative feedback),
-**Reward** (ORM/PRM/sentinel/DPO ledger) and **Curriculum** (self-play, HER,
-regression-gated LoRA) are all facets of the same self-improvement loop -
-see [Reinforcement Learning](Reinforcement-Learning.md). The thin `reward` and
-`curriculum` toolsets expose controller/KPI surfaces (`reward_generator_mix`,
-`curriculum_practice_kpi`) so personas can grant just those without the full
-learning surface.
+The `learning` toolset is deliberately large: **Mistakes** (negative feedback),
+**Reward** (outcome and process judges, sentinel, preference ledger) and
+**Curriculum** (self-play, hindsight relabel, optional LoRA gate) are facets of
+the same self-improvement loop - see [Reinforcement Learning](Reinforcement-Learning.md).
+The thin `reward` and `curriculum` toolsets expose controller and KPI surfaces
+(`reward_generator_mix`, `curriculum_practice_kpi`) so personas can grant just
+those without the full learning surface.
+
+**Store hygiene tools** (`memory_lean`, `sessions_lean`, `mistakes_lean`,
+`learning_gc_stores`) trim ephemeral or oversized state without dropping
+protected operator preferences, open mistakes, or gold outcomes.
 
 ## Dynamic tool-set slimming (`ai.agent.tool_router`)
 
 Shipping every schema on every turn overwhelms a small local model - the
-choice space is huge and it mis-routes (e.g. picks an RF tool for a git
+choice space is huge and it mis-routes (for example, picks an RF tool for a git
 question). When `ai.agent.tool_router: true` **and** `Loop.run` passes the
 user request through as `relevance:`, `Registry.definitions` shrinks the
 pool to:
@@ -51,7 +55,7 @@ CORE_TOOLS  = shell · pwn_eval · memory_remember · memory_recall
 PWN::AI::Agent::Registry.definitions(relevance: 'nmap sweep 10.0.0.0/8', top_k: 10)
 PWN::AI::Agent::Registry.rank(query: 'run a shell command')   # inspect ranking
 PWN::AI::Agent::Registry.toolsets                              # → the 12 names above
-PWN::AI::Agent::Registry.all.count                             # → 78
+PWN::AI::Agent::Registry.all.count                             # → 82
 ```
 
 Frontier engines leave `tool_router` off and receive the full set.
@@ -74,7 +78,7 @@ PWN::AI::Agent::Registry.register(
 end
 ```
 
-Drop the file in `lib/pwn/ai/agent/tools/`, it's auto-loaded on next launch.
+Drop the file in `lib/pwn/ai/agent/tools/`. It is auto-loaded on next launch.
 
 ## Restricting a persona
 

data/documentation/Configuration.md

@@ -106,6 +106,7 @@ ai:
     max_depth: 3                   # Recursion guard: how many levels deep agent_ask/agent_debate sub-agents may spawn sub-agents.
     auto_introspect: true          # Run Learning.auto_introspect (outcome logging + lesson mining) after every final answer.
     auto_extrospect: false         # Optional ambient baseline (host/repo/env ONLY - never launches burpsuite/zaproxy/msf/gqrx). Sense tools (intel/verify/watch/rf_tune/observe) stay on-demand.
+    recon_authorized: false        # Allow live subnet sweeps / raw-socket discovery tools this session
     plan_first: ~                  # Plan-then-act pre-pass: force the model to externalize a numbered tool plan BEFORE its first dispatch. nil = auto (true when ai.active == ollama).
     tool_router: false             # Dynamic tool-set slimming: ship only Registry::CORE_TOOLS + top-K keyword-relevant schemas per turn (helps small models route correctly).
     escalation_persona: ~          # Swarm persona name to ask for a 3-line corrective hint once a local model burns ≥ Loop::ESCALATE_AFTER_FAILS in-turn failures. nil = disabled.
@@ -271,6 +272,7 @@ PWN::Config.refresh_env
 | `ai.agent.auto_introspect` | Boolean | `true` | `PWN::AI::Agent::Learning.auto_introspect` | Run outcome logging + lesson mining after every final answer. Toggle live via `learning_auto_introspect_toggle`. |
 | `ai.agent.auto_extrospect` | Boolean | `false` | `PWN::AI::Agent::Extrospection.auto_extrospect` | Optional ambient baseline after every final answer (`AUTO_SECTIONS` = host/repo/env only; never spawns GUI/JVM tools). Prefer on-demand sense tools (`intel`/`verify`/`watch`/`rf_tune`/`observe`). Toggle live via `extro_auto_toggle`. |
 | `ai.agent.toolsets` | Array\<String\> \| `nil` | `nil` (all) | `bin/pwn`, `PWN::Plugins::REPL`, `PWN::AI::Agent::Registry` | Allow-list of toolsets exposed to the agent. Valid: `cron`, `curriculum`, `extrospection`, `learning`, `memory`, `metrics`, `pwn`, `reward`, `sessions`, `skills`, `swarm`, `terminal`. |
+| `ai.agent.recon_authorized` | Boolean | `false` | `PWN::AI::Agent::Loop.recon_authorized?` / shell guard | When true (or the user request contains in-scope / engagement language), live host-discovery tools may run. Default refuses unauthorized sweeps. |
 | `ai.agent.plan_first` | Boolean \| `nil` | `nil` (auto: `true` when `ai.active == ollama`) | `PWN::AI::Agent::Loop.plan_first` | Plan-then-act pre-pass: the model must emit a numbered tool plan (as an assistant message) *before* it may dispatch anything. Cheap chain-of-thought scaffolding for local models. |
 | `ai.agent.tool_router` | Boolean | `false` | `PWN::AI::Agent::Registry.definitions` | Dynamic tool-set slimming: expose only `Registry::CORE_TOOLS` + the top-K keyword-relevant schemas for *this* request. Ties break on historical `Metrics` success rate so the router itself is a learned component. |
 | `ai.agent.escalation_persona` | String \| `nil` | `nil` | `PWN::AI::Agent::Loop.escalate` → `Swarm.ask` | Circuit-breaker: once a local model accumulates ≥ `Loop::ESCALATE_AFTER_FAILS` in-turn failures, ask this Swarm persona for a 3-line corrective hint (injected as a synthetic tool result). The local model still authors the final answer so Learning/Metrics stay attributed. |

data/documentation/Contributing.md

@@ -60,3 +60,16 @@ rake            # rubocop + rspec - must be zero offenses
 `third_party/pwn_rdoc.jsonl`, runs the gates, and pushes.
 
 [← Home](Home.md)
+
+## Documentation after code changes
+
+When you change code under `/opt/pwn`:
+
+1. Clear RuboCop on the paths you touched (`bundle exec rubocop -a <paths>`).
+2. Run the test suite (`bundle exec rake`) and fix failures.
+3. Only then refresh user docs: `README.md`, `documentation/*`, and
+   `documentation/diagrams/dot/*.dot` (rebuild with `documentation/diagrams/build.sh`).
+4. Keep user-facing markdown in plain US English (ASCII hyphens, no marketing
+   filler, no internal agent backlog codes).
+
+Do not commit doc refreshes while lint or tests are still red.

data/documentation/Diagrams.md

@@ -68,7 +68,7 @@ groups) so lines never criss-cross.
 [source](diagrams/dot/ai-integration-tool-calling.dot) · doc: [AI Integration](AI-Integration.md)
 ![ai-integration-tool-calling](diagrams/ai-integration-tool-calling.svg)
 
-### Agent Tool Registry (12 toolsets · 78 tools)
+### Agent Tool Registry (12 toolsets · 82 tools)
 [source](diagrams/dot/agent-tool-registry.dot) · doc: [Agent Tool Registry](Agent-Tool-Registry.md)
 ![agent-tool-registry](diagrams/agent-tool-registry.svg)
 

data/documentation/Home.md

@@ -3,7 +3,7 @@
 > **PWN** (/pōn/) - an open-source offensive-security automation framework and
 > continuous-security-integration platform written in Ruby.
 > 66 plugins · 48 SAST rules · 90 AWS wrappers · 21 WWW drivers · 53 CLI
-> drivers · 5 LLM engines · a self-improving multi-agent AI · one REPL.
+> drivers · 6 LLM engines · a self-improving multi-agent AI · one REPL.
 
 **Repo root:** `/opt/pwn` · **This wiki:** `/opt/pwn/documentation/` ·
 **Rebuild diagrams:** `documentation/diagrams/build.sh`
@@ -35,8 +35,8 @@
 
 | | |
 |---|---|
-| [AI / LLM Integration](AI-Integration.md) | OpenAI · Anthropic · Grok (OAuth) · Gemini · Ollama |
-| [Agent Tool Registry](Agent-Tool-Registry.md) | 12 toolsets · **78** LLM-callable tools |
+| [AI / LLM Integration](AI-Integration.md) | OpenAI · Anthropic · Grok (OAuth) · Gemini · Ollama · Open WebUI |
+| [Agent Tool Registry](Agent-Tool-Registry.md) | 12 toolsets · **82** LLM-callable tools |
 | [Memory · Skills · Learning](Skills-Memory-Learning.md) | Introspection - the self-improvement loop |
 | [Mistakes](Mistakes.md) | **Negative feedback** - fingerprint failures · do-NOT-repeat · `[REPEATING]`/`[REGRESSED]` · inline self-correction |
 | [Reinforcement Learning](Reinforcement-Learning.md) | **`Reward` + `Curriculum`** - outcome/process judges · preference ledger · self-play practice · export-ready LoRA gate |
@@ -59,7 +59,7 @@
 | [SAST (48 rules)](SAST.md) | Static analysis + test-case engine |
 | [WWW (21 drivers)](WWW.md) | Browser automations for real sites |
 | [AWS (90 services)](AWS.md) | Cloud-security enumeration |
-| [SDR](SDR.md) | GQRX · FlipperZero · RFIDler · SonMicro · **Decoder::* (20)** · FFI-accelerated DSP |
+| [SDR](SDR.md) | GQRX · FlipperZero · RFIDler · SonMicro · **Decoder::* (20 protocols + Base/DSP)** · FFI-accelerated DSP |
 | [Blockchain](Blockchain.md) | BTC · ETH helpers |
 | [Bounty](Bounty.md) | Lifecycle / auth-replay tooling |
 | [Reports](Reporting.md) | HTML/JSON output + DefectDojo/Jira |

data/documentation/How-PWN-Works.md

@@ -12,7 +12,7 @@ Humans (`pwn` REPL, `pwn-ai` TUI), CI runners (`pwn --ai "..."`, `bin/pwn_*`),
 LLM providers (over HTTPS), and targets (hosts, web apps, clouds, radios,
 hardware).
 
-## L1 - Entry points  (`bin/`)
+## L1 - Entry points (`bin/`)
 
 | Entry | File | Purpose |
 |---|---|---|
@@ -23,13 +23,13 @@ hardware).
 | `bin/pwn_*` | 53 files | Thin OptionParser wrappers over one plugin each |
 | `PWN::Cron` | `lib/pwn/cron.rb` | Scheduled jobs → any of the above (nightly self-play + weekly weight-loop seeded by default) |
 
-## L2 - AI agent core  (`lib/pwn/ai/agent/`)
+## L2 - AI agent core (`lib/pwn/ai/agent/`)
 
 | Module | Role |
 |---|---|
 | `Loop` | plan → **TaskSummarizer** briefs → dispatch tool_calls → observe → repeat until final answer; tightens runway when recent turns exhausted the budget |
-| **`TaskSummarizer`** | Executive UX: English tasks primary  -  `emit_plan!` · `about_to` as `task k/n` + via tools · Loop `plan_context` injection · clearer `plan_idx` advance |
-| `Registry` | JSON-Schema function definitions grouped into 12 **toolsets** · **78 tools** |
+| **`TaskSummarizer`** | Executive UX: English tasks primary - `emit_plan!` · `about_to` as `task k/n` + via tools · Loop `plan_context` injection · clearer `plan_idx` advance |
+| `Registry` | JSON-Schema function definitions grouped into 12 **toolsets** · **82 tools** |
 | `Dispatch` / `Result` | execute a tool, capture stdout/value/error/duration |
 | `PromptBuilder` | inject MEMORY / SKILLS / LEARNING / **KNOWN MISTAKES + FIXES** / METRICS / EXTROSPECTION blocks |
 | `Metrics` · `Learning` · `Reflect` | **introspection** - how well am I doing? |
@@ -43,14 +43,14 @@ See [Agent Tool Registry](Agent-Tool-Registry.md) for every tool the LLM can
 call, and [Reinforcement Learning](Reinforcement-Learning.md) for how
 `Reward` + `Curriculum` close the weight-level loop.
 
-## L3 - Capability namespaces  (`lib/pwn/*`)
+## L3 - Capability namespaces (`lib/pwn/*`)
 
 `Plugins` (66) · `SAST` (48) · `WWW` (21) · `AWS` (90) · `SDR` · `Blockchain` ·
 `Bounty` · `Reports` · `FFI` · `Banner` · **`Setup`** · **`Migrate`**. Each is
 a plain module of `public_class_method def self.x(opts = {})` methods -
 callable the same way from the REPL, from `pwn_eval`, or from a driver.
 
-## L4 - Persistence  (`~/.pwn/`)
+## L4 - Persistence (`~/.pwn/`)
 
 Everything the framework remembers between processes lives in one directory.
 `PWN::Migrate` (schema-stamped, idempotent, dry-run capable) verifies and

data/documentation/Persistence.md

@@ -28,6 +28,7 @@ Every byte PWN remembers between processes lives here.
 | `swarm/<id>/bus.jsonl` | `Swarm` | JSON-per-line | rm -rf | append-only multi-agent chat |
 | `swarm/<id>/personas.json` | `Swarm` | JSON | rm | persona → session_id map |
 | `curriculum/` | `PWN::AI::Agent::Curriculum` | JSONL | rm -rf | self-play reproducers · train/gate A/B eval sets |
+| `curriculum_kpi.jsonl` | `PWN::AI::Agent::Curriculum.practice_kpi` | JSONL | rm | outer curriculum KPI snapshots (repeating-mistake trend) |
 | `finetune/*.jsonl` | `Learning.export_finetune` · `Reward.export_dpo` | ShareGPT / OpenAI / DPO JSONL | `rm` | supervised + preference datasets - feed to a LoRA over the local model |
 | `backup/<ts>/` | `PWN::Migrate` | tree copy | rm -rf | timestamped snapshot taken before every `--migrate --fix` |
 | `quarantine/` | `PWN::Migrate` | quarantined originals | rm -rf | corrupt/incompatible state files moved aside so the owner re-seeds |
@@ -78,3 +79,19 @@ metrics_reset(confirm: true)   # tool telemetry (optional)
 ```
 
 [← Home](Home.md) · [Configuration](Configuration.md) · [Installation](Installation.md)
+
+## Store hygiene (lean / GC)
+
+Keep `~/.pwn` small without dropping gold learning outcomes, open mistakes, or
+protected operator preferences:
+
+| Tool | What it trims |
+|---|---|
+| `memory_lean` | expired `session_*` keys and overlong values (`VALUE_MAX_CHARS`) |
+| `sessions_lean` | stub/aged transcripts not pinned by gold outcomes or open mistakes |
+| `mistakes_lean` | compact fields; age out resolved-once signatures whose fix already lives in Memory |
+| `learning_gc_stores` | coordinated lean pass across memory, learning, mistakes, and sessions |
+
+All four support `dry_run: true` (plan only). Protected `operator_pref_*`,
+`process_sop_*`, and `mistake_fix_*` memory keys are never removed by lean.
+

data/documentation/Reinforcement-Learning.md

@@ -12,30 +12,30 @@ every turn.
 ![Reinforcement-learning loop](diagrams/reinforcement-learning.svg)
 
 ```
-                       ┌────────────────────────────────────────────────┐
-        request ──────►│ Loop.run                                       │
-                       │  plan_first ─► Curriculum.red_team_plan  (S4)  │
-                       │  Dispatch ──► Reward.semantic_ok         (R4)  │
-                       │            └► Mistakes.record(cause:)    (E1)  │
-                       │  guard ────► Curriculum.counterfactual   (S2)  │──► Reward.record_preference (W1)
-                       │  final ────► Curriculum.critic           (S3)  │
-                       │            └► Reward.judge (ORM)         (R1)  │──► Reward.verify_as_reward   (E3)
-                       │            └► Reward.prm   (PRM)         (R2)  │──► Sessions[step_reward]     (C4)
-                       │            └► Curriculum.hindsight       (C3)  │
-                       │            └► Curriculum.calibrate       (W3)  │──► Metrics.calibration
-                       │            └► Reward.sentinel            (R3)  │──► Mistakes(reward_signal)
-                       └────────────────────────────────────────────────┘
-                                           │
-                    Learning.consolidate (M1 semantic-merge, M3 importance-evict)
-                    MemoryIndex.recall_semantic (M2 sim × recency × importance)
-                    Registry.rank (C1 keyword sim + advantage + UCB)
+                       +------------------------------------------------+
+        request -----> | Loop.run                                       |
+                       |  plan_first -> Curriculum.red_team_plan  (S4)  |
+                       |  Dispatch   -> Reward.semantic_ok        (R4)  |
+                       |             -> Mistakes.record(cause:)         |
+                       |  guard      -> Curriculum.counterfactual (S2)  |--> preference ledger (W1)
+                       |  final      -> Curriculum.critic         (S3)  |
+                       |             -> Reward.judge (outcome)    (R1)  |--> verify_as_reward (E3)
+                       |             -> Reward.prm (process)      (R2)  |--> Sessions[step_reward] (C4)
+                       |             -> Curriculum.hindsight      (C3)  |
+                       |             -> Curriculum.calibrate      (W3)  |--> Metrics.calibration
+                       |             -> Reward.sentinel           (R3)  |--> Mistakes(reward_signal)
+                       +------------------------------------------------+
+                                           |
+                    Learning.consolidate (M1 semantic merge, M3 importance eviction)
+                    MemoryIndex.recall_semantic (M2 similarity x recency x importance)
+                    Registry.rank (C1 keyword fit + advantage + UCB)
                     Learning.exemplars_for (C2 prioritized replay, C4 minimal trace)
-                                           │
-              nightly cron ──► Curriculum.practice (S1) ──► Mistakes.resolve ──► preference (W1)
-              weekly  cron ──► Curriculum.train_and_gate (W2) ──► LoRA vN+1 ──► A/B gate ──► promote
-                                           │
-                    Extrospection.correlate rule 9 (E2 causal lead-lag)
-                    Metrics.changepoints (E1 CUSUM) ──► Mistakes(cause: :env_drift)
+                                           |
+              nightly cron --> Curriculum.practice (S1) --> Mistakes.resolve --> preference ledger (W1)
+              weekly  cron --> Curriculum.train_and_gate (W2) --> optional LoRA --> A/B gate --> promote
+                                           |
+                    Extrospection.correlate (E2 world vs self join)
+                    Metrics.changepoints (E1) --> Mistakes(cause: :env_drift)
 ```
 
 ## Reward signal (`PWN::AI::Agent::Reward`)
@@ -48,7 +48,7 @@ every turn.
 | **R4** | `.semantic_ok` | Treats informational non-zero exits (e.g. `grep`/`rg` with no match) as benign. Metrics count them as OK; Mistakes only see true dispatch failures. |
 | - | `.warm_sentinel` | Backfills the sentinel window from scored Learning outcomes so local hosts can engage proxy distrust without waiting for live remote introspect. |
 | **W1** | `.record_preference` / `.export_dpo` | Preference ledger (`~/.pwn/preferences.jsonl`) from user corrections, resolve, counterfactual, critic, and practice. Caps per source; keeps trajectory-shaped pairs (winning traces / revised answers), not fix commentary. |
-| - | `.scrub_preferences` / `.preference_balance` / `.generator_mix` | Ledger hygiene and source-mix health so one channel cannot flood DPO export. |
+| - | `.scrub_preferences` / `.preference_balance` / `.generator_mix` | Ledger hygiene and source-mix health so one channel cannot flood preference export. |
 
 ## Credit assignment and replay
 
@@ -85,7 +85,7 @@ every turn.
 ## Budget pressure (iteration ceiling)
 
 When unresolved `agent_loop` / `assistant_answer` budget-exhaustion fingerprints
-dominate, `Loop.budget_exhaustion_hot?` tightens the live turn:
+dominate, the loop marks the budget path hot and tightens the live turn:
 
 - lower effective `max_iters` (stricter on local/ollama engines than remote)
 - **Last-iter force-final**: tools=nil on the final iteration so a text answer is required
@@ -98,11 +98,11 @@ Raising `ai.agent.max_iters` or resolving the scar returns normal runway.
 
 ## Intro and extro join
 
-| ID | Where | What |
-|----|-------|------|
-| **E1** | `Metrics.changepoints` + `Loop.attribute_cause` | Env-drift-attributed failures get `cause: :env_drift` and do not inflate `[REPEATING]`. |
-| **E2** | `Extrospection.correlate` | Lead-lag style joins ("tool X started failing after toolchain Y changed"). |
-| **E3** | `Reward.verify_as_reward` | Browser-backed claim checks can floor/cap the outcome score. |
+| Where | What |
+|-------|------|
+| `Metrics.changepoints` + `Loop.attribute_cause` (**E1**) | Env-drift-attributed failures get `cause: :env_drift` and do not inflate `[REPEATING]`. |
+| `Extrospection.correlate` (**E2**) | Lead-lag style joins ("tool X started failing after toolchain Y changed"). |
+| `Reward.verify_as_reward` (**E3**) | Browser-backed claim checks can floor/cap the outcome score. |
 
 ## Config (`PWN::Env[:ai][:agent]`)
 
@@ -113,7 +113,7 @@ Raising `ai.agent.max_iters` or resolving the scar returns normal runway.
     :critic: null            # S3 - nil = ON for remote engines, OFF for ollama
     :red_team_plan: null     # S4 - same auto policy
     :counterfactual: null    # S2 - same auto policy
-    :hindsight: true         # C3 - HER soft-relabel (default true)
+    :hindsight: true         # C3 - hindsight relabel on failed turns (default true)
     :verify_as_reward: null  # E3 - nil = auto sample on claim-shaped answers
     :reward_llm: null        # nil = outcome/process judges use LLM teacher on remote
     :local_introspect: :failure_only   # ollama cost policy; remote always introspects
@@ -168,21 +168,21 @@ keep it honest by:
 
 ## Design-priority STATUS
 
-Living flag authority for the reinforced feedback loop. Cite the Pri/ID here
-instead of inventing new milestone labels for the same theme.
+Living checklist for the reinforced feedback loop. Cite the Pri/ID here instead
+of inventing new milestone labels for the same theme.
 
 | Pri | ID | Control | Module(s) | Success criterion |
 |-----|----|---------|-----------|-------------------|
 | **P0** | W1 generator diversity | `Reward::TARGET_SOURCE_MIX` + `generator_mix` + mix-urgent force on critic/counterfactual | `reward.rb`, `curriculum.rb` | `generator_mix.healthy` OR `recommendation` not stuck on `suppress:mistakes_resolve`; trajectory_fraction ≥ 0.5 |
-| **P0** | Introspect budget | `Learning::INTROSPECT_SOFT_MS` / `HARD_MS`; stage skip under soft/hard / `budget_exhaustion_hot?` | `learning.rb` | `auto_introspect` returns `stages_skipped` when over soft; post-answer path cannot re-thrash tool critic |
-| **P1** | Local judge calibration | Heuristic score shrinkage + `confidence`; `Metrics.effective_rate` scales distrust by `judge_confidence` | `reward.rb`, `metrics.rb` | distrust×heuristic no longer fully replaces proxy; local no-trace highs capped |
-| **P1** | Practice outer KPI | `Curriculum.practice_kpi` / `repeating_trend` → `~/.pwn/curriculum_kpi.jsonl` | `curriculum.rb` | week-over-week `delta_repeating` ≤ 0 on budget fingerprints after practice nights |
-| **P2** | PRM sample efficiency | `PRM_MIN_N=5`, shrinkage to `PRM_FULL_N=20`, fleet coverage gate in `Registry.rank` | `metrics.rb`, `registry.rb` | `prm_advantage=0` until n≥5; rank delta=0 until ≥3 tools ready |
-| **P2** | STATUS over flag archaeology | This table | docs | New work cites Pri/ID here, not fresh comments for the same theme |
+| **P0** | Introspect budget | `Learning::INTROSPECT_SOFT_MS` / `HARD_MS`; stage skip under soft/hard / budget-pressure mode | `learning.rb` | `auto_introspect` returns `stages_skipped` when over soft; post-answer path cannot re-thrash tool critic |
+| **P0** | Local judge calibration | Heuristic score shrinkage + `confidence`; `Metrics.effective_rate` scales distrust by `judge_confidence` | `reward.rb`, `metrics.rb` | distrust×heuristic no longer fully replaces proxy; local no-trace highs capped |
+| **P0** | Practice outer KPI | `Curriculum.practice_kpi` / `repeating_trend` → `~/.pwn/curriculum_kpi.jsonl` | `curriculum.rb` | week-over-week `delta_repeating` ≤ 0 on budget fingerprints after practice nights |
+| **P0** | PRM sample efficiency | `PRM_MIN_N=5`, shrinkage to `PRM_FULL_N=20`, fleet coverage gate in `Registry.rank` | `metrics.rb`, `registry.rb` | `prm_advantage=0` until n≥5; rank delta=0 until ≥3 tools ready |
+| **P0** | STATUS over flag archaeology | This table | docs | New work cites Pri/ID here, not fresh comments for the same theme |
 | **ops** | Nightly diet close | `offline_judge` → `scrub_preferences` + `generator_mix` + `practice_kpi` | `curriculum.rb` | Cron path returns `scrub`/`generator_mix`/`practice_kpi`; raw resolve prose does not survive the night |
 | **ops** | Shape backfill | `Reward.infer_shape` + scrub rewrite | `reward.rb` | Legacy shapeless rows get `winning_trace`/`revised_answer` when content warrants; traj_f measurable |
 | **ops** | Mix in prompt | `Metrics.to_context` emits `W1 MIX:` when unhealthy | `metrics.rb` | Unhealthy diet visible every turn without a tool call |
-| **P0** | Budget exhaust deepen | Last-iter force-final (tools=nil); skip CF when `budget_exhaustion_hot?`; hot caps 24 ollama / 75 remote; exhaust path `append_session`+`auto_introspect` | `loop.rb` | Exhaust returns a judged final, not a bare string; CF cannot re-enter under hot; last iter cannot tool-call |
+| **P0** | Budget exhaust deepen | Last-iter force-final (tools=nil); skip CF under budget-pressure mode; hot caps 24 ollama / 75 remote; exhaust path `append_session`+`auto_introspect` | `loop.rb` | Exhaust returns a judged final, not a bare string; CF cannot re-enter under hot; last iter cannot tool-call |
 
 ### Config additions
 

data/documentation/SDR.md

@@ -15,7 +15,7 @@
 | `RFIDler` | 125 kHz RFID reader/emulator |
 | `SonMicroRFID` | SM130 13.56 MHz reader |
 | `FrequencyAllocation` | ITU/FCC band-plan lookup - "what lives at 433.92 MHz?" |
-| **`Decoder::*`** | 20 protocol demodulators/decoders - `ADSB APT Bluetooth DECT FLEX GPS GSM Iridium LoRa LTE Morse P25 Pager POCSAG RDS RFID RTL433 RTTY WiFi ZigBee` |
+| **`Decoder::*`** | 20 protocol demodulators/decoders (+ Base/DSP helpers) - `ADSB APT Bluetooth DECT FLEX GPS GSM Iridium LoRa LTE Morse P25 Pager POCSAG RDS RFID RTL433 RTTY WiFi ZigBee` |
 
 ## CLI
 

data/documentation/Sessions.md

@@ -27,6 +27,7 @@ appended as JSON-per-line to `~/.pwn/sessions/<id>.jsonl`.
 | `sessions_view(session_id:, truncate:)` | Read entries |
 | `sessions_delete(session_id:)` | Prune noisy/dev transcripts so `reflect` corpus stays clean |
 | `sessions_stats` | totals across the whole directory |
+| `sessions_lean` | prune stubs and aged unreferenced transcripts; compact oversized tool/assistant content (pins gold outcomes and open-mistake refs) |
 
 ## File format
 

data/documentation/Skills-Memory-Learning.md

@@ -113,6 +113,8 @@ the new format.
 | `learning_auto_introspect_toggle(enabled: false)` | during noisy fuzz loops |
 | `PWN::MemoryIndex.reset` | new engagement - drop the local embedding index (`memory.idx`) so it rebuilds against the fresh `memory.json` |
 | `PWN::AI::Agent::Learning.export_finetune(format: :sharegpt)` | you have enough **high-score** sessions for SFT (drops low scores / HER soft rows and compresses traces by process reward) |
+| `memory_lean` / `sessions_lean` / `mistakes_lean` | trim ephemeral or oversized state; never drop protected prefs or open mistakes |
+| `learning_gc_stores` | one coordinated lean pass across the RL stores (supports `dry_run: true`) |
 
 ## Example questions that trigger Introspection
 

data/documentation/What-is-PWN.md

@@ -16,11 +16,11 @@ with a **tool-calling AI agent** on top that can run the same methods.
 | `PWN::SAST::*` | **48** | Static-analysis rules across C/Java/Go/Python/Ruby/Scala/PHP/TS |
 | `PWN::AWS::*` | **90** | One module per AWS service for cloud enumeration |
 | `PWN::WWW::*` | **21** | Site-specific browser automations (HackerOne, BugCrowd, Google, LinkedIn, ...) |
-| `PWN::SDR::*` | **6** (+ **20** decoders) | GQRX, FlipperZero, RFIDler, SonMicro, band tables, `Decoder::{ADSB,POCSAG,RDS,LoRa,...}` |
+| `PWN::SDR::*` | **6** (+ **20** protocol decoders + Base/DSP) | GQRX, FlipperZero, RFIDler, SonMicro, band tables, `Decoder::{ADSB,POCSAG,RDS,LoRa,...}` |
 | `PWN::FFI::*` | **8** | Native DSP/RF backends: Volk · Liquid · FFTW · RTLSdr · HackRF · AdalmPluto · SoapySDR · Stdio |
-| `PWN::AI::*` | **5** engines | OpenAI, Anthropic, Grok (OAuth device-flow), Gemini, Ollama |
+| `PWN::AI::*` | **6** engines | OpenAI, Anthropic, Grok (OAuth device-flow), Gemini, Ollama, Open WebUI |
 | `bin/pwn_*` | **53** | Headless CLI drivers for CI/CD |
-| Agent toolsets | **12** · **78 tools** | terminal · pwn · memory · skills · sessions · learning · metrics · extrospection · cron · swarm · reward · curriculum |
+| Agent toolsets | **12** · **82 tools** | terminal · pwn · memory · skills · sessions · learning · metrics · extrospection · cron · swarm · reward · curriculum |
 
 ## Three ways to use it