pwn 0.5.660 → 0.5.661

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (46) hide show
  1. checksums.yaml +4 -4
  2. data/Gemfile +1 -1
  3. data/README.md +11 -6
  4. data/documentation/Agent-Tool-Registry.md +1 -1
  5. data/documentation/CLI-Drivers.md +1 -1
  6. data/documentation/Configuration.md +15 -7
  7. data/documentation/Contributing.md +1 -1
  8. data/documentation/Cron.md +4 -4
  9. data/documentation/Diagrams.md +7 -2
  10. data/documentation/Drivers.md +1 -1
  11. data/documentation/Extrospection.md +1 -1
  12. data/documentation/Home.md +4 -4
  13. data/documentation/How-PWN-Works.md +5 -4
  14. data/documentation/Mistakes.md +2 -2
  15. data/documentation/Persistence.md +12 -2
  16. data/documentation/Reinforcement-Learning.md +100 -114
  17. data/documentation/Skills-Memory-Learning.md +16 -4
  18. data/documentation/Troubleshooting.md +18 -0
  19. data/documentation/What-is-PWN.md +3 -3
  20. data/documentation/diagrams/agent-tool-registry.svg +188 -140
  21. data/documentation/diagrams/dot/agent-tool-registry.dot +8 -1
  22. data/documentation/diagrams/dot/overall-pwn-architecture.dot +3 -3
  23. data/documentation/diagrams/dot/pwn-ai-feedback-learning-loop.dot +10 -8
  24. data/documentation/diagrams/dot/reinforcement-learning.dot +17 -17
  25. data/documentation/diagrams/dot/task-summarizer.dot +63 -0
  26. data/documentation/diagrams/overall-pwn-architecture.svg +97 -95
  27. data/documentation/diagrams/pwn-ai-feedback-learning-loop.svg +417 -403
  28. data/documentation/diagrams/reinforcement-learning.svg +190 -189
  29. data/documentation/diagrams/task-summarizer.svg +230 -0
  30. data/documentation/pwn-ai-Agent.md +101 -56
  31. data/lib/pwn/ai/agent/curriculum.rb +2 -2
  32. data/lib/pwn/ai/agent/learning.rb +103 -9
  33. data/lib/pwn/ai/agent/loop.rb +328 -36
  34. data/lib/pwn/ai/agent/task_summarizer.rb +580 -14
  35. data/lib/pwn/ai/agent/tools/shell.rb +38 -14
  36. data/lib/pwn/config.rb +2 -2
  37. data/lib/pwn/memory.rb +33 -4
  38. data/lib/pwn/plugins/repl.rb +22 -3
  39. data/lib/pwn/version.rb +1 -1
  40. data/spec/integration/reinforced_feedback_loop_spec.rb +11 -9
  41. data/spec/lib/pwn/ai/agent/learning_spec.rb +21 -0
  42. data/spec/lib/pwn/ai/agent/task_summarizer_spec.rb +190 -18
  43. data/spec/lib/pwn/ai/agent/tools/shell_spec.rb +39 -1
  44. data/spec/lib/pwn/memory_spec.rb +21 -0
  45. data/third_party/pwn_rdoc.jsonl +31 -0
  46. metadata +5 -3
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: '06678f228c6acc69d9fb173f01f3825af6a7a8dcffcb8ed24c865bf281e0c65c'
4
- data.tar.gz: 9967223f974d122e30cbb4be7aea369108e28b951ace091a1ed90d54825a4720
3
+ metadata.gz: a771af76d15bb7d7bf172b6c096f5902b5dd868793593e889b2376885cf84c19
4
+ data.tar.gz: 4beecdae03ebe816005ebdefa85d131bf12ea32c9e0585f759b25b11681412c7
5
5
  SHA512:
6
- metadata.gz: 6feb604302f7e6fd428e8dcd2d1c6a888ed2e6d4c96f3e7b75b6537a5694c3053da6a69da751275c8c168f627f6b71b031ed502cf02b1b0eb8bb2dfd72e2e354
7
- data.tar.gz: 26f65f4e16312694c1163b0de7fe0545ca8ad7b8e1cdea647d931f131de6aaef89f0fe8b39be72509932295c1ad1521066c88722d5654cb6a856891c1c19e860
6
+ metadata.gz: f413d3853e97f512b1a600503f47509f0afc0918a6dbe4be2b1efd9d23e4acc267d2eba6d82a7fdf6a6b2ac6408003c084a486c2cdc8d4ea4e0cf4f14651d59f
7
+ data.tar.gz: 599d5fdeab672da7018878e4a836d4fd9fdf76fb27e797208aabee9fb3a359ae988c42f39f1c09c17352c80d826ecf35445d7b0a245c8fc15af54675a783506c
data/Gemfile CHANGED
@@ -77,7 +77,7 @@ gem 'rbvmomi2', '3.10.0'
77
77
  gem 'rdoc', '7.0.4'
78
78
  gem 'rest-client', '2.1.0'
79
79
  gem 'rex', '2.0.13'
80
- gem 'rmagick', '7.1.0'
80
+ gem 'rmagick', '7.1.1'
81
81
  gem 'rqrcode', '3.2.0'
82
82
  gem 'rspec', '3.13.2'
83
83
  gem 'rtesseract', '3.1.4'
data/README.md CHANGED
@@ -84,6 +84,11 @@ message bus:
84
84
 
85
85
  ![Swarm Multi-Agent](documentation/diagrams/swarm-multi-agent.svg)
86
86
 
87
+ Long-running turns also show **executive task briefs** (not raw commands) via
88
+ `TaskSummarizer`: one full plan on submit (`emit_plan!`), then per-batch
89
+ `about_to` lines keyed by `tool_counts_phrase` + `intent_phrase` with
90
+ `last_brief_fp` duplicate suppression. When recent turns keep hitting the iteration ceiling, the Loop tightens the remaining runway (lower `max_iters` on local engines, text-only tail, no counterfactual fork) so the agent still finishes instead of thrashing.
91
+
87
92
  Full pages: [How PWN Works](documentation/How-PWN-Works.md) ·
88
93
  [All data-flow diagrams](documentation/Diagrams.md)
89
94
 
@@ -101,11 +106,11 @@ The complete wiki lives in this repo at **[`documentation/Home.md`](documentatio
101
106
  | [Installation](documentation/Installation.md) | [Build a Driver](documentation/Drivers.md) | [Mistakes (neg-feedback)](documentation/Mistakes.md) | [WWW (21)](documentation/WWW.md) |
102
107
  | [General Usage](documentation/General-PWN-Usage.md) | | [Extrospection](documentation/Extrospection.md) | [SDR / Radio](documentation/SDR.md) |
103
108
  | [Configuration](documentation/Configuration.md) | | [Swarm (multi-agent)](documentation/Swarm.md) | [Hardware](documentation/Hardware.md) |
104
- | [Configuration](documentation/Configuration.md) | | [Sessions](documentation/Sessions.md) · [Cron](documentation/Cron.md) | [Reports](documentation/Reporting.md) |
105
- | [`~/.pwn/` Persistence](documentation/Persistence.md) | | | [BurpSuite](documentation/BurpSuite.md) · [NmapIt](documentation/NmapIt.md) |
106
- | **[All Diagrams](documentation/Diagrams.md)** | | | [Metasploit](documentation/Metasploit.md) · [Fuzzing](documentation/Fuzzing.md) |
107
- | [Troubleshooting](documentation/Troubleshooting.md) | | | [Hardware](documentation/Hardware.md) · [Blockchain](documentation/Blockchain.md) |
108
- | [Contributing](documentation/Contributing.md) | | | [Bounty](documentation/Bounty.md) · [FFI](documentation/FFI.md) · [Banner](documentation/Banner.md) |
109
+ | [`~/.pwn/` Persistence](documentation/Persistence.md) | | [Sessions](documentation/Sessions.md) · [Cron](documentation/Cron.md) | [Reports](documentation/Reporting.md) |
110
+ | **[All Diagrams](documentation/Diagrams.md)** (29) | | | [BurpSuite](documentation/BurpSuite.md) · [NmapIt](documentation/NmapIt.md) |
111
+ | [Troubleshooting](documentation/Troubleshooting.md) | | | [Metasploit](documentation/Metasploit.md) · [Fuzzing](documentation/Fuzzing.md) |
112
+ | [Contributing](documentation/Contributing.md) | | | [Blockchain](documentation/Blockchain.md) · [Bounty](documentation/Bounty.md) |
113
+ | | | | [FFI](documentation/FFI.md) · [Banner](documentation/Banner.md) |
109
114
 
110
115
  Rebuild every SVG from its Graphviz source:
111
116
  `cd documentation/diagrams && ./build.sh`
@@ -202,7 +207,7 @@ and the local [Contributing](documentation/Contributing.md) page.
202
207
  ### **Module Documentation** ###
203
208
 
204
209
  **Primary:** [`documentation/Home.md`](documentation/Home.md) - the full local
205
- wiki with 30+ pages and 26 SVG data-flow diagrams.
210
+ wiki with 30+ pages and 29 SVG data-flow diagrams.
206
211
 
207
212
  **API reference:** [rubydoc.info/gems/pwn](https://www.rubydoc.info/gems/pwn),
208
213
  or in-REPL: `PWN::Plugins::BurpSuite.help`, `show-source`, `ls`.
@@ -16,7 +16,7 @@ toolsets; the JSON-Schema for each tool is what the model actually sees.
16
16
  | `skills` | `skill_list` · `skill_view` · `skill_create` · `skill_add_reference` · `skill_delete` · `skill_migrate_legacy` | `~/.pwn/skills/<name>/SKILL.md` (**[agentskills.io](https://agentskills.io) spec**; legacy flat `*.md` auto-migrated) |
17
17
  | `sessions` | `sessions_list` · `sessions_view` · `sessions_current` · `sessions_delete` · `sessions_stats` | `PWN::Sessions` → `~/.pwn/sessions/` |
18
18
  | `learning` | `learning_note_outcome` · `learning_reflect` · `learning_distill_skill` · `learning_stats` · `learning_outcomes` · `learning_consolidate` · `learning_reset` · `learning_auto_introspect_toggle` · **`mistakes_list`** · **`mistakes_record`** · **`mistakes_resolve`** · **`mistakes_reset`** · **`reward_judge`** · **`reward_prm`** · **`reward_sentinel`** · **`reward_preferences`** · **`reward_export_dpo`** · **`reward_warm_sentinel`** · **`reward_scrub_preferences`** · **`reward_preference_balance`** · **`curriculum_practice`** · **`curriculum_train`** · **`curriculum_hindsight`** · **`curriculum_offline_judge`** · **`curriculum_preference_balance`** · **`learning_purge_noise`** | `PWN::AI::Agent::Learning` + `Mistakes` + `Reward` + `Curriculum` → `~/.pwn/learning.jsonl` + `~/.pwn/mistakes.json` + `~/.pwn/preferences.jsonl` + `~/.pwn/curriculum/` + `~/.pwn/finetune/` |
19
- | `reward` | **`reward_generator_mix`** | `PWN::AI::Agent::Reward.generator_mix` → W1 online source-mix controller (`preferences.jsonl`) |
19
+ | `reward` | **`reward_generator_mix`** | `PWN::AI::Agent::Reward.generator_mix` → online preference source-mix controller (`preferences.jsonl`) |
20
20
  | `curriculum` | **`curriculum_practice_kpi`** | `PWN::AI::Agent::Curriculum.practice_kpi` → `~/.pwn/curriculum_kpi.jsonl` |
21
21
  | `metrics` | `metrics_summary` · `metrics_reset` | `PWN::AI::Agent::Metrics` → `~/.pwn/metrics.json` |
22
22
  | `extrospection` | `extro_snapshot` · `extro_drift` · `extro_observe` · `extro_observations` · `extro_intel` · **`extro_watch`** · **`extro_verify`** · **`extro_rf_tune`** · **`extro_osint`** · **`extro_serial`** · **`extro_telecomm`** · **`extro_packet`** · **`extro_vision`** · **`extro_voice`** · `extro_correlate` · `extro_stats` · `extro_reset` · `extro_auto_toggle` | `PWN::AI::Agent::Extrospection` (+ Serial/Packet/OCR/Voice/BareSIP/TransparentBrowser/GQRX) → `~/.pwn/extrospection.json` |
@@ -1,6 +1,6 @@
1
1
  # CLI Drivers - `bin/pwn_*`
2
2
 
3
- 53 headless executables (52 `pwn_*` + `pwn`), each a thin `OptionParser`
3
+ 54 headless executables (53 `pwn_*` + `pwn`), each a thin `OptionParser`
4
4
  wrapper over one plugin (or one workflow). They exist so CI/CD can call PWN
5
5
  without a REPL or an LLM.
6
6
 
@@ -99,13 +99,17 @@ ai:
99
99
  agent:
100
100
  native_tools: true # Use provider-native tool_calls / function-calling. false → legacy text-parsed tool protocol.
101
101
  max_iters: 25 # Hard cap on tool-call rounds per user turn before a forced final answer.
102
+ task_summary: true # Executive task briefs via TaskSummarizer (plan + about_to). false disables.
103
+ task_summary_every: 5 # When task_summary_verbose: emit Progress every N completed tools.
104
+ task_summary_interval_s: 8.0 # When verbose: also emit when this many seconds elapsed.
105
+ task_summary_verbose: false # Mid-flight Progress/Finished lines (default: only plan + about_to).
102
106
  max_depth: 3 # Recursion guard: how many levels deep agent_ask/agent_debate sub-agents may spawn sub-agents.
103
107
  auto_introspect: true # Run Learning.auto_introspect (outcome logging + lesson mining) after every final answer.
104
108
  auto_extrospect: false # Optional ambient baseline (host/repo/env ONLY - never launches burpsuite/zaproxy/msf/gqrx). Sense tools (intel/verify/watch/rf_tune/observe) stay on-demand.
105
- plan_first: ~ # Plan-then-act pre-pass: force the model to externalise a numbered tool plan BEFORE its first dispatch. nil = auto (true when ai.active == ollama).
109
+ plan_first: ~ # Plan-then-act pre-pass: force the model to externalize a numbered tool plan BEFORE its first dispatch. nil = auto (true when ai.active == ollama).
106
110
  tool_router: false # Dynamic tool-set slimming: ship only Registry::CORE_TOOLS + top-K keyword-relevant schemas per turn (helps small models route correctly).
107
111
  escalation_persona: ~ # Swarm persona name to ask for a 3-line corrective hint once a local model burns ≥ Loop::ESCALATE_AFTER_FAILS in-turn failures. nil = disabled.
108
- toolsets: ~ # Allow-list of toolsets exposed to the agent. nil = all. Valid: cron, extrospection, learning, memory, metrics, pwn, sessions, skills, swarm, terminal.
112
+ toolsets: ~ # Allow-list of toolsets exposed to the agent. nil = all. Valid: cron, curriculum, extrospection, learning, memory, metrics, pwn, reward, sessions, skills, swarm, terminal.
109
113
  extrospection:
110
114
  web:
111
115
  anchors: # URLs the headless browser fingerprints on extro_snapshot(sections:[:web]). Alias: web_anchors.
@@ -124,7 +128,7 @@ ai:
124
128
  osint:
125
129
  ttl: 86400 # Observation TTL for :osint (default 1 day).
126
130
  proxy: ~ # Optional upstream proxy for OSINT HTTP feeds.
127
- api_keys: # Per-feed API keys (all optional; keyed feeds return {skipped:} when absent). ENV fallbacks also honoured.
131
+ api_keys: # Per-feed API keys (all optional; keyed feeds return {skipped:} when absent). ENV fallbacks also honored.
128
132
  shodan: ... # ← SHODAN_API_KEY
129
133
  hunter: ... # ← HUNTER_API_KEY
130
134
  abuseipdb: ... # ← ABUSEIPDB_API_KEY
@@ -224,7 +228,7 @@ PWN::Config.refresh_env
224
228
 
225
229
  ---
226
230
 
227
- ## Exhaustive key reference
231
+ ## Full key reference
228
232
 
229
233
  ### `ai` - AI engines & agent loop
230
234
 
@@ -258,11 +262,15 @@ PWN::Config.refresh_env
258
262
  | Key path | Type | Default | Consumed by | Purpose |
259
263
  |---|---|---|---|---|
260
264
  | `ai.agent.native_tools` | Boolean | `true` | `PWN::Plugins::REPL` (`pwn-ai` cmd) | Use provider-native `tool_calls` / function-calling. `false` falls back to the legacy text-parsed tool protocol. |
261
- | `ai.agent.max_iters` | Integer | `25` | `PWN::AI::Agent::Loop.run`, `PWN::AI::Agent::Swarm` | Hard cap on tool-call rounds per user turn before a forced final answer. |
265
+ | `ai.agent.max_iters` | Integer | `25` | `PWN::AI::Agent::Loop.run`, `PWN::AI::Agent::Swarm` | Hard cap on tool-call rounds per user turn before a forced final answer. When budget-exhaustion pressure is high, the effective cap is tightened (stricter on local/ollama than remote) so long multi-step goals keep a usable runway without thrashing. |
266
+ | `ai.agent.task_summary` | Boolean | `true` | `PWN::AI::Agent::TaskSummarizer`, `Loop` | Master switch for executive task briefs (`emit_plan!` / `about_to`). |
267
+ | `ai.agent.task_summary_every` | Integer | `5` | `TaskSummarizer.every_n` | Verbose progress cadence (tools). |
268
+ | `ai.agent.task_summary_interval_s` | Float | `8.0` | `TaskSummarizer.interval_s` | Verbose progress cadence (seconds). |
269
+ | `ai.agent.task_summary_verbose` | Boolean | `false` | `TaskSummarizer.verbose?` | Emit mid-flight `Progress:` / `Finished:` lines; default keeps only plan + about_to. |
262
270
  | `ai.agent.max_depth` | Integer | `3` | `PWN::AI::Agent::Swarm` | Recursion guard for `agent_ask` / `agent_debate` sub-agents spawning sub-agents. |
263
271
  | `ai.agent.auto_introspect` | Boolean | `true` | `PWN::AI::Agent::Learning.auto_introspect` | Run outcome logging + lesson mining after every final answer. Toggle live via `learning_auto_introspect_toggle`. |
264
272
  | `ai.agent.auto_extrospect` | Boolean | `false` | `PWN::AI::Agent::Extrospection.auto_extrospect` | Optional ambient baseline after every final answer (`AUTO_SECTIONS` = host/repo/env only; never spawns GUI/JVM tools). Prefer on-demand sense tools (`intel`/`verify`/`watch`/`rf_tune`/`observe`). Toggle live via `extro_auto_toggle`. |
265
- | `ai.agent.toolsets` | Array\<String\> \| `nil` | `nil` (all) | `bin/pwn`, `PWN::Plugins::REPL`, `PWN::AI::Agent::Registry` | Allow-list of toolsets exposed to the agent. Valid: `cron`, `extrospection`, `learning`, `memory`, `metrics`, `pwn`, `sessions`, `skills`, `swarm`, `terminal`. |
273
+ | `ai.agent.toolsets` | Array\<String\> \| `nil` | `nil` (all) | `bin/pwn`, `PWN::Plugins::REPL`, `PWN::AI::Agent::Registry` | Allow-list of toolsets exposed to the agent. Valid: `cron`, `curriculum`, `extrospection`, `learning`, `memory`, `metrics`, `pwn`, `reward`, `sessions`, `skills`, `swarm`, `terminal`. |
266
274
  | `ai.agent.plan_first` | Boolean \| `nil` | `nil` (auto: `true` when `ai.active == ollama`) | `PWN::AI::Agent::Loop.plan_first` | Plan-then-act pre-pass: the model must emit a numbered tool plan (as an assistant message) *before* it may dispatch anything. Cheap chain-of-thought scaffolding for local models. |
267
275
  | `ai.agent.tool_router` | Boolean | `false` | `PWN::AI::Agent::Registry.definitions` | Dynamic tool-set slimming: expose only `Registry::CORE_TOOLS` + the top-K keyword-relevant schemas for *this* request. Ties break on historical `Metrics` success rate so the router itself is a learned component. |
268
276
  | `ai.agent.escalation_persona` | String \| `nil` | `nil` | `PWN::AI::Agent::Loop.escalate` → `Swarm.ask` | Circuit-breaker: once a local model accumulates ≥ `Loop::ESCALATE_AFTER_FAILS` in-turn failures, ask this Swarm persona for a 3-line corrective hint (injected as a synthetic tool result). The local model still authors the final answer so Learning/Metrics stay attributed. |
@@ -283,7 +291,7 @@ PWN::Config.refresh_env
283
291
  | `ai.agent.extrospection.rf.ttl` | Integer | `300` | `Extrospection.rf_tune` | TTL (seconds) for `:rf` observations written by `extro_rf_tune` (ephemeral radio content). |
284
292
  | `ai.agent.extrospection.osint.ttl` | Integer | `86400` | `Extrospection.osint` | TTL (seconds) for `:osint` observations written by `extro_osint`. |
285
293
  | `ai.agent.extrospection.osint.proxy` | String | - | `Extrospection.osint` | Optional upstream proxy for OSINT HTTP feeds. |
286
- | `ai.agent.extrospection.osint.api_keys.<feed>` | String | - | `Extrospection.osint_api_keys` | Per-feed API keys for keyed OSINT sources: `shodan`, `hunter`, `abuseipdb`, `virustotal`, `greynoise`, `haveibeenpwned`, `securitytrails`, `steam`. Keyed feeds return `{skipped:}` when absent. ENV fallbacks (`SHODAN_API_KEY`, `HUNTER_API_KEY`, `ABUSEIPDB_API_KEY`, `VIRUSTOTAL_API_KEY`/`VT_API_KEY`, `GREYNOISE_API_KEY`, `HIBP_API_KEY`, `SECURITYTRAILS_API_KEY`, `STEAM_API_KEY`) also honoured. Redacted. |
294
+ | `ai.agent.extrospection.osint.api_keys.<feed>` | String | - | `Extrospection.osint_api_keys` | Per-feed API keys for keyed OSINT sources: `shodan`, `hunter`, `abuseipdb`, `virustotal`, `greynoise`, `haveibeenpwned`, `securitytrails`, `steam`. Keyed feeds return `{skipped:}` when absent. ENV fallbacks (`SHODAN_API_KEY`, `HUNTER_API_KEY`, `ABUSEIPDB_API_KEY`, `VIRUSTOTAL_API_KEY`/`VT_API_KEY`, `GREYNOISE_API_KEY`, `HIBP_API_KEY`, `SECURITYTRAILS_API_KEY`, `STEAM_API_KEY`) also honored. Redacted. |
287
295
  | `ai.agent.extrospection.osint.social.sites_file` | Path | `etc/osint/social_sites.json` | `Extrospection.osint_social_sweep` | JSON of `{sites:{Name:{url:"...{u}...",absent_status:[404],absent_body:[...],head:bool}}}` used by the `:social_sweep` presence check. Vendored subset of sherlock-project (MIT). Override to add/remove platforms. |
288
296
  | `ai.agent.extrospection.osint.social.max_threads` | Integer | `16` | `Extrospection.osint_social_sweep` | Concurrency for the presence sweep (`Concurrent::FixedThreadPool`). |
289
297
  | `ai.agent.extrospection.osint.social.max_sites` | Integer | `120` | `Extrospection.osint_social_sweep` | Hard cap on sites loaded from `sites_file`. |
@@ -9,7 +9,7 @@ lib/pwn/migrate.rb # PWN::Migrate - ~/.pwn state doctor / auto-migrator
9
9
  lib/pwn/plugins/ # 66 plugin modules
10
10
  lib/pwn/ai/agent/ # agent core
11
11
  lib/pwn/ai/agent/tools/ # LLM tool registrations
12
- bin/ # 52 pwn_* drivers + pwn (incl. pwn_setup)
12
+ bin/ # 53 pwn_* drivers + pwn (incl. pwn_setup)
13
13
  spec/ # RSpec (incl. conventions_spec)
14
14
  documentation/ # this wiki + diagrams
15
15
  ```
@@ -37,14 +37,14 @@ invokes `PWN::Cron.run(<id>)` on schedule.
37
37
  | `learning_consolidate_nightly` | `0 5 * * *` | `PWN::AI::Agent::Learning.consolidate` |
38
38
 
39
39
  - **practice** - top unresolved `Mistakes` under `Reward.judge`; auto-`resolve` with ≥2 holdouts
40
- - **offline_judge** - P3 backfill of ORM/PRM labels + W3 calibration from PLAN `p(success)=` so `:failure_only` local introspect does not starve the corpus; also runs **P10** `Reward.warm_sentinel` so the R3 ring can fill on local hosts
41
- - **dedupe (P13)** - legacy alias `offline_judge_nightly` counts as the same job; `install_defaults` will not double-seed `30 3 * * *` and disables the alias when both exist
40
+ - **offline_judge** - backfill outcome/process labels + plan calibration from PLAN `p(success)=` so `:failure_only` local introspect does not starve the corpus; also runs `Reward.warm_sentinel` so the reward-sentinel window can fill on local hosts
41
+ - **dedupe** - legacy alias `offline_judge_nightly` counts as the same job; `install_defaults` will not double-seed `30 3 * * *` and disables the alias when both exist
42
42
  - **train** - export SFT + balanced DPO, LoRA-train `pwn-vN+1`, replay Mistakes.top, promote only on win. `dry_run: false` only with a trainer+GPU
43
- - **consolidate** - M1/M3 memory GC so the injected MEMORY block stays high-signal
43
+ - **consolidate** - memory GC (semantic merge + importance eviction) so the injected MEMORY block stays high-signal
44
44
 
45
45
  See [Reinforcement Learning](Reinforcement-Learning.md).
46
46
 
47
- `cron_disable(id:)` turns either off; `install_defaults` is idempotent (including P13 alias collapse) and
47
+ `cron_disable(id:)` turns either off; `install_defaults` is idempotent (including legacy alias collapse) and
48
48
  never overwrites a job you already have with the same name.
49
49
 
50
50
  ## Example
@@ -1,6 +1,6 @@
1
1
  # PWN Data-Flow Diagrams
2
2
 
3
- 28 SVG diagrams, all rendered from Graphviz sources in
3
+ 29 SVG diagrams, all rendered from Graphviz sources in
4
4
  [`diagrams/dot/`](diagrams/dot/) with a single shared visual theme
5
5
  (see [`_THEME.md`](diagrams/dot/_THEME.md)). Rebuild everything with:
6
6
 
@@ -51,6 +51,11 @@ groups) so lines never criss-cross.
51
51
  [source](diagrams/dot/pwn-ai-feedback-learning-loop.dot) · doc: [Skills, Memory & Learning](Skills-Memory-Learning.md)
52
52
  ![pwn-ai-feedback-learning-loop](diagrams/pwn-ai-feedback-learning-loop.svg)
53
53
 
54
+
55
+ ### TaskSummarizer - Executive Task Briefs
56
+ [source](diagrams/dot/task-summarizer.dot) · doc: [pwn-ai Agent](pwn-ai-Agent.md)
57
+ ![task-summarizer](diagrams/task-summarizer.svg)
58
+
54
59
  ### Mistakes - Negative-Feedback Loop
55
60
  [source](diagrams/dot/mistakes-negative-feedback.dot) · doc: [Mistakes](Mistakes.md)
56
61
  ![mistakes-negative-feedback](diagrams/mistakes-negative-feedback.svg)
@@ -63,7 +68,7 @@ groups) so lines never criss-cross.
63
68
  [source](diagrams/dot/ai-integration-tool-calling.dot) · doc: [AI Integration](AI-Integration.md)
64
69
  ![ai-integration-tool-calling](diagrams/ai-integration-tool-calling.svg)
65
70
 
66
- ### Agent Tool Registry (10 toolsets · 71 tools)
71
+ ### Agent Tool Registry (12 toolsets · 78 tools)
67
72
  [source](diagrams/dot/agent-tool-registry.dot) · doc: [Agent Tool Registry](Agent-Tool-Registry.md)
68
73
  ![agent-tool-registry](diagrams/agent-tool-registry.svg)
69
74
 
@@ -1,7 +1,7 @@
1
1
  # Drivers - Turn a REPL Session into a Shipped Binary
2
2
 
3
3
  A **driver** is a small executable in `bin/` that wires `OptionParser` to one
4
- or more `PWN::` calls. All 52 shipped `pwn_*` binaries follow the same
4
+ or more `PWN::` calls. All 53 shipped `pwn_*` binaries follow the same
5
5
  15-line template.
6
6
 
7
7
  ![History → Driver → CI](diagrams/history-to-drivers.svg)
@@ -453,7 +453,7 @@ cron_create(name: 'memory_revalidate', schedule: '0 4 * * 0',
453
453
  ```ruby
454
454
  PWN::Env[:ai][:agent][:extrospection][:web] = {
455
455
  anchors: ['https://target.acme/api/version', ...], # else DEFAULT_WEB_ANCHORS
456
- proxy: 'tor', # or 'http://127.0.0.1:8080' (Burp) - honoured by verify/watch/probe_web
456
+ proxy: 'tor', # or 'http://127.0.0.1:8080' (Burp) - honored by verify/watch/probe_web
457
457
  max_anchors: 8,
458
458
  per_page_timeout: 15,
459
459
  screenshot: false, # → ~/.pwn/extrospection/web/<host>.png
@@ -20,15 +20,15 @@
20
20
  | [Installation](Installation.md) | `gem install pwn` → `pwn setup` doctor/provisioner · capability profiles · `--migrate` state doctor |
21
21
  | [General Usage](General-PWN-Usage.md) | Day-one cheat sheet |
22
22
  | [Configuration](Configuration.md) | `~/.pwn/pwn.yaml` (encrypted) - engines, keys, agent options · `pwn-vault` |
23
- | **[All Data-Flow Diagrams](Diagrams.md)** | **28 SVGs** in one scrollable page |
23
+ | **[All Data-Flow Diagrams](Diagrams.md)** | **29 SVGs** in one scrollable page |
24
24
 
25
25
  ## 🚪 Entry Points
26
26
 
27
27
  | | |
28
28
  |---|---|
29
29
  | [The `pwn` REPL](pwn-REPL.md) | Pry shell with the whole `PWN::` namespace pre-loaded |
30
- | [`pwn-ai` Autonomous Agent](pwn-ai-Agent.md) | Natural-language TUI + `pwn --ai PROMPT` one-shot |
31
- | [CLI Drivers `bin/pwn_*`](CLI-Drivers.md) | 53 headless executables for CI/CD |
30
+ | [`pwn-ai` Autonomous Agent](pwn-ai-Agent.md) | Natural-language TUI + `pwn --ai PROMPT` one-shot · **TaskSummarizer** briefs · iteration budget guard |
31
+ | [CLI Drivers `bin/pwn_*`](CLI-Drivers.md) | 53 `pwn_*` + `pwn` headless executables for CI/CD |
32
32
  | [Drivers (build your own)](Drivers.md) | Turn a REPL session into a shipped binary |
33
33
 
34
34
  ## 🤖 AI Subsystem (`PWN::AI`)
@@ -39,7 +39,7 @@
39
39
  | [Agent Tool Registry](Agent-Tool-Registry.md) | 12 toolsets · **78** LLM-callable tools |
40
40
  | [Memory · Skills · Learning](Skills-Memory-Learning.md) | Introspection - the self-improvement loop |
41
41
  | [Mistakes](Mistakes.md) | **Negative feedback** - fingerprint failures · do-NOT-repeat · `[REPEATING]`/`[REGRESSED]` · inline self-correction |
42
- | [Reinforcement Learning](Reinforcement-Learning.md) | **`Reward` + `Curriculum`** - ORM/PRM · P9/P14 trajectory DPO · P15 scrub · P16 warm · P17 budget skill · P18 PRM→rank · P19 diet gate · export-ready LoRA |
42
+ | [Reinforcement Learning](Reinforcement-Learning.md) | **`Reward` + `Curriculum`** - outcome/process judges · preference ledger · self-play practice · export-ready LoRA gate |
43
43
  | [Extrospection](Extrospection.md) | World-awareness - snapshot · drift · intel · **watch** · **verify** · **rf_tune** · **osint** · serial · telecomm · packet · vision · voice · correlate |
44
44
  | [Swarm (Multi-Agent)](Swarm.md) | Personas · ask · debate · broadcast · shared bus |
45
45
  | [Sessions](Sessions.md) | Transcript persistence + reflection |
@@ -20,21 +20,22 @@ hardware).
20
20
  | `pwn-ai` | `lib/pwn/ai/agent/loop.rb` | Agent TUI inside the REPL |
21
21
  | `pwn --ai PROMPT` | `bin/pwn` | Headless one-shot agent (CI-friendly) |
22
22
  | `pwn setup` | `lib/pwn/setup.rb` · `bin/pwn_setup` | Post-install doctor + capability provisioner + `--migrate` state doctor (also `pwn --setup[=PROFILE]`) |
23
- | `bin/pwn_*` | 52 files | Thin OptionParser wrappers over one plugin each |
23
+ | `bin/pwn_*` | 53 files | Thin OptionParser wrappers over one plugin each |
24
24
  | `PWN::Cron` | `lib/pwn/cron.rb` | Scheduled jobs → any of the above (nightly self-play + weekly weight-loop seeded by default) |
25
25
 
26
26
  ## L2 - AI agent core (`lib/pwn/ai/agent/`)
27
27
 
28
28
  | Module | Role |
29
29
  |---|---|
30
- | `Loop` | plan → dispatch tool_calls → observe → repeat until final answer |
30
+ | `Loop` | plan → **TaskSummarizer** briefs → dispatch tool_calls → observe → repeat until final answer; tightens runway when recent turns exhausted the budget |
31
+ | **`TaskSummarizer`** | Executive UX: `emit_plan!` (full goal once) · `about_to` per tool batch (`tool_counts_phrase` + `intent_phrase`) · `last_brief_fp` dedup · plan_idx advance |
31
32
  | `Registry` | JSON-Schema function definitions grouped into 12 **toolsets** · **78 tools** |
32
33
  | `Dispatch` / `Result` | execute a tool, capture stdout/value/error/duration |
33
34
  | `PromptBuilder` | inject MEMORY / SKILLS / LEARNING / **KNOWN MISTAKES + FIXES** / METRICS / EXTROSPECTION blocks |
34
35
  | `Metrics` · `Learning` · `Reflect` | **introspection** - how well am I doing? |
35
36
  | `Mistakes` | **negative feedback** - fingerprint failures, do NOT repeat, `[REPEATING]`/`[REGRESSED]`, inline `correction_hint` |
36
- | **`Reward`** | **R1** ORM `judge` · **R2** PRM per-step credit · **R3** `sentinel` reward-hacking detector · **R4** `semantic_ok` · **W1** DPO `preferences.jsonl` |
37
- | **`Curriculum`** | **S1** mistake-driven self-play `practice` · **S2** `counterfactual` A/B · **S3** tool-armed `critic` · **S4** `red_team_plan` · **C3** `hindsight` (HER) · **W2** `train_and_gate` regression-gated LoRA |
37
+ | **`Reward`** | outcome `judge` · per-step process credit · `sentinel` (proxy vs judge drift) · `semantic_ok` · DPO `preferences.jsonl` |
38
+ | **`Curriculum`** | mistake-driven self-play `practice` · `counterfactual` A/B · tool-armed `critic` · `red_team_plan` · `hindsight` (HER) · `train_and_gate` regression-gated LoRA |
38
39
  | `Extrospection` | **extrospection** - on-demand world sensing (`intel` · **`verify`** · **`watch`** · **`rf_tune`** · **`osint`** · `serial` · `telecomm` · `packet` · `vision` · `voice`) + ambient baseline (host · net · toolchain · repo · env · **rf** · **web**) joined to introspection via `correlate` |
39
40
  | `Swarm` | multi-agent personas over a shared JSONL bus |
40
41
 
@@ -100,7 +100,7 @@ upgraded", "the HackRF was unplugged", or "the target DOM moved". See
100
100
  [Extrospection](Extrospection.md#extro_correlate--the-point-of-the-whole-thing).
101
101
 
102
102
 
103
- ## Preference pairs from resolve (W1 / P9)
103
+ ## Preference pairs from resolve
104
104
 
105
105
  `Mistakes.resolve` writes a DPO preference into `~/.pwn/preferences.jsonl`:
106
106
 
@@ -109,7 +109,7 @@ upgraded", "the HackRF was unplugged", or "the target DOM moved". See
109
109
  present; falls back to `AVOID_ARGS` / fix prose only when no trace exists
110
110
 
111
111
  Curriculum auto-resolve always attaches a winning tool trace so the ledger
112
- learns **trajectories**, not commentary. Write-time source quotas (P9) still
112
+ learns **trajectories**, not commentary. Write-time source quotas still
113
113
  apply to pure prose resolves so resolve-monoculture cannot flood the ledger.
114
114
 
115
115
  **See also:** [Skills, Memory & Learning](Skills-Memory-Learning.md) ·
@@ -16,7 +16,7 @@ Every byte PWN remembers between processes lives here.
16
16
  | **`preferences.jsonl`** | **`PWN::AI::Agent::Reward`** | JSON-per-line `{prompt,rejected,chosen,source}` | `rm` | **DPO/KTO/ORPO preference-pair ledger - user_correction · mistakes_resolve · counterfactual · critic · curriculum** |
17
17
  | **`mistakes.json`** | **`PWN::AI::Agent::Mistakes`** | **JSON `{sig → entry}`** | **`mistakes_reset`** | **failure fingerprints · cross-session count · fix · `[REPEATING]` · `[REGRESSED]`** |
18
18
  | `metrics.json` | `PWN::AI::Agent::Metrics` | JSON | `metrics_reset` | per-tool calls · success · avg_duration · last_error · **per-engine** sub-buckets · calibration |
19
- | `reward_sentinel.json` | `PWN::AI::Agent::Reward` | JSON | `rm` | R3 - proxy vs judge vs user-correction gap history |
19
+ | `reward_sentinel.json` | `PWN::AI::Agent::Reward` | JSON | `rm` | proxy vs judge vs user-correction gap history |
20
20
  | `extrospection.json` | `PWN::AI::Agent::Extrospection` | JSON | `extro_reset` | host/net/toolchain/repo/env/**rf**/**web**/osint/serial/telecomm/packet/vision/voice snapshot + previous baseline + observations[] |
21
21
  | `extrospection/web/*.png` | `Extrospection` | PNG | `rm -rf` | headless-browser screenshots from `probe_web` / `extro_watch` (opt-in) |
22
22
  | `extrospection/packet/*.pcap` | `Extrospection` | pcap | `rm -rf` | bounded captures from `extro_packet(action: :capture)` |
@@ -27,12 +27,22 @@ Every byte PWN remembers between processes lives here.
27
27
  | `agents.yml` | `PWN::AI::Agent::Swarm` | YAML | edit / `agent_spawn` | persona registry |
28
28
  | `swarm/<id>/bus.jsonl` | `Swarm` | JSON-per-line | rm -rf | append-only multi-agent chat |
29
29
  | `swarm/<id>/personas.json` | `Swarm` | JSON | rm | persona → session_id map |
30
- | `curriculum/` | `PWN::AI::Agent::Curriculum` | JSONL | rm -rf | S1 self-play reproducers · W2 A/B eval sets |
30
+ | `curriculum/` | `PWN::AI::Agent::Curriculum` | JSONL | rm -rf | self-play reproducers · train/gate A/B eval sets |
31
31
  | `finetune/*.jsonl` | `Learning.export_finetune` · `Reward.export_dpo` | ShareGPT / OpenAI / DPO JSONL | `rm` | supervised + preference datasets - feed to a LoRA over the local model |
32
32
  | `backup/<ts>/` | `PWN::Migrate` | tree copy | rm -rf | timestamped snapshot taken before every `--migrate --fix` |
33
33
  | `quarantine/` | `PWN::Migrate` | quarantined originals | rm -rf | corrupt/incompatible state files moved aside so the owner re-seeds |
34
34
  | `~/.pwn_history` | Pry | text | rm | REPL input history |
35
35
 
36
+
37
+ ## `memory.json` safety
38
+
39
+ `PWN::Memory.save` **refuses** to overwrite a non-empty `memory.json` with `{}`
40
+ unless you pass `force: true`. That blocks the failure mode where a parse error
41
+ was rescued into an empty hash and then written back (full wipe). `memory_clear`
42
+ and deleting the last key use `force: true` on purpose. Load no longer falls
43
+ back to `{}` on parse failure for a non-empty file; it warns and keeps the bytes
44
+ on disk for repair.
45
+
36
46
  ## Verify / repair the whole tree
37
47
 
38
48
  ```bash