pwn 0.5.639 → 0.5.643

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 729dee0799d356e7a109e5ed3e5ac6cf07d322e9c136e3fc55477581e6457953
-  data.tar.gz: 398f30816941076c3ca6320526aa630875ebdae12a6274f3c0419e37db55f271
+  metadata.gz: 065c569478e763c0cc10c1dbb19f017dc06dcca1baa66c448be8f35b0572cf8e
+  data.tar.gz: 35974c5ba9ca144cc4f033a96d77b432ef8d81958ba5abb556324314af00c539
 SHA512:
-  metadata.gz: 390144f0551778087a3621784b56cadbedffcad2635811ef6141425a6879b555363254b1ed3e6047e6df7256f81d3e781681b9cfa10f3e8274ed3876b2058329
-  data.tar.gz: 4b653649d86a5669480dad3051b38f055955680f1882e746bacfc1624a6128bf47d5990c7fe9904c9f3d6879a473a86bf58f845deb360992962571a968579645
+  metadata.gz: 66cb4b9f2c195607cb300c03885d961be5c918048c4f511f55af609014a6245edb50111776fd357d35c26ca258baabc5dc061e9c229b7d16956d05a11ff6b585
+  data.tar.gz: 9a1e152a5a23441c22476bb5410db47efcdf3caee3f1029568538a15798a69905da6e0c77fd24969c08bfcadecbd009b2f6d424a720d3e98b0f2a123697cdd62

data/Gemfile

@@ -20,7 +20,7 @@ gem 'base32', '0.3.4'
 gem 'bitcoin-ruby', '0.0.20'
 gem 'brakeman', '8.0.5'
 gem 'bson', '5.2.0'
-gem 'bundler', '>=4.0.16'
+gem 'bundler', '>=4.0.17'
 gem 'bundler-audit', '>=0.9.3'
 gem 'bunny', '3.1.0'
 gem 'colorize', '1.1.0'
@@ -49,10 +49,10 @@ gem 'jwt', '3.2.0'
 gem 'libusb', '0.7.2'
 gem 'luhn', '3.0.0'
 gem 'mail', '2.9.1'
-gem 'mcp', '0.24.0'
+gem 'mcp', '1.0.0'
 gem 'meshtastic', '0.0.172'
 gem 'metasm', '1.0.6'
-gem 'mongo', '2.24.1'
+gem 'mongo', '2.25.0'
 gem 'msfrpc-client', '1.1.2'
 gem 'netaddr', '2.0.6'
 gem 'net-ldap', '0.20.0'

data/README.md

@@ -128,7 +128,7 @@ $ pwn setup --profile net --dry-run
 Also available as `pwn_setup` (standalone driver) and `pwn --setup[=PROFILE]`.
 The doctor exits non-zero when capabilities are degraded, so CI can gate on it.
 
-[![Installing the pwn Security Automation Framework](https://raw.githubusercontent.com/0dayInc/pwn/master/documentation/pwn_install.png)](https://youtu.be/G7iLUY4FzsI)
+<!--[![Installing the pwn Security Automation Framework](https://raw.githubusercontent.com/0dayInc/pwn/master/documentation/pwn_install.png)](https://youtu.be/G7iLUY4FzsI)-->
 
 Full page: [Installation](documentation/Installation.md) ·
 [Configuration](documentation/Configuration.md)

data/documentation/Cron.md

@@ -26,20 +26,22 @@ invokes `PWN::Cron.run(<id>)` on schedule.
 
 ## Seeded self-improvement jobs (`PWN::Cron.install_defaults`)
 
-`pwn setup --migrate` (schema `v1`) seeds two jobs into every fresh
-`~/.pwn/cron/jobs.yml`:
+`pwn setup --migrate` (schema `v1`) seeds four jobs into every fresh
+`~/.pwn/cron/jobs.yml` via `PWN::Cron.install_defaults`:
 
 | Name | Schedule | Ruby |
 |---|---|---|
 | `curriculum_practice_nightly` | `0 3 * * *` | `PWN::AI::Agent::Curriculum.practice(limit: 3)` |
+| `curriculum_offline_judge` | `30 3 * * *` | `PWN::AI::Agent::Curriculum.offline_judge(since_hours: 24, limit: 40)` |
 | `curriculum_train_weekly` | `0 4 * * 0` | `PWN::AI::Agent::Curriculum.train_and_gate(dry_run: true)` |
+| `learning_consolidate_nightly` | `0 5 * * *` | `PWN::AI::Agent::Learning.consolidate` |
 
-The first practises the top unresolved `Mistakes` under `Reward.judge` and
-auto-`resolve`s any it now solves; the second exports SFT + DPO datasets to
-`~/.pwn/finetune/`, LoRA-trains a `pwn-vN+1` local model, replays the
-`Mistakes.top` set on both, and only promotes the new adapter when it wins.
-Set `dry_run: false` (via `pwn-ai-cron` or `cron_create`) once a local
-trainer is installed. See [Reinforcement Learning](Reinforcement-Learning.md).
+- **practice** — top unresolved `Mistakes` under `Reward.judge`; auto-`resolve` with ≥2 holdouts
+- **offline_judge** — P3 backfill of ORM/PRM labels + W3 calibration from PLAN `p(success)=` so `:failure_only` local introspect does not starve the corpus
+- **train** — export SFT + balanced DPO, LoRA-train `pwn-vN+1`, replay Mistakes.top, promote only on win. `dry_run: false` only with a trainer+GPU
+- **consolidate** — M1/M3 memory GC so the injected MEMORY block stays high-signal
+
+See [Reinforcement Learning](Reinforcement-Learning.md).
 
 `cron_disable(id:)` turns either off; `install_defaults` is idempotent and
 never overwrites a job you already have with the same name.

data/documentation/Reinforcement-Learning.md

@@ -1,10 +1,12 @@
 # Reinforcement Learning in pwn-ai
 
-pwn-ai implements a **six-tier** in-context → weight-level RL loop that
-surpasses every published LLM-agent harness (Reflexion, Voyager, ExpeL,
-LATS, Generative Agents, PRM, RLAIF, DPO, Self-Refine, Toolformer, DSPy)
-on at least one axis, and on **five axes simultaneously** for the full
-`Curriculum.practice → Reward.export_dpo → Curriculum.train_and_gate` path.
+pwn-ai implements a **six-tier** in-context → weight-export RL loop that
+combines ORM/PRM, preference ledger, mistake curriculum, env-drift blame,
+balanced DPO export, and regression-gated LoRA promotion. On hosts **with a
+trainer + GPU**, the full
+`Curriculum.practice → Reward.export_dpo → Curriculum.train_and_gate`
+path closes the weight loop; without a trainer the path is **export-ready**
+(datasets + manual CLI) and the live learning is in-context only.
 
 ![Reinforcement-learning loop](diagrams/reinforcement-learning.svg)
 
@@ -50,7 +52,7 @@ on at least one axis, and on **five axes simultaneously** for the full
 |----|-------|------|
 | **C1** | `Registry.rank` + `Metrics.{ucb,thompson,advantage}` | score = α·keyword_sim + β·advantage + γ·UCB1. Untried tools get exploration bonus. |
 | **C2** | `Learning.exemplars_for` | priority = judge_score × e^(−Δt/30d) × keyword_sim. |
-| **C3** | `Curriculum.hindsight` | HER — relabel failed trajectory with achieved-goal as `success:true`. |
+| **C3** | `Curriculum.hindsight` | HER — relabel failed trajectory with achieved-goal as `success: 'soft'` + tags `hindsight/her/soft`. Soft rows are excluded from SFT and 0.35×-weighted in C2 exemplars. |
 | **C4** | `Learning.{compress_exemplar,build_skill_from_session}` | keep only `step_reward > 0` — minimal sufficient trace. |
 
 ## Tier 3 — Memory that stays high-signal
@@ -75,8 +77,8 @@ on at least one axis, and on **five axes simultaneously** for the full
 
 | ID | Where | What |
 |----|-------|------|
-| **W1** | `Reward.{record_preference,export_dpo}` | 5 free preference sources: user_correction, mistakes_resolve, counterfactual, curriculum, critic. |
-| **W2** | `Curriculum.train_and_gate` | SFT+DPO → unsloth/axolotl LoRA → `ollama create pwn-vN+1` → replay `Mistakes.top` on vN vs vN+1 → promote iff `resolved(N+1) > resolved(N)`. |
+| **W1** | `Reward.{record_preference,export_dpo}` | 5 free preference sources: user_correction, mistakes_resolve, counterfactual, curriculum, critic. `export_dpo` enforces ≤40% per-source (`DPO_SOURCE_CAP`); pass `balance: false` for a raw dump. |
+| **W2** | `Curriculum.train_and_gate` | SFT+DPO → unsloth/axolotl LoRA → `ollama create pwn-vN+1` → replay `Mistakes.top` on vN vs vN+1 → promote iff `resolved(N+1) > resolved(N)`. **Without a trainer: export-only** (`weight_loop: :export_ready`). |
 | **W3** | `Curriculum.calibrate` + `Metrics.{record_calibration,calibration}` | plan_first `p(success)` vs actual → per-engine Brier/overconfidence. |
 
 ## Tier 6 — Deepen the intro↔extro join
@@ -91,32 +93,37 @@ on at least one axis, and on **five axes simultaneously** for the full
 
 ```yaml
 :ai:
+  :module_reflection: false  # gates Reflect lesson writing (not ORM alone)
   :agent:
-    :critic: true            # S3
-    :red_team_plan: true     # S4
-    :counterfactual: true    # S2
-    :hindsight: true         # C3 (default true)
-    :verify_as_reward: true  # E3
+    :critic: null            # S3 — nil = ON for remote engines, OFF for ollama
+    :red_team_plan: null     # S4 — same auto policy
+    :counterfactual: null    # S2 — same auto policy
+    :hindsight: true         # C3 (default true; soft-success, 0.35× in C2)
+    :verify_as_reward: null  # E3 — nil = auto (~10% local / always remote on CLAIM_RX)
+    :reward_llm: null        # nil = ORM/PRM use LLM teacher on remote even if module_reflection is false
+    :local_introspect: :failure_only   # ollama cost policy; remote always introspects
+    :introspect_every_n: 3
 ```
 
 ## Cron self-improvement
 
 ```ruby
-PWN::Cron.create(name: 'self_play',   schedule: '0 3 * * *',
-  ruby: 'PWN::AI::Agent::Curriculum.practice(limit: 5)')
-PWN::Cron.create(name: 'weight_loop', schedule: '0 4 * * 0',
-  ruby: 'PWN::AI::Agent::Curriculum.train_and_gate(dry_run: false)')
-PWN::Cron.create(name: 'mem_gc',      schedule: '0 5 * * *',
-  ruby: 'PWN::AI::Agent::Learning.consolidate')
+# Seeded idempotently by PWN::Cron.install_defaults (pwn setup --migrate):
+PWN::Cron.install_defaults
+# → curriculum_practice_nightly   0 3  * * *  Curriculum.practice(limit: 3)
+# → curriculum_offline_judge     30 3  * * *  Curriculum.offline_judge(since_hours: 24, limit: 40)
+# → curriculum_train_weekly       0 4  * * 0  Curriculum.train_and_gate(dry_run: true)  # false only with trainer+GPU
+# → learning_consolidate_nightly  0 5  * * *  Learning.consolidate
 ```
 
 ## Tools exposed to the model
 
 `reward_judge` · `reward_prm` · `reward_sentinel` · `reward_preferences` ·
 `reward_export_dpo` · `curriculum_practice` · `curriculum_train` ·
-`curriculum_hindsight` · `learning_purge_noise`
+`curriculum_hindsight` · `curriculum_offline_judge` ·
+`curriculum_preference_balance` · `learning_purge_noise`
 
-## What no other harness does simultaneously
+## Design claims (architecture — weight promotion requires a trainer)
 
 1. **Process reward on real security tool traces** (R2)
 2. **Automatic blame attribution** self vs env-drift via CUSUM×correlate (E1+E2)
@@ -124,6 +131,21 @@ PWN::Cron.create(name: 'mem_gc',      schedule: '0 5 * * *',
 4. **Mistake-driven curriculum with regression-gated LoRA promotion** (S1+W2)
 5. **Five naturally-generated DPO sources** with zero human labelling (W1)
 
+
+## Operational controls (priority fixes)
+
+| ID | Control | What |
+|----|---------|------|
+| **P1** | `Curriculum.practice` cooldown + natural prompts | Hard-skips `reward_signal` / parked / `needs_code_change`; N-night zero-score cooldown parks thrash; reproducers are natural user tasks, never signature dumps. |
+| **P2** | R4 `semantic_ok` + structured resolve | `31f1871b8a15`-class exit≠0 phantoms stay closed via structured holdouts. |
+| **P3** | `Curriculum.offline_judge` | Scores last-24h sessions under ORM/PRM so local `:failure_only` introspect does not starve labels. Cron nightly. |
+| **P4** | `Reward.proxy_distrust` | When sentinel fires, Metrics.to_context / Registry.rank haircut proxy rates — actionable, not just another Mistakes row. |
+| **R3** | `Reward.sentinel` ring buffer | Fixed-N (`SENTINEL_WINDOW=40`) `{judge,proxy}` window replaces decaying `proxy_sum`/`proxy_n`. Means are always ∈[0,1]; `set_proxy_distrust` refuses proxy∉[0,1]; `reset_sentinel` wipes corrupt state without touching prefs. Legacy decay×`to_i` files auto-clear stuck distrust on load. |
+| **P5** | `Curriculum.preference_balance` + `export_dpo` source-cap | Surfaces W1 monoculture **and enforces** ≤40% per source at export (`DPO_SOURCE_CAP`); critic/counterfactual auto-ON for remote engines so the diet rebalances online. |
+| **P6** | W2 honesty | Docs + `train_and_gate` return `weight_loop: :export_ready` when `trainer: null`. |
+| **P7** | W3 as controller | Engine Brier > 0.35 or overconfidence > 0.25 (n≥8) → force plan_first + critic, cap max_iters at 12. `offline_judge` also records calibration from PLAN `p(success)=` so the controller can fire under `:failure_only`. |
+| **P8** | Remote reward teacher | `agent.reward_llm` nil → ORM/PRM use the LLM teacher on remote engines even when `module_reflection` is false. Local ollama stays heuristic unless explicitly enabled. PRM prompts carry R4 tags so benign recon exits score 0 not −1. |
+
 **See also:** [Skills, Memory & Learning](Skills-Memory-Learning.md) ·
 [Mistakes](Mistakes.md) · [Cron](Cron.md) · [pwn-ai Agent](pwn-ai-Agent.md)