pwn 0.5.642 → 0.5.647

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 191435688dbf322cbae6cc09116253b08831fce203ebdded672e6b19021a30ff
-  data.tar.gz: 8ecb562e1d87bf8ad036e8ced596bdd37ab76dc24464a3a2dbb6133011b263c9
+  metadata.gz: b6227e3d61730e19d8ec946eaf02247f6222412204cf1b1977fdc473c9bb83e3
+  data.tar.gz: e2fbd14507544beeecc70a7894ba0fcb26263acca09e892ddce16699ce892ce7
 SHA512:
-  metadata.gz: 27b7a67355852e7a017016acdc3e10a6929c7d679d37aff6a7d0b183e347480b340ddd9fceb39cc70f8d1c77e13eeb6d49685213a9678576cfd4f921828d9cfc
-  data.tar.gz: 846879b068546b96f8b40e54a5b454b7234823612403a4eb4bfef3be42c1682e1ef219a85097c72f6edbdf8db23905cfef38ccf8fb615302cd6ecdc489a93b83
+  metadata.gz: cd7cd23daf97bc556f97d7ed58323268d414e1a4ef119b4f6b562f0d324d808dcc7cb1be4fb2708fa957332cc42e5daaa4b410a4756afb40f6520b97e85fd56e
+  data.tar.gz: 7e1576438befe49d01f189aeebd71dac1940f95273c9933c7f51feb0fb36423ee372bfed02e43e5df9558ed971b84606eed1d3d5f82c534cc656ed307618700c

data/Gemfile

@@ -49,7 +49,7 @@ gem 'jwt', '3.2.0'
 gem 'libusb', '0.7.2'
 gem 'luhn', '3.0.0'
 gem 'mail', '2.9.1'
-gem 'mcp', '0.25.0'
+gem 'mcp', '1.0.0'
 gem 'meshtastic', '0.0.172'
 gem 'metasm', '1.0.6'
 gem 'mongo', '2.25.0'

data/README.md

@@ -128,7 +128,7 @@ $ pwn setup --profile net --dry-run
 Also available as `pwn_setup` (standalone driver) and `pwn --setup[=PROFILE]`.
 The doctor exits non-zero when capabilities are degraded, so CI can gate on it.
 
-[![Installing the pwn Security Automation Framework](https://raw.githubusercontent.com/0dayInc/pwn/master/documentation/pwn_install.png)](https://youtu.be/G7iLUY4FzsI)
+<!--[![Installing the pwn Security Automation Framework](https://raw.githubusercontent.com/0dayInc/pwn/master/documentation/pwn_install.png)](https://youtu.be/G7iLUY4FzsI)-->
 
 Full page: [Installation](documentation/Installation.md) ·
 [Configuration](documentation/Configuration.md)

data/documentation/Cron.md

@@ -37,13 +37,14 @@ invokes `PWN::Cron.run(<id>)` on schedule.
 | `learning_consolidate_nightly` | `0 5 * * *` | `PWN::AI::Agent::Learning.consolidate` |
 
 - **practice** — top unresolved `Mistakes` under `Reward.judge`; auto-`resolve` with ≥2 holdouts
-- **offline_judge** — P3 backfill of ORM/PRM labels + W3 calibration from PLAN `p(success)=` so `:failure_only` local introspect does not starve the corpus
+- **offline_judge** — P3 backfill of ORM/PRM labels + W3 calibration from PLAN `p(success)=` so `:failure_only` local introspect does not starve the corpus; also runs **P10** `Reward.warm_sentinel` so the R3 ring can fill on local hosts
+- **dedupe (P13)** — legacy alias `offline_judge_nightly` counts as the same job; `install_defaults` will not double-seed `30 3 * * *` and disables the alias when both exist
 - **train** — export SFT + balanced DPO, LoRA-train `pwn-vN+1`, replay Mistakes.top, promote only on win. `dry_run: false` only with a trainer+GPU
 - **consolidate** — M1/M3 memory GC so the injected MEMORY block stays high-signal
 
 See [Reinforcement Learning](Reinforcement-Learning.md).
 
-`cron_disable(id:)` turns either off; `install_defaults` is idempotent and
+`cron_disable(id:)` turns either off; `install_defaults` is idempotent (including P13 alias collapse) and
 never overwrites a job you already have with the same name.
 
 ## Example

data/documentation/Home.md

@@ -39,7 +39,7 @@
 | [Agent Tool Registry](Agent-Tool-Registry.md) | 10 toolsets · **71** LLM-callable tools |
 | [Memory · Skills · Learning](Skills-Memory-Learning.md) | Introspection - the self-improvement loop |
 | [Mistakes](Mistakes.md) | **Negative feedback** - fingerprint failures · do-NOT-repeat · `[REPEATING]`/`[REGRESSED]` · inline self-correction |
-| [Reinforcement Learning](Reinforcement-Learning.md) | **`Reward` + `Curriculum`** - ORM/PRM judge · sentinel · HER · self-play · DPO export · **regression-gated LoRA** |
+| [Reinforcement Learning](Reinforcement-Learning.md) | **`Reward` + `Curriculum`** - ORM/PRM · P9/P14 trajectory DPO · P15 scrub · P16 warm · P17 budget skill · P18 PRM→rank · P19 diet gate · export-ready LoRA |
 | [Extrospection](Extrospection.md) | World-awareness - snapshot · drift · intel · **watch** · **verify** · **rf_tune** · **osint** · serial · telecomm · packet · vision · voice · correlate |
 | [Swarm (Multi-Agent)](Swarm.md) | Personas · ask · debate · broadcast · shared bus |
 | [Sessions](Sessions.md) | Transcript persistence + reflection |

data/documentation/Mistakes.md

@@ -99,6 +99,19 @@ drift so the agent can distinguish "I called the API wrong" from "nmap was
 upgraded", "the HackRF was unplugged", or "the target DOM moved". See
 [Extrospection](Extrospection.md#extro_correlate--the-point-of-the-whole-thing).
 
+
+## Preference pairs from resolve (W1 / P9)
+
+`Mistakes.resolve` writes a DPO preference into `~/.pwn/preferences.jsonl`:
+
+- **rejected** — the failing snippet / action
+- **chosen** — prefers `structured_fix.winning_trace` (+ strategy/tool) when
+  present; falls back to `AVOID_ARGS` / fix prose only when no trace exists
+
+Curriculum auto-resolve always attaches a winning tool trace so the ledger
+learns **trajectories**, not commentary. Write-time source quotas (P9) still
+apply to pure prose resolves so resolve-monoculture cannot flood the ledger.
+
 **See also:** [Skills, Memory & Learning](Skills-Memory-Learning.md) ·
 [Extrospection](Extrospection.md) · [Persistence](Persistence.md) ·
 [pwn-ai Agent](pwn-ai-Agent.md)

data/documentation/Reinforcement-Learning.md

@@ -77,8 +77,8 @@ path closes the weight loop; without a trainer the path is **export-ready**
 
 | ID | Where | What |
 |----|-------|------|
-| **W1** | `Reward.{record_preference,export_dpo}` | 5 free preference sources: user_correction, mistakes_resolve, counterfactual, curriculum, critic. `export_dpo` enforces ≤40% per-source (`DPO_SOURCE_CAP`); pass `balance: false` for a raw dump. |
-| **W2** | `Curriculum.train_and_gate` | SFT+DPO → unsloth/axolotl LoRA → `ollama create pwn-vN+1` → replay `Mistakes.top` on vN vs vN+1 → promote iff `resolved(N+1) > resolved(N)`. **Without a trainer: export-only** (`weight_loop: :export_ready`). |
+| **W1** | `Reward.{record_preference,export_dpo}` | 5 free preference sources: user_correction, mistakes_resolve, counterfactual, curriculum, critic. Write-time `WRITE_SOURCE_CAP` + export `DPO_SOURCE_CAP` (≤40%). Trajectory shapes (`winning_trace` / `revised_answer` / `real_dispatch`) force-land. |
+| **W2** | `Curriculum.train_and_gate` | SFT+DPO → unsloth/axolotl LoRA → `ollama create pwn-vN+1` → **gate v2** (resolved margin + mean judge + smoke). **Without a trainer: export-only** (`weight_loop: :export_ready`). |
 | **W3** | `Curriculum.calibrate` + `Metrics.{record_calibration,calibration}` | plan_first `p(success)` vs actual → per-engine Brier/overconfidence. |
 
 ## Tier 6 — Deepen the intro↔extro join
@@ -118,7 +118,8 @@ PWN::Cron.install_defaults
 
 ## Tools exposed to the model
 
-`reward_judge` · `reward_prm` · `reward_sentinel` · `reward_preferences` ·
+`reward_judge` · `reward_prm` · `reward_sentinel` · `reward_warm_sentinel` · `reward_preferences` ·
+`reward_scrub_preferences` · `reward_preference_balance` ·
 `reward_export_dpo` · `curriculum_practice` · `curriculum_train` ·
 `curriculum_hindsight` · `curriculum_offline_judge` ·
 `curriculum_preference_balance` · `learning_purge_noise`
@@ -145,6 +146,39 @@ PWN::Cron.install_defaults
 | **P6** | W2 honesty | Docs + `train_and_gate` return `weight_loop: :export_ready` when `trainer: null`. |
 | **P7** | W3 as controller | Engine Brier > 0.35 or overconfidence > 0.25 (n≥8) → force plan_first + critic, cap max_iters at 12. `offline_judge` also records calibration from PLAN `p(success)=` so the controller can fire under `:failure_only`. |
 | **P8** | Remote reward teacher | `agent.reward_llm` nil → ORM/PRM use the LLM teacher on remote engines even when `module_reflection` is false. Local ollama stays heuristic unless explicitly enabled. PRM prompts carry R4 tags so benign recon exits score 0 not −1. |
+| **P9** | W1 pair geometry + write-time quota | Critic chosen = **revised full answer** (not `CORRECTION:` flaw prose). Resolve prefers `structured_fix.winning_trace`. Counterfactual tags `:real_dispatch` vs `:imagined`. `record_preference` enforces `WRITE_SOURCE_CAP=0.40` online (trajectory shapes force-land). |
+| **P10** | `Reward.warm_sentinel` | Backfills R3 ring from scored Learning outcomes so local `:failure_only` hosts reach `SENTINEL_WINDOW` without waiting for live remote introspect. `offline_judge` calls it every night. |
+| **P11** | W2 gate v2 | Promote only when candidate wins on resolved margin **and** mean judge **and** frozen smoke set (uname/pwd/ruby) does not regress. Coarse `resolved(N+1)>resolved(N)` alone cannot self-promote eval memorisation. |
+| **P12** | SFT quality gate | `export_finetune` drops HER/soft + score<`SFT_MIN_SCORE` (0.6), de-dupes per session by best score, PRM-compresses traces (`compress_finetune_trace`). |
+| **P13** | Cron offline_judge dedupe | `install_defaults` treats `offline_judge_nightly` as alias of `curriculum_offline_judge` and disables duplicates so the 30 3 * * * slot runs once. |
+| **P14** | Practice → DPO geometry | `Curriculum.practice` records `chosen: winning_trace` (+ final), `shape: :winning_trace` — never first-3-lines fix prose. |
+| **P15** | Ledger hygiene | `Reward.usable_preference?` / `scrub_preferences` / geometry filter in `export_dpo` drop `CORRECTION:` prose, resolve-without-trace, and chosen≪rejected. `preference_balance(scrub:true)` reports `trajectory_fraction`. |
+| **P16** | R3 warm for real | `warm_sentinel` fills from scored **and** success-boolean outcomes, returns `proxy_distrust`, runs `sentinel` once full so controllers can engage. |
+| **P17** | Budget-exhaustion skill | Practice prioritises `agent_loop`/`assistant_answer`; natural prompts teach finish-under-N; Loop hard-stops empty-final thrash and tightens max_iters when budget fingerprints dominate. |
+| **P18** | PRM → controller | `Metrics.record_step_reward` + `prm_advantage`; `Registry.rank` adds `δ·prm_advantage` so R2 biases live tool choice. |
+| **P19** | Promote diet gate | `train_and_gate` refuses promote unless scrubbed W1 diet has ≥12 pairs, no monoculture, max source ≤45%, trajectory_fraction ≥30%. Export-only stays correct otherwise. |
+| **P20** | Judge-blended Metrics | `Metrics.record_judge` / `judge_rate` / `effective_rate` fold episode ORM into per-tool UCB/Thompson/advantage when `proxy_distrust>0`. `exemplars_for` drops success rows with score<0.6 so proxy-true/judge-low cannot be few-shot. |
+| **P21** | Resolve trajectory-only | `Mistakes.resolve` writes W1 only when `structured_fix.winning_trace` ≥40 chars (`shape: :winning_trace`). Prose-only resolve still updates Memory + structured_fix; it does **not** flood DPO. |
+| **P22** | W3 calibration live | `Loop.plan_first` stashes `p(success)=` on `Thread.current[:pwn_plan_predicted]`; `Learning.recover_predicted_from_session` + `Curriculum.calibrate` light the Brier/overconfidence controller under `:failure_only`. |
+| **P23** | Short-horizon budget practice | Budget-exhaustion fingerprints get natural "finish under N" prompts; practice auto-resolve requires N≥2 holdouts at judge≥0.7 **and** a real winning_trace (no long-prose resolve). |
+| **P24** | Critic cost under budget-hot | When `budget_exhaustion_hot?`, `auto_introspect` forces `critic(text_only: true)` — single Reflect/heuristic shot, no tool-armed persona swarm that burns the remaining iteration budget. |
+| **P25** | Write-time trajectory gate | `Reward.record_preference` refuses non-`TRAJECTORY_SHAPES` unless `force:` / `user_correction`. Write-time source quota **still** applies to trajectory pairs so winning_trace cannot re-monoculture the ledger. |
+
+## Preference / trajectory signal quality (current ops posture)
+
+The in-context loop (R4 + Mistakes + prompt injection) is production-ready.
+The **weight path** is export-ready and gated; its bottleneck is pair geometry
+and source diversity, not missing tiers:
+
+1. Write-time + export caps keep resolve-monoculture from teaching "emit fix prose".
+2. Critic/resolve/counterfactual/**practice** emit **trajectory-shaped** chosen sides (P9/P14).
+3. P15 scrub + export geometry filter purge historical prose flood before DPO/LoRA.
+4. R3/W3 warm via `offline_judge` + `warm_sentinel` so controllers are not permanently cold on ollama hosts (P10/P16).
+5. SFT is filtered as hard as DPO; gate v2 needs smoke + mean, not count alone; P19 also requires trajectory diet.
+6. Budget-exhaustion is a first-class curriculum target (P17/P23); PRM step_reward biases `Registry.rank` (P18); critic is text-only when budget-hot (P24).
+7. ORM judge blends into Metrics/UCB when distrust is high (P20); W3 calibration lights from plan_first `p(success)=` (P22).
+8. Resolve/record_preference are trajectory-only at write time (P21/P25); write-time source quota still caps traj monoculture.
+9. Without unsloth/axolotl **or** a clean preference diet, `train_and_gate` stays `weight_loop: :export_ready` (honest).
 
 **See also:** [Skills, Memory & Learning](Skills-Memory-Learning.md) ·
 [Mistakes](Mistakes.md) · [Cron](Cron.md) · [pwn-ai Agent](pwn-ai-Agent.md)

data/documentation/Skills-Memory-Learning.md

@@ -39,8 +39,10 @@ its own performance, turns wins into permanent capability, and - critically -
                                        frontier engine may WRITE the lesson a local engine READS
 8. A whole workflow succeeded         → Learning.distill_skill(name, session_id, references:)
 9. Found a fix for a mistake          → mistakes_resolve(sig, fix) → Memory :lesson "AVOID X - FIX: Y"
-10. (weekly, cron) Learning.export_finetune → ~/.pwn/finetune/*.jsonl → LoRA over the local
-     model - the ONLY step that changes weights, not just the scaffold.
+10. (weekly, cron) Learning.export_finetune (P12: min_score + PRM-compress) +
+     Reward.export_dpo (≤40%/src, P15 geometry scrub) → ~/.pwn/finetune/*.jsonl → LoRA over the local
+     model via Curriculum.train_and_gate (P11 gate v2) — the ONLY step that
+     changes weights, not just the scaffold. Without a trainer this stays export-ready.
 11. Next launch: PromptBuilder injects the budgeted blocks → the model already knows:
      MEMORY · SKILLS · LEARNING · KNOWN MISTAKES/FIXES · TOOL EFFECTIVENESS · EXTROSPECTION
 ```
@@ -103,7 +105,7 @@ the new format.
 | `skill_delete(name)` | auto-distilled skill turned out low-quality |
 | `learning_auto_introspect_toggle(enabled: false)` | during noisy fuzz loops |
 | `PWN::MemoryIndex.reset` | new engagement - drop the local embedding index (`memory.idx`) so it rebuilds against the fresh `memory.json` |
-| `PWN::AI::Agent::Learning.export_finetune(format: :sharegpt)` | you have enough successful sessions to cut a supervised dataset for the local model |
+| `PWN::AI::Agent::Learning.export_finetune(format: :sharegpt)` | you have enough **high-score** sessions for SFT (P12 drops score<0.6 / HER/soft and PRM-compresses traces) |
 
 ## Example questions that trigger Introspection
 

data/documentation/diagrams/dot/pwn-ai-feedback-learning-loop.dot

@@ -34,11 +34,11 @@ Escalate [label="escalate\n≥N fails → Swarm hint",        fillcolor="#c4b5fd
     label="INTROSPECTION  (self)"; fontcolor="#a7f3d0";
     style=rounded; color="#047857"; bgcolor="#022c22"; penwidth=2;
     Metrics  [label="Metrics\nper-tool · PER-ENGINE\nsuccess · avg ms", fillcolor="#6ee7b7"];
-    Learning [label="Learning\nnote_outcome · reflect\nexemplars_for · export_finetune\nfact_check_local_final", fillcolor="#6ee7b7"];
+    Learning [label="Learning\nnote_outcome · reflect\nexemplars_for · SFT P12 gate\nfact_check_local_final", fillcolor="#6ee7b7"];
     Mistakes [label="Mistakes\nrecord · resolve\nguard · correction_hint\n[REPEATING] · [REGRESSED]", fillcolor="#6ee7b7", penwidth=2];
     ReflectT [label="Reflect\nteacher-student\n(reflect_engine)", fillcolor="#6ee7b7", penwidth=2];
-    Reward   [label="Reward\njudge (ORM) · prm (PRM)\nsentinel ring · semantic_ok\nexport_dpo ≤40%/src", fillcolor="#6ee7b7", penwidth=2];
-    Curric   [label="Curriculum\npractice · offline_judge\ncounterfactual · critic\nhindsight · train_and_gate", fillcolor="#6ee7b7", penwidth=2];
+    Reward   [label="Reward\njudge · prm · semantic_ok\nsentinel warm (P10/P16)\nDPO P9/P14/P15 scrub ≤40%", fillcolor="#6ee7b7", penwidth=2];
+    Curric   [label="Curriculum\npractice P14/P17 · offline_judge\ncritic/CF geometry · gate v2+P19 diet", fillcolor="#6ee7b7", penwidth=2];
   }
   subgraph cluster_extro {
     label="EXTROSPECTION  (world)"; fontcolor="#fde68a";

data/documentation/diagrams/dot/reinforcement-learning.dot

@@ -20,8 +20,8 @@ digraph "PWN_Reinforcement_Learning" {
     label="Tier 1 · Reward  (PWN::AI::Agent::Reward)"; fontcolor="#ddd6fe";
     style=rounded; color="#6d28d9"; bgcolor="#2e1065"; penwidth=2;
     R1 [label="R1  judge (ORM)\nscore · verdict · rationale", fillcolor="#c4b5fd", penwidth=2];
-    R2 [label="R2  prm (PRM)\nper-step +1/0/−1", fillcolor="#c4b5fd"];
-    R3 [label="R3  sentinel (ring N=40)\nproxy ≠ judge ≠ (1−corr)\n→ proxy_distrust haircut", fillcolor="#c4b5fd"];
+    R2 [label="R2  prm (PRM)\nper-step +1/0/−1\n→ Registry.rank (P18)", fillcolor="#c4b5fd"];
+    R3 [label="R3  sentinel (ring N=40)\nproxy ≠ judge ≠ (1−corr)\nwarm full (P10/P16) + distrust", fillcolor="#c4b5fd"];
     R4 [label="R4  semantic_ok\ngrep exit 1 ≠ failure", fillcolor="#c4b5fd"];
     E3 [label="E3  verify_as_reward\nextro_verify caps/floors judge", fillcolor="#c4b5fd"];
   }
@@ -31,9 +31,9 @@ digraph "PWN_Reinforcement_Learning" {
   subgraph cluster_curr {
     label="Tier 4 · Curriculum  (PWN::AI::Agent::Curriculum)"; fontcolor="#a7f3d0";
     style=rounded; color="#047857"; bgcolor="#022c22"; penwidth=2;
-    S1 [label="S1  practice\nMistakes.top → self-play\nauto-resolve on judge≥0.7", fillcolor="#6ee7b7", penwidth=2];
-    S2 [label="S2  counterfactual\nfork alt-persona A/B", fillcolor="#6ee7b7"];
-    S3 [label="S3  critic\ntool-armed constitutional", fillcolor="#6ee7b7"];
+    S1 [label="S1  practice (P14/P17)\nbudget-first + winning_trace DPO\nauto-resolve on judge≥0.7", fillcolor="#6ee7b7", penwidth=2];
+    S2 [label="S2  counterfactual\nreal_dispatch vs imagined", fillcolor="#6ee7b7"];
+    S3 [label="S3  critic\ntool-armed · revised_answer DPO", fillcolor="#6ee7b7"];
     S4 [label="S4  red_team_plan\nadversarial plan review", fillcolor="#6ee7b7"];
     C3 [label="C3  hindsight (HER)\nrelabel achieved-goal", fillcolor="#6ee7b7"];
   }
@@ -43,8 +43,8 @@ digraph "PWN_Reinforcement_Learning" {
   subgraph cluster_weight {
     label="Tier 5 · Weight Loop"; fontcolor="#fecaca";
     style=rounded; color="#b91c1c"; bgcolor="#450a0a"; penwidth=2;
-    W1 [label="W1  preference ledger\n(prompt, rejected, chosen)\n5 free sources\nexport_dpo ≤40%/source", fillcolor="#fda4af"];
-    W2 [label="W2  train_and_gate\nSFT+DPO → LoRA vN+1\nreplay Mistakes.top vN vs vN+1\npromote iff resolved(N+1) > (N)", fillcolor="#fda4af", penwidth=2];
+    W1 [label="W1  preference ledger\ntrajectory pairs (P9/P14)\nscrub+geometry (P15)\nwrite+export ≤40%/src", fillcolor="#fda4af"];
+    W2 [label="W2  train_and_gate\nSFT(P12)+DPO → LoRA vN+1\ngate v2 + diet (P11/P19)\nexport_ready without trainer", fillcolor="#fda4af", penwidth=2];
     W3 [label="W3  calibrate\np(success) vs actual → Brier", fillcolor="#fda4af"];
   }
   {rank=same; W1; W2; W3}
@@ -95,7 +95,7 @@ digraph "PWN_Reinforcement_Learning" {
 
   /* cron seeds */
   Cron [label="⏱  PWN::Cron.install_defaults\npractice · offline_judge\ntrain dry_run · consolidate", fillcolor="#7dd3fc"];
-  OJ [label="P3  offline_judge\nORM/PRM + W3 cal from PLAN", fillcolor="#6ee7b7"];
+  OJ [label="P3  offline_judge\nORM/PRM + W3 cal + P10 warm", fillcolor="#6ee7b7"];
   MemGC [label="M1/M3  Learning.consolidate\nnightly memory GC", fillcolor="#6ee7b7"];
   Cron -> S1 [label="0 3 * * *", color="#38bdf8", constraint=false];
   Cron -> OJ [label="30 3 * * *", color="#38bdf8", constraint=false];