pwn 0.5.635 → 0.5.637

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (55) hide show
  1. checksums.yaml +4 -4
  2. data/Rakefile +6 -0
  3. data/bin/pwn_setup +27 -0
  4. data/documentation/Agent-Tool-Registry.md +12 -5
  5. data/documentation/CLI-Drivers.md +37 -31
  6. data/documentation/Configuration.md +16 -3
  7. data/documentation/Contributing.md +9 -2
  8. data/documentation/Cron.md +21 -1
  9. data/documentation/Diagrams.md +6 -2
  10. data/documentation/Drivers.md +1 -1
  11. data/documentation/General-PWN-Usage.md +7 -1
  12. data/documentation/Home.md +8 -7
  13. data/documentation/How-PWN-Works.md +31 -17
  14. data/documentation/Installation.md +86 -29
  15. data/documentation/Persistence.md +34 -8
  16. data/documentation/Plugins.md +5 -5
  17. data/documentation/Reinforcement-Learning.md +7 -0
  18. data/documentation/Sessions.md +6 -2
  19. data/documentation/Skills-Memory-Learning.md +26 -10
  20. data/documentation/Troubleshooting.md +23 -0
  21. data/documentation/What-is-PWN.md +12 -6
  22. data/documentation/Why-PWN.md +3 -2
  23. data/documentation/diagrams/agent-tool-registry.svg +105 -102
  24. data/documentation/diagrams/cron-scheduling.svg +137 -78
  25. data/documentation/diagrams/dot/agent-tool-registry.dot +4 -4
  26. data/documentation/diagrams/dot/cron-scheduling.dot +25 -9
  27. data/documentation/diagrams/dot/history-to-drivers.dot +1 -1
  28. data/documentation/diagrams/dot/memory-skills-detailed.dot +10 -4
  29. data/documentation/diagrams/dot/overall-pwn-architecture.dot +2 -2
  30. data/documentation/diagrams/dot/persistence-filesystem.dot +18 -11
  31. data/documentation/diagrams/dot/pwn-ai-feedback-learning-loop.dot +19 -4
  32. data/documentation/diagrams/dot/reinforcement-learning.dot +102 -0
  33. data/documentation/diagrams/history-to-drivers.svg +6 -6
  34. data/documentation/diagrams/memory-skills-detailed.svg +233 -198
  35. data/documentation/diagrams/overall-pwn-architecture.svg +168 -168
  36. data/documentation/diagrams/persistence-filesystem.svg +212 -136
  37. data/documentation/diagrams/pwn-ai-feedback-learning-loop.svg +386 -283
  38. data/documentation/diagrams/reinforcement-learning.svg +392 -0
  39. data/documentation/pwn-REPL.md +33 -4
  40. data/documentation/pwn-ai-Agent.md +42 -30
  41. data/lib/pwn/config.rb +42 -20
  42. data/lib/pwn/migrate.rb +716 -0
  43. data/lib/pwn/plugins/repl.rb +78 -15
  44. data/lib/pwn/setup.rb +205 -6
  45. data/lib/pwn/version.rb +1 -1
  46. data/lib/pwn.rb +1 -0
  47. data/pwn.gemspec +2 -0
  48. data/spec/documentation/installation_md_spec.rb +2 -2
  49. data/spec/lib/pwn/ai/agent/mistakes_spec.rb +2 -0
  50. data/spec/lib/pwn/migrate_spec.rb +136 -0
  51. data/spec/spec_helper.rb +35 -0
  52. data/spec/support/sandbox.rb +6 -0
  53. data/third_party/pwn_rdoc.jsonl +26 -0
  54. data/third_party/terminator/pwn_shift_enter.py +137 -0
  55. metadata +8 -3
@@ -59,9 +59,9 @@ Prints a per-capability report and **exits non-zero if anything is degraded**
59
59
  (so you can gate CI on it):
60
60
 
61
61
  ```text
62
- PWN v0.5.628 · ruby 4.0.5 · linux x86_64 · pkg-manager: apt
62
+ PWN vCURRENT_VERSION · ruby 4.0.5 · linux x86_64 · pkg-manager: apt
63
63
 
64
- ~/.pwn/ ok (11 entries)
64
+ ~/.pwn/ ok (13 entries · schema v1)
65
65
  ~/.pwn/pwn.yaml ok (encrypted, decryptor present)
66
66
  AI engine ok anthropic (key set)
67
67
 
@@ -76,6 +76,12 @@ External toolchain used by
76
76
  gqrx MISSING PWN::SDR, extro_rf_tune
77
77
  ...
78
78
 
79
+ ~/.pwn state files owner
80
+ pwn.yaml ok PWN::Config
81
+ memory.json ok PWN::Memory
82
+ skills/ drift 1 legacy flat file(s) PWN::Config → run --migrate --fix
83
+ ...
84
+
79
85
  31 / 36 capabilities usable · 5 degraded
80
86
 
81
87
  Run `pwn setup --deps` to install missing OS headers/tools, or
@@ -113,7 +119,7 @@ pwn setup --list-profiles
113
119
  | `ai` | verify at least one AI engine key/oauth in `~/.pwn/pwn.yaml` |
114
120
  | `web` | `TransparentBrowser` · `BurpSuite` · `Zaproxy` · `extro_verify` · `extro_watch` · `sqlmap` · `tor` |
115
121
  | `net` | `NmapIt` · `Packet` · `extro_packet` · `extro_osint` · `tshark`/`tcpdump` |
116
- | `db` | `DAOPostgres` · `DAOSqlite3` · `DAOMongo` |
122
+ | `db` | `DAOPostgres` · `DAOSQLite3` · `DAOMongo` |
117
123
  | `sdr` | `PWN::SDR` · GQRX · `PWN::FFI` DSP backends · `extro_rf_tune` · rtl-sdr / hackrf / SoapySDR |
118
124
  | `vision` | `OCR` · `ScannableCodes` · `Reports` · `extro_vision` · tesseract / zbar / graphviz |
119
125
  | `voice` | `PWN::Plugins::Voice` · `extro_voice` · espeak-ng / sox |
@@ -125,7 +131,7 @@ pwn setup --list-profiles
125
131
 
126
132
  ```text
127
133
  pwn setup [--check] [--deps] [--profile NAME] [--list-profiles]
128
- [--yes] [--dry-run]
134
+ [--migrate [--fix]] [--yes] [--dry-run]
129
135
  ```
130
136
 
131
137
  | Flag | Meaning |
@@ -134,11 +140,63 @@ pwn setup [--check] [--deps] [--profile NAME] [--list-profiles]
134
140
  | `-d`, `--deps` | Install OS packages + rebuild native gems for `--profile` (default `full`). |
135
141
  | `-p`, `--profile NAME` | One of the profiles above. Implies `--deps`. |
136
142
  | `-l`, `--list-profiles` | Print profile table and exit. |
137
- | `-y`, `--yes` | Assume yes; non-interactive (CI / Docker / Packer). |
143
+ | **`-m`, `--migrate`** | **Verify every `~/.pwn` state file against this pwn version; apply schema migrations. See below.** |
144
+ | **`-f`, `--fix`** | **With `--migrate`: also autofix incompatible files (timestamped backup taken first).** |
145
+ | `-y`, `--yes` | Assume yes; non-interactive (CI / Docker / Packer). Also implies `--fix` when combined with `--migrate`. |
138
146
  | `-n`, `--dry-run` | Print the commands that *would* run, do nothing. |
139
147
 
140
148
  ---
141
149
 
150
+ ## Upgrading — `~/.pwn` state migration (`PWN::Migrate`)
151
+
152
+ PWN persists a growing set of files under `~/.pwn` (encrypted config,
153
+ memory, learning, metrics, mistakes, extrospection, cron, agents, skills,
154
+ sessions, swarm, curriculum, finetune, …). Each file is owned by a different
155
+ module and each release can add keys or change shape. `PWN::Migrate` closes
156
+ that gap so `gem update pwn` never leaves you with a `KeyError` or a silent
157
+ empty-fallback.
158
+
159
+ ```bash
160
+ gem update pwn
161
+ pwn setup --migrate # verify + apply schema migrations (backup taken)
162
+ pwn setup --migrate --fix # also autofix incompatible files
163
+ pwn setup --migrate --dry-run # report only, write nothing
164
+ ```
165
+
166
+ What `--migrate` does:
167
+
168
+ 1. **Stamp** `~/.pwn/.schema` with `{ schema:, pwn_version:, at: }`. Any
169
+ release that changes the on-disk shape of a `~/.pwn` file bumps
170
+ `PWN::Migrate::SCHEMA_VERSION` and appends an idempotent lambda to
171
+ `PWN::Migrate::MIGRATIONS`.
172
+ 2. **Verify** every state file against its owning module's *own* loader
173
+ (`PWN::Migrate::STATE_FILES` - no re-implemented parsers). If the raw
174
+ file has bytes but the owner returned its empty-fallback, the file is
175
+ flagged incompatible.
176
+ 3. **Autofix** (with `--fix`): timestamped backup under `~/.pwn/backup/<ts>/`
177
+ → ordered schema migrations → per-file repair (quarantine corrupt to
178
+ `~/.pwn/quarantine/`, re-seed missing dirs, strip unparsable JSONL lines,
179
+ convert legacy flat skills to the [agentskills.io](https://agentskills.io)
180
+ directory layout) → **deep-merge** any keys the current
181
+ `PWN::Config.env_template` added into your encrypted `~/.pwn/pwn.yaml`
182
+ **without overwriting your values** (re-encrypted with the same key/IV).
183
+
184
+ Everything is idempotent and dry-run capable. The plain `pwn` launcher also
185
+ prints a one-line drift warning on startup whenever `~/.pwn/.schema`
186
+ predates the running gem (`PWN::Migrate.needed?`).
187
+
188
+ Schema `v1` also seeds `PWN::Cron.install_defaults` — the nightly
189
+ `curriculum_practice` and weekly `curriculum_train` self-improvement jobs
190
+ (see [Reinforcement Learning](Reinforcement-Learning.md)).
191
+
192
+ From a checkout:
193
+
194
+ ```bash
195
+ cd /opt/pwn && git pull && rake install && pwn setup --migrate --fix
196
+ ```
197
+
198
+ ---
199
+
142
200
  ## From a git checkout (contributors)
143
201
 
144
202
  ```bash
@@ -178,21 +236,6 @@ pwn setup --profile ${PWN_PROFILE:-full} --yes
178
236
 
179
237
  ---
180
238
 
181
- ## Upgrading
182
-
183
- ```bash
184
- gem update pwn # or: gem uninstall --all --executables pwn && gem install pwn
185
- pwn setup # re-doctor - new versions may add capabilities
186
- ```
187
-
188
- From a checkout:
189
-
190
- ```bash
191
- cd /opt/pwn && git pull && rake install && pwn setup
192
- ```
193
-
194
- ---
195
-
196
239
  ## First-run configuration
197
240
 
198
241
  The first `pwn` launch creates `~/.pwn/` and an **encrypted**
@@ -203,25 +246,38 @@ row as `MISSING` until a key is set.
203
246
 
204
247
  ---
205
248
 
206
- ## Programmatic API - `PWN::Setup`
249
+ ## Programmatic API - `PWN::Setup` / `PWN::Migrate`
207
250
 
208
- Everything above is a thin CLI over one autoloaded module:
251
+ Everything above is a thin CLI over two autoloaded modules:
209
252
 
210
253
  ```ruby
211
- PWN::Setup.check # → { ok:, native_gems_missing:, toolchain_missing:, pkg_manager:, os:, arch: }
254
+ PWN::Setup.check # → { ok:, native_gems_missing:, toolchain_missing:, state:, pkg_manager:, os:, arch: }
212
255
  PWN::Setup.deps(profile: :web, yes: true) # install, then re-check
213
256
  PWN::Setup.list_profiles
257
+ PWN::Setup.migrate(fix: true) # thin wrapper → PWN::Migrate.run
214
258
  PWN::Setup.pkg_manager # → { key: :apt, install: 'sudo apt-get install -y', sudo: true }
215
259
 
216
260
  # The data tables - single source of truth, versioned with the gem:
217
261
  PWN::Setup::NATIVE_GEMS # native ext → { apt:, dnf:, pacman:, brew:, port:, plugins: }
218
262
  PWN::Setup::TOOLCHAIN # external bin → { apt:, dnf:, pacman:, brew:, port:, plugins: }
219
263
  PWN::Setup::PROFILES # profile → { desc:, gems:, bins: }
264
+
265
+ # ~/.pwn state doctor / auto-migrator (lib/pwn/migrate.rb)
266
+ PWN::Migrate.needed? # cheap: schema stamp < SCHEMA_VERSION ?
267
+ PWN::Migrate.status # per-file compatibility rows (never writes)
268
+ PWN::Migrate.check # human report + { compatible:[], incompatible:[], missing:[] }
269
+ PWN::Migrate.run(fix: true, dry_run:false)# backup → ordered migrations → per-file repair → vault backfill
270
+ PWN::Migrate.vault_drift # keys env_template has that your pwn.yaml is missing
271
+ PWN::Migrate.backfill_vault # deep-merge those keys under your values, re-encrypt
272
+ PWN::Migrate::STATE_FILES # declarative registry of every ~/.pwn artefact
273
+ PWN::Migrate::SCHEMA_VERSION # bump when a state file changes shape
220
274
  ```
221
275
 
222
276
  Adding a new native dependency or wrapped binary? Add **one row** to the
223
- appropriate constant in `lib/pwn/setup.rb` - every install path (gem, git,
224
- Docker, Packer, Vagrant, CI) picks it up automatically.
277
+ appropriate constant in `lib/pwn/setup.rb`. Adding a new persisted state
278
+ file under `~/.pwn`? Add **one entry** to `PWN::Migrate::STATE_FILES` in
279
+ `lib/pwn/migrate.rb` - every install path (gem, git, Docker, Packer,
280
+ Vagrant, CI) picks it up automatically.
225
281
 
226
282
  ---
227
283
 
@@ -229,11 +285,12 @@ Docker, Packer, Vagrant, CI) picks it up automatically.
229
285
 
230
286
  ```ruby
231
287
  pwn[CURRENT_VERSION]:001 >>> PWN::Setup.check[:ok] # => true
232
- pwn[CURRENT_VERSION]:002 >>> PWN::Plugins.constants.count # => 66
233
- pwn[CURRENT_VERSION]:003 >>> PWN::SAST.constants.count # => 48
234
- pwn[CURRENT_VERSION]:004 >>> pwn-ai # launches agent TUI
288
+ pwn[CURRENT_VERSION]:002 >>> PWN::Migrate.needed? # => false
289
+ pwn[CURRENT_VERSION]:003 >>> PWN::Plugins.constants.count # => 66
290
+ pwn[CURRENT_VERSION]:004 >>> PWN::SAST.constants.count # => 48
291
+ pwn[CURRENT_VERSION]:005 >>> pwn-ai # launches agent TUI
235
292
  ```
236
293
 
237
- **Next:** [Configuration](Configuration.md) · [General Usage](General-PWN-Usage.md)
294
+ **Next:** [Configuration](Configuration.md) · [General Usage](General-PWN-Usage.md) · [Persistence](Persistence.md)
238
295
 
239
296
  [← Home](Home.md)
@@ -8,29 +8,55 @@ Every byte PWN remembers between processes lives here.
8
8
  |---|---|---|---|---|
9
9
  | `pwn.yaml` | `PWN::Config` · `PWN::Plugins::Vault` | AES-encrypted YAML | `pwn-vault` | engines, keys, agent options |
10
10
  | `pwn.yaml.decryptor` | `PWN::Plugins::Vault` | key/IV | - | decrypts `pwn.yaml` (or set `PWN_DECRYPTOR_KEY`/`_IV`) |
11
+ | **`.schema`** | **`PWN::Migrate`** | JSON `{schema:,pwn_version:,at:}` | `pwn setup --migrate` | **schema stamp - drives the one-line drift warning on launch** |
11
12
  | `memory.json` | `PWN::Memory` | JSON array | `memory_clear` | facts · prefs · lessons · env - injected into every prompt |
12
13
  | `memory.idx` | `PWN::MemoryIndex` | JSON `{key → {sha,vec}}` | `PWN::MemoryIndex.reset` | local embedding index over `memory.json` - powers **relevance-ranked** MEMORY injection (incremental; only re-embeds changed entries) |
13
- | `finetune/*.jsonl` | `PWN::AI::Agent::Learning.export_finetune` | ShareGPT / OpenAI JSONL | `rm` | supervised dataset cut from every successful session - feed to a LoRA over the local model |
14
- | `skills/*.md` | `PWN::Config.load_skills` | Markdown + YAML front-matter | `skill_delete` | reusable procedures + `references:` (CWE/CVE/ATT&CK/NIST) |
14
+ | `skills/<name>/SKILL.md` | `PWN::Config.load_skills` | **[agentskills.io](https://agentskills.io) spec** - `name`+`description` YAML front-matter | `skill_delete` | reusable procedures + `metadata.references:` (CWE/CVE/ATT&CK/NIST). Legacy flat `skills/*.md` auto-migrated by `skill_migrate_legacy` / `pwn setup --migrate --fix`. |
15
15
  | `learning.jsonl` | `PWN::AI::Agent::Learning` | JSON-per-line | `learning_reset` | task outcome log → success_rate |
16
+ | **`preferences.jsonl`** | **`PWN::AI::Agent::Reward`** | JSON-per-line `{prompt,rejected,chosen,source}` | `rm` | **DPO/KTO/ORPO preference-pair ledger - user_correction · mistakes_resolve · counterfactual · critic · curriculum** |
16
17
  | **`mistakes.json`** | **`PWN::AI::Agent::Mistakes`** | **JSON `{sig → entry}`** | **`mistakes_reset`** | **failure fingerprints · cross-session count · fix · `[REPEATING]` · `[REGRESSED]`** |
17
- | `metrics.json` | `PWN::AI::Agent::Metrics` | JSON | `metrics_reset` | per-tool calls · success · avg_duration · last_error · **per-engine** sub-buckets |
18
- | `extrospection.json` | `PWN::AI::Agent::Extrospection` | JSON | `extro_reset` | host/net/toolchain/repo/env/**rf**/**web** snapshot + previous baseline + observations[] |
19
- | `extrospection/web/*.png` | `PWN::AI::Agent::Extrospection` | PNG | `rm -rf` | headless-browser screenshots from `probe_web` / `extro_watch` (opt-in) |
20
- | `sessions/*.jsonl` | `PWN::Sessions` | JSON-per-line | `sessions_delete` | full transcript per pwn-ai run |
21
- | `cron/jobs.yml` | `PWN::Cron` | YAML | `cron_remove` | scheduled prompt/ruby/script jobs |
18
+ | `metrics.json` | `PWN::AI::Agent::Metrics` | JSON | `metrics_reset` | per-tool calls · success · avg_duration · last_error · **per-engine** sub-buckets · calibration |
19
+ | `reward_sentinel.json` | `PWN::AI::Agent::Reward` | JSON | `rm` | R3 - proxy vs judge vs user-correction gap history |
20
+ | `extrospection.json` | `PWN::AI::Agent::Extrospection` | JSON | `extro_reset` | host/net/toolchain/repo/env/**rf**/**web**/osint/serial/telecomm/packet/vision/voice snapshot + previous baseline + observations[] |
21
+ | `extrospection/web/*.png` | `Extrospection` | PNG | `rm -rf` | headless-browser screenshots from `probe_web` / `extro_watch` (opt-in) |
22
+ | `extrospection/packet/*.pcap` | `Extrospection` | pcap | `rm -rf` | bounded captures from `extro_packet(action: :capture)` |
23
+ | `extrospection/voice/*` | `Extrospection` | wav / txt | `rm -rf` | TTS/STT artifacts from `extro_voice` |
24
+ | `sessions/*.jsonl` | `PWN::Sessions` | JSON-per-line | `sessions_delete` | full transcript per pwn-ai run (with per-step `step_reward` from `Reward.prm`) |
25
+ | `cron/jobs.yml` | `PWN::Cron` | YAML | `cron_remove` | scheduled prompt/ruby/script jobs (**seeded** with `curriculum_practice_nightly` + `curriculum_train_weekly`) |
22
26
  | `cron/log/*.log` | `PWN::Cron` | text | rm | last_run output |
23
27
  | `agents.yml` | `PWN::AI::Agent::Swarm` | YAML | edit / `agent_spawn` | persona registry |
24
28
  | `swarm/<id>/bus.jsonl` | `Swarm` | JSON-per-line | rm -rf | append-only multi-agent chat |
25
29
  | `swarm/<id>/personas.json` | `Swarm` | JSON | rm | persona → session_id map |
30
+ | `curriculum/` | `PWN::AI::Agent::Curriculum` | JSONL | rm -rf | S1 self-play reproducers · W2 A/B eval sets |
31
+ | `finetune/*.jsonl` | `Learning.export_finetune` · `Reward.export_dpo` | ShareGPT / OpenAI / DPO JSONL | `rm` | supervised + preference datasets - feed to a LoRA over the local model |
32
+ | `backup/<ts>/` | `PWN::Migrate` | tree copy | rm -rf | timestamped snapshot taken before every `--migrate --fix` |
33
+ | `quarantine/` | `PWN::Migrate` | quarantined originals | rm -rf | corrupt/incompatible state files moved aside so the owner re-seeds |
26
34
  | `~/.pwn_history` | Pry | text | rm | REPL input history |
27
35
 
36
+ ## Verify / repair the whole tree
37
+
38
+ ```bash
39
+ pwn setup --migrate # per-file compatibility report + apply schema migrations
40
+ pwn setup --migrate --fix # + autofix (backup → quarantine/repair → vault backfill)
41
+ ```
42
+
43
+ ```ruby
44
+ PWN::Migrate.status # machine-readable rows
45
+ PWN::Migrate.needed? # ~/.pwn/.schema older than PWN::Migrate::SCHEMA_VERSION ?
46
+ ```
47
+
48
+ See [Installation § Upgrading](Installation.md#upgrading--pwn-state-migration-pwnmigrate)
49
+ for the full `PWN::Migrate` API.
50
+
28
51
  ## Back it up
29
52
 
30
53
  ```bash
31
54
  tar czf pwn-state-$(date +%F).tgz -C "$HOME" .pwn
32
55
  ```
33
56
 
57
+ `pwn setup --migrate --fix` also writes one automatically to
58
+ `~/.pwn/backup/<timestamp>/` before touching anything.
59
+
34
60
  ## Start fresh for a new engagement
35
61
 
36
62
  ```ruby
@@ -41,4 +67,4 @@ learning_reset(confirm: true) # task outcomes (optional)
41
67
  metrics_reset(confirm: true) # tool telemetry (optional)
42
68
  ```
43
69
 
44
- [← Home](Home.md) · [Configuration](Configuration.md)
70
+ [← Home](Home.md) · [Configuration](Configuration.md) · [Installation](Installation.md)
@@ -54,7 +54,7 @@ show-source PWN::Plugins::Fuzz.generate
54
54
  ### Data Access / Parsing
55
55
  | Module | Purpose |
56
56
  |---|---|
57
- | `DAOLdap` · `DAOMongo` · `DAOPostgres` · `DAOSqlite3` | DB clients |
57
+ | `DAOLDAP` · `DAOMongo` · `DAOPostgres` · `DAOSQLite3` | DB clients |
58
58
  | `JSONPathify` | JSONPath query |
59
59
  | `PDFParse` · `OCR` | document extraction |
60
60
  | `ScannableCodes` | QR/barcode gen/read |
@@ -92,17 +92,17 @@ show-source PWN::Plugins::Fuzz.generate
92
92
  | `RabbitMQ` | AMQP |
93
93
 
94
94
  ### Utility
95
- `FileFu` · `ThreadPool` · `Log` · `PwnLogger` · `Char` · `XXD` · `DetectOS` ·
95
+ `FileFu` · `ThreadPool` · `Log` · `PWNLogger` · `Char` · `XXD` · `DetectOS` ·
96
96
  `CreditCard` · `SSN` · `EIN` · `VIN` · `PS` · `MonkeyPatch` · `REPL`
97
97
 
98
98
  ## Full alphabetical list
99
99
 
100
100
  `Android Assembly AuthenticationHelper BareSIP BasicAuth BeEF
101
- BlackDuckBinaryAnalysis BurpSuite BusPirate Char CreditCard DAOLdap DAOMongo
102
- DAOPostgres DAOSqlite3 DefectDojo DetectOS EIN FileFu Fuzz Git Github
101
+ BlackDuckBinaryAnalysis BurpSuite BusPirate Char CreditCard DAOLDAP DAOMongo
102
+ DAOPostgres DAOSQLite3 DefectDojo DetectOS EIN FileFu Fuzz Git Github
103
103
  HackerOne Hunter IPInfo IRC Jenkins JiraDataCenter JSONPathify Log MailAgent
104
104
  Metasploit MonkeyPatch MSR206 NessusCloud NexposeVulnScan NmapIt OAuth2 OCR
105
- OpenAPI OpenVAS Packet PDFParse Pony PS PwnLogger RabbitMQ REPL ScannableCodes
105
+ OpenAPI OpenVAS Packet PDFParse Pony PS PWNLogger RabbitMQ REPL ScannableCodes
106
106
  Serial Shodan SlackClient Sock Spider SSN ThreadPool Tor TransparentBrowser
107
107
  TwitterAPI URIScheme Vault VIN Voice Vsphere XXD Zaproxy`
108
108
 
@@ -6,6 +6,8 @@ LATS, Generative Agents, PRM, RLAIF, DPO, Self-Refine, Toolformer, DSPy)
6
6
  on at least one axis, and on **five axes simultaneously** for the full
7
7
  `Curriculum.practice → Reward.export_dpo → Curriculum.train_and_gate` path.
8
8
 
9
+ ![Reinforcement-learning loop](diagrams/reinforcement-learning.svg)
10
+
9
11
  ```
10
12
  ┌────────────────────────────────────────────────┐
11
13
  request ──────►│ Loop.run │
@@ -121,3 +123,8 @@ PWN::Cron.create(name: 'mem_gc', schedule: '0 5 * * *',
121
123
  3. **Reward-hacking self-detection** (R3)
122
124
  4. **Mistake-driven curriculum with regression-gated LoRA promotion** (S1+W2)
123
125
  5. **Five naturally-generated DPO sources** with zero human labelling (W1)
126
+
127
+ **See also:** [Skills, Memory & Learning](Skills-Memory-Learning.md) ·
128
+ [Mistakes](Mistakes.md) · [Cron](Cron.md) · [pwn-ai Agent](pwn-ai-Agent.md)
129
+
130
+ [← Home](Home.md)
@@ -9,7 +9,11 @@ appended as JSON-per-line to `~/.pwn/sessions/<id>.jsonl`.
9
9
 
10
10
  - `learning_reflect(session_id:)` mines them for durable lessons → Memory.
11
11
  - `learning_distill_skill(name:, session_id:)` turns a winning run into a
12
- reusable Skill.
12
+ reusable Skill (only steps with `step_reward > 0` are kept — the minimal
13
+ sufficient trace).
14
+ - `Reward.prm(request:, session_id:)` back-labels every tool step in the
15
+ transcript with `step_reward: +1/0/−1` for credit assignment.
16
+ - `Reward.judge(request:, final:, session_id:)` grades the whole run.
13
17
  - `sessions_view` lets you (or a later agent) re-read exactly what happened.
14
18
  - Swarm maps each persona to its own session so its private context survives
15
19
  across `agent_ask` calls.
@@ -30,7 +34,7 @@ appended as JSON-per-line to `~/.pwn/sessions/<id>.jsonl`.
30
34
  {"role":"system","ts":"2026-07-07T22:08:02Z","content":"..."}
31
35
  {"role":"user","ts":"...","content":"..."}
32
36
  {"role":"assistant","ts":"...","content":"...","tool_calls":[...]}
33
- {"role":"tool","ts":"...","name":"shell","content":"..."}
37
+ {"role":"tool","ts":"...","name":"shell","content":"...","step_reward":1}
34
38
  ```
35
39
 
36
40
  **See also:** [Skills, Memory & Learning](Skills-Memory-Learning.md) ·
@@ -11,7 +11,7 @@ its own performance, turns wins into permanent capability, and - critically -
11
11
  | Store | File | Write tool | Read tool | Injected as |
12
12
  |---|---|---|---|---|
13
13
  | **Memory** | `memory.json` (+ `memory.idx`) | `memory_remember` | `memory_recall` · `PWN::MemoryIndex.recall_semantic` | `MEMORY` block - durable facts / prefs / lessons / env. **Relevance-ranked** for the current request via a local embedding index when `ai.ollama.embed_model` is available; falls back to newest-first otherwise. |
14
- | **Skills** | `skills/*.md` | `skill_create` · `learning_distill_skill` | `skill_list` · `skill_view` | `SKILLS` list - reusable procedures + `references:` (CWE/CVE/ATT&CK/NIST/URL) |
14
+ | **Skills** | `skills/<name>/SKILL.md` | `skill_create` · `skill_migrate_legacy` · `learning_distill_skill` | `skill_list` · `skill_view` | `SKILLS` list - reusable procedures + `references:` (CWE/CVE/ATT&CK/NIST/URL) |
15
15
  | **Learning** | `learning.jsonl` | `learning_note_outcome` · `learning_reflect` | `learning_outcomes` · `learning_stats` · `Learning.exemplars_for` | `LEARNING` block - recent outcomes + success_rate. Prior *successful* traces are also spliced in as **few-shot exemplars** for local models. |
16
16
  | **Mistakes** | `mistakes.json` | `mistakes_record` · `mistakes_resolve` · *auto on failure* | `mistakes_list` | `KNOWN MISTAKES` + `KNOWN FIXES` blocks - do-NOT-repeat + do-THIS-instead |
17
17
  | **Metrics** | `metrics.json` | *automatic* (every Dispatch) | `metrics_summary` | `TOOL EFFECTIVENESS` block - steer tool choice. **Segmented per engine** (`engine=...`) so a local model's telemetry never blends with a frontier model's. |
@@ -51,16 +51,26 @@ dongle unplugged, target DOM moved). See **[Mistakes](Mistakes.md)** for the
51
51
  negative-feedback mechanics and **[Extrospection](Extrospection.md)** for the
52
52
  outward half.
53
53
 
54
- ## Skill file format
54
+ ## Skill file format ([agentskills.io](https://agentskills.io) spec)
55
+
56
+ Each skill lives in its own directory as `~/.pwn/skills/<name>/SKILL.md`.
57
+ The name is sanitised to `[a-z0-9-]{1,64}`. Front-matter **must** carry
58
+ `name` + `description`; `metadata.references` and `allowed-tools` are
59
+ optional.
55
60
 
56
61
  ```markdown
57
62
  ---
58
- references:
59
- - CWE-89
60
- - T1190
61
- - https://portswigger.net/web-security/sql-injection
63
+ name: sqli-union-enum
64
+ description: Manual UNION-based SQLi column enumeration and data extraction.
65
+ license: MIT
66
+ allowed-tools: [terminal, pwn, extrospection]
67
+ metadata:
68
+ references:
69
+ - CWE-89
70
+ - T1190
71
+ - https://portswigger.net/web-security/sql-injection
62
72
  ---
63
- # sqli_union_enum
73
+ # sqli-union-enum
64
74
 
65
75
  1. Confirm injection with `' AND 1=1 --`.
66
76
  2. Find column count with `ORDER BY n`.
@@ -71,10 +81,16 @@ references:
71
81
  - T1190
72
82
  ```
73
83
 
74
- `PWN::Config.parse_skill_references` reads both the YAML front-matter **and**
75
- the `## References` section, deduplicates, and exposes them via
84
+ `PWN::Config.parse_skill_references` reads both the YAML `metadata.references`
85
+ **and** the `## References` section, deduplicates, and exposes them via
76
86
  `skill_view(name)[:references]`.
77
87
 
88
+ Legacy flat `~/.pwn/skills/*.md` files are still read, but
89
+ `skill_migrate_legacy` (also run by `pwn setup --migrate --fix`) converts
90
+ them in-place to the spec-conformant `<name>/SKILL.md` layout with
91
+ back-filled `name`/`description` front-matter. `skill_create` always writes
92
+ the new format.
93
+
78
94
  ## Housekeeping
79
95
 
80
96
  | Tool | When |
@@ -135,7 +151,7 @@ when deciding which side of the loop to exercise.
135
151
 
136
152
  - "Which tools have the lowest success rate right now?"
137
153
  - "How often has `shell` been called, and what's its avg duration?"
138
- - "Is `extro_rf_tune` healtheir than the old GQRX helpers by metrics?"
154
+ - "Is `extro_rf_tune` healthier than the old GQRX helpers by metrics?"
139
155
  - "Reset metrics after we fixed the broken tool so the 0 % doesn't steer us away." *(destructive)*
140
156
 
141
157
  ### Sessions / transcripts (`sessions_list`, `sessions_view`, `sessions_current`, `sessions_stats`, `sessions_delete`)
@@ -20,6 +20,29 @@ $ pwn setup --deps --dry-run # show the apt/dnf/pacman/brew/port commands wi
20
20
  See [Installation](Installation.md#pwn-setup--the-post-install-doctor--provisioner)
21
21
  for the full profile table and `PWN::Setup` API.
22
22
 
23
+ ## `[pwn] ~/.pwn schema is stale — run pwn setup --migrate` on launch
24
+
25
+ Your `~/.pwn` state files were written by an older PWN release than the one
26
+ now running. The launcher checks `PWN::Migrate.needed?` on every start.
27
+
28
+ ```bash
29
+ $ pwn setup --migrate # verify each state file against its owning module
30
+ $ pwn setup --migrate --fix # + backup → schema migrations → repair → vault backfill
31
+ $ pwn setup --migrate --dry-run # report only
32
+ ```
33
+
34
+ Corrupt / incompatible files are moved to `~/.pwn/quarantine/` (never
35
+ deleted); a full timestamped backup is written to `~/.pwn/backup/<ts>/`
36
+ first. Missing config keys added by the new release are deep-merged into
37
+ your encrypted `~/.pwn/pwn.yaml` **under** your existing values. See
38
+ [Installation § Upgrading](Installation.md#upgrading--pwn-state-migration-pwnmigrate).
39
+
40
+ ## `KeyError` / `NoMethodError for nil` reading memory / metrics / mistakes / cron / agents
41
+
42
+ Same cause — the on-disk shape predates the current loader.
43
+ `pwn setup --migrate --fix` will quarantine or repair the offending file
44
+ and re-seed a valid one on next write.
45
+
23
46
  ## `LoadError` / `cannot load such file -- pg` (or rmagick, pcaprub, ...)
24
47
 
25
48
  A native extension failed to compile because its OS headers weren't present
@@ -18,10 +18,11 @@ tool-calling AI agent** on top of it.
18
18
  | `PWN::SAST::*` | **48** | Static-analysis rules across C/Java/Go/Python/Ruby/Scala/PHP/TS |
19
19
  | `PWN::AWS::*` | **90** | One module per AWS service for cloud enumeration |
20
20
  | `PWN::WWW::*` | **21** | Site-specific browser automations (HackerOne, BugCrowd, Google, LinkedIn, ...) |
21
- | `PWN::SDR::*` | **6** | GQRX, FlipperZero, RFIDler, SonMicro, decoders, band tables |
21
+ | `PWN::SDR::*` | **6** (+ **20** decoders) | GQRX, FlipperZero, RFIDler, SonMicro, band tables, `Decoder::{ADSB,POCSAG,RDS,LoRa,...}` |
22
+ | `PWN::FFI::*` | **8** | Native DSP/RF backends: Volk · Liquid · FFTW · RTLSdr · HackRF · AdalmPluto · SoapySDR · Stdio |
22
23
  | `PWN::AI::*` | **5** engines | OpenAI, Anthropic, Grok (OAuth device-flow), Gemini, Ollama |
23
24
  | `bin/pwn_*` | **53** | Headless CLI drivers for CI/CD |
24
- | Agent toolsets | **10** | terminal · pwn · memory · skills · sessions · learning · metrics · extrospection · cron · swarm |
25
+ | Agent toolsets | **10** · **71 tools** | terminal · pwn · memory · skills · sessions · learning · metrics · extrospection · cron · swarm |
25
26
 
26
27
  ## The three ways to use it
27
28
 
@@ -37,13 +38,18 @@ tool-calling AI agent** on top of it.
37
38
  - **Everything is Ruby, everything is a method.** No YAML DSLs, no plugins-in-a-
38
39
  black-box. If you can call it in the REPL, the AI agent can call it, a driver
39
40
  can call it, and a cron job can call it.
40
- - **Closed self-improvement loop.** Metrics + Learning (introspection) and
41
- Snapshot + Drift + Intel (extrospection) feed `extro_correlate`, which tells
42
- the agent whether a failure was *its* fault or *the world* changed - and
43
- writes the lesson back into the prompt for next time.
41
+ - **Closed self-improvement loop.** Metrics + Learning + **Reward** (ORM/PRM
42
+ judge, sentinel) + **Curriculum** (mistake-driven self-play, HER,
43
+ regression-gated LoRA) on the introspection side; Snapshot + Drift + Intel
44
+ + Verify on the extrospection side; joined by `extro_correlate`, which
45
+ tells the agent whether a failure was *its* fault or *the world* changed -
46
+ and writes the lesson back into the prompt for next time.
44
47
  - **Native multi-agent.** `PWN::AI::Agent::Swarm` runs personas (each a full
45
48
  tool-calling agent, optionally on a *different* LLM engine) that debate,
46
49
  broadcast and share an append-only bus - no IRC daemon, no external service.
50
+ - **Self-healing state.** `PWN::Setup` (doctor/provisioner) + `PWN::Migrate`
51
+ (schema-stamped `~/.pwn` verifier/auto-migrator) make `gem install pwn` →
52
+ `pwn setup` the entire install *and* upgrade story on every OS.
47
53
 
48
54
  ![Overall Architecture](diagrams/overall-pwn-architecture.svg)
49
55
 
@@ -40,8 +40,9 @@ when the caller is an autonomous agent.
40
40
  A pentest framework that doesn't learn repeats the same dead-end scans forever.
41
41
  PWN records **per-tool success rate**, **per-task outcome**, **host drift**,
42
42
  and **external CVE intel**, then correlates them so tomorrow's run starts where
43
- today's finished. See [Skills, Memory & Learning](Skills-Memory-Learning.md)
44
- and [Extrospection](Extrospection.md).
43
+ today's finished. See [Skills, Memory & Learning](Skills-Memory-Learning.md),
44
+ [Reinforcement Learning](Reinforcement-Learning.md) and
45
+ [Extrospection](Extrospection.md).
45
46
 
46
47
  **Next:** [How PWN Works](How-PWN-Works.md)
47
48