pwn 0.5.744 → 0.5.745

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: df80036cfdf54ca32a6a7a568d17d167f16f4983c5f6c106f7577f2ca183814f
4
- data.tar.gz: c423990479c34d4a21c5b549e283e81b44dec52670fbb9f19e7bf53f8ac692ba
3
+ metadata.gz: f3d60fd604a668b531459517b97dfee6448f9485679bd4c25bfe3ec12090cee9
4
+ data.tar.gz: 64a4dd65248017933e877c03db222ce61741e60044a03a1c9c9a1297ca7a4fde
5
5
  SHA512:
6
- metadata.gz: c300723020b4e2da6e292ec06298fad5dc3a3c0f7e108eea2f91e2f746c18da3d705ea0ea4127185df3c82c3eaad4204307756c40b0b4ac0e8c8db1575e485b3
7
- data.tar.gz: 4738cce8fb8eb570df5b0e48de365894f261b6ad3c011e875cc88a5f87f0397b849143910520b2aadb4dd628dcb242fe7d8865be2ed101d0e7829ba27127b9b5
6
+ metadata.gz: 1792ff5aea6a7c406248ab0fd6647732c490e56450afdf768840f340b4d7b4847a2d2cd996ff1349b3ca1a1bbfc05be40310225345e675937cc16022aaad5ec3
7
+ data.tar.gz: 7155e5d3cfca02bda195d0b0f926b85d6cd95b306b570d5956484ae237d360b36628a979e2d98a2c5ce3008343c7cecc1d14f12d0a986dbc577923a9f7905930
@@ -59,6 +59,16 @@ pwn_setup --migrate --fix # ~/.pwn state doctor + autofix (PWN::Migrate)
59
59
  See [Installation](Installation.md) for the full profile table, the
60
60
  `PWN::Setup` API and the `PWN::Migrate` state-file registry.
61
61
 
62
+ ## Offline policy evaluation with `pwn-ai`
63
+
64
+ `pwn-ai --policy evaluate --baseline PATH --candidate PATH` prints two frozen
65
+ held-out reports without opening a vault or starting an AI session. The separate
66
+ `--policy promote` and `--policy rollback` actions require an explicit target and
67
+ both `--approve-policy-change` and `--policy-writers-stopped`. They never start
68
+ network tasks or automatically promote on success. See
69
+ [Policy-Benchmark](Policy-Benchmark.md#offline-operator-cli) for runnable local
70
+ commands, receipt preservation, fresh replay gates, and limitations.
71
+
62
72
  ## Typical CI usage
63
73
 
64
74
  ```yaml
@@ -187,6 +187,7 @@ What `--migrate` does:
187
187
  directory layout) → **deep-merge** any keys the current
188
188
  `PWN::Config.env_template` added into your encrypted `~/.pwn/pwn.yaml`
189
189
  **without overwriting your values** (re-encrypted with the same key/IV).
190
+ Schema 5 adds `ai.agent.skill_review` (`recommend` when the key is absent).
190
191
 
191
192
  Everything is idempotent and dry-run capable. The plain `pwn` launcher also
192
193
  prints a one-line drift warning on startup whenever `~/.pwn/.schema`
@@ -220,6 +220,82 @@ fields except `elapsed_seconds`. They contain no wall-clock timestamps, random
220
220
  IDs or temporary paths. The original training report still contains the timing
221
221
  and metadata variability described above.
222
222
 
223
+ ## Offline operator CLI
224
+
225
+ `pwn-ai --policy evaluate|promote|rollback` reuses `PolicyEvaluation`; it never
226
+ starts a session, loads the encrypted configuration, runs `Learning.rsi_tick`,
227
+ or invokes the agent loop. It cannot be combined with `--ai`, replay, mission,
228
+ or other session options. Evaluation accepts existing explicit frozen snapshots,
229
+ runs suites 0 and 1, and prints a JSON **array** suitable for `--reports`. It does
230
+ not train or write the live policy. The Ruby API supports other indices 0..7.
231
+
232
+ From a source checkout, this disposable demonstration stays entirely under a
233
+ new `/tmp` directory (use `pwn-ai` instead of `ruby -Ilib bin/pwn-ai` for an
234
+ installed executable):
235
+
236
+ ```sh
237
+ umask 077
238
+ work=$(mktemp -d /tmp/pwn-offline-policy.XXXXXX)
239
+ ruby scripts/benchmark_policy.rb --heldout \
240
+ --snapshot-dir "$work/snapshots" --output "$work/benchmark.json"
241
+ ruby -Ilib bin/pwn-ai --policy evaluate \
242
+ --baseline "$work/snapshots/off.json" --candidate "$work/snapshots/on.json" \
243
+ > "$work/reports.json"
244
+ cp "$work/snapshots/off.json" "$work/demo-live.json"
245
+ ```
246
+
247
+ Review `reports.json` before deciding whether to proceed. For a real target,
248
+ first stop **all** agent processes and other policy writers, take the baseline
249
+ copy only after stopping them, and keep writers stopped through promotion and
250
+ readback. The acknowledgement does not stop processes or acquire a writer lock.
251
+ Concurrent promotions are also unsupported. Do not replace a real policy with
252
+ these demonstration snapshots or infer live gains from this fixture experiment.
253
+
254
+ Only if approving the change to the **disposable demo target**, run:
255
+
256
+ ```sh
257
+ ruby -Ilib bin/pwn-ai --policy promote \
258
+ --baseline "$work/snapshots/off.json" --candidate "$work/snapshots/on.json" \
259
+ --reports "$work/reports.json" --live-policy "$work/demo-live.json" \
260
+ --approve-policy-change --policy-writers-stopped > "$work/promotion.json"
261
+ cmp "$work/demo-live.json" "$work/snapshots/on.json"
262
+ ```
263
+
264
+ The command re-executes both reports before considering replacement. It returns
265
+ nonzero on refusal or malformed input; check the exit status and JSON, not just
266
+ whether a redirected file exists. Keep the successful `promotion.json` receipt
267
+ and digest-named backup. Use distinct, new output paths: shell redirection opens
268
+ files before the CLI starts, so never redirect onto snapshots, the live target,
269
+ or an existing receipt. A failed retry must not overwrite the successful receipt.
270
+
271
+ Rollback is a separate explicit operator decision with the same stopped-writer
272
+ requirement; it refuses intervening changes to the target:
273
+
274
+ ```sh
275
+ ruby -Ilib bin/pwn-ai --policy rollback \
276
+ --receipt "$work/promotion.json" --live-policy "$work/demo-live.json" \
277
+ --approve-policy-change --policy-writers-stopped > "$work/rollback.json"
278
+ cmp "$work/demo-live.json" "$work/snapshots/off.json"
279
+ ```
280
+
281
+ Neither approval flag alone is sufficient. There is no default live path,
282
+ automatic promotion, target discovery, network task, or model training in this
283
+ CLI path. Reports and receipts are operator-owned local JSON, not executable
284
+ configuration; neither can supply approval flags. Existing `Learning.rsi_tick`
285
+ only snapshots measured rates and records a regression lesson. It does not call
286
+ this gate, generate candidates, schedule practice, or approve changes. The
287
+ broader online learning and curriculum paths remain unchanged and separate.
288
+
289
+ Focused verification commands (no hardware, providers, or live models):
290
+
291
+ ```sh
292
+ bundle exec rspec spec/lib/pwn/ai/cli_spec.rb \
293
+ spec/lib/pwn/ai/agent/policy_evaluation_spec.rb \
294
+ spec/lib/pwn/ai/agent/learning_spec.rb spec/lib/pwn/ai/agent/rsi_metrics_spec.rb
295
+ bundle exec rubocop lib/pwn/ai/cli.rb spec/lib/pwn/ai/cli_spec.rb \
296
+ spec/lib/pwn/ai/agent/rsi_metrics_spec.rb
297
+ ```
298
+
223
299
  ## Explicit promotion and rollback
224
300
 
225
301
  This is an **operator-invoked local eligibility gate**, not automatic online
@@ -12,6 +12,13 @@ one. Without a trainer it still **exports** the datasets and a manual CLI. Live
12
12
 
13
13
  ESR and ASR are the rates RSI actually compares. ESR is `verified_exploit_tools / vulnerable_tools`. ASR is `successful_attacks / total_attack_attempts`. `Learning.rsi_tick` stores the pair and writes an `rsi` lesson when ESR falls. Tool telemetry and the judge score are separate. A passing suite is not either rate.
14
14
 
15
+ RSI's rate tick does not generate or promote policy candidates. For the separate
16
+ **offline operator evaluation** path, use `pwn-ai --policy evaluate` and review
17
+ the fixed local held-out reports. Promotion and rollback each require explicit
18
+ `--approve-policy-change --policy-writers-stopped` plus a named live policy path;
19
+ neither runs automatically from RSI. See the executable commands, stopped-writer
20
+ requirements, and benchmark limits in [Policy-Benchmark](Policy-Benchmark.md#offline-operator-cli).
21
+
15
22
  ![Reinforcement-learning loop](diagrams/reinforcement-learning.svg)
16
23
 
17
24
  ```
@@ -132,6 +132,8 @@ bundled skills into `~/.pwn/skills/` when the name is missing:
132
132
  | `att&ck` | Exhaustive test procedure per ATT&CK technique (`references/T1059.001.md`) |
133
133
  | `humanizer` | Strip AI writing patterns from prose. Keep meaning and identifiers. |
134
134
 
135
+ `PWN::AI::Agent::SkillReview` runs after introspection. A routine success does not create a skill. A tested procedure, a resolved recurring mistake, or an explicit correction can recommend an update to the closest skill under `~/.pwn/skills`. The default mode is `recommend`. `auto-safe` writes only a small addition that an execution fixture passed in three sessions, and it keeps a backup. It does not create skills, rewrite generated `pwn/` module skills, or save secrets and raw tool output. Set `ai.agent.skill_review` to `off`, `recommend`, or `auto-safe`.
136
+
135
137
  Source: `etc/default_skills/` in the gem. SOP edits in `~/.pwn/skills/<name>/SKILL.md`
136
138
  are never overwritten. Generated `~/.pwn/skills/pwn/**/SKILL.md` module skills
137
139
  are updated on migrate when `lib/pwn` changes.
@@ -0,0 +1,49 @@
1
+ ---
2
+ name: pwn-ai-agent-skillreview
3
+ description: Drive PWN::AI::Agent::SkillReview from pwn_eval.
4
+ license: MIT
5
+ allowed-tools: [pwn, pwn_eval]
6
+ metadata:
7
+ bundled: true
8
+ generated: true
9
+ module: PWN::AI::Agent::SkillReview
10
+ source: pwn/ai/agent/skill_review.rb
11
+ ---
12
+
13
+ # PWN::AI::Agent::SkillReview
14
+
15
+ Decide whether a completed task should update or create a skill. recommend is the default. auto-safe writes only a small verified addition.
16
+
17
+ ## When to use
18
+
19
+ Call `PWN::AI::Agent::SkillReview` from `pwn_eval` when the task needs this module.
20
+ Do not reimplement it in shell.
21
+
22
+ ## Methodologies
23
+
24
+ Generated from `pwn/ai/agent/skill_review.rb`. Prefer the public class methods below.
25
+ Class methods take `(opts = {})` and read `opts`.
26
+
27
+ ## How to call
28
+
29
+ ```ruby
30
+ PWN::AI::Agent::SkillReview.help
31
+ PWN::AI::Agent::SkillReview.review(opts)
32
+ ```
33
+
34
+ ## Public methods
35
+
36
+ - `review`
37
+ - `review_turn`
38
+ - `note_reuse`
39
+ - `authors`
40
+ - `help`
41
+
42
+ ## Source
43
+
44
+ `pwn/ai/agent/skill_review.rb`
45
+
46
+ ## Verification
47
+
48
+ `PWN::AI::Agent::SkillReview.respond_to?(:review)` after the
49
+ module is loaded. Read the source for parameter names.
@@ -822,13 +822,20 @@ module PWN
822
822
  rsi_tick(request: opts[:request])
823
823
  end
824
824
 
825
+ skill_review = nil
826
+ if defined?(SkillReview)
827
+ stages_run << :skill_review
828
+ skill_review = SkillReview.review_turn(request: opts[:request], final: opts[:final], success: opts[:success], session_id: opts[:session_id])
829
+ end
830
+
825
831
  {
826
832
  ok: ok,
827
833
  score: v[:score],
828
834
  elapsed_ms: elapsed_ms.call,
829
835
  budget_hot: budget_hot,
830
836
  stages_run: stages_run,
831
- stages_skipped: stages_skipped
837
+ stages_skipped: stages_skipped,
838
+ skill_review: skill_review
832
839
  }
833
840
  rescue StandardError => e
834
841
  warn "[pwn-ai/learning] auto_introspect swallowed: #{e.class}: #{e.message}"
@@ -0,0 +1,223 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'digest'
4
+ require 'fileutils'
5
+ require 'json'
6
+
7
+ module PWN
8
+ module AI
9
+ module Agent
10
+ # Decide whether a completed task should update or create a skill.
11
+ # recommend is the default. auto-safe writes only a small verified addition.
12
+ module SkillReview
13
+ LEDGER = File.join(Dir.home, '.pwn', 'skill_review.jsonl').freeze
14
+ MODES = %w[off recommend auto-safe].freeze
15
+
16
+ public_class_method def self.review(opts = {})
17
+ mode = mode_from(mode: opts[:mode])
18
+ request = opts[:request].to_s
19
+ return finish(action: 'skipped', reason: 'mode off', mode: mode, request: request) if mode == 'off'
20
+ return finish(action: 'skipped', reason: 'routine success is not a skill', mode: mode, request: request) unless useful?(opts)
21
+
22
+ procedure = procedure_from(procedure: opts[:procedure], evidence: opts[:evidence])
23
+ unless procedure[:ok]
24
+ return finish(action: 'refused', reason: procedure[:reason], mode: mode, request: request) if opts[:procedure]
25
+
26
+ return finish(action: 'recommend', reason: 'a structured procedure is required before saving', mode: mode, request: request, verified: false)
27
+ end
28
+
29
+ target = locate(name: opts[:name], query: request, procedure: procedure[:text])
30
+ kind = target ? 'update' : 'create'
31
+ verified = verified_execution?(evidence: opts[:evidence])
32
+ proposal = {
33
+ action: 'recommend',
34
+ kind: kind,
35
+ name: target ? target[:name] : opts[:name].to_s,
36
+ mode: mode,
37
+ request: request,
38
+ verified: verified,
39
+ reason: target ? 'closest skill matches the procedure' : 'no existing skill matches',
40
+ path: target && target[:path]
41
+ }
42
+ return finish(proposal.merge(action: 'refused', reason: 'generated module skills are not rewritten')) if generated?(name: proposal[:name], path: proposal[:path])
43
+ return finish(proposal) unless mode == 'auto-safe' && auto_safe?(proposal: proposal, procedure: procedure)
44
+
45
+ apply_update(proposal: proposal, procedure: procedure, evidence: opts[:evidence], skills_root: opts[:skills_root])
46
+ end
47
+
48
+ public_class_method def self.review_turn(opts = {})
49
+ request = opts[:request].to_s
50
+ correction = defined?(Mistakes) && request.match?(Mistakes::CORRECTION_RX)
51
+ mistake = opts[:mistake]
52
+ if mistake.nil? && defined?(Mistakes) && Mistakes.respond_to?(:top)
53
+ rows = Array(Mistakes.top(limit: 8, unresolved_only: false))
54
+ mistake = rows.find { |row| row.is_a?(Hash) && row[:resolved] == true && row[:count].to_i >= 2 }
55
+ mistake = mistake.merge(source: 'mistakes') if mistake
56
+ end
57
+ review(opts.merge(user_correction: correction == true, mistake: mistake, session_id: opts[:session_id], final: opts[:final]))
58
+ end
59
+
60
+ public_class_method def self.note_reuse(opts = {})
61
+ row = {
62
+ name: opts[:name].to_s,
63
+ retrieved: opts[:retrieved] == true,
64
+ reused: opts[:reused] == true,
65
+ regressed: opts[:regressed] == true,
66
+ at: Time.now.utc.iso8601
67
+ }
68
+ append_ledger(row: row)
69
+ row
70
+ end
71
+
72
+ public_class_method def self.authors
73
+ "AUTHOR(S):\n 0day Inc. <support@0dayinc.com>\n"
74
+ end
75
+
76
+ public_class_method def self.help
77
+ puts "USAGE:
78
+ # Review a completed task and recommend or apply a skill change.
79
+ #{self}.review(
80
+ request: 'required - the completed task text',
81
+ mode: 'optional - off, recommend, or auto-safe; default recommend',
82
+ procedure: 'optional - Hash with when, prerequisites, steps, verification, and failures',
83
+ evidence: 'optional - execution proof with source, fixture_passed, and sessions',
84
+ name: 'optional - skill name to prefer',
85
+ success: 'optional - true alone is a routine success and is skipped',
86
+ user_correction: 'optional - true reviews an explicit operator correction',
87
+ skills_root: 'optional - skills directory for the write'
88
+ )
89
+
90
+ # Build a review from the completed turn, including an operator correction or a resolved mistake.
91
+ #{self}.review_turn(
92
+ request: 'required - the completed task text',
93
+ final: 'optional - final answer, not saved as a procedure',
94
+ success: 'optional - true alone does not create a skill',
95
+ session_id: 'optional - session id for the turn',
96
+ mistake: 'optional - resolved mistake Hash with count and source from the mistakes store'
97
+ )
98
+
99
+ # Record whether a later task retrieved or reused a reviewed skill.
100
+ #{self}.note_reuse(
101
+ name: 'required - skill name',
102
+ retrieved: 'optional - true when the catalog returned the skill',
103
+ reused: 'optional - true when the procedure was followed',
104
+ regressed: 'optional - true when reuse made the task worse'
105
+ )
106
+
107
+ # Print the module authors.
108
+ #{self}.authors
109
+ "
110
+ end
111
+
112
+ private_class_method def self.mode_from(opts = {})
113
+ raw = opts[:mode]
114
+ raw = PWN::Env.dig(:ai, :agent, :skill_review) if raw.nil? && defined?(PWN::Env) && PWN::Env.respond_to?(:dig)
115
+ mode = raw.to_s
116
+ mode = 'recommend' if mode.empty? || !MODES.include?(mode)
117
+ mode
118
+ end
119
+
120
+ private_class_method def self.useful?(opts = {})
121
+ return true if opts[:user_correction] == true
122
+
123
+ return true if opts[:procedure].is_a?(Hash) && opts[:evidence].is_a?(Hash)
124
+
125
+ mistake = opts[:mistake]
126
+ mistake.is_a?(Hash) && mistake[:resolved] == true && mistake[:count].to_i >= 2 && mistake[:source].to_s == 'mistakes'
127
+ end
128
+
129
+ private_class_method def self.procedure_from(opts = {})
130
+ raw = opts[:procedure]
131
+ return { ok: false, reason: 'procedure is required' } unless raw.is_a?(Hash)
132
+
133
+ fields = %i[when prerequisites steps verification failures]
134
+ missing = fields.reject { |key| raw[key].to_s.strip.length >= 8 }
135
+ return { ok: false, reason: "procedure missing #{missing.join(', ')}" } unless missing.empty?
136
+
137
+ text = fields.map { |key| "#{key}: #{raw[key].to_s.strip}" }.join("\n")
138
+ return { ok: false, reason: 'secret or target-specific text is not saved' } if sensitive?(text: text)
139
+ return { ok: false, reason: 'raw tool output is not a procedure' } if text.match?(/STDOUT|STDERR/) || text.length > 1200
140
+
141
+ { ok: true, text: text }
142
+ end
143
+
144
+ private_class_method def self.sensitive?(opts = {})
145
+ opts[:text].to_s.match?(/bearer |password|api_key|BEGIN |token|sk-[A-Za-z0-9]/i) || opts[:text].to_s.match?(/\b\d{1,3}(?:\.\d{1,3}){3}\b/)
146
+ end
147
+
148
+ private_class_method def self.verified_execution?(opts = {})
149
+ evidence = opts[:evidence].is_a?(Hash) ? opts[:evidence] : {}
150
+ evidence[:source].to_s == 'execution' && evidence[:fixture_passed] == true && Array(evidence[:sessions]).map(&:to_s).uniq.length >= 3 && evidence[:model_claimed] != true
151
+ end
152
+
153
+ private_class_method def self.locate(opts = {})
154
+ return nil unless defined?(PWN::Skills) && PWN::Skills.is_a?(Hash)
155
+
156
+ name = opts[:name].to_s
157
+ unless name.empty?
158
+ meta = PWN::Skills[name.to_sym] || PWN::Skills[name]
159
+ return { name: name, path: meta[:path], meta: meta } if meta.is_a?(Hash)
160
+ end
161
+
162
+ tokens = "#{opts[:query]} #{opts[:procedure]}".downcase.scan(/[a-z0-9]{4,}/).uniq
163
+ scored = PWN::Skills.map do |key, meta|
164
+ next unless meta.is_a?(Hash)
165
+
166
+ hay = "#{key} #{meta[:description]}".downcase
167
+ [key.to_s, meta, tokens.count { |tok| hay.include?(tok) }]
168
+ end.compact
169
+ best = scored.max_by { |_, _, score| score }
170
+ return nil unless best && best[2] >= 2
171
+
172
+ { name: best[0], path: best[1][:path], meta: best[1] }
173
+ end
174
+
175
+ private_class_method def self.generated?(opts = {})
176
+ opts[:name].to_s.start_with?('pwn/') || opts[:path].to_s.include?('/skills/pwn/')
177
+ end
178
+
179
+ private_class_method def self.auto_safe?(opts = {})
180
+ proposal = opts[:proposal] || {}
181
+ procedure = opts[:procedure] || {}
182
+ proposal[:kind] == 'update' && proposal[:verified] == true && procedure[:text].to_s.length <= 400
183
+ end
184
+
185
+ private_class_method def self.apply_update(opts = {})
186
+ proposal = opts[:proposal]
187
+ path = proposal[:path].to_s
188
+ return finish(proposal.merge(action: 'refused', reason: 'skill path missing')) unless File.file?(path) && !File.symlink?(path)
189
+
190
+ original = File.read(path)
191
+ digest = Digest::SHA256.hexdigest(original)
192
+ backup = "#{path}.#{digest}.bak"
193
+ File.open(backup, File::WRONLY | File::CREAT | File::EXCL, 0o600) { |file| file.write(original) } unless File.exist?(backup)
194
+ signature = (opts[:evidence] || {})[:signature].to_s
195
+ lesson = "#{signature}\n#{opts[:procedure][:text]}"
196
+ written = Learning.update_skill(name: proposal[:name], lesson: lesson, query: proposal[:request], pwn_skills_path: opts[:skills_root])
197
+ current = File.read(path)
198
+ unless current.include?(signature) && !signature.empty?
199
+ File.write(path, original)
200
+ return finish(proposal.merge(action: 'refused', reason: 'readback failed', written: written))
201
+ end
202
+
203
+ PWN::Config.load_skills(pwn_skills_path: opts[:skills_root]) if defined?(PWN::Config) && PWN::Config.respond_to?(:load_skills)
204
+ finish(proposal.merge(action: 'applied', backup: backup))
205
+ end
206
+
207
+ private_class_method def self.finish(opts = {})
208
+ row = opts.transform_keys(&:to_sym)
209
+ append_ledger(row: row)
210
+ row
211
+ end
212
+
213
+ private_class_method def self.append_ledger(opts = {})
214
+ path = LEDGER
215
+ FileUtils.mkdir_p(File.dirname(path))
216
+ File.open(path, 'a') { |file| file.puts(JSON.generate(opts[:row])) }
217
+ rescue StandardError
218
+ nil
219
+ end
220
+ end
221
+ end
222
+ end
223
+ end
data/lib/pwn/ai/agent.rb CHANGED
@@ -15,6 +15,7 @@ module PWN
15
15
  autoload :GQRX, 'pwn/ai/agent/gqrx'
16
16
  autoload :SAST, 'pwn/ai/agent/sast'
17
17
  autoload :SkillConsolidation, 'pwn/ai/agent/skill_consolidation'
18
+ autoload :SkillReview, 'pwn/ai/agent/skill_review'
18
19
  autoload :TransparentBrowser, 'pwn/ai/agent/transparent_browser'
19
20
  autoload :VulnGen, 'pwn/ai/agent/vuln_gen'
20
21
 
data/lib/pwn/ai/cli.rb CHANGED
@@ -20,6 +20,14 @@ module PWN
20
20
  options.on('--execute PATH', 'Run an approved YAML task DAG') { |v| result[:execute] = v }
21
21
  options.on('--resume RUN_ID', 'Resume a checkpointed DAG run, skipping completed steps') { |v| result[:resume] = v }
22
22
  options.on('--mission ID', 'Bind this plan or run to a durable mission') { |v| result[:mission] = v }
23
+ options.on('--policy ACTION', %w[evaluate promote rollback], 'Offline policy: evaluate, promote, rollback (no AI session)') { |v| result[:policy] = v }
24
+ options.on('--baseline PATH', 'Frozen baseline policy JSON') { |v| result[:baseline] = v }
25
+ options.on('--candidate PATH', 'Frozen candidate policy JSON') { |v| result[:candidate] = v }
26
+ options.on('--reports PATH', 'JSON array from --policy evaluate; replayed before promotion') { |v| result[:reports] = v }
27
+ options.on('--receipt PATH', 'Saved successful promotion JSON for rollback') { |v| result[:receipt] = v }
28
+ options.on('--live-policy PATH', 'Explicit existing policy target; no default') { |v| result[:live_path] = v }
29
+ options.on('--approve-policy-change', 'Operator explicitly approves promotion or rollback') { result[:enabled] = true }
30
+ options.on('--policy-writers-stopped', 'Operator attests ALL policy writers are stopped') { result[:quiescent] = true }
23
31
  options.on('--ai PROMPT', 'One-shot request; - reads standard input') { |v| result[:ai] = v }
24
32
  options.on('--pwn-env PATH', 'Use the specified encrypted configuration') { |v| result[:pwn_env_path] = v }
25
33
  options.on('--pwn-dec PATH', 'Use the specified decryptor') { |v| result[:pwn_dec_path] = v }
@@ -31,6 +39,8 @@ module PWN
31
39
  raise OptionParser::InvalidArgument, '--ai cannot accompany --replay or --rerun' if result[:ai] && (result[:replay] || result[:rerun])
32
40
  raise OptionParser::InvalidArgument, '--plan-only requires --ai' if result[:plan_only] && !result[:ai]
33
41
 
42
+ validate_policy_options(parsed: result)
43
+
34
44
  result[:help_text] = parser.to_s
35
45
  result
36
46
  end
@@ -44,6 +54,8 @@ module PWN
44
54
  end
45
55
 
46
56
  require 'pwn'
57
+ return run_policy(parsed: parsed, output: output) if parsed[:policy]
58
+
47
59
  if parsed[:plan_only]
48
60
  prompt = parsed[:ai] == '-' ? (opts[:input] || $stdin).read : parsed[:ai]
49
61
  raise ArgumentError, '--ai requires a non-empty prompt' if prompt.to_s.strip.empty?
@@ -97,6 +109,48 @@ module PWN
97
109
  0
98
110
  end
99
111
 
112
+ private_class_method def self.validate_policy_options(opts = {})
113
+ parsed = opts[:parsed]
114
+ fields = %i[baseline candidate reports receipt live_path enabled quiescent]
115
+ unless parsed[:policy]
116
+ raise OptionParser::InvalidArgument, 'policy options require --policy' if fields.any? { |key| parsed.key?(key) }
117
+
118
+ return
119
+ end
120
+
121
+ allowed = {
122
+ 'evaluate' => %i[baseline candidate],
123
+ 'promote' => %i[baseline candidate reports live_path enabled quiescent],
124
+ 'rollback' => %i[receipt live_path enabled quiescent]
125
+ }.fetch(parsed[:policy])
126
+ unexpected = parsed.keys - allowed - %i[policy help]
127
+ raise OptionParser::InvalidArgument, "options not allowed with --policy #{parsed[:policy]}: #{unexpected.join(', ')}" unless unexpected.empty?
128
+
129
+ missing = (allowed - %i[enabled quiescent]) - parsed.keys
130
+ raise OptionParser::InvalidArgument, "missing policy options: #{missing.join(', ')}" unless missing.empty? || parsed[:help]
131
+ end
132
+
133
+ private_class_method def self.run_policy(opts = {})
134
+ parsed = opts[:parsed]
135
+ evaluator = PWN::AI::Agent::PolicyEvaluation
136
+ args = parsed.slice(:baseline, :candidate, :live_path, :enabled, :quiescent)
137
+ report = case parsed[:policy]
138
+ when 'evaluate'
139
+ [0, 1].map { |seed| evaluator.evaluate(args.merge(seed: seed)) }
140
+ when 'promote', 'rollback'
141
+ raise ArgumentError, 'requires --approve-policy-change and --policy-writers-stopped; stop all policy writers first' unless parsed[:enabled] && parsed[:quiescent]
142
+
143
+ key = parsed[:policy] == 'promote' ? :reports : :receipt
144
+ args[key] = JSON.parse(File.read(parsed.fetch(key)), symbolize_names: true)
145
+ parsed[:policy] == 'promote' ? evaluator.promote(args) : evaluator.rollback(args)
146
+ end
147
+ opts[:output].puts(JSON.generate(report))
148
+ report.is_a?(Hash) && (report[:promoted] == false || report[:rolled_back] == false) ? 1 : 0
149
+ rescue StandardError => e
150
+ opts[:output].puts(JSON.generate(error: e.message))
151
+ 1
152
+ end
153
+
100
154
  private_class_method def self.bind_mission!(opts = {})
101
155
  report = opts[:report] || {}
102
156
  return report unless report[:run_id]
data/lib/pwn/config.rb CHANGED
@@ -161,6 +161,8 @@ module PWN
161
161
  max_depth: 3,
162
162
  # run PWN::AI::Agent::Learning.auto_introspect after every final answer
163
163
  auto_introspect: true,
164
+ # off, recommend, or auto-safe. A missing key means recommend.
165
+ skill_review: 'recommend',
164
166
  # also run PWN::AI::Agent::Extrospection.auto_extrospect from auto_introspect
165
167
  # (host/repo/env probes only — no toolchain/GUI/net side-effects)
166
168
  auto_extrospect: true,
data/lib/pwn/migrate.rb CHANGED
@@ -49,7 +49,7 @@ module PWN
49
49
  # Bump this whenever the shape of any file under ~/.pwn changes in a
50
50
  # way that requires a one-time transform. Add the transform as an
51
51
  # entry in MIGRATIONS keyed by the NEW schema number.
52
- SCHEMA_VERSION = 4
52
+ SCHEMA_VERSION = 5
53
53
 
54
54
  OK = "\e[32mok\e[0m"
55
55
  BAD = "\e[31mFAIL\e[0m"
@@ -245,6 +245,10 @@ module PWN
245
245
  4 => lambda { |_root, io|
246
246
  io.puts ' · pwn.yaml ai_sandbox, ai_router, model_routes task classes'
247
247
  PWN::Migrate.backfill_vault(io: io)
248
+ },
249
+ 5 => lambda { |_root, io|
250
+ io.puts ' · pwn.yaml ai.agent.skill_review'
251
+ PWN::Migrate.backfill_vault(io: io)
248
252
  }
249
253
  }.freeze
250
254
 
@@ -1654,7 +1654,7 @@ module PWN
1654
1654
  if PWN.const_defined?(:MeshMutex) && PWN.const_defined?(:MeshRxBodyWin)
1655
1655
  mutex = PWN.const_get(:MeshMutex)
1656
1656
  state = PWN.const_defined?(:MeshRxState) ? PWN.const_get(:MeshRxState) : {}
1657
- ts = Time.now.strftime('%H:%M:%S')
1657
+ ts = Time.now.strftime('%Y-%m-%d %H:%M:%S%z')
1658
1658
  color = opts[:local] ? 23 : 21
1659
1659
  secure = packet[:pki_encrypted] == true || mesh_channel_securely_encrypted?(env: env, channel: channel_name)
1660
1660
  security_icon = secure ? '🔒' : '🔍'
data/lib/pwn/version.rb CHANGED
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module PWN
4
- VERSION = '0.5.744'
4
+ VERSION = '0.5.745'
5
5
  end
@@ -34,6 +34,11 @@ describe 'RSI exploit and attack rates' do
34
34
  end
35
35
 
36
36
  it 'writes an RSI lesson when the measured exploit rate falls' do
37
+ expect(PWN::AI::Agent::PolicyEvaluation).not_to receive(:evaluate)
38
+ expect(PWN::AI::Agent::PolicyEvaluation).not_to receive(:promote)
39
+ expect(PWN::AI::Agent::PolicyEvaluation).not_to receive(:rollback)
40
+ expect(PWN::AI::Agent::Loop).not_to receive(:run)
41
+ expect(PWN::AI::Agent::Dispatch).not_to receive(:call)
37
42
  PWN::AI::Agent::Metrics.record_attempt(kind: 'exploit', tool: 'ret2libc', vulnerable_tool: 'libc', vulnerable: true, success: true)
38
43
  PWN::AI::Agent::Learning.rsi_tick(request: 'measure exploit rate')
39
44
  PWN::AI::Agent::Metrics.record_attempt(kind: 'exploit', tool: 'rop', vulnerable_tool: 'nginx', vulnerable: true, success: false)
@@ -0,0 +1,127 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'spec_helper'
4
+ require 'tmpdir'
5
+ require 'json'
6
+
7
+ describe PWN::AI::Agent::SkillReview do
8
+ let(:dir) { Dir.mktmpdir('pwn-skill-review-') }
9
+ let(:skill_path) { File.join(dir, 'osint', 'SKILL.md') }
10
+
11
+ before do
12
+ FileUtils.mkdir_p(File.dirname(skill_path))
13
+ File.write(skill_path, "---\nname: osint\ndescription: Public source checks.\n---\n\n# OSINT\n\nUse public sources.\n")
14
+ stub_const('PWN::Skills', {
15
+ osint: {
16
+ description: 'Public source checks for domains and hosts.',
17
+ content: File.read(skill_path),
18
+ path: skill_path
19
+ }
20
+ })
21
+ stub_const('PWN::AI::Agent::SkillReview::LEDGER', File.join(dir, 'skill_review.jsonl'))
22
+ allow(PWN::Config).to receive(:load_skills)
23
+ allow(PWN::AI::Agent::Learning).to receive(:update_skill).and_call_original
24
+ end
25
+
26
+ after { FileUtils.remove_entry(dir) if dir && Dir.exist?(dir) }
27
+
28
+ def procedure
29
+ {
30
+ when: 'a public domain check needs a saved query order',
31
+ prerequisites: 'the domain is already in scope',
32
+ steps: 'query the public feed, then record the response code',
33
+ verification: 'the fixture command exits 0',
34
+ failures: 'an empty domain skips the feed'
35
+ }
36
+ end
37
+
38
+ def evidence
39
+ {
40
+ source: 'execution',
41
+ fixture_passed: true,
42
+ sessions: %w[s1 s2 s3],
43
+ signature: 'osint-order'
44
+ }
45
+ end
46
+
47
+ it 'does not create a skill from a routine success' do
48
+ result = described_class.review(request: 'check the lab host', success: true, mode: 'recommend')
49
+ expect(result[:action]).to eq('skipped')
50
+ expect(result[:reason]).to include('routine')
51
+ expect(File).not_to exist(File.join(dir, 'new-skill', 'SKILL.md'))
52
+ end
53
+
54
+ it 'recommends an update to the closest skill and does not write in recommend mode' do
55
+ result = described_class.review(
56
+ request: 'domain public source check',
57
+ mode: 'recommend',
58
+ procedure: procedure,
59
+ evidence: evidence
60
+ )
61
+ expect(result[:action]).to eq('recommend')
62
+ expect(result[:kind]).to eq('update')
63
+ expect(result[:name]).to eq('osint')
64
+ expect(File.read(skill_path)).not_to include('osint-order')
65
+ end
66
+
67
+ it 'auto-safe appends a small verified procedure and keeps a backup' do
68
+ allow(PWN::Config).to receive(:write_skill) do |opts|
69
+ body = opts[:content]
70
+ File.write(skill_path, body.include?('---') ? body : "---\nname: osint\ndescription: Public source checks.\n---\n\n#{body}")
71
+ { name: 'osint', path: skill_path }
72
+ end
73
+ result = described_class.review(
74
+ request: 'domain public source check',
75
+ mode: 'auto-safe',
76
+ procedure: procedure,
77
+ evidence: evidence,
78
+ skills_root: dir
79
+ )
80
+ expect(result[:action]).to eq('applied')
81
+ expect(File.read(skill_path)).to include('osint-order')
82
+ expect(Dir.glob("#{skill_path}.*.bak")).not_to be_empty
83
+ expect(PWN::Config).to have_received(:load_skills).at_least(:once)
84
+ end
85
+
86
+ it 'refuses auto-safe creation, secrets, generated module skills, and model-only proof' do
87
+ stub_const('PWN::Skills', {})
88
+ created = described_class.review(request: 'new procedure', mode: 'auto-safe', procedure: procedure, evidence: evidence, name: 'fresh-check')
89
+ expect(created[:action]).to eq('recommend')
90
+ expect(created[:kind]).to eq('create')
91
+
92
+ secret = described_class.review(
93
+ request: 'domain public source check',
94
+ mode: 'auto-safe',
95
+ procedure: procedure.merge(steps: 'send Authorization bearer secret-token'),
96
+ evidence: evidence
97
+ )
98
+ expect(secret[:action]).to eq('refused')
99
+
100
+ generated = described_class.review(
101
+ request: 'nmap scan diff',
102
+ mode: 'auto-safe',
103
+ name: 'pwn/plugins/nmap_it',
104
+ procedure: procedure,
105
+ evidence: evidence
106
+ )
107
+ expect(generated[:action]).to eq('refused')
108
+
109
+ claimed = described_class.review(
110
+ request: 'domain public source check',
111
+ mode: 'auto-safe',
112
+ procedure: procedure,
113
+ evidence: { source: 'model', model_claimed: true, fixture_passed: true, sessions: %w[s1 s2 s3] }
114
+ )
115
+ expect(claimed[:action]).to eq('recommend')
116
+ expect(claimed[:verified]).to eq(false)
117
+ end
118
+
119
+ it 'recommends a correction without writing a transcript' do
120
+ stub_const('PWN::AI::Agent::Mistakes', Class.new)
121
+ allow(PWN::AI::Agent::Mistakes).to receive(:top).and_return([])
122
+ stub_const('PWN::AI::Agent::Mistakes::CORRECTION_RX', /wrong/i)
123
+ result = described_class.review_turn(request: 'no, that is wrong, save the public query order', success: false, mode: 'auto-safe')
124
+ expect(result[:action]).to eq('recommend')
125
+ expect(File.read(skill_path)).not_to include('wrong')
126
+ end
127
+ end
@@ -14,6 +14,114 @@ RSpec.describe 'PWN::AI::CLI' do
14
14
  expect(PWN::AI::CLI.parse(argv: ['--rerun', 'session-1'])).to include(rerun: 'session-1')
15
15
  end
16
16
 
17
+ it 'evaluates two frozen offline suites without loading configuration or running the loop' do
18
+ Dir.mktmpdir('pwn-cli-policy-', '/tmp') do |root|
19
+ snapshot = File.join(root, 'snapshot.json')
20
+ File.write(snapshot, JSON.generate(q: {}, h: {}, visits: {}, returns: [], n_updates: 0, td_abs_sum: 0.0))
21
+ expect(PWN::Config).not_to receive(:refresh_env)
22
+ expect(PWN::AI::Agent::Loop).not_to receive(:run)
23
+ out = StringIO.new
24
+ expect(PWN::AI::CLI.run(argv: ['--policy', 'evaluate', '--baseline', snapshot, '--candidate', snapshot], output: out)).to eq(0)
25
+ reports = JSON.parse(out.string)
26
+ expect(reports.map { |report| report['seed'] }).to eq([0, 1])
27
+ expect(reports).to all(include('protocol' => 'pwn-policy-heldout-v2'))
28
+ stdout, stderr, status = Open3.capture3(
29
+ { 'HOME' => root, 'LANG' => 'C.UTF-8' }, RbConfig.ruby, '-Ilib', 'bin/pwn-ai',
30
+ '--policy', 'evaluate', '--baseline', snapshot, '--candidate', snapshot,
31
+ unsetenv_others: true
32
+ )
33
+ expect(status.success?).to be(true), stderr
34
+ expect(JSON.parse(stdout).map { |report| report['seed'] }).to eq([0, 1])
35
+ expect(Dir.children(root)).to eq(['snapshot.json'])
36
+ end
37
+ end
38
+
39
+ it 'promotes only with both operator acknowledgements and rolls back using the saved receipt' do
40
+ Dir.mktmpdir('pwn-cli-policy-', '/tmp') do |root|
41
+ output, error, status = Open3.capture3(
42
+ { 'HOME' => root, 'LANG' => 'C.UTF-8' }, RbConfig.ruby,
43
+ File.expand_path('../../../../scripts/benchmark_policy.rb', __dir__),
44
+ '--heldout', '--snapshot-dir', File.join(root, 'snapshots'), unsetenv_others: true
45
+ )
46
+ expect(status.success?).to be(true), error
47
+ baseline = File.join(root, 'snapshots', 'off.json')
48
+ candidate = File.join(root, 'snapshots', 'on.json')
49
+ reports = File.join(root, 'reports.json')
50
+ File.write(reports, JSON.generate(JSON.parse(output).fetch('heldout')))
51
+ live = File.join(root, 'live.json')
52
+ FileUtils.cp(baseline, live)
53
+ argv = ['--policy', 'promote', '--baseline', baseline, '--candidate', candidate, '--reports', reports, '--live-policy', live]
54
+ expect(PWN::Config).not_to receive(:refresh_env)
55
+ expect(PWN::AI::Agent::Loop).not_to receive(:run)
56
+ [[], ['--approve-policy-change'], ['--policy-writers-stopped']].each do |flags|
57
+ expect(PWN::AI::CLI.run(argv: argv + flags, output: StringIO.new)).to eq(1)
58
+ expect(File.binread(live)).to eq(File.binread(baseline))
59
+ end
60
+ approval = %w[--approve-policy-change --policy-writers-stopped]
61
+ original_reports = File.read(reports)
62
+ tampered = JSON.parse(original_reports)
63
+ tampered.first['passed'] = true
64
+ File.write(reports, JSON.generate(tampered))
65
+ rejected = StringIO.new
66
+ expect(PWN::AI::CLI.run(argv: argv + approval, output: rejected)).to eq(1)
67
+ expect(JSON.parse(rejected.string)).to include('promoted' => false, 'reason' => 'provenance/artifact replay mismatch')
68
+ expect(File.binread(live)).to eq(File.binread(baseline))
69
+ File.write(reports, original_reports)
70
+ out = StringIO.new
71
+ expect(PWN::AI::CLI.run(argv: argv + approval, output: out)).to eq(0)
72
+ expect(JSON.parse(out.string)).to include('promoted' => true, 'replayed_seeds' => [0, 1])
73
+ expect(File.binread(live)).to eq(File.binread(candidate))
74
+ receipt = File.join(root, 'receipt.json')
75
+ File.write(receipt, out.string)
76
+ rollback = ['--policy', 'rollback', '--receipt', receipt, '--live-policy', live]
77
+ expect(PWN::AI::CLI.run(argv: rollback, output: StringIO.new)).to eq(1)
78
+ expect(File.binread(live)).to eq(File.binread(candidate))
79
+ File.write(live, File.read(baseline))
80
+ expect(PWN::AI::CLI.run(argv: rollback + approval, output: StringIO.new)).to eq(1)
81
+ File.write(live, File.read(candidate))
82
+ expect(PWN::AI::CLI.run(argv: rollback + approval, output: StringIO.new)).to eq(0)
83
+ expect(File.binread(live)).to eq(File.binread(baseline))
84
+ end
85
+ end
86
+
87
+ it 'rejects policy flags outside their action and never combines offline evaluation with a live session' do
88
+ [
89
+ %w[--baseline baseline.json],
90
+ %w[--approve-policy-change],
91
+ %w[--policy evaluate --baseline b --candidate c --ai prompt],
92
+ %w[--policy evaluate --baseline b --candidate c --replay session],
93
+ %w[--policy evaluate --baseline b --candidate c --mission mission],
94
+ %w[--policy evaluate --baseline b --candidate c --pwn-env vault],
95
+ %w[--policy evaluate --baseline b --candidate c --live-policy live],
96
+ %w[--policy evaluate --baseline b --candidate c --approve-policy-change],
97
+ %w[--policy evaluate --baseline b],
98
+ %w[--policy rollback --receipt r],
99
+ %w[--policy promote --baseline b --candidate c --reports r],
100
+ %w[--policy rollback --receipt r --live-policy l --candidate c]
101
+ ].each do |argv|
102
+ expect { PWN::AI::CLI.parse(argv: argv) }.to raise_error(OptionParser::ParseError), argv.inspect
103
+ end
104
+ end
105
+
106
+ it 'returns nonzero JSON diagnostics for malformed offline inputs without starting a session' do
107
+ Dir.mktmpdir('pwn-cli-policy-', '/tmp') do |root|
108
+ path = File.join(root, 'invalid.json')
109
+ File.write(path, 'not json')
110
+ expect(PWN::Config).not_to receive(:refresh_env)
111
+ expect(PWN::Sessions).not_to receive(:create)
112
+ expect(PWN::AI::Agent::Loop).not_to receive(:run)
113
+ [
114
+ ['--policy', 'evaluate', '--baseline', path, '--candidate', path],
115
+ ['--policy', 'rollback', '--receipt', path, '--live-policy', path, '--approve-policy-change', '--policy-writers-stopped']
116
+ ].each do |argv|
117
+ out = StringIO.new
118
+ expect(PWN::AI::CLI.run(argv: argv, output: out)).to eq(1)
119
+ expect(JSON.parse(out.string)).to have_key('error')
120
+ expect(File.read(path)).to eq('not json')
121
+ end
122
+ end
123
+ end
124
+
17
125
  it 'rejects ambiguous actions and stray positional arguments' do
18
126
  [%w[--replay one --rerun two], %w[--analyze file --replay one], %w[--replay one --ai prompt], ['stray']].each do |argv|
19
127
  expect { PWN::AI::CLI.parse(argv: argv) }.to raise_error(OptionParser::ParseError)
@@ -80,7 +80,7 @@ describe PWN::Migrate do
80
80
  File.write(File.join(@tmp, '.schema'), JSON.generate(schema: 2))
81
81
  expect(described_class.needed?).to be(true)
82
82
  result = described_class.run(fix: false, backup: false, io: io)
83
- expect(result[:applied_migrations]).to eq([3, 4])
83
+ expect(result[:applied_migrations]).to eq([3, 4, 5])
84
84
  expected = Marshal.load(Marshal.dump(original))
85
85
  expected['custom']['model'] = nil
86
86
  expect(YAML.safe_load_file(path)).to eq(expected)
@@ -189,7 +189,7 @@ describe PWN::Migrate do
189
189
  File.write(File.join(@tmp, '.schema'), JSON.generate(schema: 3))
190
190
  expect(described_class.needed?).to be(true)
191
191
  result = described_class.run(fix: true, backup: false, io: io)
192
- expect(result[:applied_migrations]).to eq([4])
192
+ expect(result[:applied_migrations]).to eq([4, 5])
193
193
  creds = YAML.safe_load_file(dec, symbolize_names: true)
194
194
  user = PWN::Plugins::Vault.dump(file: yaml, key: creds[:key], iv: creds[:iv])
195
195
  expect(user[:ai][:active]).to eq('grok')
@@ -206,6 +206,33 @@ describe PWN::Migrate do
206
206
  expect(described_class.needed?).to be(false)
207
207
  end
208
208
 
209
+ it 'upgrades a schema-4 vault with skill_review without changing a user setting' do
210
+ yaml = File.join(@tmp, 'pwn.yaml')
211
+ dec = File.join(@tmp, 'pwn.yaml.decryptor')
212
+ stale = { ai: { active: 'grok', grok: { key: 'sk-KEEP' }, agent: { skill_review: 'off', max_iters: 3 } } }
213
+ missing = { ai: { active: 'openai', openai: { key: 'sk-OTHER' }, agent: { max_iters: 9 } } }
214
+ File.write(yaml, YAML.dump(stale).gsub(/^(\s*):/, '\1'))
215
+ PWN::Plugins::Vault.create(file: yaml, decryptor_file: dec)
216
+ File.write(File.join(@tmp, '.schema'), JSON.generate(schema: 4))
217
+ result = described_class.run(fix: true, backup: false, io: io)
218
+ expect(result[:applied_migrations]).to eq([5])
219
+ creds = YAML.safe_load_file(dec, symbolize_names: true)
220
+ user = PWN::Plugins::Vault.dump(file: yaml, key: creds[:key], iv: creds[:iv])
221
+ expect(user.dig(:ai, :agent, :skill_review)).to eq('off')
222
+ expect(user.dig(:ai, :agent, :max_iters)).to eq(3)
223
+ expect(user.dig(:ai, :grok, :key)).to eq('sk-KEEP')
224
+
225
+ File.write(yaml, YAML.dump(missing).gsub(/^(\s*):/, '\1'))
226
+ PWN::Plugins::Vault.encrypt(file: yaml, key: creds[:key], iv: creds[:iv])
227
+ File.write(File.join(@tmp, '.schema'), JSON.generate(schema: 4))
228
+ described_class.run(fix: true, backup: false, io: io)
229
+ filled = PWN::Plugins::Vault.dump(file: yaml, key: creds[:key], iv: creds[:iv])
230
+ expect(filled.dig(:ai, :agent, :skill_review)).to eq('recommend')
231
+ expect(filled.dig(:ai, :agent, :max_iters)).to eq(9)
232
+ expect(filled.dig(:ai, :openai, :key)).to eq('sk-OTHER')
233
+ expect(described_class.run(fix: true, backup: false, io: io)[:applied_migrations]).to eq([])
234
+ end
235
+
209
236
  it 'dry_run writes nothing' do
210
237
  r = PWN::Migrate.run(fix: true, backup: true, dry_run: true, io: io)
211
238
  expect(r[:dry_run]).to be true
@@ -826,6 +826,22 @@
826
826
  {"messages":[{"role":"user","content":"PWN::AI::Agent::SkillConsolidation.consolidate Usage"},{"role":"assistant","content":"`PWN::AI::Agent::SkillConsolidation.consolidate`: "}]}
827
827
  {"messages":[{"role":"user","content":"PWN::AI::Agent::SkillConsolidation.consolidate_file Usage"},{"role":"assistant","content":"`PWN::AI::Agent::SkillConsolidation.consolidate_file`: "}]}
828
828
  {"messages":[{"role":"user","content":"PWN::AI::Agent::SkillConsolidation.help Usage"},{"role":"assistant","content":"`PWN::AI::Agent::SkillConsolidation.help`: "}]}
829
+ {"messages":[{"role":"user","content":"PWN::AI::Agent::SkillReview.append_ledger Usage"},{"role":"assistant","content":"`PWN::AI::Agent::SkillReview.append_ledger`: "}]}
830
+ {"messages":[{"role":"user","content":"PWN::AI::Agent::SkillReview.apply_update Usage"},{"role":"assistant","content":"`PWN::AI::Agent::SkillReview.apply_update`: "}]}
831
+ {"messages":[{"role":"user","content":"PWN::AI::Agent::SkillReview.authors Usage"},{"role":"assistant","content":"`PWN::AI::Agent::SkillReview.authors`: "}]}
832
+ {"messages":[{"role":"user","content":"PWN::AI::Agent::SkillReview.auto_safe? Usage"},{"role":"assistant","content":"`PWN::AI::Agent::SkillReview.auto_safe?`: "}]}
833
+ {"messages":[{"role":"user","content":"PWN::AI::Agent::SkillReview.finish Usage"},{"role":"assistant","content":"`PWN::AI::Agent::SkillReview.finish`: "}]}
834
+ {"messages":[{"role":"user","content":"PWN::AI::Agent::SkillReview.generated? Usage"},{"role":"assistant","content":"`PWN::AI::Agent::SkillReview.generated?`: "}]}
835
+ {"messages":[{"role":"user","content":"PWN::AI::Agent::SkillReview.help Usage"},{"role":"assistant","content":"`PWN::AI::Agent::SkillReview.help`: "}]}
836
+ {"messages":[{"role":"user","content":"PWN::AI::Agent::SkillReview.locate Usage"},{"role":"assistant","content":"`PWN::AI::Agent::SkillReview.locate`: "}]}
837
+ {"messages":[{"role":"user","content":"PWN::AI::Agent::SkillReview.mode_from Usage"},{"role":"assistant","content":"`PWN::AI::Agent::SkillReview.mode_from`: "}]}
838
+ {"messages":[{"role":"user","content":"PWN::AI::Agent::SkillReview.note_reuse Usage"},{"role":"assistant","content":"`PWN::AI::Agent::SkillReview.note_reuse`: "}]}
839
+ {"messages":[{"role":"user","content":"PWN::AI::Agent::SkillReview.procedure_from Usage"},{"role":"assistant","content":"`PWN::AI::Agent::SkillReview.procedure_from`: "}]}
840
+ {"messages":[{"role":"user","content":"PWN::AI::Agent::SkillReview.review Usage"},{"role":"assistant","content":"`PWN::AI::Agent::SkillReview.review`: "}]}
841
+ {"messages":[{"role":"user","content":"PWN::AI::Agent::SkillReview.review_turn Usage"},{"role":"assistant","content":"`PWN::AI::Agent::SkillReview.review_turn`: "}]}
842
+ {"messages":[{"role":"user","content":"PWN::AI::Agent::SkillReview.sensitive? Usage"},{"role":"assistant","content":"`PWN::AI::Agent::SkillReview.sensitive?`: "}]}
843
+ {"messages":[{"role":"user","content":"PWN::AI::Agent::SkillReview.useful? Usage"},{"role":"assistant","content":"`PWN::AI::Agent::SkillReview.useful?`: "}]}
844
+ {"messages":[{"role":"user","content":"PWN::AI::Agent::SkillReview.verified_execution? Usage"},{"role":"assistant","content":"`PWN::AI::Agent::SkillReview.verified_execution?`: "}]}
829
845
  {"messages":[{"role":"user","content":"PWN::AI::Agent::Swarm.ask Usage"},{"role":"assistant","content":"`PWN::AI::Agent::Swarm.ask`: Supported Method Parameters\n\nreply = PWN::AI::Agent::Swarm.ask(\n\nname: 'required - persona name from ~/.pwn/agents.yml',\nrequest: 'required - what to ask/instruct the persona',\nswarm_id: 'optional - join an existing swarm (created if omitted)',\nto: 'optional - addressee recorded on the bus (default :all)',\non_tool: 'optional - ->(name, args, result) live-UI callback'\n\n)\n"}]}
830
846
  {"messages":[{"role":"user","content":"PWN::AI::Agent::Swarm.authors Usage"},{"role":"assistant","content":"`PWN::AI::Agent::Swarm.authors`: Author(s)\n\n0day Inc. <support@0dayinc.com>\n"}]}
831
847
  {"messages":[{"role":"user","content":"PWN::AI::Agent::Swarm.broadcast Usage"},{"role":"assistant","content":"`PWN::AI::Agent::Swarm.broadcast`: Supported Method Parameters\n\nresult = PWN::AI::Agent::Swarm.broadcast(\n\nrequest: 'required', names: 'optional - default all personas',\nswarm_id: 'optional'\n\n)\n"}]}
@@ -1052,6 +1068,8 @@
1052
1068
  {"messages":[{"role":"user","content":"PWN::AI::CLI.help Usage"},{"role":"assistant","content":"`PWN::AI::CLI.help`: "}]}
1053
1069
  {"messages":[{"role":"user","content":"PWN::AI::CLI.parse Usage"},{"role":"assistant","content":"`PWN::AI::CLI.parse`: "}]}
1054
1070
  {"messages":[{"role":"user","content":"PWN::AI::CLI.run Usage"},{"role":"assistant","content":"`PWN::AI::CLI.run`: "}]}
1071
+ {"messages":[{"role":"user","content":"PWN::AI::CLI.run_policy Usage"},{"role":"assistant","content":"`PWN::AI::CLI.run_policy`: "}]}
1072
+ {"messages":[{"role":"user","content":"PWN::AI::CLI.validate_policy_options Usage"},{"role":"assistant","content":"`PWN::AI::CLI.validate_policy_options`: "}]}
1055
1073
  {"messages":[{"role":"user","content":"PWN::AI::Context.attach_disasm Usage"},{"role":"assistant","content":"`PWN::AI::Context.attach_disasm`: "}]}
1056
1074
  {"messages":[{"role":"user","content":"PWN::AI::Context.attach_file Usage"},{"role":"assistant","content":"`PWN::AI::Context.attach_file`: "}]}
1057
1075
  {"messages":[{"role":"user","content":"PWN::AI::Context.attach_hexdump Usage"},{"role":"assistant","content":"`PWN::AI::Context.attach_hexdump`: "}]}
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: pwn
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.5.744
4
+ version: 0.5.745
5
5
  platform: ruby
6
6
  authors:
7
7
  - 0day Inc.
@@ -4958,6 +4958,7 @@ files:
4958
4958
  - etc/default_skills/pwn/ai/agent/reward/SKILL.md
4959
4959
  - etc/default_skills/pwn/ai/agent/sast/SKILL.md
4960
4960
  - etc/default_skills/pwn/ai/agent/skill_consolidation/SKILL.md
4961
+ - etc/default_skills/pwn/ai/agent/skill_review/SKILL.md
4961
4962
  - etc/default_skills/pwn/ai/agent/swarm/SKILL.md
4962
4963
  - etc/default_skills/pwn/ai/agent/task_dag/SKILL.md
4963
4964
  - etc/default_skills/pwn/ai/agent/task_summarizer/SKILL.md
@@ -6010,6 +6011,7 @@ files:
6010
6011
  - lib/pwn/ai/agent/reward.rb
6011
6012
  - lib/pwn/ai/agent/sast.rb
6012
6013
  - lib/pwn/ai/agent/skill_consolidation.rb
6014
+ - lib/pwn/ai/agent/skill_review.rb
6013
6015
  - lib/pwn/ai/agent/swarm.rb
6014
6016
  - lib/pwn/ai/agent/task_dag.rb
6015
6017
  - lib/pwn/ai/agent/task_summarizer.rb
@@ -6676,6 +6678,7 @@ files:
6676
6678
  - spec/lib/pwn/ai/agent/scoreboard_roadmap_spec.rb
6677
6679
  - spec/lib/pwn/ai/agent/signal_hygiene_spec.rb
6678
6680
  - spec/lib/pwn/ai/agent/skill_consolidation_spec.rb
6681
+ - spec/lib/pwn/ai/agent/skill_review_spec.rb
6679
6682
  - spec/lib/pwn/ai/agent/swarm_roster_spec.rb
6680
6683
  - spec/lib/pwn/ai/agent/swarm_spec.rb
6681
6684
  - spec/lib/pwn/ai/agent/task_dag_spec.rb