wordpress-agent-kit 0.2.2 → 0.3.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (51) hide show
  1. package/.github/agents/wp-architect.agent.md +1 -0
  2. package/.github/skills/blueprint/SKILL.md +418 -0
  3. package/.github/skills/wordpress-router/SKILL.md +1 -0
  4. package/.github/skills/wp-abilities-api/SKILL.md +13 -0
  5. package/.github/skills/wp-abilities-api/references/delegate-helper-pattern.md +241 -0
  6. package/.github/skills/wp-abilities-api/references/domain-vs-projection.md +113 -0
  7. package/.github/skills/wp-abilities-api/references/error-code-vocabulary.md +123 -0
  8. package/.github/skills/wp-abilities-api/references/grouping-heuristic.md +89 -0
  9. package/.github/skills/wp-abilities-api/references/input-schema-gotchas.md +265 -0
  10. package/.github/skills/wp-abilities-api/references/php-registration.md +47 -20
  11. package/.github/skills/wp-abilities-api/references/plugin-family-patterns.md +233 -0
  12. package/.github/skills/wp-abilities-api/references/shared-core-service.md +184 -0
  13. package/.github/skills/wp-abilities-audit/SKILL.md +199 -0
  14. package/.github/skills/wp-abilities-audit/references/audit-schema.md +300 -0
  15. package/.github/skills/wp-abilities-audit/references/capability-gate-tracing.md +197 -0
  16. package/.github/skills/wp-abilities-audit/references/controller-enumeration.md +116 -0
  17. package/.github/skills/wp-abilities-verify/SKILL.md +215 -0
  18. package/.github/skills/wp-abilities-verify/references/annotation-correctness.md +154 -0
  19. package/.github/skills/wp-abilities-verify/references/audit-schema-validation.md +131 -0
  20. package/.github/skills/wp-abilities-verify/references/permission-roundtrip.md +190 -0
  21. package/.github/skills/wp-abilities-verify/references/runtime-harness.md +462 -0
  22. package/.github/skills/wp-abilities-verify/references/schema-lints.md +118 -0
  23. package/.github/skills/wp-abilities-verify/references/static-enumeration.md +126 -0
  24. package/.github/skills/wp-block-development/SKILL.md +1 -0
  25. package/.github/skills/wp-block-themes/SKILL.md +1 -0
  26. package/.github/skills/wp-interactivity-api/SKILL.md +1 -0
  27. package/.github/skills/wp-performance/SKILL.md +1 -0
  28. package/.github/skills/wp-phpstan/SKILL.md +1 -0
  29. package/.github/skills/wp-playground/SKILL.md +1 -0
  30. package/.github/skills/wp-plugin-development/SKILL.md +1 -0
  31. package/.github/skills/wp-plugin-directory-guidelines/SKILL.md +133 -0
  32. package/.github/skills/wp-plugin-directory-guidelines/references/gpl-compliance.md +217 -0
  33. package/.github/skills/wp-plugin-directory-guidelines/references/guideline-review-checklist.md +592 -0
  34. package/.github/skills/wp-plugin-directory-guidelines/references/naming-rules.md +121 -0
  35. package/.github/skills/wp-project-triage/SKILL.md +1 -0
  36. package/.github/skills/wp-project-triage/scripts/detect_wp_project.mjs +22 -4
  37. package/.github/skills/wp-rest-api/SKILL.md +1 -0
  38. package/.github/skills/wp-wpcli-and-ops/SKILL.md +1 -0
  39. package/.github/skills/wpds/SKILL.md +1 -0
  40. package/AGENTS.md +33 -10
  41. package/AGENTS.template.md +63 -18
  42. package/README.md +226 -124
  43. package/biome.json +1 -1
  44. package/dist/commands/install.js +47 -6
  45. package/dist/commands/upgrade.js +34 -5
  46. package/dist/lib/api.js +93 -27
  47. package/dist/lib/installer.js +113 -7
  48. package/dist/lib/updater.js +260 -0
  49. package/extensions/wp-agent-kit/index.ts +452 -0
  50. package/package.json +21 -3
  51. package/kit-learnings.md +0 -192
@@ -0,0 +1,190 @@
1
+ # Permission Roundtrip
2
+
3
+ Verify that the registered `permission_callback` on every ability
4
+ actually gates on a real capability — statically (by source
5
+ inspection) and, in runtime mode, by exercising the gate against
6
+ unauthenticated, subscriber, and admin contexts.
7
+
8
+ ## Background — what `permission_callback` actually receives
9
+
10
+ When an ability is invoked, the registered `permission_callback` is
11
+ called through `WP_Ability::check_permissions( $input )`, which
12
+ dispatches to `WP_Ability::invoke_callback( $callback, $input )` (both
13
+ defined in `class-wp-ability.php` in WordPress core).
14
+
15
+ `invoke_callback`'s contract:
16
+
17
+ - If the ability declares a non-empty `input_schema`, the callback is
18
+ invoked with one positional argument: the validated `$input` value
19
+ (whatever the schema's root type produced — array, string, integer,
20
+ boolean, etc.).
21
+ - If `input_schema` is empty or absent, the callback is invoked **with
22
+ no arguments**.
23
+
24
+ In particular, the callback **never receives a `WP_REST_Request`**,
25
+ even when the ability is reached via the REST bridge. The bridge
26
+ unwraps the request, runs schema validation, and passes the validated
27
+ value down. Permission callbacks built around `WP_REST_Request`
28
+ patterns (e.g. `$request->get_method()`) cannot work as-is when copied
29
+ from a REST controller — flag any such usage as a static FAIL.
30
+
31
+ ## Static check — classify each callback's shape
32
+
33
+ Read each ability's `permission_callback` body and classify:
34
+
35
+ | Shape | Body | Result | Notes |
36
+ |---|---|---|---|
37
+ | A | `return current_user_can( 'cap' );` | OK | Preferred. Record the resolved capability. |
38
+ | B | Branches on `$input` — different cap for different shapes | OK with smell | Two distinct user actions usually want two abilities, each with its own Shape A. See `../../wp-abilities-api/references/domain-vs-projection.md`. |
39
+ | B-bad | Branches on `$request->get_method()` or other `WP_REST_Request` calls | FAIL | The argument is the validated input value, not a request object. Almost always a copy from a REST controller without translation. |
40
+ | C | `'__return_true'` | WARN | Deliberate public ability. Document the reason in code or in the audit doc's `risks` array. |
41
+ | D | Delegates to a helper that resolves to `current_user_can(...)` | OK | Trace the helper. If it returns `true` unconditionally, treat as Shape C. |
42
+ | E | `return true;` (literal) | FAIL | Functionally Shape C but harder to grep for. Change to `'__return_true'` or add a real cap check. |
43
+ | F | `return is_user_logged_in();` | WARN | Lets any authenticated user — including subscribers — call. Rarely intended. Document or tighten. |
44
+
45
+ Record per ability: `(shape, resolved_cap)`. Any Shape B-bad or
46
+ Shape E → static FAIL. Shapes C and F → WARN. Shapes A, B, D → OK.
47
+
48
+ ## Runtime check
49
+
50
+ Exercise the gate against three user contexts using
51
+ `WP_Ability::check_permissions( $input )`.
52
+
53
+ `check_permissions()` accepts an optional input value and passes it
54
+ through to the registered `permission_callback`. Shape A callbacks
55
+ (`return current_user_can('cap')`) don't read it. Shape B callbacks
56
+ that branch on `$input` (the smell the static check flags) need a
57
+ representative value to exercise the real gate; otherwise they
58
+ receive `null` and the roundtrip result is misleading. The snippet
59
+ below passes `array()` — the minimal safe input for object-typed
60
+ schemas. For abilities with a non-object root schema, substitute a
61
+ representative value of the declared root type.
62
+
63
+ ```bash
64
+ <env-cli> wp --user=admin eval '
65
+ $ability = wp_get_ability( "<plugin>/<ability-name>" );
66
+ if ( ! $ability ) {
67
+ echo "ability not registered" . PHP_EOL;
68
+ exit( 1 );
69
+ }
70
+
71
+ $input = array(); // representative input; substitute for non-object root schemas.
72
+ $results = array();
73
+
74
+ // Unauthenticated.
75
+ wp_set_current_user( 0 );
76
+ $results["anon"] = $ability->check_permissions( $input );
77
+
78
+ // Subscriber (create a fresh user).
79
+ $sub_login = "verify_sub_" . time();
80
+ $sub_id = wp_create_user( $sub_login, "x", $sub_login . "@example.com" );
81
+ if ( ! is_wp_error( $sub_id ) ) {
82
+ $sub_user = get_user_by( "id", $sub_id );
83
+ $sub_user->set_role( "subscriber" );
84
+ wp_set_current_user( $sub_id );
85
+ $results["subscriber"] = $ability->check_permissions( $input );
86
+ }
87
+
88
+ // Admin.
89
+ wp_set_current_user( 1 );
90
+ $results["admin"] = $ability->check_permissions( $input );
91
+
92
+ foreach ( $results as $context => $result ) {
93
+ if ( true === $result ) {
94
+ $printable = "true";
95
+ } elseif ( is_wp_error( $result ) ) {
96
+ $printable = "WP_Error(" . $result->get_error_code() . ")";
97
+ } else {
98
+ $printable = var_export( $result, true );
99
+ }
100
+ echo $context . "=" . $printable . PHP_EOL;
101
+ }
102
+
103
+ // Cleanup the test subscriber so repeated harness runs don't accumulate users
104
+ // on shared dev environments. Already running as admin (line above), so the
105
+ // caller has the delete_users capability. On multisite, wp_delete_user() only
106
+ // removes the user from the current site's membership — wpmu_delete_user() in
107
+ // wp-admin/includes/ms.php is the network-wide delete.
108
+ if ( isset( $sub_id ) && ! is_wp_error( $sub_id ) ) {
109
+ if ( is_multisite() ) {
110
+ require_once ABSPATH . "wp-admin/includes/ms.php";
111
+ wpmu_delete_user( $sub_id );
112
+ } else {
113
+ require_once ABSPATH . "wp-admin/includes/user.php";
114
+ wp_delete_user( $sub_id );
115
+ }
116
+ }
117
+ '
118
+ ```
119
+
120
+ Notes on interpretation:
121
+
122
+ - `check_permissions()` returns `bool|WP_Error`. Treat `true` as
123
+ *allowed*; treat `false` or any `WP_Error` as *denied*.
124
+ - A `WP_Error` with code `ability_invalid_permission_callback` means
125
+ the registration didn't supply a valid callable — hard FAIL.
126
+ - A `WP_Error` with code `ability_callback_exception` means the
127
+ callback threw — hard FAIL; capture the underlying message.
128
+
129
+ Expected for a standard (non-public) ability:
130
+
131
+ ```
132
+ anon=false
133
+ subscriber=false
134
+ admin=true
135
+ ```
136
+
137
+ Expected for a deliberate public ability (Shape C):
138
+
139
+ ```
140
+ anon=true
141
+ subscriber=true
142
+ admin=true
143
+ ```
144
+
145
+ Any deviation → FAIL. Common causes: cap reference an admin doesn't
146
+ hold, callback bug, or permission too permissive (Shape E or F when it
147
+ should have been Shape A).
148
+
149
+ ## Audit cross-check
150
+
151
+ If an audit doc was provided, the audit's `capability_gate` (or each
152
+ ability's `permission.resolves_to`) declares what the gate should be.
153
+ Compare:
154
+
155
+ - Audit and registration resolve to the same cap → OK.
156
+ - Audit and registration disagree → FAIL. Either the audit is wrong or
157
+ the registration drifted.
158
+ - Audit declares a compound `{read, write}` gate, registration uses
159
+ Shape B with both caps → OK.
160
+ - Audit declares a compound gate, registration uses Shape A (single
161
+ cap) → FAIL. Write paths would inherit the read gate (or vice
162
+ versa), under- or over-authorizing.
163
+
164
+ See `../../wp-abilities-audit/references/capability-gate-tracing.md`
165
+ for the tracing mechanics; this skill re-derives the same trace and
166
+ diffs.
167
+
168
+ ## Output format
169
+
170
+ ```markdown
171
+ ## Permission gates
172
+
173
+ | Ability | Shape | Resolved cap(s) | anon | subscriber | admin | Audit match |
174
+ |---|---|---|---|---|---|---|
175
+ | <ability> | A | manage_options | false | false | true | OK |
176
+ | <ability> | B | edit_posts (read), delete_posts (destructive) | false | false | true | OK |
177
+ | <ability> | C | __return_true (public) | true | true | true | WARN |
178
+ | <ability> | E | (literal true) | true | true | true | FAIL |
179
+ ```
180
+
181
+ ## Static-only mode caveats
182
+
183
+ Without runtime mode, only the source-inspection columns are
184
+ populated:
185
+
186
+ | Ability | Shape | Resolved cap(s) | Audit match |
187
+ |---|---|---|---|
188
+
189
+ Roundtrip columns are omitted rather than guessed. Flag in the section
190
+ header: `Permission gates (static inspection only)`.
@@ -0,0 +1,462 @@
1
+ # Runtime Harness
2
+
3
+ The runtime-mode procedure. Everything static-mode does, plus six canonical
4
+ checks that need a live `wp_get_abilities()` call.
5
+
6
+ Static mode catches structural problems (annotation lies, schema lint
7
+ failures, audit mismatches). Runtime mode catches the class of bug that
8
+ only surfaces against a booted WordPress: missing constructor arguments,
9
+ bootstrap-ordering issues, schema-validator paths, capability-roundtrip
10
+ failures, and idempotency violations at the response-byte level.
11
+
12
+ ## Harness rule: stop on first fatal
13
+
14
+ If any step below produces a PHP fatal, STOP. Later steps won't produce
15
+ meaningful output. Capture the failure, escalate to the plugin's
16
+ implementer to fix, then re-run from step 1.
17
+
18
+ A `WP_Error` return is acceptable — any `<plugin>_*` or upstream-prefixed
19
+ error means the execute callback handled the error path gracefully. Only
20
+ PHP fatals block.
21
+
22
+ ## Step 0 — identify the env-up command
23
+
24
+ Read the plugin's `AGENTS.md` for the canonical env bring-up command. Do
25
+ NOT assume `npm run wp-env start` works for every plugin. Common patterns
26
+ seen in real plugin trees:
27
+
28
+ - `npm run wp-env start` — projects using `@wordpress/env` with a checked-in
29
+ `.wp-env.json`.
30
+ - `npx wp-env start` — projects using `@wordpress/env` without a custom
31
+ npm script wrapper.
32
+ - `composer install && composer test-php --setup-only` — package-local
33
+ test bootstraps that don't run a full WordPress install.
34
+ - `docker-compose up -d` — plugin-specific dev Docker stacks.
35
+
36
+ Plugin families with their own dev tooling will have their own bring-up
37
+ command in `AGENTS.md`; follow it as documented.
38
+
39
+ If `AGENTS.md` doesn't document it, ask the user rather than guessing.
40
+ Record the env-up command + the corresponding wp-cli invocation (e.g.
41
+ `npx wp-env run cli wp`) and use them uniformly for the rest of the
42
+ harness.
43
+
44
+ In this file, `<env-cli>` is shorthand for whatever wp-cli invocation the
45
+ plugin uses.
46
+
47
+ ## Step 1 — bring up the env and sanity-check
48
+
49
+ ```bash
50
+ <env-up-command>
51
+ <env-cli> wp core version
52
+ <env-cli> wp plugin list --status=active
53
+ <env-cli> wp eval 'var_export( function_exists( "wp_get_abilities" ) );'
54
+ ```
55
+
56
+ Confirm:
57
+
58
+ - WordPress version >= 6.9 (Abilities API available in core).
59
+ - The plugin being verified is active.
60
+ - `wp_get_abilities` exists (true if WP >= 6.9, else the Abilities API
61
+ feature plugin/package must be active).
62
+
63
+ Any "no" answer halts the harness.
64
+
65
+ ## Check 1 — ability names match source-expected list
66
+
67
+ Enumerate runtime abilities and diff against the static inventory from
68
+ `static-enumeration.md`:
69
+
70
+ ```bash
71
+ <env-cli> wp --user=admin eval '
72
+ $names = array_filter(
73
+ array_keys( (array) wp_get_abilities() ),
74
+ function ( $n ) {
75
+ return strpos( $n, "<plugin-slug>/" ) === 0;
76
+ }
77
+ );
78
+ sort( $names );
79
+ echo "count=" . count( $names ) . PHP_EOL;
80
+ echo implode( PHP_EOL, $names ) . PHP_EOL;
81
+ '
82
+ ```
83
+
84
+ Compare against the static inventory:
85
+
86
+ - Source contains ability, runtime missing → FAIL. Registration hook
87
+ isn't firing; check init hook timing and plugin activation.
88
+ - Runtime contains ability, source missing → WARN. Dynamic registration
89
+ path the enumerator couldn't follow. Document but don't block.
90
+ - Counts match → OK.
91
+
92
+ ## Check 2 — annotations read back as declared
93
+
94
+ ```bash
95
+ <env-cli> wp --user=admin eval '
96
+ $names = array_filter(
97
+ array_keys( (array) wp_get_abilities() ),
98
+ function ( $n ) {
99
+ return strpos( $n, "<plugin-slug>/" ) === 0;
100
+ }
101
+ );
102
+ sort( $names );
103
+ foreach ( $names as $name ) {
104
+ $a = wp_get_ability( $name );
105
+ $m = $a->get_meta();
106
+ printf(
107
+ "%s | readonly=%s | destructive=%s | idempotent=%s | category=%s" . PHP_EOL,
108
+ $name,
109
+ var_export( $m["annotations"]["readonly"], true ),
110
+ var_export( $m["annotations"]["destructive"], true ),
111
+ var_export( $m["annotations"]["idempotent"], true ),
112
+ $a->get_category()
113
+ );
114
+ }
115
+ '
116
+ ```
117
+
118
+ Cross-reference each annotation against the audit's declared value (if an
119
+ audit was provided) AND against the static inventory's declared value.
120
+ Mismatch on either axis → FAIL.
121
+
122
+ This check complements the static adversarial check from
123
+ `annotation-correctness.md`: static checks the callback's actual
124
+ behavior; runtime checks what the registration hook resolved to at boot
125
+ time. Both must agree for the annotations to be trustworthy.
126
+
127
+ ## Check 3 — each read ability's `execute()` behaves as the contract claims
128
+
129
+ Two verification levels. The smoke level catches bootstrap and gross-error
130
+ regressions; the high-confidence level is the one that actually exercises
131
+ the ability against the data shape it will see in production. Run both
132
+ when the audit doc provides `seed_data_needs`; run the smoke level alone
133
+ when it doesn't.
134
+
135
+ ### Level 1 — smoke execution (synthetic inputs)
136
+
137
+ Confirm each read returns `OK` or a vocabulary `WP_Error`. Catches PHP
138
+ fatals, un-bootstrapped services, registration failures.
139
+
140
+ ```bash
141
+ <env-cli> wp --user=admin eval '
142
+ $reads = array(
143
+ "<plugin-slug>/<read-ability-1>",
144
+ "<plugin-slug>/<read-ability-2>",
145
+ // ...
146
+ );
147
+ foreach ( $reads as $name ) {
148
+ $r = wp_get_ability( $name )->execute();
149
+ echo $name . ": " . ( is_wp_error( $r ) ? "WP_Error(" . $r->get_error_code() . ")" : "OK" ) . PHP_EOL;
150
+ }
151
+ '
152
+ ```
153
+
154
+ Acceptable outcomes:
155
+
156
+ - `OK` — the ability returned without error.
157
+ - `WP_Error(<plugin>_not_initialized)` — bootstrap guard fired (e.g.
158
+ un-bootstrapped service). Happy error path.
159
+ - `WP_Error(<plugin>_<resource>_data_unavailable)` — transient backend
160
+ error. Acceptable.
161
+ - `WP_Error(<upstream_code>)` — upstream third-party error bubbled
162
+ through. Document, don't block.
163
+
164
+ Unacceptable:
165
+
166
+ - PHP fatal → stop the harness.
167
+ - `WP_Error` with a non-vocabulary code → WARN. Cross-reference
168
+ `../../wp-abilities-api/references/error-code-vocabulary.md`.
169
+
170
+ Abilities with required input get invoked separately with a
171
+ synthetic-but-plausible value:
172
+
173
+ ```bash
174
+ <env-cli> wp --user=admin eval '
175
+ $r = wp_get_ability( "<plugin>/<ability-with-required-input>" )
176
+ ->execute( array( "<field>" => "<synthetic-id>" ) );
177
+ echo "<ability>: " . ( is_wp_error( $r ) ? "WP_Error(" . $r->get_error_code() . ")" : "OK" ) . PHP_EOL;
178
+ '
179
+ ```
180
+
181
+ A synthetic ID on a fresh install typically triggers
182
+ `<plugin>_<resource>_data_unavailable` or an upstream-equivalent code.
183
+ Both are acceptable for Level 1 — they mean "ability dispatched cleanly,
184
+ the backing reported no data," which is the smoke signal.
185
+
186
+ ### Level 2 — high-confidence verification (representative seeded data)
187
+
188
+ Synthetic inputs catch fatals and gross errors. They do NOT catch wrong
189
+ IDs returning the wrong record, cached sentinel values being served
190
+ instead of fresh data, permission gates appearing correct against a
191
+ synthetic ID but not against a real one, filtered labels diverging from
192
+ the unfiltered claim, missing capabilities surfacing only when a real
193
+ record is in scope, or curated output drift (a new field added to the
194
+ controller that the ability inherits and now leaks). These are the
195
+ failure modes that matter and they only fall out when the ability runs
196
+ against the data shape it will see in production.
197
+
198
+ For each ability whose audit entry declares a non-null `seed_data_needs`,
199
+ seed the environment per that field, then call the ability with
200
+ representative inputs (a real id, a real slug — not a synthetic
201
+ placeholder), and assert the output shape AND the privacy contract:
202
+
203
+ ```bash
204
+ <env-cli> wp --user=admin eval '
205
+ // Seed once at the top of the harness run, per the audit doc:
206
+ // e.g. wp post create, wp user create, wp option update, factory helpers.
207
+
208
+ $ability = wp_get_ability( "<plugin>/<read-ability>" );
209
+ $result = $ability->execute( array( "<field>" => "<real-id-from-the-seeded-data>" ) );
210
+
211
+ if ( is_wp_error( $result ) ) {
212
+ echo "FAIL: " . $result->get_error_code() . PHP_EOL;
213
+ exit;
214
+ }
215
+
216
+ // Assert the documented output shape — keys present, types correct.
217
+ $expected_keys = array( "id", "label", "status" ); // from the ability schema or audit return_type
218
+ foreach ( $expected_keys as $k ) {
219
+ if ( ! array_key_exists( $k, (array) $result ) ) {
220
+ echo "FAIL: missing output key " . $k . PHP_EOL;
221
+ exit;
222
+ }
223
+ }
224
+
225
+ // Assert the privacy contract: sensitive fields the contract does NOT
226
+ // promise must not appear (e.g. full PAN, full bank account, raw token,
227
+ // internal-only debug fields).
228
+ $forbidden_keys = array( "full_pan", "card_number", "bank_account_number", "iban" );
229
+ foreach ( $forbidden_keys as $k ) {
230
+ if ( array_key_exists( $k, (array) $result ) ) {
231
+ echo "FAIL: privacy leak — " . $k . " present in output." . PHP_EOL;
232
+ exit;
233
+ }
234
+ }
235
+
236
+ echo "PASS: shape OK, privacy OK." . PHP_EOL;
237
+ '
238
+ ```
239
+
240
+ Adapt `$expected_keys` and `$forbidden_keys` to each ability. The audit
241
+ doc's `return_type` field hints at the shape; the privacy keys depend on
242
+ the plugin's domain. For payments-family plugins, full PANs / bank
243
+ numbers / raw tokens are the canonical forbidden set; other families
244
+ substitute appropriately. The plugin's in-tree contract tests (the
245
+ overlay's `test-the-public-contract.md` for WooCommerce extensions) are
246
+ the durable home for these assertions on every CI run — this Level 2
247
+ check is the one-off harness run that produces the PR artifact.
248
+
249
+ When the audit doc declares `seed_data_needs: null`, Level 2 is not yet
250
+ runnable: the auditor has not identified the seed shape. The harness
251
+ reports `LEVEL 2: pending (seed_data_needs is null — ask the
252
+ implementer)` and proceeds. When `seed_data_needs` is a string, the
253
+ harness operator seeds per that description before running the
254
+ representative-input block above.
255
+
256
+ ## Check 4 — each write ability's missing-input returns `ability_invalid_input` or `<plugin>_missing_<field>`
257
+
258
+ ```bash
259
+ <env-cli> wp --user=admin eval '
260
+ $a = wp_get_ability( "<plugin>/<write-ability>" );
261
+
262
+ $r1 = $a->execute( array() );
263
+ echo "missing: " . ( is_wp_error( $r1 ) ? "WP_Error(" . $r1->get_error_code() . ")" : "UNEXPECTED_OK" ) . PHP_EOL;
264
+
265
+ $r2 = $a->execute( array( "<required_field>" => 123 ) );
266
+ echo "non-string: " . ( is_wp_error( $r2 ) ? "WP_Error(" . $r2->get_error_code() . ")" : "UNEXPECTED_OK" ) . PHP_EOL;
267
+ '
268
+ ```
269
+
270
+ Acceptable codes, per
271
+ `../../wp-abilities-api/references/error-code-vocabulary.md`:
272
+
273
+ - `ability_invalid_input` — the Abilities API's schema validator fired
274
+ first (normal REST-bridge path; emitted by
275
+ `WP_Ability::validate_input()` in core).
276
+ - `<plugin>_missing_<field>` — the execute callback's own guard fired
277
+ (direct-invocation path).
278
+ - `<plugin>_invalid_<field>` — same, for the wrong-type case.
279
+
280
+ `UNEXPECTED_OK` → FAIL. Validation is missing; the ability accepted
281
+ no-input and proceeded to do something it shouldn't have.
282
+
283
+ ## Check 5 — permission gate denies subscriber, allows admin
284
+
285
+ ```bash
286
+ <env-cli> wp --user=admin eval '
287
+ $ability = wp_get_ability( "<plugin>/<any-ability>" );
288
+ $input = array(); // representative input; substitute for non-object root schemas.
289
+
290
+ // Admin path.
291
+ wp_set_current_user( 1 );
292
+ $admin_result = $ability->check_permissions( $input );
293
+
294
+ // Subscriber path.
295
+ $sub_login = "verify_sub_" . time();
296
+ $sub_id = wp_create_user( $sub_login, "x", $sub_login . "@example.com" );
297
+ if ( ! is_wp_error( $sub_id ) ) {
298
+ $user = get_user_by( "id", $sub_id );
299
+ $user->set_role( "subscriber" );
300
+ wp_set_current_user( $sub_id );
301
+ $sub_result = $ability->check_permissions( $input );
302
+ } else {
303
+ $sub_result = $sub_id; // surface the create_user error in the report
304
+ }
305
+
306
+ foreach ( array( "admin" => $admin_result, "subscriber" => $sub_result ) as $context => $r ) {
307
+ if ( true === $r ) {
308
+ $printable = "true";
309
+ } elseif ( is_wp_error( $r ) ) {
310
+ $printable = "WP_Error(" . $r->get_error_code() . ")";
311
+ } else {
312
+ $printable = var_export( $r, true );
313
+ }
314
+ echo $context . ": " . $printable . PHP_EOL;
315
+ }
316
+
317
+ // Cleanup the test subscriber so repeated harness runs don't accumulate users
318
+ // on shared dev environments. Switch back to admin first since we last ran as
319
+ // the subscriber and that role doesn't hold delete_users. On multisite,
320
+ // wp_delete_user() only removes the user from the current site's membership —
321
+ // wpmu_delete_user() in wp-admin/includes/ms.php is the network-wide delete.
322
+ if ( isset( $sub_id ) && ! is_wp_error( $sub_id ) ) {
323
+ wp_set_current_user( 1 );
324
+ if ( is_multisite() ) {
325
+ require_once ABSPATH . "wp-admin/includes/ms.php";
326
+ wpmu_delete_user( $sub_id );
327
+ } else {
328
+ require_once ABSPATH . "wp-admin/includes/user.php";
329
+ wp_delete_user( $sub_id );
330
+ }
331
+ }
332
+ '
333
+ ```
334
+
335
+ Expected:
336
+
337
+ ```
338
+ admin: true
339
+ subscriber: false
340
+ ```
341
+
342
+ Any inversion is a bug in the registered `permission_callback`. See
343
+ `permission-roundtrip.md` for the deeper cross-checks (audit's declared
344
+ capability → registered callback → resolved `current_user_can(...)`).
345
+
346
+ Public abilities (deliberately ungated) expect `subscriber: true` AND
347
+ `admin: true`. Verify that the ability's `permission_callback` is
348
+ `'__return_true'` in source before accepting this outcome.
349
+
350
+ `check_permissions()` returns `bool|WP_Error` per `WP_Ability::check_permissions()`
351
+ in WordPress core. A `WP_Error` with code `ability_invalid_permission_callback`
352
+ means the registration didn't supply a valid callable — a hard FAIL. A
353
+ `WP_Error` with code `ability_callback_exception` means the callback
354
+ threw — also a hard FAIL.
355
+
356
+ ## Check 6 — twin-invocation heuristic for idempotent abilities
357
+
358
+ Only apply this to abilities annotated `idempotent: true` whose `execute()`
359
+ returned without error in Check 3. Per `annotation-correctness.md` step
360
+ "Runtime check complement", this is a *heuristic*: idempotent in core
361
+ means "no additional effect on the environment" (per the `idempotent`
362
+ annotation's docblock in `class-wp-ability.php`), not "byte-identical
363
+ return values."
364
+
365
+ ```bash
366
+ <env-cli> wp --user=admin eval '
367
+ $a = wp_get_ability( "<plugin>/<idempotent-ability>" );
368
+ $r1 = $a->execute();
369
+ $r2 = $a->execute();
370
+
371
+ if ( is_wp_error( $r1 ) || is_wp_error( $r2 ) ) {
372
+ echo "skipped: one or both invocations returned WP_Error" . PHP_EOL;
373
+ if ( is_wp_error( $r1 ) ) { echo "r1=" . $r1->get_error_code() . PHP_EOL; }
374
+ if ( is_wp_error( $r2 ) ) { echo "r2=" . $r2->get_error_code() . PHP_EOL; }
375
+ } else {
376
+ $h1 = md5( serialize( $r1 ) );
377
+ $h2 = md5( serialize( $r2 ) );
378
+ echo "match=" . var_export( $h1 === $h2, true ) . PHP_EOL;
379
+ echo "h1=" . $h1 . PHP_EOL;
380
+ echo "h2=" . $h2 . PHP_EOL;
381
+ }
382
+ '
383
+ ```
384
+
385
+ Interpretation:
386
+
387
+ - `match=true` → cheap PASS. Same input produced the same response, and
388
+ any environmental writes (write abilities) were the same on both calls.
389
+ - `match=false` → inspect what changed before deciding:
390
+ - Response embeds a per-call timestamp, nonce, or random ID →
391
+ environment unchanged. Still idempotent under core's reading.
392
+ Optionally remove the field if the agent doesn't need it; the
393
+ annotation stays `idempotent: true`.
394
+ - Response reflects a counter or sequence that grew between calls →
395
+ real environmental change. FAIL: drop the `idempotent: true`
396
+ annotation or fix the underlying write to be input-determined.
397
+
398
+ For ambiguous cases (response varies but no obvious counter), supplement
399
+ with a state diff: snapshot a representative table or option before call
400
+ 1, snapshot after call 2, diff. If state changed by more than the input
401
+ writes would explain, the ability is non-idempotent.
402
+
403
+ ## Output format
404
+
405
+ The runtime harness writes a dedicated section in the run report:
406
+
407
+ ```markdown
408
+ ## Runtime harness
409
+
410
+ **Env:** wp-env (Docker), WordPress 6.9, <plugin> <version>
411
+ **Captured:** <YYYY-MM-DD HH:MM>
412
+
413
+ ### Check 1 — enumeration
414
+
415
+ count=7 (expected 7 from static inventory)
416
+ <sorted list>
417
+
418
+ ### Check 2 — annotations
419
+
420
+ | Ability | readonly | destructive | idempotent | Matches source? |
421
+ |---|---|---|---|---|
422
+
423
+ ### Check 3 — read execution
424
+
425
+ | Ability | Result |
426
+ |---|---|
427
+
428
+ ### Check 4 — write input validation
429
+
430
+ | Ability | Missing-input code | Wrong-type code |
431
+ |---|---|---|
432
+
433
+ ### Check 5 — permission gate
434
+
435
+ | Ability | admin | subscriber | Expected |
436
+ |---|---|---|---|
437
+
438
+ ### Check 6 — idempotency
439
+
440
+ | Ability | match | h1 | h2 |
441
+ |---|---|---|---|
442
+
443
+ ### Notes / surprises
444
+
445
+ <Anything unexpected that didn't block.>
446
+ ```
447
+
448
+ ## When the harness catches a bug
449
+
450
+ Observed pattern:
451
+
452
+ 1. Harness surfaces a PHP fatal (e.g. `ArgumentCountError: Too few
453
+ arguments to function <controller>::__construct`).
454
+ 2. Implementer fixes the bug in a focused commit.
455
+ 3. Harness re-runs; write the post-fix output as the headline status.
456
+ 4. Preserve the pre-fix trace in the report under a "Pre-fix status"
457
+ section so reviewers can see what verify caught.
458
+
459
+ This is the highest-leverage signal the runtime harness produces:
460
+ bootstrap-ordering and missing-dependency bugs that pass static review
461
+ because the source declares the right shape — they only manifest when
462
+ the registration runs against a booted WordPress.