wordpress-agent-kit 0.3.0 → 0.3.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.github/skills/blueprint/SKILL.md +418 -0
- package/.github/skills/wp-abilities-api/SKILL.md +12 -0
- package/.github/skills/wp-abilities-api/references/delegate-helper-pattern.md +241 -0
- package/.github/skills/wp-abilities-api/references/domain-vs-projection.md +113 -0
- package/.github/skills/wp-abilities-api/references/error-code-vocabulary.md +123 -0
- package/.github/skills/wp-abilities-api/references/grouping-heuristic.md +89 -0
- package/.github/skills/wp-abilities-api/references/input-schema-gotchas.md +265 -0
- package/.github/skills/wp-abilities-api/references/php-registration.md +47 -20
- package/.github/skills/wp-abilities-api/references/plugin-family-patterns.md +233 -0
- package/.github/skills/wp-abilities-api/references/shared-core-service.md +184 -0
- package/.github/skills/wp-abilities-audit/SKILL.md +199 -0
- package/.github/skills/wp-abilities-audit/references/audit-schema.md +300 -0
- package/.github/skills/wp-abilities-audit/references/capability-gate-tracing.md +197 -0
- package/.github/skills/wp-abilities-audit/references/controller-enumeration.md +116 -0
- package/.github/skills/wp-abilities-verify/SKILL.md +215 -0
- package/.github/skills/wp-abilities-verify/references/annotation-correctness.md +154 -0
- package/.github/skills/wp-abilities-verify/references/audit-schema-validation.md +131 -0
- package/.github/skills/wp-abilities-verify/references/permission-roundtrip.md +190 -0
- package/.github/skills/wp-abilities-verify/references/runtime-harness.md +462 -0
- package/.github/skills/wp-abilities-verify/references/schema-lints.md +118 -0
- package/.github/skills/wp-abilities-verify/references/static-enumeration.md +126 -0
- package/.github/skills/wp-plugin-directory-guidelines/SKILL.md +133 -0
- package/.github/skills/wp-plugin-directory-guidelines/references/gpl-compliance.md +217 -0
- package/.github/skills/wp-plugin-directory-guidelines/references/guideline-review-checklist.md +592 -0
- package/.github/skills/wp-plugin-directory-guidelines/references/naming-rules.md +121 -0
- package/.github/skills/wp-project-triage/scripts/detect_wp_project.mjs +22 -4
- package/README.md +6 -3
- package/dist/lib/api.js +30 -19
- package/dist/lib/installer.js +0 -2
- package/extensions/wp-agent-kit/index.ts +146 -324
- package/package.json +1 -1
|
@@ -0,0 +1,190 @@
|
|
|
1
|
+
# Permission Roundtrip
|
|
2
|
+
|
|
3
|
+
Verify that the registered `permission_callback` on every ability
|
|
4
|
+
actually gates on a real capability — statically (by source
|
|
5
|
+
inspection) and, in runtime mode, by exercising the gate against
|
|
6
|
+
unauthenticated, subscriber, and admin contexts.
|
|
7
|
+
|
|
8
|
+
## Background — what `permission_callback` actually receives
|
|
9
|
+
|
|
10
|
+
When an ability is invoked, the registered `permission_callback` is
|
|
11
|
+
called through `WP_Ability::check_permissions( $input )`, which
|
|
12
|
+
dispatches to `WP_Ability::invoke_callback( $callback, $input )` (both
|
|
13
|
+
defined in `class-wp-ability.php` in WordPress core).
|
|
14
|
+
|
|
15
|
+
`invoke_callback`'s contract:
|
|
16
|
+
|
|
17
|
+
- If the ability declares a non-empty `input_schema`, the callback is
|
|
18
|
+
invoked with one positional argument: the validated `$input` value
|
|
19
|
+
(whatever the schema's root type produced — array, string, integer,
|
|
20
|
+
boolean, etc.).
|
|
21
|
+
- If `input_schema` is empty or absent, the callback is invoked **with
|
|
22
|
+
no arguments**.
|
|
23
|
+
|
|
24
|
+
In particular, the callback **never receives a `WP_REST_Request`**,
|
|
25
|
+
even when the ability is reached via the REST bridge. The bridge
|
|
26
|
+
unwraps the request, runs schema validation, and passes the validated
|
|
27
|
+
value down. Permission callbacks built around `WP_REST_Request`
|
|
28
|
+
patterns (e.g. `$request->get_method()`) cannot work as-is when copied
|
|
29
|
+
from a REST controller — flag any such usage as a static FAIL.
|
|
30
|
+
|
|
31
|
+
## Static check — classify each callback's shape
|
|
32
|
+
|
|
33
|
+
Read each ability's `permission_callback` body and classify:
|
|
34
|
+
|
|
35
|
+
| Shape | Body | Result | Notes |
|
|
36
|
+
|---|---|---|---|
|
|
37
|
+
| A | `return current_user_can( 'cap' );` | OK | Preferred. Record the resolved capability. |
|
|
38
|
+
| B | Branches on `$input` — different cap for different shapes | OK with smell | Two distinct user actions usually want two abilities, each with its own Shape A. See `../../wp-abilities-api/references/domain-vs-projection.md`. |
|
|
39
|
+
| B-bad | Branches on `$request->get_method()` or other `WP_REST_Request` calls | FAIL | The argument is the validated input value, not a request object. Almost always a copy from a REST controller without translation. |
|
|
40
|
+
| C | `'__return_true'` | WARN | Deliberate public ability. Document the reason in code or in the audit doc's `risks` array. |
|
|
41
|
+
| D | Delegates to a helper that resolves to `current_user_can(...)` | OK | Trace the helper. If it returns `true` unconditionally, treat as Shape C. |
|
|
42
|
+
| E | `return true;` (literal) | FAIL | Functionally Shape C but harder to grep for. Change to `'__return_true'` or add a real cap check. |
|
|
43
|
+
| F | `return is_user_logged_in();` | WARN | Lets any authenticated user — including subscribers — call. Rarely intended. Document or tighten. |
|
|
44
|
+
|
|
45
|
+
Record per ability: `(shape, resolved_cap)`. Any Shape B-bad or
|
|
46
|
+
Shape E → static FAIL. Shapes C and F → WARN. Shapes A, B, D → OK.
|
|
47
|
+
|
|
48
|
+
## Runtime check
|
|
49
|
+
|
|
50
|
+
Exercise the gate against three user contexts using
|
|
51
|
+
`WP_Ability::check_permissions( $input )`.
|
|
52
|
+
|
|
53
|
+
`check_permissions()` accepts an optional input value and passes it
|
|
54
|
+
through to the registered `permission_callback`. Shape A callbacks
|
|
55
|
+
(`return current_user_can('cap')`) don't read it. Shape B callbacks
|
|
56
|
+
that branch on `$input` (the smell the static check flags) need a
|
|
57
|
+
representative value to exercise the real gate; otherwise they
|
|
58
|
+
receive `null` and the roundtrip result is misleading. The snippet
|
|
59
|
+
below passes `array()` — the minimal safe input for object-typed
|
|
60
|
+
schemas. For abilities with a non-object root schema, substitute a
|
|
61
|
+
representative value of the declared root type.
|
|
62
|
+
|
|
63
|
+
```bash
|
|
64
|
+
<env-cli> wp --user=admin eval '
|
|
65
|
+
$ability = wp_get_ability( "<plugin>/<ability-name>" );
|
|
66
|
+
if ( ! $ability ) {
|
|
67
|
+
echo "ability not registered" . PHP_EOL;
|
|
68
|
+
exit( 1 );
|
|
69
|
+
}
|
|
70
|
+
|
|
71
|
+
$input = array(); // representative input; substitute for non-object root schemas.
|
|
72
|
+
$results = array();
|
|
73
|
+
|
|
74
|
+
// Unauthenticated.
|
|
75
|
+
wp_set_current_user( 0 );
|
|
76
|
+
$results["anon"] = $ability->check_permissions( $input );
|
|
77
|
+
|
|
78
|
+
// Subscriber (create a fresh user).
|
|
79
|
+
$sub_login = "verify_sub_" . time();
|
|
80
|
+
$sub_id = wp_create_user( $sub_login, "x", $sub_login . "@example.com" );
|
|
81
|
+
if ( ! is_wp_error( $sub_id ) ) {
|
|
82
|
+
$sub_user = get_user_by( "id", $sub_id );
|
|
83
|
+
$sub_user->set_role( "subscriber" );
|
|
84
|
+
wp_set_current_user( $sub_id );
|
|
85
|
+
$results["subscriber"] = $ability->check_permissions( $input );
|
|
86
|
+
}
|
|
87
|
+
|
|
88
|
+
// Admin.
|
|
89
|
+
wp_set_current_user( 1 );
|
|
90
|
+
$results["admin"] = $ability->check_permissions( $input );
|
|
91
|
+
|
|
92
|
+
foreach ( $results as $context => $result ) {
|
|
93
|
+
if ( true === $result ) {
|
|
94
|
+
$printable = "true";
|
|
95
|
+
} elseif ( is_wp_error( $result ) ) {
|
|
96
|
+
$printable = "WP_Error(" . $result->get_error_code() . ")";
|
|
97
|
+
} else {
|
|
98
|
+
$printable = var_export( $result, true );
|
|
99
|
+
}
|
|
100
|
+
echo $context . "=" . $printable . PHP_EOL;
|
|
101
|
+
}
|
|
102
|
+
|
|
103
|
+
// Cleanup the test subscriber so repeated harness runs don't accumulate users
|
|
104
|
+
// on shared dev environments. Already running as admin (line above), so the
|
|
105
|
+
// caller has the delete_users capability. On multisite, wp_delete_user() only
|
|
106
|
+
// removes the user from the current site's membership — wpmu_delete_user() in
|
|
107
|
+
// wp-admin/includes/ms.php is the network-wide delete.
|
|
108
|
+
if ( isset( $sub_id ) && ! is_wp_error( $sub_id ) ) {
|
|
109
|
+
if ( is_multisite() ) {
|
|
110
|
+
require_once ABSPATH . "wp-admin/includes/ms.php";
|
|
111
|
+
wpmu_delete_user( $sub_id );
|
|
112
|
+
} else {
|
|
113
|
+
require_once ABSPATH . "wp-admin/includes/user.php";
|
|
114
|
+
wp_delete_user( $sub_id );
|
|
115
|
+
}
|
|
116
|
+
}
|
|
117
|
+
'
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
Notes on interpretation:
|
|
121
|
+
|
|
122
|
+
- `check_permissions()` returns `bool|WP_Error`. Treat `true` as
|
|
123
|
+
*allowed*; treat `false` or any `WP_Error` as *denied*.
|
|
124
|
+
- A `WP_Error` with code `ability_invalid_permission_callback` means
|
|
125
|
+
the registration didn't supply a valid callable — hard FAIL.
|
|
126
|
+
- A `WP_Error` with code `ability_callback_exception` means the
|
|
127
|
+
callback threw — hard FAIL; capture the underlying message.
|
|
128
|
+
|
|
129
|
+
Expected for a standard (non-public) ability:
|
|
130
|
+
|
|
131
|
+
```
|
|
132
|
+
anon=false
|
|
133
|
+
subscriber=false
|
|
134
|
+
admin=true
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
Expected for a deliberate public ability (Shape C):
|
|
138
|
+
|
|
139
|
+
```
|
|
140
|
+
anon=true
|
|
141
|
+
subscriber=true
|
|
142
|
+
admin=true
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
Any deviation → FAIL. Common causes: cap reference an admin doesn't
|
|
146
|
+
hold, callback bug, or permission too permissive (Shape E or F when it
|
|
147
|
+
should have been Shape A).
|
|
148
|
+
|
|
149
|
+
## Audit cross-check
|
|
150
|
+
|
|
151
|
+
If an audit doc was provided, the audit's `capability_gate` (or each
|
|
152
|
+
ability's `permission.resolves_to`) declares what the gate should be.
|
|
153
|
+
Compare:
|
|
154
|
+
|
|
155
|
+
- Audit and registration resolve to the same cap → OK.
|
|
156
|
+
- Audit and registration disagree → FAIL. Either the audit is wrong or
|
|
157
|
+
the registration drifted.
|
|
158
|
+
- Audit declares a compound `{read, write}` gate, registration uses
|
|
159
|
+
Shape B with both caps → OK.
|
|
160
|
+
- Audit declares a compound gate, registration uses Shape A (single
|
|
161
|
+
cap) → FAIL. Write paths would inherit the read gate (or vice
|
|
162
|
+
versa), under- or over-authorizing.
|
|
163
|
+
|
|
164
|
+
See `../../wp-abilities-audit/references/capability-gate-tracing.md`
|
|
165
|
+
for the tracing mechanics; this skill re-derives the same trace and
|
|
166
|
+
diffs.
|
|
167
|
+
|
|
168
|
+
## Output format
|
|
169
|
+
|
|
170
|
+
```markdown
|
|
171
|
+
## Permission gates
|
|
172
|
+
|
|
173
|
+
| Ability | Shape | Resolved cap(s) | anon | subscriber | admin | Audit match |
|
|
174
|
+
|---|---|---|---|---|---|---|
|
|
175
|
+
| <ability> | A | manage_options | false | false | true | OK |
|
|
176
|
+
| <ability> | B | edit_posts (read), delete_posts (destructive) | false | false | true | OK |
|
|
177
|
+
| <ability> | C | __return_true (public) | true | true | true | WARN |
|
|
178
|
+
| <ability> | E | (literal true) | true | true | true | FAIL |
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
## Static-only mode caveats
|
|
182
|
+
|
|
183
|
+
Without runtime mode, only the source-inspection columns are
|
|
184
|
+
populated:
|
|
185
|
+
|
|
186
|
+
| Ability | Shape | Resolved cap(s) | Audit match |
|
|
187
|
+
|---|---|---|---|
|
|
188
|
+
|
|
189
|
+
Roundtrip columns are omitted rather than guessed. Flag in the section
|
|
190
|
+
header: `Permission gates (static inspection only)`.
|
|
@@ -0,0 +1,462 @@
|
|
|
1
|
+
# Runtime Harness
|
|
2
|
+
|
|
3
|
+
The runtime-mode procedure. Everything static-mode does, plus six canonical
|
|
4
|
+
checks that need a live `wp_get_abilities()` call.
|
|
5
|
+
|
|
6
|
+
Static mode catches structural problems (annotation lies, schema lint
|
|
7
|
+
failures, audit mismatches). Runtime mode catches the class of bug that
|
|
8
|
+
only surfaces against a booted WordPress: missing constructor arguments,
|
|
9
|
+
bootstrap-ordering issues, schema-validator paths, capability-roundtrip
|
|
10
|
+
failures, and idempotency violations at the response-byte level.
|
|
11
|
+
|
|
12
|
+
## Harness rule: stop on first fatal
|
|
13
|
+
|
|
14
|
+
If any step below produces a PHP fatal, STOP. Later steps won't produce
|
|
15
|
+
meaningful output. Capture the failure, escalate to the plugin's
|
|
16
|
+
implementer to fix, then re-run from step 1.
|
|
17
|
+
|
|
18
|
+
A `WP_Error` return is acceptable — any `<plugin>_*` or upstream-prefixed
|
|
19
|
+
error means the execute callback handled the error path gracefully. Only
|
|
20
|
+
PHP fatals block.
|
|
21
|
+
|
|
22
|
+
## Step 0 — identify the env-up command
|
|
23
|
+
|
|
24
|
+
Read the plugin's `AGENTS.md` for the canonical env bring-up command. Do
|
|
25
|
+
NOT assume `npm run wp-env start` works for every plugin. Common patterns
|
|
26
|
+
seen in real plugin trees:
|
|
27
|
+
|
|
28
|
+
- `npm run wp-env start` — projects using `@wordpress/env` with a checked-in
|
|
29
|
+
`.wp-env.json`.
|
|
30
|
+
- `npx wp-env start` — projects using `@wordpress/env` without a custom
|
|
31
|
+
npm script wrapper.
|
|
32
|
+
- `composer install && composer test-php --setup-only` — package-local
|
|
33
|
+
test bootstraps that don't run a full WordPress install.
|
|
34
|
+
- `docker-compose up -d` — plugin-specific dev Docker stacks.
|
|
35
|
+
|
|
36
|
+
Plugin families with their own dev tooling will have their own bring-up
|
|
37
|
+
command in `AGENTS.md`; follow it as documented.
|
|
38
|
+
|
|
39
|
+
If `AGENTS.md` doesn't document it, ask the user rather than guessing.
|
|
40
|
+
Record the env-up command + the corresponding wp-cli invocation (e.g.
|
|
41
|
+
`npx wp-env run cli wp`) and use them uniformly for the rest of the
|
|
42
|
+
harness.
|
|
43
|
+
|
|
44
|
+
In this file, `<env-cli>` is shorthand for whatever wp-cli invocation the
|
|
45
|
+
plugin uses.
|
|
46
|
+
|
|
47
|
+
## Step 1 — bring up the env and sanity-check
|
|
48
|
+
|
|
49
|
+
```bash
|
|
50
|
+
<env-up-command>
|
|
51
|
+
<env-cli> wp core version
|
|
52
|
+
<env-cli> wp plugin list --status=active
|
|
53
|
+
<env-cli> wp eval 'var_export( function_exists( "wp_get_abilities" ) );'
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
Confirm:
|
|
57
|
+
|
|
58
|
+
- WordPress version >= 6.9 (Abilities API available in core).
|
|
59
|
+
- The plugin being verified is active.
|
|
60
|
+
- `wp_get_abilities` exists (true if WP >= 6.9, else the Abilities API
|
|
61
|
+
feature plugin/package must be active).
|
|
62
|
+
|
|
63
|
+
Any "no" answer halts the harness.
|
|
64
|
+
|
|
65
|
+
## Check 1 — ability names match source-expected list
|
|
66
|
+
|
|
67
|
+
Enumerate runtime abilities and diff against the static inventory from
|
|
68
|
+
`static-enumeration.md`:
|
|
69
|
+
|
|
70
|
+
```bash
|
|
71
|
+
<env-cli> wp --user=admin eval '
|
|
72
|
+
$names = array_filter(
|
|
73
|
+
array_keys( (array) wp_get_abilities() ),
|
|
74
|
+
function ( $n ) {
|
|
75
|
+
return strpos( $n, "<plugin-slug>/" ) === 0;
|
|
76
|
+
}
|
|
77
|
+
);
|
|
78
|
+
sort( $names );
|
|
79
|
+
echo "count=" . count( $names ) . PHP_EOL;
|
|
80
|
+
echo implode( PHP_EOL, $names ) . PHP_EOL;
|
|
81
|
+
'
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
Compare against the static inventory:
|
|
85
|
+
|
|
86
|
+
- Source contains ability, runtime missing → FAIL. Registration hook
|
|
87
|
+
isn't firing; check init hook timing and plugin activation.
|
|
88
|
+
- Runtime contains ability, source missing → WARN. Dynamic registration
|
|
89
|
+
path the enumerator couldn't follow. Document but don't block.
|
|
90
|
+
- Counts match → OK.
|
|
91
|
+
|
|
92
|
+
## Check 2 — annotations read back as declared
|
|
93
|
+
|
|
94
|
+
```bash
|
|
95
|
+
<env-cli> wp --user=admin eval '
|
|
96
|
+
$names = array_filter(
|
|
97
|
+
array_keys( (array) wp_get_abilities() ),
|
|
98
|
+
function ( $n ) {
|
|
99
|
+
return strpos( $n, "<plugin-slug>/" ) === 0;
|
|
100
|
+
}
|
|
101
|
+
);
|
|
102
|
+
sort( $names );
|
|
103
|
+
foreach ( $names as $name ) {
|
|
104
|
+
$a = wp_get_ability( $name );
|
|
105
|
+
$m = $a->get_meta();
|
|
106
|
+
printf(
|
|
107
|
+
"%s | readonly=%s | destructive=%s | idempotent=%s | category=%s" . PHP_EOL,
|
|
108
|
+
$name,
|
|
109
|
+
var_export( $m["annotations"]["readonly"], true ),
|
|
110
|
+
var_export( $m["annotations"]["destructive"], true ),
|
|
111
|
+
var_export( $m["annotations"]["idempotent"], true ),
|
|
112
|
+
$a->get_category()
|
|
113
|
+
);
|
|
114
|
+
}
|
|
115
|
+
'
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
Cross-reference each annotation against the audit's declared value (if an
|
|
119
|
+
audit was provided) AND against the static inventory's declared value.
|
|
120
|
+
Mismatch on either axis → FAIL.
|
|
121
|
+
|
|
122
|
+
This check complements the static adversarial check from
|
|
123
|
+
`annotation-correctness.md`: static checks the callback's actual
|
|
124
|
+
behavior; runtime checks what the registration hook resolved to at boot
|
|
125
|
+
time. Both must agree for the annotations to be trustworthy.
|
|
126
|
+
|
|
127
|
+
## Check 3 — each read ability's `execute()` behaves as the contract claims
|
|
128
|
+
|
|
129
|
+
Two verification levels. The smoke level catches bootstrap and gross-error
|
|
130
|
+
regressions; the high-confidence level is the one that actually exercises
|
|
131
|
+
the ability against the data shape it will see in production. Run both
|
|
132
|
+
when the audit doc provides `seed_data_needs`; run the smoke level alone
|
|
133
|
+
when it doesn't.
|
|
134
|
+
|
|
135
|
+
### Level 1 — smoke execution (synthetic inputs)
|
|
136
|
+
|
|
137
|
+
Confirm each read returns `OK` or a vocabulary `WP_Error`. Catches PHP
|
|
138
|
+
fatals, un-bootstrapped services, registration failures.
|
|
139
|
+
|
|
140
|
+
```bash
|
|
141
|
+
<env-cli> wp --user=admin eval '
|
|
142
|
+
$reads = array(
|
|
143
|
+
"<plugin-slug>/<read-ability-1>",
|
|
144
|
+
"<plugin-slug>/<read-ability-2>",
|
|
145
|
+
// ...
|
|
146
|
+
);
|
|
147
|
+
foreach ( $reads as $name ) {
|
|
148
|
+
$r = wp_get_ability( $name )->execute();
|
|
149
|
+
echo $name . ": " . ( is_wp_error( $r ) ? "WP_Error(" . $r->get_error_code() . ")" : "OK" ) . PHP_EOL;
|
|
150
|
+
}
|
|
151
|
+
'
|
|
152
|
+
```
|
|
153
|
+
|
|
154
|
+
Acceptable outcomes:
|
|
155
|
+
|
|
156
|
+
- `OK` — the ability returned without error.
|
|
157
|
+
- `WP_Error(<plugin>_not_initialized)` — bootstrap guard fired (e.g.
|
|
158
|
+
un-bootstrapped service). Happy error path.
|
|
159
|
+
- `WP_Error(<plugin>_<resource>_data_unavailable)` — transient backend
|
|
160
|
+
error. Acceptable.
|
|
161
|
+
- `WP_Error(<upstream_code>)` — upstream third-party error bubbled
|
|
162
|
+
through. Document, don't block.
|
|
163
|
+
|
|
164
|
+
Unacceptable:
|
|
165
|
+
|
|
166
|
+
- PHP fatal → stop the harness.
|
|
167
|
+
- `WP_Error` with a non-vocabulary code → WARN. Cross-reference
|
|
168
|
+
`../../wp-abilities-api/references/error-code-vocabulary.md`.
|
|
169
|
+
|
|
170
|
+
Abilities with required input get invoked separately with a
|
|
171
|
+
synthetic-but-plausible value:
|
|
172
|
+
|
|
173
|
+
```bash
|
|
174
|
+
<env-cli> wp --user=admin eval '
|
|
175
|
+
$r = wp_get_ability( "<plugin>/<ability-with-required-input>" )
|
|
176
|
+
->execute( array( "<field>" => "<synthetic-id>" ) );
|
|
177
|
+
echo "<ability>: " . ( is_wp_error( $r ) ? "WP_Error(" . $r->get_error_code() . ")" : "OK" ) . PHP_EOL;
|
|
178
|
+
'
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
A synthetic ID on a fresh install typically triggers
|
|
182
|
+
`<plugin>_<resource>_data_unavailable` or an upstream-equivalent code.
|
|
183
|
+
Both are acceptable for Level 1 — they mean "ability dispatched cleanly,
|
|
184
|
+
the backing reported no data," which is the smoke signal.
|
|
185
|
+
|
|
186
|
+
### Level 2 — high-confidence verification (representative seeded data)
|
|
187
|
+
|
|
188
|
+
Synthetic inputs catch fatals and gross errors. They do NOT catch wrong
|
|
189
|
+
IDs returning the wrong record, cached sentinel values being served
|
|
190
|
+
instead of fresh data, permission gates appearing correct against a
|
|
191
|
+
synthetic ID but not against a real one, filtered labels diverging from
|
|
192
|
+
the unfiltered claim, missing capabilities surfacing only when a real
|
|
193
|
+
record is in scope, or curated output drift (a new field added to the
|
|
194
|
+
controller that the ability inherits and now leaks). These are the
|
|
195
|
+
failure modes that matter and they only fall out when the ability runs
|
|
196
|
+
against the data shape it will see in production.
|
|
197
|
+
|
|
198
|
+
For each ability whose audit entry declares a non-null `seed_data_needs`,
|
|
199
|
+
seed the environment per that field, then call the ability with
|
|
200
|
+
representative inputs (a real id, a real slug — not a synthetic
|
|
201
|
+
placeholder), and assert the output shape AND the privacy contract:
|
|
202
|
+
|
|
203
|
+
```bash
|
|
204
|
+
<env-cli> wp --user=admin eval '
|
|
205
|
+
// Seed once at the top of the harness run, per the audit doc:
|
|
206
|
+
// e.g. wp post create, wp user create, wp option update, factory helpers.
|
|
207
|
+
|
|
208
|
+
$ability = wp_get_ability( "<plugin>/<read-ability>" );
|
|
209
|
+
$result = $ability->execute( array( "<field>" => "<real-id-from-the-seeded-data>" ) );
|
|
210
|
+
|
|
211
|
+
if ( is_wp_error( $result ) ) {
|
|
212
|
+
echo "FAIL: " . $result->get_error_code() . PHP_EOL;
|
|
213
|
+
exit;
|
|
214
|
+
}
|
|
215
|
+
|
|
216
|
+
// Assert the documented output shape — keys present, types correct.
|
|
217
|
+
$expected_keys = array( "id", "label", "status" ); // from the ability schema or audit return_type
|
|
218
|
+
foreach ( $expected_keys as $k ) {
|
|
219
|
+
if ( ! array_key_exists( $k, (array) $result ) ) {
|
|
220
|
+
echo "FAIL: missing output key " . $k . PHP_EOL;
|
|
221
|
+
exit;
|
|
222
|
+
}
|
|
223
|
+
}
|
|
224
|
+
|
|
225
|
+
// Assert the privacy contract: sensitive fields the contract does NOT
|
|
226
|
+
// promise must not appear (e.g. full PAN, full bank account, raw token,
|
|
227
|
+
// internal-only debug fields).
|
|
228
|
+
$forbidden_keys = array( "full_pan", "card_number", "bank_account_number", "iban" );
|
|
229
|
+
foreach ( $forbidden_keys as $k ) {
|
|
230
|
+
if ( array_key_exists( $k, (array) $result ) ) {
|
|
231
|
+
echo "FAIL: privacy leak — " . $k . " present in output." . PHP_EOL;
|
|
232
|
+
exit;
|
|
233
|
+
}
|
|
234
|
+
}
|
|
235
|
+
|
|
236
|
+
echo "PASS: shape OK, privacy OK." . PHP_EOL;
|
|
237
|
+
'
|
|
238
|
+
```
|
|
239
|
+
|
|
240
|
+
Adapt `$expected_keys` and `$forbidden_keys` to each ability. The audit
|
|
241
|
+
doc's `return_type` field hints at the shape; the privacy keys depend on
|
|
242
|
+
the plugin's domain. For payments-family plugins, full PANs / bank
|
|
243
|
+
numbers / raw tokens are the canonical forbidden set; other families
|
|
244
|
+
substitute appropriately. The plugin's in-tree contract tests (the
|
|
245
|
+
overlay's `test-the-public-contract.md` for WooCommerce extensions) are
|
|
246
|
+
the durable home for these assertions on every CI run — this Level 2
|
|
247
|
+
check is the one-off harness run that produces the PR artifact.
|
|
248
|
+
|
|
249
|
+
When the audit doc declares `seed_data_needs: null`, Level 2 is not yet
|
|
250
|
+
runnable: the auditor has not identified the seed shape. The harness
|
|
251
|
+
reports `LEVEL 2: pending (seed_data_needs is null — ask the
|
|
252
|
+
implementer)` and proceeds. When `seed_data_needs` is a string, the
|
|
253
|
+
harness operator seeds per that description before running the
|
|
254
|
+
representative-input block above.
|
|
255
|
+
|
|
256
|
+
## Check 4 — each write ability's missing-input returns `ability_invalid_input` or `<plugin>_missing_<field>`
|
|
257
|
+
|
|
258
|
+
```bash
|
|
259
|
+
<env-cli> wp --user=admin eval '
|
|
260
|
+
$a = wp_get_ability( "<plugin>/<write-ability>" );
|
|
261
|
+
|
|
262
|
+
$r1 = $a->execute( array() );
|
|
263
|
+
echo "missing: " . ( is_wp_error( $r1 ) ? "WP_Error(" . $r1->get_error_code() . ")" : "UNEXPECTED_OK" ) . PHP_EOL;
|
|
264
|
+
|
|
265
|
+
$r2 = $a->execute( array( "<required_field>" => 123 ) );
|
|
266
|
+
echo "non-string: " . ( is_wp_error( $r2 ) ? "WP_Error(" . $r2->get_error_code() . ")" : "UNEXPECTED_OK" ) . PHP_EOL;
|
|
267
|
+
'
|
|
268
|
+
```
|
|
269
|
+
|
|
270
|
+
Acceptable codes, per
|
|
271
|
+
`../../wp-abilities-api/references/error-code-vocabulary.md`:
|
|
272
|
+
|
|
273
|
+
- `ability_invalid_input` — the Abilities API's schema validator fired
|
|
274
|
+
first (normal REST-bridge path; emitted by
|
|
275
|
+
`WP_Ability::validate_input()` in core).
|
|
276
|
+
- `<plugin>_missing_<field>` — the execute callback's own guard fired
|
|
277
|
+
(direct-invocation path).
|
|
278
|
+
- `<plugin>_invalid_<field>` — same, for the wrong-type case.
|
|
279
|
+
|
|
280
|
+
`UNEXPECTED_OK` → FAIL. Validation is missing; the ability accepted
|
|
281
|
+
no-input and proceeded to do something it shouldn't have.
|
|
282
|
+
|
|
283
|
+
## Check 5 — permission gate denies subscriber, allows admin
|
|
284
|
+
|
|
285
|
+
```bash
|
|
286
|
+
<env-cli> wp --user=admin eval '
|
|
287
|
+
$ability = wp_get_ability( "<plugin>/<any-ability>" );
|
|
288
|
+
$input = array(); // representative input; substitute for non-object root schemas.
|
|
289
|
+
|
|
290
|
+
// Admin path.
|
|
291
|
+
wp_set_current_user( 1 );
|
|
292
|
+
$admin_result = $ability->check_permissions( $input );
|
|
293
|
+
|
|
294
|
+
// Subscriber path.
|
|
295
|
+
$sub_login = "verify_sub_" . time();
|
|
296
|
+
$sub_id = wp_create_user( $sub_login, "x", $sub_login . "@example.com" );
|
|
297
|
+
if ( ! is_wp_error( $sub_id ) ) {
|
|
298
|
+
$user = get_user_by( "id", $sub_id );
|
|
299
|
+
$user->set_role( "subscriber" );
|
|
300
|
+
wp_set_current_user( $sub_id );
|
|
301
|
+
$sub_result = $ability->check_permissions( $input );
|
|
302
|
+
} else {
|
|
303
|
+
$sub_result = $sub_id; // surface the create_user error in the report
|
|
304
|
+
}
|
|
305
|
+
|
|
306
|
+
foreach ( array( "admin" => $admin_result, "subscriber" => $sub_result ) as $context => $r ) {
|
|
307
|
+
if ( true === $r ) {
|
|
308
|
+
$printable = "true";
|
|
309
|
+
} elseif ( is_wp_error( $r ) ) {
|
|
310
|
+
$printable = "WP_Error(" . $r->get_error_code() . ")";
|
|
311
|
+
} else {
|
|
312
|
+
$printable = var_export( $r, true );
|
|
313
|
+
}
|
|
314
|
+
echo $context . ": " . $printable . PHP_EOL;
|
|
315
|
+
}
|
|
316
|
+
|
|
317
|
+
// Cleanup the test subscriber so repeated harness runs don't accumulate users
|
|
318
|
+
// on shared dev environments. Switch back to admin first since we last ran as
|
|
319
|
+
// the subscriber and that role doesn't hold delete_users. On multisite,
|
|
320
|
+
// wp_delete_user() only removes the user from the current site's membership —
|
|
321
|
+
// wpmu_delete_user() in wp-admin/includes/ms.php is the network-wide delete.
|
|
322
|
+
if ( isset( $sub_id ) && ! is_wp_error( $sub_id ) ) {
|
|
323
|
+
wp_set_current_user( 1 );
|
|
324
|
+
if ( is_multisite() ) {
|
|
325
|
+
require_once ABSPATH . "wp-admin/includes/ms.php";
|
|
326
|
+
wpmu_delete_user( $sub_id );
|
|
327
|
+
} else {
|
|
328
|
+
require_once ABSPATH . "wp-admin/includes/user.php";
|
|
329
|
+
wp_delete_user( $sub_id );
|
|
330
|
+
}
|
|
331
|
+
}
|
|
332
|
+
'
|
|
333
|
+
```
|
|
334
|
+
|
|
335
|
+
Expected:
|
|
336
|
+
|
|
337
|
+
```
|
|
338
|
+
admin: true
|
|
339
|
+
subscriber: false
|
|
340
|
+
```
|
|
341
|
+
|
|
342
|
+
Any inversion is a bug in the registered `permission_callback`. See
|
|
343
|
+
`permission-roundtrip.md` for the deeper cross-checks (audit's declared
|
|
344
|
+
capability → registered callback → resolved `current_user_can(...)`).
|
|
345
|
+
|
|
346
|
+
Public abilities (deliberately ungated) expect `subscriber: true` AND
|
|
347
|
+
`admin: true`. Verify that the ability's `permission_callback` is
|
|
348
|
+
`'__return_true'` in source before accepting this outcome.
|
|
349
|
+
|
|
350
|
+
`check_permissions()` returns `bool|WP_Error` per `WP_Ability::check_permissions()`
|
|
351
|
+
in WordPress core. A `WP_Error` with code `ability_invalid_permission_callback`
|
|
352
|
+
means the registration didn't supply a valid callable — a hard FAIL. A
|
|
353
|
+
`WP_Error` with code `ability_callback_exception` means the callback
|
|
354
|
+
threw — also a hard FAIL.
|
|
355
|
+
|
|
356
|
+
## Check 6 — twin-invocation heuristic for idempotent abilities
|
|
357
|
+
|
|
358
|
+
Only apply this to abilities annotated `idempotent: true` whose `execute()`
|
|
359
|
+
returned without error in Check 3. Per `annotation-correctness.md` step
|
|
360
|
+
"Runtime check complement", this is a *heuristic*: idempotent in core
|
|
361
|
+
means "no additional effect on the environment" (per the `idempotent`
|
|
362
|
+
annotation's docblock in `class-wp-ability.php`), not "byte-identical
|
|
363
|
+
return values."
|
|
364
|
+
|
|
365
|
+
```bash
|
|
366
|
+
<env-cli> wp --user=admin eval '
|
|
367
|
+
$a = wp_get_ability( "<plugin>/<idempotent-ability>" );
|
|
368
|
+
$r1 = $a->execute();
|
|
369
|
+
$r2 = $a->execute();
|
|
370
|
+
|
|
371
|
+
if ( is_wp_error( $r1 ) || is_wp_error( $r2 ) ) {
|
|
372
|
+
echo "skipped: one or both invocations returned WP_Error" . PHP_EOL;
|
|
373
|
+
if ( is_wp_error( $r1 ) ) { echo "r1=" . $r1->get_error_code() . PHP_EOL; }
|
|
374
|
+
if ( is_wp_error( $r2 ) ) { echo "r2=" . $r2->get_error_code() . PHP_EOL; }
|
|
375
|
+
} else {
|
|
376
|
+
$h1 = md5( serialize( $r1 ) );
|
|
377
|
+
$h2 = md5( serialize( $r2 ) );
|
|
378
|
+
echo "match=" . var_export( $h1 === $h2, true ) . PHP_EOL;
|
|
379
|
+
echo "h1=" . $h1 . PHP_EOL;
|
|
380
|
+
echo "h2=" . $h2 . PHP_EOL;
|
|
381
|
+
}
|
|
382
|
+
'
|
|
383
|
+
```
|
|
384
|
+
|
|
385
|
+
Interpretation:
|
|
386
|
+
|
|
387
|
+
- `match=true` → cheap PASS. Same input produced the same response, and
|
|
388
|
+
any environmental writes (write abilities) were the same on both calls.
|
|
389
|
+
- `match=false` → inspect what changed before deciding:
|
|
390
|
+
- Response embeds a per-call timestamp, nonce, or random ID →
|
|
391
|
+
environment unchanged. Still idempotent under core's reading.
|
|
392
|
+
Optionally remove the field if the agent doesn't need it; the
|
|
393
|
+
annotation stays `idempotent: true`.
|
|
394
|
+
- Response reflects a counter or sequence that grew between calls →
|
|
395
|
+
real environmental change. FAIL: drop the `idempotent: true`
|
|
396
|
+
annotation or fix the underlying write to be input-determined.
|
|
397
|
+
|
|
398
|
+
For ambiguous cases (response varies but no obvious counter), supplement
|
|
399
|
+
with a state diff: snapshot a representative table or option before call
|
|
400
|
+
1, snapshot after call 2, diff. If state changed by more than the input
|
|
401
|
+
writes would explain, the ability is non-idempotent.
|
|
402
|
+
|
|
403
|
+
## Output format
|
|
404
|
+
|
|
405
|
+
The runtime harness writes a dedicated section in the run report:
|
|
406
|
+
|
|
407
|
+
```markdown
|
|
408
|
+
## Runtime harness
|
|
409
|
+
|
|
410
|
+
**Env:** wp-env (Docker), WordPress 6.9, <plugin> <version>
|
|
411
|
+
**Captured:** <YYYY-MM-DD HH:MM>
|
|
412
|
+
|
|
413
|
+
### Check 1 — enumeration
|
|
414
|
+
|
|
415
|
+
count=7 (expected 7 from static inventory)
|
|
416
|
+
<sorted list>
|
|
417
|
+
|
|
418
|
+
### Check 2 — annotations
|
|
419
|
+
|
|
420
|
+
| Ability | readonly | destructive | idempotent | Matches source? |
|
|
421
|
+
|---|---|---|---|---|
|
|
422
|
+
|
|
423
|
+
### Check 3 — read execution
|
|
424
|
+
|
|
425
|
+
| Ability | Result |
|
|
426
|
+
|---|---|
|
|
427
|
+
|
|
428
|
+
### Check 4 — write input validation
|
|
429
|
+
|
|
430
|
+
| Ability | Missing-input code | Wrong-type code |
|
|
431
|
+
|---|---|---|
|
|
432
|
+
|
|
433
|
+
### Check 5 — permission gate
|
|
434
|
+
|
|
435
|
+
| Ability | admin | subscriber | Expected |
|
|
436
|
+
|---|---|---|---|
|
|
437
|
+
|
|
438
|
+
### Check 6 — idempotency
|
|
439
|
+
|
|
440
|
+
| Ability | match | h1 | h2 |
|
|
441
|
+
|---|---|---|---|
|
|
442
|
+
|
|
443
|
+
### Notes / surprises
|
|
444
|
+
|
|
445
|
+
<Anything unexpected that didn't block.>
|
|
446
|
+
```
|
|
447
|
+
|
|
448
|
+
## When the harness catches a bug
|
|
449
|
+
|
|
450
|
+
Observed pattern:
|
|
451
|
+
|
|
452
|
+
1. Harness surfaces a PHP fatal (e.g. `ArgumentCountError: Too few
|
|
453
|
+
arguments to function <controller>::__construct`).
|
|
454
|
+
2. Implementer fixes the bug in a focused commit.
|
|
455
|
+
3. Harness re-runs; write the post-fix output as the headline status.
|
|
456
|
+
4. Preserve the pre-fix trace in the report under a "Pre-fix status"
|
|
457
|
+
section so reviewers can see what verify caught.
|
|
458
|
+
|
|
459
|
+
This is the highest-leverage signal the runtime harness produces:
|
|
460
|
+
bootstrap-ordering and missing-dependency bugs that pass static review
|
|
461
|
+
because the source declares the right shape — they only manifest when
|
|
462
|
+
the registration runs against a booted WordPress.
|