@deeeed/metamask-harness 0.47.0 → 0.47.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/site/recipes.html DELETED
@@ -1,430 +0,0 @@
1
- <!DOCTYPE html>
2
- <html lang="en">
3
- <head>
4
- <meta charset="utf-8">
5
- <meta name="viewport" content="width=device-width, initial-scale=1">
6
- <title>What is a recipe?</title>
7
- <meta name="description" content="The anatomy of a recipe: a JSON graph of typed actions, human-authored intent, first-class assertions, and a verdict derived from the trace.">
8
- <link rel="icon" href="assets/metamask-fox.svg" type="image/svg+xml">
9
- <link rel="stylesheet" href="assets/style.css?v=4">
10
- </head>
11
- <body data-progress-page="recipes">
12
- <a class="skip" href="#anatomy">Skip to the anatomy</a>
13
-
14
- <header class="topbar">
15
- <div class="wrap topbar-inner">
16
- <a class="brand" href="index.html">
17
- <img class="brand-mark" src="assets/metamask-fox.svg" alt="" width="22" height="22">
18
- <span class="brand-name">recipes</span>
19
- </a>
20
- <nav class="nav" aria-label="Main">
21
- <a class="nav-cta" href="index.html#prompt">Quick start</a>
22
- <a href="how-it-works.html">How it works</a>
23
- <a href="perps.html">Team / Perps</a>
24
- <a href="cheatsheet.html">Cheatsheet</a>
25
- <a href="architecture.html">Architecture</a>
26
- <a href="tutorials/index.html">Tutorials</a>
27
- </nav>
28
- </div>
29
- </header>
30
-
31
- <main>
32
- <section class="wrap hero" style="padding-bottom:1rem">
33
- <span class="eyebrow">The trust mechanism</span>
34
- <h1>What is a recipe?</h1>
35
- <p class="lede">
36
- A recipe turns "I checked, it works" into something proved on a device as the change is made —
37
- and something that objects later if the behaviour breaks.
38
- </p>
39
- </section>
40
-
41
- <section class="wrap" id="agent-method">
42
- <div class="cmd cmd-hero" data-help-recipes>
43
- <pre><code>Loading version-matched recipe help…</code></pre>
44
- </div>
45
- </section>
46
-
47
- <section class="wrap">
48
- <div class="doctrine">
49
- <p>
50
- <span class="k">A recipe proves a task with actions.</span> A JSON graph whose nodes are typed
51
- actions drawn only from what this checkout discovered, each carrying a human-written intent,
52
- executed against a real build, ending in a verdict computed from what happened rather than
53
- asserted in prose.
54
- </p>
55
- </div>
56
-
57
- <p>Every clause there does a job against a specific failure mode. Here they are, one at a time.</p>
58
-
59
- <h2 id="why">What it is for</h2>
60
- <p>
61
- <strong>Proof.</strong> Actions drive a real build and assertions check what should be true —
62
- counted, where a count is what distinguishes working from nearly working. What lands is on-device
63
- evidence, not a claim in a pull request description.
64
- </p>
65
- <p>
66
- <strong>Self-checking, in the same context.</strong> The agent does not need a fresh session, a
67
- reviewer, or a CI round-trip to know whether its change held. It replays the recipe mid-task, where
68
- it is already working, and reads the verdict itself — the inner loop, closed before handoff.
69
- </p>
70
- <p>
71
- <strong>Prevents regressions.</strong> The outer loop is the same recipe replayed against every
72
- later change. Written once, it keeps refusing the bug it was written for, which is why a recipe is
73
- worth authoring rather than checking by hand: the check survives the task.
74
- </p>
75
- <p>
76
- <strong>Trust.</strong> Proven, self-checked, and still proving — so the change is trustworthy
77
- while it is being built. Everything downstream, review included, inherits that, and every gate
78
- stays yours to steer.
79
- </p>
80
- </section>
81
-
82
- <section class="wrap">
83
- <h2 id="anatomy">The anatomy of a real one</h2>
84
- <p>
85
- <code>perps.lifecycle</code> ships with the harness. Abridged here — the full graph runs the whole
86
- lifecycle, clean state through teardown — but nothing is invented; read it all with
87
- <code>mm-harness run perps.lifecycle --describe</code>.
88
- </p>
89
-
90
- <div class="annot">
91
- <div class="out">{
92
- "title": "MetaMask Perps lifecycle smoke",
93
- "description": "Proves a deterministic Perps prestate,
94
- opens a small ETH testnet position, verifies it,
95
- closes it, and tears down to a clean state.",
96
- "workflow": {
97
- "entry": <span class="hint">"status"</span>,
98
- "nodes": {
99
- "status": {
100
- "action": <span class="hint">"app.status"</span>,
101
- "intent": "Read runner compatibility status",
102
- "next": "fixture"
103
- },
104
- <span class="dim">… wallet fixture, unlock, account select …</span>
105
-
106
- "start-state": {
107
- "action": "metamask.perps.start_state",
108
- "market": "ETH",
109
- "positions": { "state": "none" },
110
- "intent": "Prepare clean ETH testnet Perps state",
111
- "next": "assert-clean-position"
112
- },
113
- <span class="hint">"assert-clean-position"</span>: {
114
- "action": "metamask.perps.assert_positions",
115
- "market": "ETH",
116
- "state": "none",
117
- "intent": "Confirm ETH position state is clean",
118
- "next": "assert-clean-orders"
119
- },
120
- <span class="dim">… assert clean orders …</span>
121
-
122
- "place-order": {
123
- "action": "metamask.perps.place_order",
124
- "market": "ETH", "side": "long",
125
- "amount": "11", "leverage": 3,
126
- "intent": "Open a small ETH 3x long position",
127
- "next": "assert-open"
128
- },
129
- <span class="hint">"assert-open"</span>: {
130
- "action": "metamask.perps.assert_positions",
131
- "market": "ETH",
132
- <span class="ok">"state": "open"</span>,
133
- "intent": "Confirm the ETH position opened",
134
- "next": "read-orders"
135
- },
136
- <span class="dim">… close orders, close position, assert closed …</span>
137
-
138
- "teardown-state": {
139
- "action": "metamask.perps.teardown_state",
140
- "intent": "Return Perps to globally clean home state",
141
- "next": "done"
142
- },
143
- "done": {
144
- "action": <span class="hint">"end"</span>,
145
- "status": "pass"
146
- }
147
- }
148
- }
149
- }</div>
150
-
151
- <div class="annot-notes">
152
- <div class="annot-note">
153
- <b>entry + next</b>
154
- <p>It is a graph, not a script. Execution starts at <code>entry</code> and follows each node's <code>next</code>. Branching exists too — a <code>switch</code> node picks an edge — which is why the trace records the path actually taken rather than assuming the file's reading order.</p>
155
- </div>
156
- <div class="annot-note">
157
- <b>action</b>
158
- <p>Always a name from the discovered manifest. <code>app.status</code>, <code>metamask.perps.place_order</code>, and <code>end</code> are all real entries you can look up with <code>mm-harness actions --action &lt;name&gt;</code>. You cannot write an action that does not exist and have it run.</p>
159
- </div>
160
- <div class="annot-note">
161
- <b>intent</b>
162
- <p>Written by a human, before the run. It is not a comment — it is carried into the trace, so the evidence reads as an argument rather than a log.</p>
163
- </div>
164
- <div class="annot-note">
165
- <b>assert-clean-position</b>
166
- <p>The pattern worth stealing: assert the <em>starting</em> state before acting. Without it, a position that was already open would make the next step look like a success it did not cause.</p>
167
- </div>
168
- <div class="annot-note">
169
- <b>assert-open</b>
170
- <p>And assert the effect afterwards, independently. <code>place_order</code> reporting success is the action's own word for it; <code>assert_positions</code> goes and looks. These are deliberately two different nodes.</p>
171
- </div>
172
- <div class="annot-note">
173
- <b>end</b>
174
- <p>The terminal node. Reaching it is what a passing run means — and it is only reachable by passing through every assertion on the path.</p>
175
- </div>
176
- </div>
177
- </div>
178
- </section>
179
-
180
- <section class="wrap">
181
- <h2 id="vocabulary">Actions come only from the discovered manifest</h2>
182
- <p>
183
- The available actions are fixed and discoverable, and a recipe may use nothing else. Validation
184
- happens against that manifest <em>before</em> anything executes:
185
- </p>
186
-
187
- <div class="cmd"><pre><code><span class="p">$ </span>mm-harness run my-recipe.json --plan</code></pre></div>
188
-
189
- <div class="out"><span class="ok">✓</span> [static] validate.schema: recipe document schema + action existence/platform vs the adapter manifest</div>
190
-
191
- <p>
192
- A misspelled or imagined action fails validation with exit code 5, and nothing touches the app.
193
- That is the cheap check — run it before the expensive one.
194
- </p>
195
-
196
- <div class="note blue">
197
- <span class="note-title">The constraint is the point</span>
198
- <p>
199
- A tool free to invent action names is free to invent plausible ones, and a recipe full of
200
- plausible steps that never ran is worse than no recipe. Bounding the vocabulary to what exists is
201
- what makes a green run mean something — and what makes this safe to hand to an agent.
202
- </p>
203
- </div>
204
- </section>
205
-
206
- <section class="wrap">
207
- <h2 id="intent">Intent turns a trace into a narrative</h2>
208
- <p>
209
- The runner copies each node's <code>intent</code> into the trace beside its result. Read them in
210
- order and you have the argument the recipe makes:
211
- </p>
212
-
213
- <div class="out"><span class="ok">PASS</span> status — Read runner compatibility status
214
- <span class="ok">PASS</span> fixture — Verify wallet fixture availability
215
- <span class="ok">PASS</span> assert-clean-position — Confirm ETH position state is clean
216
- <span class="ok">PASS</span> place-order — Open a small ETH 3x long position
217
- <span class="ok">PASS</span> assert-open — Confirm the ETH position opened</div>
218
-
219
- <p>
220
- A reviewable claim in five lines, generated from the recipe rather than written about it
221
- afterwards. It gives review a precise question: if the intents do not add up to what the pull
222
- request claims, the bundle does not support it — however green.
223
- </p>
224
- </section>
225
-
226
- <section class="wrap">
227
- <h2 id="assertions">Assertions are first-class — and counting matters</h2>
228
- <p>
229
- An assertion is a node like any other: it occupies a position in the graph, carries its own intent,
230
- and failing it stops the path to <code>end</code>.
231
- </p>
232
-
233
- <div class="table-scroll" data-copy-cells>
234
- <table>
235
- <thead><tr><th>Assertion</th><th>Proves</th></tr></thead>
236
- <tbody>
237
- <tr><td><code>assert_file</code></td><td>A project file exists, and optionally contains given text.</td></tr>
238
- <tr><td><code>assert_json</code></td><td>A JSON value satisfies a selector and operator.</td></tr>
239
- <tr><td><code>assert_output</code></td><td>Captured stdout or stderr from an earlier command node matches.</td></tr>
240
- <tr><td><code>assert_exit_code</code></td><td>An earlier command node exited as expected.</td></tr>
241
- <tr><td><code>metamask.perps.assert_positions</code></td><td>Live positions are present or absent for an explicit selection.</td></tr>
242
- <tr><td><code>metamask.analytics.assert_events</code></td><td>Event names, property values, <strong>and counts</strong>.</td></tr>
243
- </tbody>
244
- </table>
245
- </div>
246
-
247
- <div class="doctrine">
248
- <p>
249
- <span class="k">Counted assertions earn their keep.</span> The analytics assertion takes an
250
- <code>exact</code> count, and its own description says why: an exact count catches a double emit
251
- that a presence check cannot. "Did the event fire?" answers yes to a bug that fires it twice.
252
- </p>
253
- </div>
254
-
255
- <p>
256
- On Core, <code>assert_positions</code> and <code>assert_orders</code> take
257
- <code>expect_count</code>; the Extension and Mobile manifests do not declare it, and a recipe that
258
- passes it there fails validation with exit 5. <code>assert_orders</code> can also assert a matching
259
- order's trigger price, execution mode, and reduce-only flag. Reach for the most specific assertion
260
- the surface supports — a vague one passes for the wrong reasons.
261
- </p>
262
- </section>
263
-
264
- <section class="wrap">
265
- <h2 id="composition">Recipes call recipes</h2>
266
- <p>
267
- <code>call</code> is itself an action: it invokes a named recipe by reference and runs its steps
268
- inline, so shared setup is written once instead of copied into every proof. The manifest's own
269
- example:
270
- </p>
271
-
272
- <div class="out">{
273
- "action": <span class="hint">"call"</span>,
274
- "ref": "mydev.open_perps_setup",
275
- "intent": "Run a personal Perps setup recipe",
276
- "next": "done"
277
- }</div>
278
-
279
- <p>
280
- Running inline means the called recipe's nodes appear in the trace like any others: composition is
281
- an author's convenience, not a hole in the evidence. <code>--describe</code> reports both the
282
- actions and the recipes pulled in:
283
- </p>
284
-
285
- <div class="cmd"><pre><code><span class="p">$ </span>mm-harness run &lt;recipe&gt; --describe</code></pre></div>
286
-
287
- <div class="out"> actions (3)
288
- app.status
289
- assert_file
290
- end
291
- called recipes: none</div>
292
- </section>
293
-
294
- <section class="wrap">
295
- <h2 id="verdict">The verdict is derived, never declared</h2>
296
- <p>
297
- Status is computed from the trace, not asserted by the author or written by an agent afterwards.
298
- Every node has an <code>ok</code>; the run passes only if the path reached <code>end</code> with all
299
- of them true:
300
- </p>
301
-
302
- <div class="out">{
303
- "status": <span class="ok">"pass"</span>,
304
- "total": 6,
305
- "passed": 6,
306
- "failed": 0
307
- }</div>
308
-
309
- <p>
310
- Two outcomes only: <strong>pass</strong> or <strong>fail</strong>. No "inconclusive" hiding place at
311
- this level — either the graph reached its terminal node with every assertion satisfied, or it did
312
- not.
313
- </p>
314
-
315
- <div class="note">
316
- <span class="note-title">Where "cannot be proven here" lives</span>
317
- <p>
318
- One level up. When a review workflow maps acceptance criteria onto proofs, a criterion the surface
319
- cannot demonstrate is reported as needing a human rather than quietly passed. That third answer
320
- belongs to the review layer; a recipe either proved the thing or did not.
321
- </p>
322
- </div>
323
-
324
- <p>
325
- So every number you cite comes from <code>trace.json</code>; prose around a bundle is commentary.
326
- <a href="reviewers.html">The reviewer's guide</a> walks a real one.
327
- </p>
328
- </section>
329
-
330
- <section class="wrap">
331
- <h2 id="falsifiability">The test of a good recipe</h2>
332
-
333
- <div class="doctrine">
334
- <p>
335
- <span class="k">A good recipe fails if the change it proves is reverted.</span> That is the whole
336
- quality bar.
337
- </p>
338
- </div>
339
-
340
- <p>
341
- A sharper question than "does it pass?", because passing is easy and uninformative. A recipe that
342
- navigates to a screen, screenshots it, and ends will pass forever — including after someone deletes
343
- the feature. It proves the app still starts.
344
- </p>
345
-
346
- <p>
347
- So the check is adversarial: imagine the change gone and ask which node goes red. If the answer is
348
- "none", the recipe is decoration. Common ways this goes wrong:
349
- </p>
350
-
351
- <div class="table-scroll">
352
- <table>
353
- <thead><tr><th>Shape</th><th>Why it survives a revert</th><th>Fix</th></tr></thead>
354
- <tbody>
355
- <tr>
356
- <td>Screenshot with no assertion</td>
357
- <td>An image is evidence for a human, but nothing in the graph inspects it.</td>
358
- <td>Add an assertion on the state the screenshot illustrates.</td>
359
- </tr>
360
- <tr>
361
- <td>Presence check where a count belongs</td>
362
- <td>Still true when the bug fires the event twice, or leaves two positions open.</td>
363
- <td>Use <code>exact</code>; on Core, <code>expect_count</code>.</td>
364
- </tr>
365
- <tr>
366
- <td>No starting-state assertion</td>
367
- <td>Passes off pre-existing state as the effect of the action under test.</td>
368
- <td>Assert clean before acting, as <code>perps.lifecycle</code> does.</td>
369
- </tr>
370
- <tr>
371
- <td>Asserting the action's own report</td>
372
- <td>Confirms the call returned, not that the world changed.</td>
373
- <td>Read the state back with a separate node.</td>
374
- </tr>
375
- </tbody>
376
- </table>
377
- </div>
378
-
379
- <p>
380
- Cheapest way to run this test rather than reason about it: revert your change locally, run the
381
- recipe, watch it fail, restore. A recipe you have seen fail for the right reason is one you can
382
- trust when it passes.
383
- </p>
384
- </section>
385
-
386
- <section class="wrap">
387
- <hr class="sep">
388
- <h2 id="author">Author one yourself</h2>
389
- <p>
390
- You do not have to write the JSON by hand, and mostly should not. Point the recipe-authoring skill
391
- at an acceptance criterion and it discovers the actions, composes the graph, validates with
392
- <code>--plan</code>, and runs it — following the rules on this page because the harness enforces
393
- them. <a href="tutorials/v4.html">V4 walks that flow.</a>
394
- </p>
395
- <p>Expect one failure-and-fix loop on the way; that is the error messages doing their job.</p>
396
-
397
- <div class="grid grid-2">
398
- <div class="card">
399
- <h3>Before you author</h3>
400
- <p>Learn the vocabulary for your checkout — you are choosing from this list, not inventing.</p>
401
- <div class="cmd"><pre><code><span class="p">$ </span>mm-harness actions --categories
402
- <span class="p">$ </span>mm-harness actions --action assert_orders</code></pre></div>
403
- </div>
404
- <div class="card">
405
- <h3>Read one that works</h3>
406
- <p>The bundled recipes are worked examples. Start from the closest one to your problem.</p>
407
- <div class="cmd"><pre><code><span class="p">$ </span>mm-harness run --list
408
- <span class="p">$ </span>mm-harness run perps.lifecycle --describe</code></pre></div>
409
- </div>
410
- </div>
411
-
412
- <div class="btn-row">
413
- <a class="btn btn-primary" href="how-it-works.html#steps">Run your first recipe →</a>
414
- <a class="btn btn-ghost" href="reviewers.html">Read the evidence it produces</a>
415
- <a class="btn btn-ghost" href="architecture.html">Where skills fit</a>
416
- </div>
417
- </section>
418
- </main>
419
-
420
- <footer class="footer">
421
- <div class="wrap">
422
- <p>Internal getting-started guide for proving MetaMask changes with recipes. Not official MetaMask product documentation.</p>
423
- <p>The annotated recipe is a bundled library recipe, abridged. Sample output moves between releases; trust your terminal over this page.</p>
424
- </div>
425
- </footer>
426
-
427
- <script type="module" src="assets/help-recipes.mjs?v=1"></script>
428
- <script type="module" src="assets/progress.mjs?v=4"></script>
429
- </body>
430
- </html>