@alejoamiras/aztec-benchmark 5.0.0-rc.2 → 5.0.1-revision.1

package/README.md

@@ -1,5 +1,5 @@
 # Aztec Benchmark
-[![npm version](https://badge.fury.io/js/%40defi-wonderland%2Faztec-benchmark.svg)](https://www.npmjs.com/package/@alejoamiras/aztec-benchmark)
+[![npm version](https://img.shields.io/npm/v/@alejoamiras/aztec-benchmark.svg)](https://www.npmjs.com/package/@alejoamiras/aztec-benchmark)
 
 **CLI tool and reusable CI workflows for running Aztec contract benchmarks.**
 
@@ -27,7 +27,7 @@ Use the CLI to execute benchmark files written in TypeScript. For CI integration
 ## Installation
 
 ```sh
-yarn add --dev @alejoamiras/aztec-benchmark
+bun add -D @alejoamiras/aztec-benchmark
 # or
 npm install --save-dev @alejoamiras/aztec-benchmark
 ```
@@ -134,6 +134,9 @@ export default class MyContractBenchmark extends Benchmark {
     const fullConfig = { ...config, l1Contracts };
     // IMPORTANT: true enables proof generation for the benchmark, set it to false when using --skip-proving
     fullConfig.proverEnabled = true;
+    // PXE wipes any store whose schema version differs from the running PXE's (Aztec 5.0.0+
+    // resets on upgrade) — a hardcoded version can resurrect or mask a stale store, so use a
+    // fresh dataDirectory per toolchain version when in doubt.
     const pxeVersion = 2;
     const store = await createStore('pxe', pxeVersion, {
       dataDirectory: 'store',
@@ -214,9 +217,9 @@ If you provide a `NamedBenchmarkedInteraction` object, its `name` field will be
 If you provide a plain `ContractFunctionInteractionCallIntent`, the tool will attempt to derive a name from the interaction (e.g., the method name).
 If you return a `feePaymentMethod` in the `BenchmarkContext`, it is automatically passed to every transaction the profiler sends — no changes to `getMethods` are needed.
 
-### Wonderland's Usage Example
+### Usage Example
 
-You can find how we use this tool for benchmarking our Aztec contracts in [`aztec-standards`](https://github.com/defi-wonderland/aztec-standards/tree/dev/benchmarks).
+See how this monorepo benchmarks its own contracts in [`packages/aztec-fee-payment/benchmarks`](https://github.com/alejoamiras/ecosystem-tooling/tree/main/packages/aztec-fee-payment/benchmarks).
 
 ---
 
@@ -227,114 +230,35 @@ Each entry in the output will be identified by the custom `name` you provided (i
 
 ---
 
-## Reusable Workflows
+## CI Integration
 
-This repository ships two **reusable GitHub workflows** (`workflow_call`) that handle the full CI benchmark cycle. Consumer repos call them with a single `uses:` line — no need to copy workflow YAML or wire up artifact management manually.
+### Inside this monorepo
 
-### PR Benchmark (`pr-benchmark.yml`)
+The per-package gate (`aztec-fee-payment.yml`) calls the repo's reusable workflows:
 
-Runs benchmarks on the PR head, downloads the baseline from the base branch, generates a comparison report, comments it on the PR (hiding any previous benchmark comments as outdated), and uploads the new results as a baseline artifact for the PR branch.
+- **`_pr-benchmark.yml`** — on every PR: runs the suites on the PR head (`--skip-proving`), downloads the `main` baseline artifact, generates a comparison report (regressions beyond 2.5% highlighted), and posts it as a PR comment. Security split: the job executing PR code is read-only; a separate write-permission job posts the comment without executing anything from the PR.
+- **`_update-baseline.yml`** — on pushes to `main` (+ monthly cron via `update-baselines.yml`): refreshes the baseline artifacts (`benchmark-baseline-<package>-main`).
 
-**Usage:**
+Wiring a new package: add a `benchmark` job to its gate workflow:
 
 ```yaml
-# .github/workflows/pr-checks.yml
-name: PR Checks
-
-on:
-  pull_request:
-    branches: [dev, main]
-
-jobs:
-  benchmark:
-    uses: defi-wonderland/aztec-benchmark/.github/workflows/pr-benchmark.yml@v0
-    permissions:
-      pull-requests: write
-      issues: write
-      actions: read
+benchmark:
+  needs: changes
+  if: needs.changes.outputs.relevant == 'true' && github.event_name == 'pull_request'
+  permissions:
+    contents: read
+    pull-requests: write
+    issues: write
+    actions: read
+  uses: ./.github/workflows/_pr-benchmark.yml
+  with:
+    package-dir: packages/<your-package>
+    pr-workflow: <your-package>.yml
 ```
 
-**Inputs:**
+### From other repositories
 
-| Input | Type | Default | Description |
-|---|---|---|---|
-| `runner` | `string` | `ubuntu-latest-m` | GitHub runner label |
-| `timeout` | `number` | `120` | Job timeout in minutes |
-| `bench-dir` | `string` | `./benchmarks` | Directory for benchmark files |
+The reusable workflows reference this repo's local `setup-aztec` action, so they are not directly callable cross-repo. External projects have two options:
 
-**With custom inputs:**
-
-```yaml
-jobs:
-  benchmark:
-    uses: defi-wonderland/aztec-benchmark/.github/workflows/pr-benchmark.yml@v0
-    permissions:
-      pull-requests: write
-      issues: write
-      actions: read
-    with:
-      runner: ubuntu-latest-l
-      timeout: 180
-      bench-dir: ./my-benchmarks
-```
-
-### Update Baseline (`update-baseline.yml`)
-
-Runs benchmarks on the current branch and uploads the results as a baseline artifact. This should be triggered on pushes to your default branches so that PR benchmarks have a baseline to compare against.
-
-**Usage:**
-
-```yaml
-# .github/workflows/update-baseline.yml
-name: Update Baseline
-
-on:
-  push:
-    branches: [dev, main]
-
-jobs:
-  update-baseline:
-    uses: defi-wonderland/aztec-benchmark/.github/workflows/update-baseline.yml@v0
-    permissions:
-      contents: read
-      actions: write
-```
-
-**Inputs:**
-
-| Input | Type | Default | Description |
-|---|---|---|---|
-| `runner` | `string` | `ubuntu-latest-m` | GitHub runner label |
-| `timeout` | `number` | `120` | Job timeout in minutes |
-| `bench-dir` | `string` | `./benchmarks` | Directory for benchmark files |
-
-### How Baselines Work
-
-The workflows use GitHub Actions artifacts to store and retrieve baseline benchmark results:
-
-1. **`update-baseline.yml`** runs benchmarks with the `_latest` suffix and uploads the results as `benchmark-baseline-<branch>`.
-2. **`pr-benchmark.yml`** runs benchmarks with the `_new` suffix on the PR head, then downloads the `benchmark-baseline-<base-branch>` artifact to get the `_latest` files. It compares `_latest` (baseline) vs `_new` (PR) and comments a Markdown diff table on the PR.
-3. Before posting the new comment, the workflow finds all previous benchmark comments on the PR (identified by a unique marker in the comment body) and hides them as **Outdated** via the GitHub GraphQL API, so the PR timeline stays clean.
-4. After comparison, the PR workflow renames `_new` files to `_latest` and uploads them as `benchmark-baseline-<head-branch>`, so stacked PRs can also compare against each other.
-
-Artifacts are retained for **90 days** by default.
-
----
-
-## Action Usage (Advanced)
-
-> **Note:** For most projects, the [reusable workflows](#reusable-workflows) above are the recommended approach. The action below is a lower-level building block for projects that need a custom CI setup.
-
-This repository also includes a GitHub Action (defined in `action/action.yml`) that runs `aztec-benchmark` and compares results. It automatically finds benchmark reports (named with `_base` and `_latest` suffixes) and produces a Markdown comparison report.
-
-### Inputs
-
-- `threshold`: Regression threshold percentage (default: `2.5`).
-- `output_markdown_path`: Path to save the generated Markdown comparison report (default: `benchmark-comparison.md`).
-
-### Outputs
-
-- `comparison_markdown`: The generated Markdown report content.
-- `markdown_file_path`: Path to the saved Markdown file.
-
-Refer to the `action/action.yml` file for the definitive inputs and description.
+1. **Use the CLI directly** (recommended): install `@alejoamiras/aztec-benchmark` as a devDependency, add a `[benchmark]` table to your `Nargo.toml`, and run `aztec-benchmark --suffix _base` in a job that has an Aztec local network running. The comparison helper is importable as `require('@alejoamiras/aztec-benchmark/action/comparison.cjs')` — see `_pr-benchmark.yml` in this repo as a reference implementation for baselines, comparison and PR comments.
+2. **Vendor the workflows**: copy `_pr-benchmark.yml`, `_update-baseline.yml` and `.github/actions/setup-aztec/` into your repo and adjust `package-dir`.

package/dist/cli.js

@@ -126,7 +126,9 @@ program
                     await pxe.blockStateSynchronizer.blockStream?.stop();
                     await pxe.blockStateSynchronizer.store?.close();
                 }
-                await runContext.wallet.stop();
+                // stop() is EmbeddedWallet lifecycle, not part of the Wallet interface —
+                // narrow structurally so any wallet implementation remains accepted.
+                await runContext.wallet.stop?.();
                 // Close leftover HTTP keep-alive sockets
                 for (const h of process._getActiveHandles()) {
                     if (h?.constructor?.name === 'Socket' && !h.destroyed)

package/dist/profiler.d.ts

@@ -1,6 +1,6 @@
 import type { ContractFunctionInteractionCallIntent } from '@aztec/aztec.js/authorization';
 import type { FeePaymentMethod } from '@aztec/aztec.js/fee';
-import type { EmbeddedWallet } from '@aztec/wallets/embedded';
+import type { Wallet } from '@aztec/aztec.js/wallet';
 import type { NamedBenchmarkedInteraction, ProfileResult } from './types.js';
 interface ProfilerOptions {
     skipProving?: boolean;
@@ -13,7 +13,7 @@ interface ProfilerOptions {
 export declare class Profiler {
     #private;
     /** @param _wallet - Unused, kept for backward compatibility with existing callers. */
-    constructor(_wallet?: EmbeddedWallet, options?: ProfilerOptions);
+    constructor(_wallet?: Wallet, options?: ProfilerOptions);
     /**
      * Profiles a list of contract function interactions.
      * Items can be plain interactions or objects with an interaction and a custom name.

package/dist/types.d.ts

@@ -1,7 +1,7 @@
 import type { AztecAddress } from '@aztec/aztec.js/addresses';
 import type { ContractFunctionInteractionCallIntent } from '@aztec/aztec.js/authorization';
 import type { FeePaymentMethod } from '@aztec/aztec.js/fee';
-import type { EmbeddedWallet } from '@aztec/wallets/embedded';
+import type { Wallet } from '@aztec/aztec.js/wallet';
 import type { SystemInfo } from './systemInfo.js';
 export type { SystemInfo } from './systemInfo.js';
 /** Simplified Gas type (contains actual gas values) */
@@ -18,7 +18,9 @@ export type GasLimits = {
 };
 /** Benchmark specific setup/teardown context */
 export interface BenchmarkContext {
-    wallet?: EmbeddedWallet;
+    /** Typed as the Wallet interface so suite contexts can extend with any wallet
+     *  implementation; the CLI narrows structurally for lifecycle calls (stop). */
+    wallet?: Wallet;
     /** Optional fee payment method used when sending profiled transactions.
      *  When set, the profiler passes this to every send/prove call.
      *  When unset, the sender must have pre-existing Fee Juice. */

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@alejoamiras/aztec-benchmark",
-  "version": "5.0.0-rc.2",
+  "version": "5.0.1-revision.1",
   "description": "CLI tool and GitHub Action for Aztec contract benchmarking",
   "repository": {
     "type": "git",
@@ -23,7 +23,8 @@
   ],
   "scripts": {
     "build": "tsc && ncc build action/index.cjs -o action/dist -m -C",
-    "start": "tsx cli/cli.ts"
+    "start": "tsx cli/cli.ts",
+    "typecheck": "tsc --noEmit -p tsconfig.check.json"
   },
   "dependencies": {
     "@actions/core": "1.10.1",
@@ -35,15 +36,15 @@
     "typescript": "5.8.3"
   },
   "devDependencies": {
-    "@aztec/aztec.js": "5.0.0-rc.2",
-    "@aztec/stdlib": "5.0.0-rc.2",
-    "@aztec/wallets": "5.0.0-rc.2",
+    "@aztec/aztec.js": "5.0.1",
+    "@aztec/stdlib": "5.0.1",
+    "@aztec/wallets": "5.0.1",
     "@types/node": "22.15.3",
     "@vercel/ncc": "0.38.3"
   },
   "peerDependencies": {
-    "@aztec/aztec.js": "5.0.0-rc.2",
-    "@aztec/wallets": "5.0.0-rc.2"
+    "@aztec/aztec.js": "5.0.1",
+    "@aztec/wallets": "5.0.1"
   },
   "engines": {
     "node": ">=22"