rightmodeler 0.2.1 → 0.4.0

package/docs/github-actions.md

@@ -0,0 +1,328 @@
+# GitHub Actions
+
+This workflow runs rightmodeler in GitHub Actions with the job's built-in `GITHUB_TOKEN`. It needs no GitHub App and no personal access token.
+
+## What it does
+
+- Runs `init` every Monday and whenever a push to `main` changes `traces/`.
+- Opens the draft pull request only when a person runs the workflow with `command=apply`.
+- Runs `watch` every 6 hours on each swap pull request that is still being watched.
+- Never merges. A person reviews and merges the draft.
+
+## Setup
+
+1. Let GitHub Actions open pull requests. In the repository settings, under Actions, General, Workflow permissions, turn on "Allow GitHub Actions to create and approve pull requests", or run `gh api -X PUT repos/<owner>/<repo>/actions/permissions/workflow -F can_approve_pull_request_reviews=true`. This is the only repository setting the workflow needs. In a repository owned by an organization, the organization must allow it first.
+2. Set the repository variable `RIGHTMODELER_PROVIDER_BASE_URL` and the repository secret `RIGHTMODELER_PROVIDER_API_KEY`, for example with `gh variable set` and `gh secret set`.
+3. Commit traces under `traces/`, or change `RIGHTMODELER_TRACES` and the `paths` filter together.
+4. Change `main` if the default branch has another name, and adjust `RIGHTMODELER_MAX_COST_USD`.
+5. Save the workflow below as `.github/workflows/rightmodeler.yml`.
+
+Each job grants `GITHUB_TOKEN` only what its command needs: `apply` gets `contents: write` and `pull-requests: write`, and `watch` gets `contents: read`, `pull-requests: write`, `checks: read` and `statuses: read`. See [GitHub](github.md) for what each command does with them.
+
+## The workflow
+
+```yaml
+name: rightmodeler
+
+on:
+  schedule:
+    - cron: "17 5 * * 1"
+    - cron: "43 */6 * * *"
+  push:
+    branches: [main]
+    paths:
+      - "traces/**"
+  workflow_dispatch:
+    inputs:
+      command:
+        description: Which rightmodeler command to run
+        type: choice
+        options: [init, apply, watch]
+        default: init
+
+permissions:
+  contents: read
+
+concurrency:
+  group: rightmodeler-store
+  cancel-in-progress: false
+
+env:
+  RIGHTMODELER_VERSION: "0.4.0"
+  RIGHTMODELER_TRACES: traces
+  RIGHTMODELER_MAX_COST_USD: "5"
+  RM_ANNOTATE: |
+    const { readFileSync } = require("node:fs");
+    const data = (s) => String(s).replace(/%/g, "%25").replace(/\r/g, "%0D").replace(/\n/g, "%0A");
+    const prop = (s) => data(s).replace(/:/g, "%3A").replace(/,/g, "%2C");
+    for (const file of process.argv.slice(1)) {
+      let text = "";
+      try { text = readFileSync(file, "utf8"); } catch { continue; }
+      for (const line of text.split("\n")) {
+        let value;
+        try { value = JSON.parse(line); } catch { continue; }
+        if (value === null || typeof value !== "object") continue;
+        if (value.event === "result") value = value.result;
+        if (value.event === "warning") {
+          console.log(`::warning title=${prop(`rightmodeler ${value.code}`)}::${data(value.message)}`);
+        } else if (value.status === "refused" && Array.isArray(value.reasons)) {
+          for (const reason of value.reasons) {
+            console.log(`::error title=${prop(`rightmodeler ${reason.code}`)}::${data(reason.message)}`);
+          }
+        } else if (typeof value.code === "string" && typeof value.message === "string") {
+          console.log(`::error title=${prop(`rightmodeler ${value.code}`)}::${data(`${value.message} Remedy: ${value.remedy ?? ""}`)}`);
+        }
+      }
+    }
+
+jobs:
+  init:
+    if: >-
+      github.event_name == 'push' ||
+      (github.event_name == 'schedule' && github.event.schedule == '17 5 * * 1') ||
+      (github.event_name == 'workflow_dispatch' && inputs.command == 'init')
+    runs-on: ubuntu-latest
+    timeout-minutes: 60
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          fetch-depth: 0
+          persist-credentials: false
+      - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
+        with:
+          node-version: 24
+      - uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
+        with:
+          path: .rightmodeler
+          key: rightmodeler-store-${{ github.run_id }}-${{ github.run_attempt }}
+          restore-keys: rightmodeler-store-
+      - id: init
+        name: Find and prove cheaper models
+        env:
+          RIGHTMODELER_PROVIDER_BASE_URL: ${{ vars.RIGHTMODELER_PROVIDER_BASE_URL }}
+          RIGHTMODELER_PROVIDER_API_KEY: ${{ secrets.RIGHTMODELER_PROVIDER_API_KEY }}
+        run: |
+          out="$RUNNER_TEMP/rightmodeler"
+          mkdir -p "$out"
+          npx --yes "rightmodeler@${RIGHTMODELER_VERSION}" --version
+          set +e
+          npx --yes "rightmodeler@${RIGHTMODELER_VERSION}" init \
+            --traces "$RIGHTMODELER_TRACES" \
+            --base-url "$RIGHTMODELER_PROVIDER_BASE_URL" \
+            --api-key-env RIGHTMODELER_PROVIDER_API_KEY \
+            --max-cost-usd "$RIGHTMODELER_MAX_COST_USD" \
+            --output jsonl --repo "$GITHUB_WORKSPACE" \
+            >"$out/init.jsonl" 2>"$out/init.err"
+          code=$?
+          set -e
+          node -e "$RM_ANNOTATE" "$out/init.jsonl" "$out/init.err"
+          report="$GITHUB_WORKSPACE/.rightmodeler/project/reports/report.md"
+          if [ -f "$report" ]; then
+            cp "$report" "$out/report.md"
+            cat "$report" >>"$GITHUB_STEP_SUMMARY"
+          fi
+          case "$code" in
+            0) echo "recommendation=false" >>"$GITHUB_OUTPUT" ;;
+            1)
+              echo "recommendation=true" >>"$GITHUB_OUTPUT"
+              echo "::notice title=rightmodeler::A proven swap is ready. Run this workflow with command=apply to open the draft pull request."
+              ;;
+            *)
+              echo "::error title=rightmodeler::init exited $code; the annotations above name the cause and the fix."
+              exit 1
+              ;;
+          esac
+      - if: always()
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: rightmodeler-init
+          path: ${{ runner.temp }}/rightmodeler/
+          if-no-files-found: ignore
+      - if: always()
+        uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
+        with:
+          path: .rightmodeler
+          key: rightmodeler-store-${{ github.run_id }}-${{ github.run_attempt }}
+
+  apply:
+    if: github.event_name == 'workflow_dispatch' && inputs.command == 'apply'
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+    permissions:
+      contents: write
+      pull-requests: write
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          fetch-depth: 0
+          persist-credentials: false
+      - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
+        with:
+          node-version: 24
+      - uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
+        with:
+          path: .rightmodeler
+          key: rightmodeler-store-${{ github.run_id }}-${{ github.run_attempt }}
+          restore-keys: rightmodeler-store-
+      - id: apply
+        name: Open the draft pull request
+        env:
+          RIGHTMODELER_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: |
+          out="$RUNNER_TEMP/rightmodeler"
+          mkdir -p "$out"
+          npx --yes "rightmodeler@${RIGHTMODELER_VERSION}" --version
+          run_apply() {
+            npx --yes "rightmodeler@${RIGHTMODELER_VERSION}" apply "$@" \
+              --owner "$GITHUB_REPOSITORY_OWNER" \
+              --github-repo "${GITHUB_REPOSITORY#*/}" \
+              --github-base-url "$GITHUB_API_URL" \
+              --github-token-env RIGHTMODELER_GITHUB_TOKEN \
+              --output json --repo "$GITHUB_WORKSPACE"
+          }
+          for mode in dry-run apply; do
+            set +e
+            if [ "$mode" = "dry-run" ]; then
+              run_apply --dry-run >"$out/$mode.json" 2>"$out/$mode.err"
+            else
+              run_apply >"$out/$mode.json" 2>"$out/$mode.err"
+            fi
+            code=$?
+            set -e
+            node -e "$RM_ANNOTATE" "$out/$mode.json" "$out/$mode.err"
+            if [ "$code" -ne 0 ]; then
+              echo "::error title=rightmodeler::apply ($mode) exited $code; the annotations above name the cause and the fix."
+              exit 1
+            fi
+          done
+          pr="$(node -p 'JSON.parse(require("node:fs").readFileSync(process.argv[1], "utf8")).prNumber' "$out/apply.json")"
+          echo "pr-number=$pr" >>"$GITHUB_OUTPUT"
+          echo "Draft pull request #$pr is open for review. rightmodeler never merges it." >>"$GITHUB_STEP_SUMMARY"
+      - if: always()
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: rightmodeler-apply
+          path: ${{ runner.temp }}/rightmodeler/
+          if-no-files-found: ignore
+      - if: always()
+        uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
+        with:
+          path: .rightmodeler
+          key: rightmodeler-store-${{ github.run_id }}-${{ github.run_attempt }}
+
+  watch:
+    if: >-
+      (github.event_name == 'schedule' && github.event.schedule == '43 */6 * * *') ||
+      (github.event_name == 'workflow_dispatch' && inputs.command == 'watch')
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+    permissions:
+      contents: read
+      pull-requests: write
+      checks: read
+      statuses: read
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          fetch-depth: 0
+          persist-credentials: false
+      - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
+        with:
+          node-version: 24
+      - uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
+        with:
+          path: .rightmodeler
+          key: rightmodeler-store-${{ github.run_id }}-${{ github.run_attempt }}
+          restore-keys: rightmodeler-store-
+      - id: watch
+        name: Reconcile open swap pull requests
+        env:
+          RIGHTMODELER_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: |
+          out="$RUNNER_TEMP/rightmodeler"
+          mkdir -p "$out"
+          npx --yes "rightmodeler@${RIGHTMODELER_VERSION}" --version
+          set +e
+          npx --yes "rightmodeler@${RIGHTMODELER_VERSION}" status --output json \
+            --repo "$GITHUB_WORKSPACE" >"$out/status.json" 2>"$out/status.err"
+          code=$?
+          set -e
+          node -e "$RM_ANNOTATE" "$out/status.err"
+          if [ "$code" -ne 0 ]; then
+            echo "::error title=rightmodeler::status exited $code"
+            exit 1
+          fi
+          failed=0
+          for pr in $(node -p 'JSON.parse(require("node:fs").readFileSync(process.argv[1], "utf8")).pullRequests.map((entry) => entry.prNumber).join(" ")' "$out/status.json"); do
+            set +e
+            npx --yes "rightmodeler@${RIGHTMODELER_VERSION}" watch --pr "$pr" \
+              --owner "$GITHUB_REPOSITORY_OWNER" \
+              --github-repo "${GITHUB_REPOSITORY#*/}" \
+              --github-base-url "$GITHUB_API_URL" \
+              --github-token-env RIGHTMODELER_GITHUB_TOKEN \
+              --output json --repo "$GITHUB_WORKSPACE" \
+              >"$out/watch-$pr.json" 2>"$out/watch-$pr.err"
+            code=$?
+            set -e
+            node -e "$RM_ANNOTATE" "$out/watch-$pr.json" "$out/watch-$pr.err"
+            case "$code" in
+              0) ;;
+              1) echo "::notice title=rightmodeler::watch acted on pull request #$pr" ;;
+              2)
+                held="$(node -p 'try { JSON.parse(require("node:fs").readFileSync(process.argv[1], "utf8")).status } catch { "" }' "$out/watch-$pr.json")"
+                if [ "$held" = "lock_held" ]; then
+                  echo "::warning title=rightmodeler::another watcher holds the lock for pull request #$pr; the next run retries"
+                else
+                  failed=1
+                fi
+                ;;
+              *) failed=1 ;;
+            esac
+          done
+          exit "$failed"
+      - if: always()
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: rightmodeler-watch
+          path: ${{ runner.temp }}/rightmodeler/
+          if-no-files-found: ignore
+      - if: always()
+        uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
+        with:
+          path: .rightmodeler
+          key: rightmodeler-store-${{ github.run_id }}-${{ github.run_attempt }}
+```
+
+## How it behaves
+
+- **The store.** `.rightmodeler/` lives in the Actions cache. Each run saves it under a new key and restores the newest `rightmodeler-store-` entry. GitHub removes entries unused for 7 days, and the 6-hourly watch keeps the store in use. Anyone with read access to the repository can read cache contents and uploaded artifacts, so use this workflow in private repositories.
+- **One run at a time.** All runs share one concurrency group. If a run is already waiting, a newly queued run replaces it, so dispatch again if yours was replaced.
+- **Annotations.** Every error and warning the CLI prints becomes an annotation that names its code, and errors also carry the remedy. `init` adds the report to the job summary, and every job uploads its output files as an artifact.
+- **Stale evidence.** If `main` moves between `init` and `apply`, `apply` refuses with `stale_evidence`. Run `init` again.
+- **Other branches.** A dispatch from another branch works on that branch. It starts from the default branch's store, saves its own copy that only that branch's runs restore, and `apply` opens the draft against that branch.
+- **The token.** The draft is authored by `github-actions[bot]`, and the owners of the swapped files are requested as reviewers. GitHub starts no workflow for a push made with `GITHUB_TOKEN`. For a pull request that `GITHUB_TOKEN` opens, GitHub creates the `pull_request` workflow runs in an approval-required state, and a person with write access starts them with "Approve workflows to run" on the pull request.
+- **Schedules.** GitHub can delay scheduled runs at busy times, and disables schedules in public repositories after 60 days without activity.
+
+## Optional: a GitHub App token
+
+The workflow above runs on the built-in `GITHUB_TOKEN`, the supported default. A GitHub App installation token is an optional upgrade: the draft's `pull_request` workflows start without approval, and the draft is authored by `<app-slug>[bot]`. To use one:
+
+1. Create a GitHub App with the permissions in [GitHub](github.md) and install it on the repository.
+2. Store its client ID in the repository variable `RIGHTMODELER_APP_CLIENT_ID` and its private key in the repository secret `RIGHTMODELER_APP_PRIVATE_KEY`.
+3. In the `apply` and `watch` jobs, add this step before the rightmodeler step, and change that step's `RIGHTMODELER_GITHUB_TOKEN` to `${{ steps.app-token.outputs.token }}`:
+
+   ```
+   - id: app-token
+     uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
+     with:
+       client-id: ${{ vars.RIGHTMODELER_APP_CLIENT_ID }}
+       private-key: ${{ secrets.RIGHTMODELER_APP_PRIVATE_KEY }}
+   ```
+
+## Cloud Mode B
+
+To confirm in Vercel Sandbox, add `--modeb-config <file>` to the `init` command and map the `VERCEL_TOKEN`, `VERCEL_TEAM_ID` and `VERCEL_PROJECT_ID` secrets into that step's `env`. See [Mode B](modeb.md).
+
+## Upgrading
+
+Change `RIGHTMODELER_VERSION`. Each rightmodeler step first runs the CLI with `--version`, so a version npm cannot install fails the step with npm's error in its log instead of being mistaken for a rightmodeler exit code. Each release's copy of this guide pins that release, and `rightmodeler docs github-actions` prints the copy for the installed version.

package/docs/github.md

@@ -0,0 +1,110 @@
+# GitHub
+
+Three commands talk to GitHub. `apply` opens a draft pull request that changes model identifiers only and carries an evidence table. `watch` reconciles one of those pull requests per run. `rollback` opens a draft pull request that restores a merged swap. The CLI never merges a pull request: a person reviews and merges.
+
+## Tokens
+
+Pass the name of the environment variable that holds the token with `--github-token-env`, never the token itself.
+
+| Token                                                                    | What works                                                                                                                                                                                                                                                                                                |
+| ------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| GitHub App installation token (recommended)                              | `apply` and `watch` fully.                                                                                                                                                                                                                                                                                |
+| Classic personal access token, or `gh auth token`, with the `repo` scope | `apply` and `watch` fully.                                                                                                                                                                                                                                                                                |
+| Fine-grained personal access token                                       | `apply` fully. `watch` works but cannot read check runs, so each pass prints the `github_checks_unavailable` warning and uses commit statuses only.                                                                                                                                                       |
+| GitHub Actions `GITHUB_TOKEN`                                            | `apply` and `watch` when the job grants `contents: write`, `pull-requests: write` and `statuses: read` (add `checks: read` for check runs) and the repository allows GitHub Actions to create pull requests. CI on the pull requests it opens waits for approval, which an App installation token avoids. |
+
+Give a GitHub App these repository permissions: Contents read and write, Pull requests read and write, Checks read, Commit statuses read, and Metadata read. Pull requests it opens are authored by `<app-slug>[bot]`.
+
+Give a fine-grained personal access token access to the repository with Contents read and write, Pull requests read and write, Commit statuses read, and Metadata read. GitHub offers no check-run permission for these tokens.
+
+Commenting on a pull request needs only Pull requests write, so neither token needs the Issues permission.
+
+For a ready workflow that uses `GITHUB_TOKEN`, see [GitHub Actions](github-actions.md).
+
+## API host
+
+`--github-base-url` defaults to `https://api.github.com`. For GitHub Enterprise Server, pass its API URL, such as `https://github.example.com/api/v3`. Enterprise Server 3.21 or newer is needed, because the CLI sends REST API version `2026-03-10` and older servers answer every call with HTTP 400.
+
+`--github-repo` defaults to the name of the directory given to `--repo` on `apply`, `watch` and `rollback`. Pass it when the directory name differs from the repository name on GitHub.
+
+## Apply
+
+`apply` needs:
+
+- a completed `init` run whose report recommends at least one swap;
+- local `HEAD` on a branch and equal to the evidence revision, and that branch on GitHub at the same revision (the branch becomes the pull request base);
+- no uncommitted change to any file the swap touches.
+
+`--dry-run` runs every check and reads GitHub but writes nothing. Its result lists the branch, title, body, files and reviewers it would use. The reviewer list is shown before the pull request author is removed, because the author is known only once the pull request exists.
+
+Reviewers:
+
+- Owners of each swapped file come from the first of `.github/CODEOWNERS`, `CODEOWNERS` and `docs/CODEOWNERS` that exists, and the last matching rule wins.
+- A file with no matching rule falls back to its three most recent `git blame` authors, matched to GitHub users by commit email. Blame needs full history, so fetch with depth 0 in CI.
+- At most five users and five teams are requested, and never the pull request author.
+- If GitHub rejects the batch with HTTP 422, each reviewer is requested alone and the rejected ones are dropped.
+- Team reviewers need a repository owned by an organization, and every reviewer needs access to the repository.
+- GitHub does not request code owners on draft pull requests by itself, so every review request on the draft comes from `apply`.
+
+The branch is `<prefix>swap-<family>-<digest prefix>`, where the prefix is the one most common among the repository's recent branches, or `rightmodeler/` when no branch has one. The title is `perf(models): swap <families>` when the repository uses conventional commits, and `Swap <families> models` otherwise. The body is the repository's pull request template, if it has one, followed by the `## Rightmodeler evidence` table: revision, corpus version, and per family the decision, evaluators, cascade status, worst-case bound, models, cost per case, latency, caps and case IDs. Case IDs are SHA-256 digests of the replayed cases, never prompts. With `--code-graph <path>`, a `## Code context (Graphify)` section for the swapped call sites follows the table; the owners it lists are never requested as reviewers.
+
+Rerunning `apply` for the same evidence returns the same open pull request with `status: "existing"` and requests reviewers again only if the first attempt left no record of them.
+
+## Refusal codes
+
+A refusal exits `1` and prints a result with `status: "refused"` and one or more `reasons`, each with a `code`, a `message` and a `detail`.
+
+Apply:
+
+- `no_confirmed_recommendation`: no recommended family has confirmed or not-required cascade evidence.
+- `release_gate_failed`: at least one release gate is not green.
+- `inconsistent_evidence`: the selected families do not share one evidence revision, corpus and gate policy.
+- `previously_rejected`: this evidence and swap set was previously rejected and needs new evidence before it can be proposed again.
+- `stale_evidence`: the evidence revision does not match the repository `HEAD`, or the remote base moved beyond it; re-prove before applying.
+- `detached_head`: the repository has no current branch to use as the pull request base.
+- `dirty_worktree`: a file the swap touches has uncommitted changes; commit or stash them before applying.
+- `stale_location`: a proposed swap no longer matches its scan-time file digest or no longer has one fresh source location.
+- `diff_lint_failed`: the proposed diff changes something other than model identifiers.
+- `formatter_blocked`: the repository's formatter changed content outside the proposed swap.
+- `host_conventions_unreadable`: one or more repository instructions could not be read unambiguously.
+- `invalid_repository_revision`: the evidence revision is not a 40- or 64-character lowercase hex object ID.
+- `apply_branch_unowned`: the swap branch exists without a recorded apply start.
+- `apply_branch_scope_mismatch`: the existing swap branch changes files outside its declared scope.
+- `apply_resume_state_mismatch`: the recorded pre-apply state does not match the resumed change.
+- `apply_restore_failed`: after a failed apply, a file on the swap branch could not be restored.
+
+Rollback:
+
+- `missing_remediation_evidence`: the pull request has no valid recorded apply evidence.
+- `original_pr_not_merged`: only a merged swap pull request can be rolled back.
+- `pre_apply_revision_unavailable`: the recorded pre-apply file revision is unavailable or does not match its digest.
+- `post_apply_digest_mismatch`: the affected files no longer match the recorded post-apply state.
+- `rollback_branch_unowned`: the rollback branch exists without a recorded rollback start or base revision.
+- `rollback_branch_scope_mismatch`: the existing rollback branch changes files outside the recorded scope.
+- `rollback_restore_mismatch`: the rollback did not restore the recorded pre-apply state.
+- `rollback_restore_failed`: after a failed rollback, a file could not be restored to the base state.
+
+## Watch
+
+Each `watch` run makes one pass over one pull request under a lock kept in the store, so overlapping runs do not act twice. Run it on a schedule or from repository events. A pass:
+
+- records a merge and ends watching;
+- records a close without merge as a rejection and ends watching, after which `apply` refuses that swap with `previously_rejected`;
+- answers each human review and comment once with the stored evidence for the families it names, else for the families in the file it comments on, else for every family in the pull request;
+- marks families for re-proof when a reviewer requests changes, or when the base branch changes a swapped file, and says so in a comment; the re-proof itself happens on the next pipeline run;
+- on a failing check run or commit status, comments once, and if a check with the same name fails again under a new run on the same head commit, closes the pull request.
+
+Exit codes:
+
+- `0`: nothing needed doing.
+- `1`: the pass took an action, listed in the result's `actions`.
+- `2`: another watcher holds the lock, or the store has no completed run. With `--output json` or `jsonl`, a held lock prints a result with `"status":"lock_held"` on standard output, while a missing run prints an error with code `stage_not_completed` on standard error and nothing on standard output.
+- `10` or greater: runtime failure.
+
+When GitHub refuses to list check runs with HTTP 403, the pass still reconciles reviews, comments, commit statuses, merges and base-branch changes, and prints the `github_checks_unavailable` warning. Fine-grained personal access tokens always cause it. To include check runs, use a GitHub App installation token with Checks read or a classic token with the `repo` scope.
+
+## Rollback
+
+`rollback --pr <number>` opens a draft pull request that restores the pre-apply contents of the files a merged swap changed. It refuses unless the original pull request merged, and a rerun returns the same rollback pull request.
+
+See [Exit codes](exit-codes.md) and [Commands](commands.md) for every option.

package/docs/modeb.md

@@ -6,6 +6,7 @@ Mode B runs confirmation cases inside a container when a recommendation can affe
 {
   "version": "1",
   "image": "my-agent:latest",
+  "backend": "docker",
   "appSpec": {
     "mountPath": ".",
     "command": ["node", "/rightmodeler/app/driver.mjs", "{caseFile}"],
@@ -26,6 +27,39 @@ Mode B runs confirmation cases inside a container when a recommendation can affe
 - `appSpec.command` is a non-empty array of non-empty arguments. At least one argument must contain `{caseFile}`; the harness replaces every occurrence with the in-container case file path.
 - `appSpec.installCommand` is optional. When present, it is a non-empty array of non-empty arguments run before the workload.
 - `stepMap` maps at least one canonical scanner step ID to the runtime step header emitted by the application. Runtime headers must be unique.
+- `backend` is optional and is either `"docker"` (the default) or `"cloud"`. The cloud backend runs each case in a remote sandbox, so `image` must name an image that sandbox platform can pull, and the run fails before any case starts when the sandbox SDK or its credentials are absent.
 - `confirmMaxRunSets` is optional and must be a non-negative integer.
 
+## Runtime contract
+
+- The container receives `OPENAI_BASE_URL=http://127.0.0.1:8787/v1`, a placeholder `OPENAI_API_KEY`, and the `RM_RUN_ID`, `RM_CASE_ID`, and `RM_EXECUTION_ID` identifiers.
+- `/rightmodeler/scratch/driver/case.json` contains `{ "caseId", "input", "headers"? }`. The `headers` field is omitted when the case has no headers.
+- Every model request must carry `x-rm-step` exactly once. Its value is the runtime step header selected through `stepMap`.
+- Every model request must carry `x-rm-call`. Use one ID per logical call and reuse that ID when the SDK retries the call.
+- The request body must be a JSON object with a non-empty `model` string.
+- On a step the candidate does not replace, the request must name that step's current model by the id the provider catalog lists for it. The proxy holds prices only for the models the run's steps call, so a request for any other model is recorded as lost with `missing_pricing` and never forwarded.
+- `max_completion_tokens` or `max_tokens` is accepted when present. When both are absent, the proxy reserves against the catalog maximum output when known, or 4096 tokens otherwise, and does not add a limit to the forwarded body.
+- Streamed requests are sent with `stream_options.include_usage: true` and metered from the trailing usage chunk. A stream without a usage chunk is charged at its reservation.
+- The last non-empty stdout line must be `{ "runId", "caseId", "executionId", "finalOutput" }`.
+- A request is recorded as lost and never forwarded for `missing_correlation`, `duplicate_step_correlation`, `request_too_large` at 10 MiB, `malformed_json`, `invalid_request`, or `missing_pricing`.
+- A case with any lost request is a lost execution. Lost reason counts are reported on the Mode B result.
+- A request that the case lease cannot cover receives HTTP 402. The case is blocked on budget without an execution fact and is retried on rerun.
+- If the host cannot observe a container exit within the configured timeout plus ten seconds, it force-removes the container and records the case as lost under `container_lifecycle`.
+- The `docker` CLI must reach the Docker daemon. A missing daemon, a failed egress listener, or a failed container launch blocks affected cases with a named reason instead of recording executions, so a rerun retries them.
+- The in-container proxy runs on both backends. The workload always reaches it at `OPENAI_BASE_URL`, and it meters the case lease and records every attempt. Only the hop after it differs: the Docker backend forwards to a host listener over `host.docker.internal`, while the cloud backend forwards straight to the provider and the sandbox platform's egress firewall attaches the model credential in flight. The credential never enters the sandbox on either backend.
+- The workload is killed at the configured timeout by both the host and an in-container deadline.
+
+## Cloud backend
+
+- The cloud backend runs each confirmation case in a short-lived Vercel Sandbox microVM in your own Vercel project. It is separate from the Vercel AI Gateway: the `VERCEL_*` variables only authorize creating sandboxes, and the model credential is always the variable named by `--api-key-env`, whatever the provider.
+- Install: `npx rightmodeler` installs `@vercel/sandbox` as an optional dependency. Installing with `--omit=optional` leaves it out; the CLI still runs and the cloud backend reports `modeb_cloud_unavailable`.
+- Credentials: either `VERCEL_OIDC_TOKEN` (from `vercel link` then `vercel env pull`; it expires after 12 hours) or all of `VERCEL_TOKEN`, `VERCEL_TEAM_ID` and `VERCEL_PROJECT_ID`. Use the access token in CI.
+- Image: `image` must be a Vercel Container Registry reference or a managed image such as `vercel/sandbox/node:24`; a Docker Hub name does not work, and a custom image's entrypoint and command do not run. Commands run as the image's default user, which is not root (`ubuntu` on `vercel/sandbox/node:24`). An image the platform cannot start blocks its case with `launch-failed`, so a rerun retries it.
+- Provider: the base URL must be HTTPS, because the sandbox firewall matches the provider host by TLS server name before it attaches the credential. Its path is kept, so a base URL such as `https://openrouter.ai/api/v1` reaches the same endpoints as on the Docker backend.
+- Credential handling: the firewall adds the model credential to requests for the provider host only, and its policy lets every other host through, so this is credential brokering, not an egress allowlist.
+- Metering: the in-sandbox proxy asks the provider for uncompressed responses so it can read usage from every answer. There is no host listener to mark answers of its own, so every HTTP answer is attributed to the provider, including one the platform firewall produces itself; a connection that fails is attributed to the network path and its case is recorded as a lost execution.
+- Timing: the 60 second case deadline starts when the workload command starts and includes `appSpec.installCommand`; every case starts a fresh microVM, so bake dependencies into the image. The sandbox itself lives for the deadline plus 60 seconds, which also bounds how long an interrupted run can leave one running.
+- Cleanup: sandboxes are created non-persistent and deleted after each case.
+- Scoring: sandboxes never score. The host judges every case; sandbox output is only the candidate text.
+
 See [Commands](commands.md) for where `--modeb-config` is accepted and [Exit codes](exit-codes.md) for blocked or failed runs.

package/docs/model-routes.md

@@ -0,0 +1,111 @@
+# Model routes
+
+Replay sends each recorded request to cheaper candidate models, and the built-in judge grades every answer against the recorded one. Candidates and the judge each run on a route: an API endpoint with a key, or a plan you are signed in to on this machine through its command-line tool.
+
+## Routes
+
+- `--route api` sends candidate calls to the `--base-url` endpoint with the key named by `--api-key-env`. It is the default when `--base-url` is given, so commands from earlier releases behave as before.
+- `--route claude-login` runs candidate calls through the `claude` CLI you are signed in to, under your Claude plan.
+- `--route codex-login` runs candidate calls through the `codex` CLI you are signed in to, under your ChatGPT plan.
+- `--judge-route api`, `--judge-route claude-login` or `--judge-route codex-login` picks where the built-in judge runs. With `--base-url` and no route flag, the judge uses the API route too.
+
+The judge must come from a vendor other than both the candidate's and the recorded model's. A plan route serves one vendor's models, so a plan `--route` needs an explicit `--judge-route`; without one, rightmodeler stops with `invalid_option`. Candidates from the judge route's own vendor are left out with the warning `judge_vendor_candidates_dropped`, and the run stops before any model call with `no_neutral_judge` when no candidate is left or the recorded model comes from the judge's vendor.
+
+`--base-url`, `--api-key-env` and `--header` configure the API route, so they are refused when both `--route` and `--judge-route` name a plan route. `--detach` and `--modeb-config` are refused when either role uses a plan route: detached replay and Mode B confirmation call models only through an API endpoint. `--evaluator` works with every route; the built-in judge on `--judge-route` grades when the evaluator is unreachable.
+
+```sh
+npx rightmodeler init --traces traces.jsonl --route claude-login --judge-route api --base-url https://openrouter.ai/api/v1 --api-key-env OPENROUTER_API_KEY
+npx rightmodeler init --traces traces.jsonl --base-url https://api.openai.com/v1 --api-key-env OPENAI_API_KEY --catalog-reference https://ai-gateway.vercel.sh/v1/models --judge-route claude-login
+npx rightmodeler init --traces traces.jsonl --route codex-login --judge-route claude-login
+```
+
+## Choosing at the start
+
+Run in a terminal, `init` and `estimate` first ask how to call models, before the trace question and before any stage runs.
+
+- **Your plans:** rightmodeler lists the `claude` and `codex` CLIs on this machine, each with its status and, when it cannot be used, how to fix it. The status comes from each CLI's version and login commands, run with the same API key variables kept away as for a call, and with the variable a saved answer names kept away too. You choose the CLI that replays candidates (choose the vendor your app calls today) and where the judge runs: the other CLI, or an API key for OpenRouter, Vercel AI Gateway or another OpenAI-compatible endpoint. The first time a plan route is chosen, rightmodeler shows what it sends and asks `[y/N]`; any other answer than `y` continues with no route.
+- **An API key:** OpenRouter, Vercel AI Gateway, OpenAI, Anthropic, or another OpenAI-compatible endpoint, then the name of the environment variable that holds the key. The question checks only whether that variable is set, never its value. It accepts only a name made of capital letters, digits and underscores, so a pasted key is refused and never repeated or saved, and it refuses a base URL that carries a user name, a password or a query string.
+
+The answer is saved in the store as `project/setup/model-route.json`: route names, a base URL and a variable name, never a key. The next interactive run shows it as flags and keeps it when you press Enter; type `c` to choose again. `--yes` applies it without asking. The question and the saved answer are skipped with `--output json` or `jsonl`, without a terminal, when `--route`, `--judge-route`, `--base-url`, `--api-key-env` or `--header` is passed, with `init --plan`, and with `--through` before `replay`. Scripts pass the flags the question prints. With `--modeb-config`, only API routes through a multi-vendor endpoint are offered, because Mode B confirmation runs only through an API key. Ctrl-C or Ctrl-D at a question continues with no route: the free stages run and replay stops with `missing_provider_configuration`.
+
+## API routes
+
+| Choice            | `--base-url`                      | Key variable, by default | Also passed                                                                                         |
+| ----------------- | --------------------------------- | ------------------------ | --------------------------------------------------------------------------------------------------- |
+| OpenRouter        | `https://openrouter.ai/api/v1`    | `OPENROUTER_API_KEY`     | nothing                                                                                             |
+| Vercel AI Gateway | `https://ai-gateway.vercel.sh/v1` | `AI_GATEWAY_API_KEY`     | nothing                                                                                             |
+| OpenAI            | `https://api.openai.com/v1`       | `OPENAI_API_KEY`         | `--route api --judge-route claude-login --catalog-reference https://ai-gateway.vercel.sh/v1/models` |
+| Anthropic         | `https://api.anthropic.com/v1`    | `ANTHROPIC_API_KEY`      | `--route api --judge-route codex-login --catalog-reference https://ai-gateway.vercel.sh/v1/models`  |
+| Another endpoint  | the URL you type                  | `RIGHTMODELER_API_KEY`   | nothing                                                                                             |
+
+OpenAI's and Anthropic's APIs serve only their own models and list no prices, so the judge runs through the other vendor's CLI signed in on this machine and prices come from the public list; without that CLI, choose a gateway. A variable that is not set yet is still saved: set it in your own shell before replay.
+
+## Use your Claude plan
+
+`--route claude-login` and `--judge-route claude-login` run the `claude` CLI (Claude Code) already installed and signed in on this machine, version 2.1.282 or newer. Rightmodeler runs your own unmodified binary as a child process under the login it already holds. It never reads, stores or forwards a token, and never opens `~/.claude`, the macOS Keychain or a credential file.
+
+Before any model call, rightmodeler runs `claude --version` and `claude auth status`. It accepts only a Claude plan login: signed in, through Anthropic directly, with a claude.ai login or a `claude setup-token` token. Anything else, such as an API key, Amazon Bedrock or Google Vertex, stops with `plan_login_required`.
+
+Each call runs in a fresh, empty temporary directory with no tools, no MCP servers, no settings files, no skills, no auto memory, no session file and one turn: `claude -p --model <id> --system-prompt-file <file> --tools "" --strict-mcp-config --disable-slash-commands --setting-sources "" --no-session-persistence --max-turns 1 --settings {"switchModelsOnFlag":false} --output-format stream-json --verbose`, with the recorded user message on standard input. Because no session file is written, replays never appear in the Claude Code transcripts rightmodeler reads as traces.
+
+When `claude` runs the built-in judge, rightmodeler adds `--json-schema` with the verdict schema and sets `--max-turns 3`. `claude` returns the verdict through its StructuredOutput tool, checks it against the schema and asks the model again when it does not match; rightmodeler reads the verdict from the `structured_output` field. A verdict that still does not match after two corrections, or a call that ends without `structured_output`, counts as a malformed judge answer, as on the API route. Candidate replays get no schema and keep `--max-turns 1`. In rightmodeler's check on `claude` 2.1.283 on 2026-09-26, `--max-turns 3` allowed the first answer and two corrections, and the schema added about 540 input tokens to each Opus 5 judge call; cost estimates use the request's own tokens.
+
+An API key variable would make `claude` bill the key instead of your plan: in non-interactive mode the key is always used when present. Rightmodeler removes every `ANTHROPIC_*` and `OPENAI_*` variable, `CODEX_API_KEY`, the run's `--api-key-env` variable, the other key variables the first-run question offers (`OPENROUTER_API_KEY`, `AI_GATEWAY_API_KEY` and `RIGHTMODELER_API_KEY`), and the variables of a parent Claude Code session from the child's environment, and warns with `plan_route_key_withheld` when `ANTHROPIC_API_KEY` or `ANTHROPIC_AUTH_TOKEN` was set, naming the variable and never its value. It keeps `CLAUDE_CODE_OAUTH_TOKEN` and `CLAUDE_CONFIG_DIR`. If `claude` still reports that a call would be paid by a key, rightmodeler stops that call at once with `plan_login_required`.
+
+Rightmodeler runs at most `--max-concurrency` `claude` processes at a time (default 2) and stops any still running when it exits. It refuses plan routes when the `CI` environment variable is set: a plan is for your own machine, not for continuous integration.
+
+## Use your ChatGPT plan through Codex
+
+`--route codex-login` and `--judge-route codex-login` run the `codex` CLI already installed and signed in on this machine, version 0.153.3 or newer. As with Claude, rightmodeler runs your own unmodified binary under the login it already holds.
+
+- What runs: `codex exec --json --ephemeral --ignore-user-config --ignore-rules --strict-config --skip-git-repo-check --sandbox read-only`, in a fresh, empty temporary directory, with the recorded system and developer messages in a temporary instructions file (`model_instructions_file`), the recorded user message on standard input, and settings that turn off the shell and other tools, web search, apps, plugins, skills, memories, the injected permission, app and environment context, project instructions and history. Code execution is off too (`features.code_mode_host=false`). A case with no system or developer message runs with `instructions=""` instead of the file, because Codex refuses an empty instructions file and would otherwise add its own default instructions. `--ephemeral` keeps the session file out of `~/.codex/sessions`, where rightmodeler reads Codex traces, and `--ignore-user-config` keeps your `config.toml`, MCP servers and plugins out of the call. `--strict-config` makes a newer `codex` that renamed one of these settings stop with `plan_cli_unavailable` instead of quietly turning a tool back on. When `codex` runs the built-in judge, rightmodeler also writes the verdict schema to that directory and passes `--output-schema`, so Codex's final message is the verdict as JSON.
+- What it never touches: `auth.json`, the keyring or any token. Rightmodeler picks the credential store by name (`file`, else `keyring`) from what `codex login status` reports, and passes that name to each call. `CODEX_API_KEY`, every `OPENAI_*` and every `ANTHROPIC_*` variable, and the key variables listed for Claude above are kept away from `codex`, and `plan_route_key_withheld` names a set `CODEX_API_KEY`, `OPENAI_API_KEY`, `OPENAI_FEDERATION_RULE_ID` or `OPENAI_IDENTITY_TOKEN_FILE`, never its value. `CODEX_HOME` and `CODEX_ACCESS_TOKEN` are passed through unread. A login with an API key, workload identity or Amazon Bedrock stops with `plan_login_required`, because it would not use your ChatGPT plan.
+- What a Codex route measures: Codex adds about 2,000 input tokens of its own context to each call. That includes your global instructions file, `$CODEX_HOME/AGENTS.md` (or `AGENTS.override.md`), which Codex always adds and cannot be told to leave out; rightmodeler checks only that the file exists, never reads it, and warns once with `codex_global_instructions`. Cost estimates use the recorded input tokens, so this context does not change the savings. Codex cannot set temperature or an output limit, and each model runs at its default reasoning effort, as on the API route, which sends none. Codex reports no latency, so the p50 latency of a Codex answer reads n/a.
+- What Codex does not report: which model answered. Rightmodeler records the requested model, and leaves a call out of the evidence when Codex reports that it rerouted the call to another model.
+- Tools: Codex keeps a code tool and a patch tool registered that no setting removes, so a call can include a tool step its output does not show. A call where Codex reports a tool step is left out of the evidence. In rightmodeler's check on `codex-cli` 0.153.3 on 2026-09-25, a prompt asking the model to read a file with its code tool used 13,325 input tokens with code execution on and 3,826 with it off; in both runs the model answered that it could not run anything, did not return the file's contents, and the output showed no tool step.
+- Models: the list comes from `codex debug models` of the same binary that makes the calls, so update Codex to see newer models. GPT-5.5 retires from ChatGPT sign-in on October 14, 2026; it stays on the OpenAI API.
+- Limits: a limit on one model blocks that candidate, or moves the judge to its next model. An account usage limit, a workspace out of credits or a spend cap starts no new call and exits `2` with `plan_usage_limit`, quoting Codex's reset time as Codex wrote it. Rerun after the reset: finished calls are kept. `codex exec` reports no remaining allowance, so rightmodeler cannot warn before the limit.
+- Concurrency: at most `--max-concurrency` `codex` processes run at a time (default 2), each stopped when rightmodeler exits. Plan routes are refused when `CI` is set.
+
+## What a plan route measures
+
+A plan route measures the model inside a coding CLI, not the API request your application makes:
+
+- Claude Code adds its own instructions to every call, even when rightmodeler replaces the system prompt: 380 to 539 input tokens in testing, including the signed-in account's email address and today's date.
+- The CLI cannot set temperature or an output limit, so the recorded values are not applied, and an answer can be longer than on the API route.
+- It sends one user turn. Recorded cases with an earlier assistant or tool turn, or more than one user message, are left out of the replay sample with the warning `plan_route_cases_left_out`.
+- With `claude`, latency is the API time the CLI reports, without its start-up time.
+- With `claude`, rightmodeler checks which model answered on every call, and records a substitution when `claude` answers with another model, takes more than one turn or calls a tool. A judge call is expected to use the StructuredOutput tool and to report up to four turns for the first answer and two corrections, so it counts as a substitution only when `claude` calls another tool or takes more than four turns.
+
+The report's "Model routes" section, and the pull request that `apply` opens, say when a result was measured through a plan.
+
+With `claude`, your recorded prompts are sent to Anthropic under your plan account's data settings. Anthropic's data-usage page says: "We will train new models using data from Free, Pro, and Max accounts when this setting is on (including when you use Claude Code from these accounts)" (`https://code.claude.com/docs/en/data-usage`).
+
+## Costs, the cap and usage limits
+
+Calls through a plan are not billed in dollars: they use your plan's usage allowance; with `claude`, the same 5-hour and weekly limits as your own Claude Code sessions. To budget and compare them, rightmodeler prices each call at API list prices from a public price list, `https://ai-gateway.vercel.sh/v1/models`, read without a key. `--catalog-reference <url-or-path>` replaces that list, including with a local file when the public list is unreachable. `--pricing-file` only overrides the prices of the ids it names.
+
+A call's cost is the recorded request's input tokens plus the output tokens the CLI reports, at list price, and is always marked as an estimate. The CLI's added instructions stay out of it, so savings compare with the recorded calls. The report and `estimate` label these amounts as list-price equivalents and show dollars billed through an API route separately.
+
+Plan routes have no default limit on calls or spend: a run makes as many calls as it needs, and the vendor's own usage limit is the only stop (the run resumes after the reset). `--max-cost-usd` is optional; when you set it on a plan route it caps the list-price equivalent, as a soft cap: the CLI sets no output limit, so a call can cost more than its reservation.
+
+When `claude` reports your plan near its limit, rightmodeler warns once with `plan_usage_warning`, giving the share used and the reset time. When the plan reaches its limit, or further calls would bill usage credits (`credits_required`, or `claude` reporting overage), rightmodeler starts no new call and exits `2` with `plan_usage_limit`, quoting the reset time. Calls already running finish and are kept, so up to `--max-concurrency` calls can still use credits when overage begins. Rerun the same command after the reset: finished replay and judge calls are reused.
+
+Anthropic announced, then paused, a change that would take `claude -p` usage off your plan's limits and onto a monthly credit, then usage credits (`https://support.claude.com/en/articles/15036540-use-the-claude-agent-sdk-with-your-claude-plan`). For now, the page says, `claude -p` still draws from your subscription's usage limits. If the change takes effect, rightmodeler stops at `credits_required` or overage as described above.
+
+Rightmodeler leaves out Claude Fable models, because in non-interactive mode "When a Fable request there would bill to usage credits, Claude Code bills it without asking" (`https://code.claude.com/docs/en/model-config`), and `[1m]` variants, whose 1M context can require usage credits. Models the price list does not price are left out with `plan_model_unpriced`.
+
+## Errors
+
+- `plan_cli_unavailable` (exit `2`): `claude` is missing or older than 2.1.282, `codex` is missing or older than 0.153.3, `CI` is set, the CLI changed an output shape rightmodeler relies on, or `codex` rejected one of rightmodeler's isolation settings.
+- `plan_login_required` (exit `2`): `claude` is not signed in with a Claude plan, `codex` is not signed in with ChatGPT, the CLI would use an API key, or it lost its login during the run.
+- `plan_usage_limit` (exit `2`): the plan reached its usage limit.
+- `no_neutral_judge` and `judge_family_unknown` (exit `2`): no judge from a third vendor is available. See [Exit codes](exit-codes.md).
+
+## Terms
+
+Anthropic's legal page says OAuth authentication "is designed to support ordinary use of Claude Code and other native Anthropic applications", that third-party developers may not "route requests through Free, Pro, or Max plan credentials on behalf of their users", and that this does not "prevent an end user from signing in to the unmodified Claude Code binary with their own Claude subscription" (`https://code.claude.com/docs/en/legal-and-compliance`). The Agent SDK page adds: "Unless previously approved, Anthropic does not allow third party developers to offer claude.ai login or rate limits for their products" (`https://code.claude.com/docs/en/agent-sdk/overview`). Anthropic's Consumer Terms restrict access "through automated or non-human means, whether through a bot, script, or otherwise" except "where we otherwise explicitly permit it" (`https://www.anthropic.com/legal/consumer-terms`).
+
+OpenAI's non-interactive guide says "`codex exec` reuses saved CLI authentication by default." and "API keys are the right default for automation because they are simpler to provision and rotate. Use this path only if you specifically need to run as your Codex account." (`https://learn.chatgpt.com/docs/non-interactive-mode`). OpenAI's pricing page lists "Codex SDK, `codex exec`, and scriptable workflows" as available on Plus, Pro, Business and Enterprise (`https://learn.chatgpt.com/docs/pricing`). OpenAI's Terms of Use list "Automatically or programmatically extract data or Output" among what you may not do (`https://openai.com/policies/terms-of-use/`). OpenAI's CI guide says "Do not use this workflow for public or open-source repositories" about seeding `auth.json` on CI runners (`https://learn.chatgpt.com/docs/auth/ci-cd-auth`).
+
+Refusing plan routes when `CI` is set is rightmodeler's own product choice: a plan route is local and opt-in. Rightmodeler runs on your machine, for you, with your own unmodified `claude` or `codex` and your own login, and never handles a credential. Whether a replay fits your plan's terms is your decision; the API route is always available.