metergraph-cli 0.2.0-preview.4 → 0.2.0-preview.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,105 @@
1
+ ---
2
+ name: metergraph-investigate
3
+ description: Investigate a failed or anomalous LLM trace, usage spike or incident through an already connected Metergraph Agent Access server, answering only from bounded, cited evidence. Use after setup is verified; use the metergraph skill first when not connected.
4
+ ---
5
+
6
+ # Investigate with Metergraph evidence
7
+
8
+ This skill expands the bounded investigation section of the public `metergraph`
9
+ skill. It adds no tools, scopes or authority. When no Metergraph MCP server is
10
+ connected, or the user has not chosen a client, runtime and deployment, stop and
11
+ use the `metergraph` skill for setup routing. Never ask for, echo or store a
12
+ key.
13
+
14
+ ## 1. Discover before reading data
15
+
16
+ 1. Call `metergraph_get_workspace_context`, then `metergraph_get_capabilities`.
17
+ Do not call any other Metergraph tool first.
18
+ 2. Confirm with the user that `provenance.workspace_id`, the workspace name and
19
+ `provenance.deployment_profile` match the installation they intend. A staging
20
+ or demo workspace is not their production workspace. Stop on a mismatch,
21
+ an authorization error, or revoked or expired access, and tell the human to
22
+ reconnect through their client's secure flow.
23
+ 3. Build the tool plan only from what the server returns: the workspace
24
+ context's `access.scopes`, and each capability under the capabilities
25
+ response's `agent` object with its `available`, `privacy_class` (metadata,
26
+ content or replay), `required_scope`, `mutates` and `external_calls`, plus
27
+ `bounds` (`max_days`, `max_rows`). Treat the tools your client lists as the
28
+ only callable tools. A tool or capability that is absent, or has
29
+ `available: false`, is a blocker to report, not something to
30
+ guess, retry under another name or replace with a dashboard API, generic SQL
31
+ or a shell command. Some deployments are content-blind or have no replay,
32
+ incidents, ingestion health or reports.
33
+
34
+ ## 2. Fix the question and the bound
35
+
36
+ Restate the question, the route or workload if known, and an explicit window in
37
+ whole days (tools accept `days` from 1 to 90; default to the shortest window
38
+ that answers the question). Then choose the smallest tool that can answer:
39
+
40
+ | Question | First tool | Bound |
41
+ | --- | --- | --- |
42
+ | How much, how many, which route costs most | `metergraph_get_usage` | `days`, small `limit` |
43
+ | Which routes exist or are replay eligible | `metergraph_list_routes` | none needed |
44
+ | Is data arriving | `metergraph_get_ingestion_health` | `days` |
45
+ | Known alert or anomaly | `metergraph_list_incidents` | small `limit` |
46
+ | One failed, slow or costly call | `metergraph_query_traces` with `status`, `route`, `trace_id` or `request_id` | `limit` 1 to 5 |
47
+ | A finished analysis | `metergraph_list_reports`, then `metergraph_get_report` | small `limit` |
48
+
49
+ Follow `next_cursor` only while you are still inside the agreed bound, and say
50
+ how many pages you read. Never request every trace, every report or unbounded
51
+ history.
52
+
53
+ ## 3. Escalate privacy classes one step at a time
54
+
55
+ - Metadata tools answer most questions. Stay there unless the answer needs
56
+ captured prompts, outputs, tool calls or report evidence.
57
+ - `metergraph_get_trace` and `metergraph_get_report_evidence` return captured
58
+ content (`agent:read`). Ask the user before calling them, name the single trace
59
+ or workload you will read, and do not quote more captured content than the
60
+ answer needs. Never paste secrets you find in content.
61
+ - `metergraph_replay_trace` (`agent:replay`) sends a captured trace to a model
62
+ provider and can spend money. It needs separate, explicit approval that names
63
+ the trace, the overrides and the timeout. Replay is non-persistent: it does not
64
+ change production telemetry, prove a fix or substitute for an evaluation run.
65
+
66
+ Read access never grants evaluation writes, analysis launch, dashboard changes,
67
+ key rotation or production configuration changes. Refuse those requests and
68
+ point to the human action in the Metergraph app instead.
69
+
70
+ ## 4. Answer only from returned evidence
71
+
72
+ Every answer cites, from the tool responses you actually received:
73
+
74
+ - **Workspace**: workspace name and `provenance.workspace_id`, plus
75
+ `deployment_profile`.
76
+ - **Window**: `window.since` to `window.until` (or `days`) for each read.
77
+ - **Provenance**: the tool name, `provenance.source` and `generated_at`.
78
+ - **Evidence and completeness**: `evidence.rows`, `evidence.complete`,
79
+ `page.truncated` and whether a `next_cursor` was left unread.
80
+ - **Warnings**: every returned `warnings[].code` with its message. Say "no
81
+ warnings returned" when the list is empty.
82
+ - **Identifiers and links**: trace, request, incident or analysis IDs, and
83
+ `metergraph_links` values exactly as returned. Never construct a link.
84
+
85
+ Keep these states distinct and name the one that applies: no data in the
86
+ window, incomplete or truncated evidence, capability unavailable in this
87
+ deployment, classification pending, ambiguous request match
88
+ (`request_match.ambiguous`), failed analysis, and missing report. An empty page
89
+ does not prove there was no traffic; check ingestion health or a longer bounded
90
+ window before saying so. Do not round, extrapolate or invent totals.
91
+
92
+ ## 5. Worked workflow: one failed trace
93
+
94
+ 1. Discover: context, then capabilities. Confirm the workspace with the user.
95
+ 2. Find: `metergraph_query_traces` with `status` set to the failure status, the
96
+ agreed `days` and `limit` 1. If the user gave a request ID and the match is
97
+ ambiguous, list the candidates and ask which one.
98
+ 3. Explain from metadata: route, model, provider, tokens, cost, latency, status
99
+ and span count, with the citations above. If metadata is enough, stop here.
100
+ 4. Optional content: with consent, `metergraph_get_trace` for that one trace to
101
+ read the error, tool calls and root-cause classification.
102
+ 5. Optional replay: only with explicit approval and when the trace is replay
103
+ eligible, run one bounded `metergraph_replay_trace`, report the returned
104
+ status and comparison, and state that nothing in production changed.
105
+ 6. Close with what is known, what is unknown, and the next bounded check.
@@ -0,0 +1,73 @@
1
+ ---
2
+ name: metergraph-iterate
3
+ description: Refine and rerun a Metergraph model-swap analysis after the person has inspected the report in the app - turn their findings into a specific change (eval revision, cases, candidates, traffic or route settings), start a new approved run linked to the previous one, and compare the two honestly. Use when someone returns with "the grader is wrong on these cases", "drop that model", "rerun it", "what changed since last run", or stage 6 of the metergraph-model-swap loop.
4
+ ---
5
+
6
+ # Refine and rerun
7
+
8
+ A rerun is a new run with one deliberate change. The earlier run, report and
9
+ eval revision stay as they are.
10
+
11
+ ## 0. Confirm the connection
12
+
13
+ If this conversation has not already confirmed the workspace, call
14
+ `metergraph_get_workspace_context`, then `metergraph_get_capabilities`, before
15
+ anything else. Say which workspace you are connected to in one line, and plan
16
+ only with the tools the capabilities advertise.
17
+
18
+ ## 1. Recover where the loop is
19
+
20
+ 1. Read the loop record (in the conversation or `.metergraph/model-swap.md`). If
21
+ there is none, `metergraph_list_analysis_runs` and `metergraph_get_report`
22
+ on the latest run for this workload, and rebuild it.
23
+ 2. Confirm the workload's `selection_reference` still exists with
24
+ `metergraph_list_classified_workloads`. A newer classification may have
25
+ replaced it; say so and pick the matching workload by its traces, not by its
26
+ name alone.
27
+
28
+ ## 2. Turn the finding into one change
29
+
30
+ Ask what the person saw in the app, then map it:
31
+
32
+ | Finding | Change | Skill |
33
+ | --- | --- | --- |
34
+ | A grade is wrong on some cases | Fix the check or rubric criterion; save a new eval revision; add the cases as reference cases | `metergraph-evals` |
35
+ | The bar is missing something important | Add a check or criterion, test it on the preview, save a new revision | `metergraph-evals` |
36
+ | A candidate is clearly out, or one is missing | Change candidate selection on the Opportunities page | `metergraph-candidates` |
37
+ | Too few samples, or `insufficient_samples` | More production traffic for this workload, then rerun | `metergraph-workloads` |
38
+ | Traces were skipped | Replay opt-out or residency on a route; the person changes it in the app | `metergraph-candidates` |
39
+ | The winner looks right | No rerun. Plan the code change and a canary instead | — |
40
+
41
+ Change one thing per rerun when you can, so the comparison means something.
42
+ If several must change, list them all in the confirmation.
43
+
44
+ ## 3. Rerun only with a new yes
45
+
46
+ A rerun spends money and sends captured prompts to providers again. Get the
47
+ same explicit confirmation as the first run (see `metergraph-analyze`), plus
48
+ what changed. If the request already names the change and approves a budget and
49
+ time, that is the confirmation: restate it, make the change (for an eval fix,
50
+ save the new revision first, as in `metergraph-evals`), then start. Then call `metergraph_start_analysis` with:
51
+
52
+ - a **new** `request_key`, for example `rerun-<pattern_id>-<yyyymmdd>-<n>`;
53
+ - the same `pinned_workload`, unless the workload itself changed;
54
+ - `rerun_of_run_id` set to the previous run ID;
55
+ - `reference_binding_id` when reference cases were bound.
56
+
57
+ Poll and read the report as in `metergraph-analyze` and `metergraph-report`.
58
+
59
+ ## 4. Compare honestly
60
+
61
+ Show the two runs side by side only for what is comparable:
62
+
63
+ ```
64
+ run <a> (<date>) run <b> (<date>)
65
+ eval <revision a> <revision b>
66
+ samples <n> <n>
67
+ <candidate> <grade, cost ratio> <grade, cost ratio>
68
+ ```
69
+
70
+ If the eval revision, the sample or the candidate set changed, say that a
71
+ different grade may come from the change, not from the model. Then give the
72
+ decision line from `metergraph-report` with the new report link, and update the
73
+ loop record.
@@ -0,0 +1,283 @@
1
+ ---
2
+ name: metergraph-local-bundle
3
+ description: Verify and start the signed Metergraph customer-local bundle on a person's own machine, preserve existing local data and secrets, then hand over to metergraph setup. Use when customer-local setup reports that the bundle_started_verified prerequisite is incomplete or when installing the commercial local bundle from a public release.
4
+ ---
5
+
6
+ # Start the signed customer-local bundle
7
+
8
+ Stand up the released bundle only on the person's own machine, then run
9
+ `metergraph setup`. The CLI does not provision it, and
10
+ `--confirm-prerequisites` records confirmation rather than installing anything.
11
+
12
+ Keep registry, admin and database secrets out of chat, command arguments and
13
+ output. Never read or print `.env` values, `~/.docker/config.json`,
14
+ `docker compose config` output or container environment from `docker inspect`.
15
+ Do not configure model-provider keys.
16
+
17
+ ## Stop in remote runtimes
18
+
19
+ Before changing anything, determine where the agent executes. Check only
20
+ whether these variables exist, never their values:
21
+
22
+ ```sh
23
+ for name in SSH_CONNECTION SSH_CLIENT SSH_TTY CODESPACES GITPOD_WORKSPACE_ID CLOUD_SHELL CI; do
24
+ if printenv "$name" >/dev/null 2>&1; then printf '%s is set\n' "$name"; fi
25
+ done
26
+ ```
27
+
28
+ If any is set, or this is otherwise a hosted/cloud runtime, download, verify,
29
+ extract and start nothing. Prefer an installed `metergraph` whose `--help`
30
+ lists `setup`; otherwise state that the fallback fetches the preview CLI and
31
+ use `npx --yes metergraph-cli@next` as the prefix. Run only:
32
+
33
+ ```sh
34
+ metergraph setup --runtime cloud --deployment customer-local --url http://localhost:8080 --workspace 11111111-1111-4111-8111-111111111111 --skip-skill --json
35
+ ```
36
+
37
+ Relay its `data.next_action.kind: "run_on_customer_machine"` handoff and tell
38
+ the person to run this skill on their own machine. The placeholder workspace is
39
+ intentional: the runtime check refuses before making a request or writing a
40
+ file. When an SSH, cloud or CI marker is set, even `--runtime local` is refused
41
+ with reason `run_on_local_machine` and the same handoff. Valid runtimes are
42
+ `local`, `cloud` and `cloud-no-shell`. Never expose localhost to the remote
43
+ session.
44
+
45
+ ## Preflight and reuse
46
+
47
+ Ask for the coding client and bundle version. Default to the newest version at
48
+ https://www.metergraph.dev/docs/releases/ and record its `v`-prefixed value.
49
+ Record the application repository directory to return to for setup. Run:
50
+
51
+ ```sh
52
+ docker info >/dev/null
53
+ docker compose version
54
+ cosign version
55
+ curl -fsS http://localhost:8080/v1/deployment
56
+ lsof -nP -iTCP:8080 -sTCP:LISTEN
57
+ ```
58
+
59
+ Docker with Compose must be running and `cosign` installed. If the deployment
60
+ endpoint returns the `local` profile, reuse that stack and download nothing.
61
+ Find its bundle directory using only the Compose working-directory label:
62
+
63
+ ```sh
64
+ container_id="$(docker ps --filter publish=8080 --format '{{.ID}}' | head -n 1)"
65
+ bundle_dir="$(docker inspect --format '{{ index .Config.Labels "com.docker.compose.project.working_dir" }}' "$container_id")"
66
+ (cd "$bundle_dir" && ./bin/status)
67
+ ```
68
+
69
+ Do not inspect container environment. Read the workspace ID from the
70
+ `workspace` check and skip to **Run setup**. If the directory cannot be found,
71
+ ask for the workspace ID from Settings. If nothing listens on 8080, continue.
72
+ If anything else owns it, stop and ask; never stop or reconfigure it.
73
+
74
+ Use `~/metergraph/<version>` and set
75
+ `bundle_dir="$HOME/metergraph/$version/local"`. If it contains `.env`, reuse
76
+ that existing installation: never overwrite, delete or read the file, and
77
+ never extract over `local/`. Continue at **Log in and start**. Otherwise check:
78
+
79
+ ```sh
80
+ docker volume inspect local_pgdata >/dev/null 2>&1
81
+ ```
82
+
83
+ Because the extracted directory and Compose project are `local`, an existing
84
+ volume is `local_pgdata`. If it exists without the matching `local/.env`, stop
85
+ and ask for the prior bundle/configuration. Postgres fixed its password when
86
+ the volume was created; a new `.env` can cause `password authentication failed`.
87
+ Never remove the volume, delete `.env` or use `down --volumes`.
88
+
89
+ ## Download and verify
90
+
91
+ Require a fresh, empty version directory. Before creating it, tell the person
92
+ the directory and four public files that will be written. No account is needed.
93
+ Set `version` to the recorded version and run:
94
+
95
+ ```sh
96
+ release_dir="$HOME/metergraph/$version"; mkdir -p "$release_dir"; cd "$release_dir"
97
+ archive="metergraph-customer-local-$version.tar.gz"
98
+ base="https://www.metergraph.dev/releases/$version"
99
+ curl -fsSLO "$base/$archive"
100
+ curl -fsSLO "$base/metergraph-byoc-release.json"
101
+ curl -fsSLO "$base/metergraph-byoc-release.json.sig"
102
+ curl -fsSLO "$base/metergraph-byoc-release.json.pem"
103
+ ```
104
+
105
+ Do not list, extract or run archive contents. Verify the manifest signature
106
+ with exactly this command. The issuer and identity are pinned here, not read
107
+ from the release page, so a changed download site cannot change who must have
108
+ signed the release:
109
+
110
+ ```sh
111
+ cosign verify-blob \
112
+ --certificate metergraph-byoc-release.json.pem \
113
+ --signature metergraph-byoc-release.json.sig \
114
+ --certificate-oidc-issuer https://token.actions.githubusercontent.com \
115
+ --certificate-identity-regexp '^https://github\.com/PioneerSquareLabs/metergraph-internal/\.github/workflows/byoc-release\.yml@refs/(heads/main|tags/v.+)$' \
116
+ metergraph-byoc-release.json
117
+ ```
118
+
119
+ Never loosen or drop the issuer or identity, and never add
120
+ `--insecure-ignore-tlog` or a similar bypass. If the release page publishes a
121
+ different identity, stop and tell the person rather than using it.
122
+
123
+ If cosign fails, stop: "The manifest signature did not verify. Do not extract
124
+ or run this bundle." After it succeeds, check signed fields:
125
+
126
+ ```sh
127
+ python3 - "$version" "$archive" <<'EOF'
128
+ import json, sys
129
+ version, archive = sys.argv[1:]
130
+ m = json.load(open("metergraph-byoc-release.json"))
131
+ required = ("release", "local_bundle", "local_bundle_sha256",
132
+ "local_image_digest", "local_analysis_image_digest")
133
+ missing = [key for key in required if not m.get(key)]
134
+ if missing: sys.exit("Signed manifest is missing: " + ", ".join(missing))
135
+ if m["release"] != version: sys.exit("Signed manifest release mismatch. Do not run it.")
136
+ if m["local_bundle"] != archive: sys.exit("Signed archive name mismatch. Do not run it.")
137
+ print("Signed manifest release, archive, and image digests are present.")
138
+ EOF
139
+ ```
140
+
141
+ Then run this digest check, still before any `tar` command:
142
+
143
+ ```sh
144
+ python3 - <<'EOF'
145
+ import hashlib, json, sys
146
+ manifest = json.load(open("metergraph-byoc-release.json"))
147
+ name = manifest["local_bundle"]
148
+ actual = hashlib.sha256(open(name, "rb").read()).hexdigest()
149
+ if actual != manifest["local_bundle_sha256"]:
150
+ sys.exit(f"{name} does not match the signed manifest. Do not run it.")
151
+ print(f"{name} matches the signed manifest.")
152
+ EOF
153
+ ```
154
+
155
+ Stop on failure. After both checks pass, list and validate archive paths:
156
+
157
+ ```sh
158
+ tar -tzf "$archive"
159
+ python3 - "$archive" <<'EOF'
160
+ from pathlib import PurePosixPath
161
+ import sys, tarfile
162
+ bad = []
163
+ with tarfile.open(sys.argv[1], "r:gz") as bundle:
164
+ for member in bundle.getmembers():
165
+ path = PurePosixPath(member.name)
166
+ if path.is_absolute() or not path.parts or path.parts[0] != "local" or ".." in path.parts:
167
+ bad.append(member.name)
168
+ if bad: sys.exit("Unsafe archive paths; do not extract: " + ", ".join(bad[:10]))
169
+ print("Every archive entry is beneath local/ with a safe relative path.")
170
+ EOF
171
+ ```
172
+
173
+ Only now tell the person extraction will create `local/`, then run:
174
+
175
+ ```sh
176
+ tar xzf "$archive"
177
+ mv metergraph-byoc-release.json* local/
178
+ cd local
179
+ ```
180
+
181
+ ## Configure without exposing passwords
182
+
183
+ Reuse an existing `.env` unchanged. For a new installation, state that a
184
+ mode-0600 file with random admin/database passwords and empty provider keys
185
+ will be created. Confirm `local_pgdata` did not exist, then run:
186
+
187
+ ```sh
188
+ umask 077; cp .env.example .env; chmod 600 .env
189
+ python3 - <<'EOF'
190
+ from pathlib import Path
191
+ import secrets
192
+ path = Path(".env")
193
+ keys = ("METERGRAPH_LOCAL_ADMIN_PASSWORD", "METERGRAPH_DB_PASSWORD")
194
+ values = {key: secrets.token_urlsafe(24) for key in keys}
195
+ lines, found = path.read_text().splitlines(), set()
196
+ for i, line in enumerate(lines):
197
+ for key in keys:
198
+ if line.startswith(key + "="):
199
+ lines[i], found = key + "=" + values[key], found | {key}
200
+ if found != set(keys): raise SystemExit("Password keys missing from .env.example")
201
+ path.write_text("\n".join(lines) + "\n")
202
+ print("Set random values for: " + ", ".join(keys))
203
+ EOF
204
+ ```
205
+
206
+ Do not display, source or parse `.env`. Leave `AI_GATEWAY_API_KEY`,
207
+ `FIREWORKS_API_KEY`, `ANTHROPIC_API_KEY`, `OPENAI_API_KEY` and `GOOGLE_API_KEY`
208
+ empty.
209
+
210
+ ## Log in and start
211
+
212
+ Stop for the person to log in to the registry in their own terminal. They use
213
+ the pull-only invitation values, never chat or an agent-visible command:
214
+
215
+ ```sh
216
+ echo "$METERGRAPH_PULL_TOKEN" | docker login ghcr.io -u "$METERGRAPH_PULL_USER" --password-stdin
217
+ ```
218
+
219
+ Wait for confirmation only; never read Docker's credential file. State that
220
+ `./bin/start` will re-verify the manifest and both image digests, pull by digest,
221
+ start containers, and on a new database send one synthetic demo trace labeled
222
+ `environment=local-demo`. Then run:
223
+
224
+ ```sh
225
+ ./bin/start
226
+ ./bin/status
227
+ ```
228
+
229
+ An image-pull failure usually means login is missing or expired. On other
230
+ failure, read only the log named by `./bin/start` and `./bin/status`; never read
231
+ rendered Compose configuration or container environment. On password
232
+ authentication failure with `local_pgdata`, stop and ask for its matching
233
+ `.env`. Never delete data. Read the workspace ID and suggested command from the
234
+ status JSON's `workspace` check. The service listens only on localhost:8080.
235
+
236
+ ## Run setup
237
+
238
+ Return to the application repository. Prefer installed `metergraph` only when
239
+ its help lists `setup`; otherwise state that `npx --yes metergraph-cli@next`
240
+ fetches the preview CLI and use it as the prefix. Run with `--json`:
241
+
242
+ ```sh
243
+ metergraph setup --runtime local --deployment customer-local --url http://localhost:8080 --workspace 11111111-1111-4111-8111-111111111111 --confirm-prerequisites --client <client> --json
244
+ ```
245
+
246
+ Replace the example UUID and client only. Setup writes an ingest key to the
247
+ application `.env`, installs the skill and requests Metadata access only; never
248
+ print that file. Setup opens the deployment's browser consent page and waits;
249
+ the person signs in as the local admin and completes two approvals. If the
250
+ browser cannot open, follow the `metergraph` skill's guidance for handing the
251
+ person the approval URL. Continue until `data.status` is `ok`. For
252
+ `operator_handoff`, relay `data.next_action.kind` (`complete_prerequisite`,
253
+ `run_on_customer_machine`, `connection_guide`, or other); never report a
254
+ handoff as success.
255
+
256
+ The agent never runs a command that reads `.env`. To see the admin login without
257
+ printing it, the person runs these no-output pipelines themselves, one at a
258
+ time, in their own terminal or in Claude Code with the `!` prefix. Fill in the
259
+ absolute bundle directory for them; their shell does not have `bundle_dir`:
260
+
261
+ ```sh
262
+ sed -n 's/^METERGRAPH_LOCAL_ADMIN_EMAIL=//p' ~/metergraph/<version>/local/.env | pbcopy
263
+ sed -n 's/^METERGRAPH_LOCAL_ADMIN_PASSWORD=//p' ~/metergraph/<version>/local/.env | pbcopy
264
+ ```
265
+
266
+ They may instead use their own editor. Never use `cat`, `grep` or any other
267
+ command that prints `.env`, particularly with `!`, because `!` output lands in
268
+ the transcript. A new configuration keeps `admin@example.com` as the email.
269
+
270
+ ## Recovery and stopping
271
+
272
+ | State | Action |
273
+ | --- | --- |
274
+ | Signature, manifest, digest or path check fails | Stop; extract and run nothing |
275
+ | Port 8080 belongs to something else | Stop and ask; do not alter it |
276
+ | Registry pull is denied | Person repeats private login; never request the token |
277
+ | Existing `.env` | Reuse unchanged; never regenerate passwords |
278
+ | `local_pgdata` lacks its matching `.env` | Stop and find prior configuration; never remove it |
279
+ | Startup fails | Use status and only the named log; preserve secrets and data |
280
+ | Setup returns a handoff | Relay exact status/next action; do not report success |
281
+
282
+ To stop later, follow the bundle README's `stop`. Prefer it over `down`, which
283
+ loses raw payloads in local object storage. Never use `down --volumes`.
@@ -0,0 +1,81 @@
1
+ ---
2
+ name: metergraph-model-swap
3
+ description: Run the Metergraph model-swap loop from a coding agent - choose a workload, choose traces and candidate models, define or refine the eval, start the analysis, summarize the report with its app link, then refine and rerun. Use when someone asks whether a workload could use a cheaper or better model, asks to plan or run a model swap analysis, asks "what should my largest workload run on", or comes back after inspecting a Metergraph report. Use the metergraph skill first when Metergraph is not connected.
4
+ ---
5
+
6
+ # Model-swap loop
7
+
8
+ This skill is the map. Each stage has its own skill; load it when you reach the
9
+ stage. The person stays in the coding agent for choosing, defining and
10
+ rerunning, and goes to the Metergraph app to read the detailed report and
11
+ inspect evidence.
12
+
13
+ ```
14
+ choose workload → choose traces and models → define or refine eval
15
+ → run analysis → summary + report link → (person inspects evidence in the app)
16
+ → refine and rerun ↺
17
+ ```
18
+
19
+ | Stage | Skill | Ends with |
20
+ | --- | --- | --- |
21
+ | 1. Choose workload | `metergraph-workloads` | One classified workload, with its `selection_reference` and why it was picked |
22
+ | 2. Choose traces and models | `metergraph-candidates` | A run plan: environment, window, candidate models, route settings that must change |
23
+ | 3. Define or refine the eval | `metergraph-evals` | A saved eval revision bound to the workload, or a written rubric the person enters in the app |
24
+ | 4. Run the analysis | `metergraph-analyze` | A run ID, polled to a final state |
25
+ | 5. Summarize | `metergraph-report` | A short summary and the report link |
26
+ | 6. Refine and rerun | `metergraph-iterate` | What changed and a new, separately approved run |
27
+
28
+ ## Start every session the same way
29
+
30
+ 1. Call `metergraph_get_workspace_context`, then `metergraph_get_capabilities`.
31
+ Say which workspace and deployment you are connected to in one line. Stop if
32
+ it is not the one the person means.
33
+ 2. Call `metergraph_list_analysis_runs` with a small limit. If a run already
34
+ exists for this workload, the person is probably on stage 6: read its report
35
+ first instead of starting over, and give the report link
36
+ (`<origin>/#analysis?report=<analysis_id>`, as in `metergraph-report`).
37
+ 3. Write down the tools that are advertised. Plan only with those. When a stage
38
+ needs a tool that is not advertised, say which one, keep the selection you
39
+ have, and hand that step to the app page named in the stage skill. Never
40
+ invent a tool, call a dashboard API with an agent key, or drive the app in a
41
+ browser unless the person asks for that specific change.
42
+
43
+ ## Rules that hold across stages
44
+
45
+ - **Workloads are not routes.** Routes are labels on traffic. An analysis runs
46
+ on a classified workload, identified by `source_run_id`, `pattern_id` and
47
+ `pattern_set_version`. Do not rank or plan on routes and then discover this at
48
+ the start step.
49
+ - **Check the data before planning.** A workspace created today, data that all
50
+ arrived in one batch, fewer than 10 samples, or totals that disagree between
51
+ tools and the app all change the plan. Say so first, with the numbers.
52
+ - **One decision at a time.** End each turn with the single decision you need
53
+ from the person, not a list of everything that could be configured.
54
+ - **Spending and data egress need a yes.** Starting an analysis sends copies of
55
+ captured prompts to other model providers and costs money. Stage 4 has the
56
+ exact confirmation to get. A connected key, a configured provider key or an
57
+ earlier yes for a different run is not approval.
58
+ - **Settings are the person's.** Replay opt-out, data residency, candidate
59
+ models, cost and latency limits and the size floor are changed in the app.
60
+ Explain the change and give the page; do not make it for them.
61
+ - **Keep answers short.** Lead with the answer or the blocker. Tables only for
62
+ three or more rows. Give confidence once, where it matters.
63
+
64
+ ## Keep a loop record
65
+
66
+ Keep these values in the conversation, and, if you can write files in the
67
+ person's project and they agree, in `.metergraph/model-swap.md`. They are not
68
+ secret. They let a later session pick up at stage 6.
69
+
70
+ ```
71
+ workspace: <name> (<workspace_id>) at <origin>
72
+ workload: <display_name> source_run_id=<…> pattern_id=<…> pattern_set_version=<…>
73
+ environment: <production|…> window: <from>–<to>
74
+ candidates: <model ids>
75
+ eval: <suite or rubric revision id, or "rubric in chat, not saved">
76
+ runs: <run_id> (<status>) → report <analysis_id> <link>
77
+ decision: <switch | no switch | more testing> because <reason>
78
+ next: <the one next step>
79
+ ```
80
+
81
+ Never put keys, `.env` values or captured prompt or response text in it.