@withgauge/cli 0.17.0 → 0.18.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +65 -21
- package/dist/index.js +2409 -2129
- package/package.json +3 -4
- package/scripts/install-skill.mjs +0 -64
package/README.md
CHANGED
|
@@ -14,13 +14,9 @@ gauge orgs use <org> # set your default organization
|
|
|
14
14
|
|
|
15
15
|
Requires Node.js ≥ 22.12.
|
|
16
16
|
|
|
17
|
-
|
|
17
|
+
To install the optional
|
|
18
18
|
[`gauge-agents` skill](https://github.com/gauge-sh/gauge-skills/tree/main/gauge-agents)
|
|
19
|
-
globally
|
|
20
|
-
agent detection for tools such as Codex and Claude Code. Skill installation
|
|
21
|
-
needs network access; failures leave the CLI installed and print a retry command.
|
|
22
|
-
It is skipped in CI or when `GAUGE_SKIP_SKILL_INSTALL=1` is set. If you install
|
|
23
|
-
with `--ignore-scripts`, install the skill manually:
|
|
19
|
+
globally, run:
|
|
24
20
|
|
|
25
21
|
```sh
|
|
26
22
|
npx --yes skills@1.5.0 add gauge-sh/gauge-skills --skill gauge-agents --global --yes
|
|
@@ -80,7 +76,7 @@ config:
|
|
|
80
76
|
Add a /health endpoint and run the relevant app checks.
|
|
81
77
|
```
|
|
82
78
|
|
|
83
|
-
Check your
|
|
79
|
+
Check your setup, then commit the files before planning and running with Git inputs:
|
|
84
80
|
|
|
85
81
|
```sh
|
|
86
82
|
gauge evals verify # offline; includes uncommitted working-tree edits
|
|
@@ -91,7 +87,8 @@ gauge run-requests wait <request-id> -o json
|
|
|
91
87
|
```
|
|
92
88
|
|
|
93
89
|
Cases default to `gauge-evals/**/*.md`. `--files` accepts quoted paths/globs and overrides
|
|
94
|
-
that convention for `plan` and `run
|
|
90
|
+
that convention for `plan` and `run`. By default, these commands read committed Git
|
|
91
|
+
HEAD. With `--input`, they read your current files, including uncommitted edits.
|
|
95
92
|
Each case owns its agents, models, consumer repository/ref, Persona and criteria.
|
|
96
93
|
The optional `config.inputs` selects named inputs; all attach by default.
|
|
97
94
|
|
|
@@ -108,7 +105,7 @@ event filters and runtime conditions are not evaluated against a candidate PR.
|
|
|
108
105
|
Live input delivery, artifact contents, sandbox execution, App activation, and
|
|
109
106
|
account settings are outside this command's scope and do not create unknown rows.
|
|
110
107
|
Exit 1 means at least one `no`; exit 0 means no definite static problem was found.
|
|
111
|
-
No workflow code is executed.
|
|
108
|
+
No workflow code is executed.
|
|
112
109
|
|
|
113
110
|
`gauge evals verify -o json` writes exactly one JSON object to stdout, including on
|
|
114
111
|
validation failure. The versioned report provides:
|
|
@@ -154,15 +151,19 @@ alongside your product:
|
|
|
154
151
|
}
|
|
155
152
|
```
|
|
156
153
|
|
|
157
|
-
Gauge waits for
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
154
|
+
Gauge waits for your PR build and tests its uploaded artifacts. Keep the workflow's
|
|
155
|
+
existing checkout settings, including GitHub's default merge checkout. Omit
|
|
156
|
+
`source.artifact` when the workflow uploads one artifact, or select it by name.
|
|
157
|
+
|
|
158
|
+
For automatic checks, connect the Gauge GitHub App with repository access and
|
|
159
|
+
Actions read permission. Set `org` and enable the repository and target branches
|
|
160
|
+
in **Settings → GitHub**. Waiting for inputs starts no sessions and spends no
|
|
161
|
+
session credits.
|
|
162
|
+
Fork pull requests are disabled by default. Choose **Require approval** to let
|
|
163
|
+
workspace members authorize an evaluation from its GitHub check. Authorization
|
|
164
|
+
covers one PR revision and uses workspace credits; new revisions and explicit
|
|
165
|
+
reruns require fresh approval. Configured inputs determine when an authorized
|
|
166
|
+
evaluation is ready to run.
|
|
166
167
|
A failed run or absent/expired artifact returns an actionable input failure.
|
|
167
168
|
Absent runs have a five-minute discovery grace period; known running or queued
|
|
168
169
|
CI can continue past an hour. Abandoned requests expire after seven days. Reruns retain captured bytes. CI metadata reports a revision; it
|
|
@@ -235,7 +236,7 @@ Actions artifact (or small committed Git files):
|
|
|
235
236
|
}
|
|
236
237
|
```
|
|
237
238
|
|
|
238
|
-
Paths are relative to the uploaded artifact root.
|
|
239
|
+
Paths are relative to the uploaded artifact root. Use your normal PR build;
|
|
239
240
|
for `actions/upload-artifact@v7`, use `archive: true`. Include companion files in
|
|
240
241
|
`paths` and explicitly map 1–32 command names to ordinary selected files. Gauge
|
|
241
242
|
validates Linux x86-64 ELF executables in its isolated artifact processor, restores
|
|
@@ -261,6 +262,49 @@ the source root. Gauge validates and extracts selected archives in isolation and
|
|
|
261
262
|
stages the Skill in the coding agent's native discovery location. File inputs use
|
|
262
263
|
`paths` to select committed files or files inside an Actions artifact.
|
|
263
264
|
|
|
265
|
+
Gauge tests the artifact uploaded by your PR's successful build. Keep your
|
|
266
|
+
existing checkout settings. The GitHub check links to the selected build;
|
|
267
|
+
rerun Gauge to test the latest build.
|
|
268
|
+
|
|
269
|
+
### Run from your own CI or local build
|
|
270
|
+
|
|
271
|
+
With CLI 0.18.0 or later, supply built inputs directly to run without the GitHub App:
|
|
272
|
+
|
|
273
|
+
```sh
|
|
274
|
+
gauge --org my-org evals run --input product-analytics=./dist/skill.zip --yes -o json > request.json
|
|
275
|
+
gauge --org my-org run-requests wait "$(jq -er '.runRequestId' request.json)" -o json > results.json
|
|
276
|
+
```
|
|
277
|
+
|
|
278
|
+
Set `GAUGE_API_TOKEN` as a CI secret and replace `my-org` with your workspace slug.
|
|
279
|
+
This can run immediately after your build in the same CI job.
|
|
280
|
+
|
|
281
|
+
Repeat `--input name=path` for each input. Skills accept a directory containing
|
|
282
|
+
`SKILL.md` or a ZIP/tar.gz archive; npm inputs accept a packed `.tgz`. Files and
|
|
283
|
+
binary inputs accept a directory, with configured paths relative to that directory.
|
|
284
|
+
Use a dedicated build directory: links are rejected, and common credential files
|
|
285
|
+
and `node_modules` are excluded. Each input is limited to 512 MiB and 20,000 files.
|
|
286
|
+
|
|
287
|
+
With `--input`, Gauge uses your current evaluation files, including uncommitted
|
|
288
|
+
edits. No Git checkout is required when all selected inputs are supplied locally.
|
|
289
|
+
Inputs without an override keep their configured source. You can omit `source`
|
|
290
|
+
in `gauge.json` to default to Git; the override replaces it for this run.
|
|
291
|
+
Website previews and private consumer repositories still require their configured
|
|
292
|
+
access. Public consumer repositories and the default workspace need no GitHub App.
|
|
293
|
+
|
|
294
|
+
`wait -o json` returns the verdict, session results, criterion explanations, and
|
|
295
|
+
session links. It exits nonzero for failed evaluations or execution errors and
|
|
296
|
+
still writes the results JSON. Automatic PR checks continue to use configured
|
|
297
|
+
sources; disable those checks in Gauge's GitHub settings if CI owns the trigger.
|
|
298
|
+
Save `results.json` as a CI artifact even when the check fails.
|
|
299
|
+
|
|
300
|
+
### Authenticated MCP servers
|
|
301
|
+
|
|
302
|
+
Connect your account in Gauge's MCP setup, completing OAuth in your browser when
|
|
303
|
+
required. Add the server's versioned reference to each case's `config.mcpRefs`.
|
|
304
|
+
Gauge manages credentials for the session; keep tokens out of prompts and Git.
|
|
305
|
+
|
|
306
|
+
### Website previews
|
|
307
|
+
|
|
264
308
|
Website inputs use `type: "website"` and a canonical HTTPS origin in `url`.
|
|
265
309
|
The preview can come from `source: { "type": "github-deployment", "environment": "preview" }`
|
|
266
310
|
or a `github-pr-comment` source with an exact `author`, literal `bodyIncludes`
|
|
@@ -272,8 +316,8 @@ They are fetched live rather than snapshotted. See the [preview source examples
|
|
|
272
316
|
limitations](../specs/docs-web-inputs.md), including Astro's existing bot comment.
|
|
273
317
|
|
|
274
318
|
Editor schemas are published at
|
|
275
|
-
`https://agents.withgauge.com/schemas/gauge/v2.
|
|
276
|
-
`https://agents.withgauge.com/schemas/gauge/v2.
|
|
319
|
+
`https://agents.withgauge.com/schemas/gauge/v2.7.0/gauge.json` and
|
|
320
|
+
`https://agents.withgauge.com/schemas/gauge/v2.7.0/case.json`. Associate them in editor
|
|
277
321
|
settings; `gauge.json` does not accept a `$schema` field. These release URLs are
|
|
278
322
|
immutable. Update earlier pilot configs by moving workflows into input sources,
|
|
279
323
|
replacing `use` with input `type` and `path`/`paths`, and moving cases into
|