@withgauge/cli 0.15.0 → 0.16.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +21 -3
- package/dist/index.js +5 -4
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -181,7 +181,25 @@ Paths are repository-relative, case-sensitive globs; matching any pattern runs
|
|
|
181
181
|
the checks. `**` matches nested directories, including hidden files. Use 1–32
|
|
182
182
|
patterns; exclusions (`!`) are not supported. Changes to `gauge.json` or your
|
|
183
183
|
configured case paths always count. Deleted files and both names of renamed
|
|
184
|
-
files are included. Without `checks`, every eligible PR update
|
|
184
|
+
files are included. Without root `checks`, every eligible PR update reaches case
|
|
185
|
+
selection.
|
|
186
|
+
|
|
187
|
+
With CLI 0.16.0 or later, a case can narrow its own automatic PR triggers in its
|
|
188
|
+
Markdown frontmatter (alongside its other `config` settings):
|
|
189
|
+
|
|
190
|
+
```yaml
|
|
191
|
+
config:
|
|
192
|
+
agents:
|
|
193
|
+
- agent: CODEX_CLI
|
|
194
|
+
checks:
|
|
195
|
+
paths: ["cli/**", "packages/shared/**"]
|
|
196
|
+
```
|
|
197
|
+
|
|
198
|
+
Root `checks.paths`, when present, must match first. The App then runs matching
|
|
199
|
+
cases and all cases without a filter. Changes to `gauge.json` select every case;
|
|
200
|
+
editing a case selects that case regardless of its filter. If no cases match,
|
|
201
|
+
the check is skipped before preparing inputs or reserving credits. `config.inputs`
|
|
202
|
+
only chooses inputs; it does not define change triggers.
|
|
185
203
|
|
|
186
204
|
The first check uses the PR diff. After a passing evaluation, later updates
|
|
187
205
|
compare against that evaluated commit, so unrelated follow-up edits can skip.
|
|
@@ -254,8 +272,8 @@ They are fetched live rather than snapshotted. See the [preview source examples
|
|
|
254
272
|
limitations](../specs/docs-web-inputs.md), including Astro's existing bot comment.
|
|
255
273
|
|
|
256
274
|
Editor schemas are published at
|
|
257
|
-
`https://agents.withgauge.com/schemas/gauge/v2.
|
|
258
|
-
`https://agents.withgauge.com/schemas/gauge/v2.
|
|
275
|
+
`https://agents.withgauge.com/schemas/gauge/v2.5.0/gauge.json` and
|
|
276
|
+
`https://agents.withgauge.com/schemas/gauge/v2.5.0/case.json`. Associate them in editor
|
|
259
277
|
settings; `gauge.json` does not accept a `$schema` field. These release URLs are
|
|
260
278
|
immutable. Update earlier pilot configs by moving workflows into input sources,
|
|
261
279
|
replacing `use` with input `type` and `path`/`paths`, and moving cases into
|
package/dist/index.js
CHANGED
|
@@ -4311,6 +4311,7 @@ var uncredentialedRepoUrlSchema = z42.string().min(1).refine((raw) => {
|
|
|
4311
4311
|
return false;
|
|
4312
4312
|
}
|
|
4313
4313
|
}, "Repository URL must not contain embedded credentials or query parameters");
|
|
4314
|
+
var runDefinitionChecksSchema = z42.object({ paths: z42.array(inputPathSchema).min(1).max(32) }).strict();
|
|
4314
4315
|
var runDefinitionConfigSchema = z42.object({
|
|
4315
4316
|
agents: z42.array(agentConfigInputSchema).min(1),
|
|
4316
4317
|
sampleCount: z42.number().int().min(1).max(50).optional(),
|
|
@@ -4321,14 +4322,13 @@ var runDefinitionConfigSchema = z42.object({
|
|
|
4321
4322
|
mcpRefs: assetRefListSchema.optional(),
|
|
4322
4323
|
connectionIds: z42.array(z42.string().min(1)).max(32).optional(),
|
|
4323
4324
|
addons: runAddonsSchema.optional(),
|
|
4324
|
-
inputs: z42.array(z42.string()).optional()
|
|
4325
|
+
inputs: z42.array(z42.string()).optional(),
|
|
4326
|
+
checks: runDefinitionChecksSchema.optional()
|
|
4325
4327
|
}).strict();
|
|
4326
4328
|
var runDefinitionFileConfigSchema = z42.object({
|
|
4327
4329
|
version: z42.literal(2).default(2),
|
|
4328
4330
|
org: z42.string().min(1).optional(),
|
|
4329
|
-
checks:
|
|
4330
|
-
paths: z42.array(inputPathSchema).min(1).max(32)
|
|
4331
|
-
}).strict().optional(),
|
|
4331
|
+
checks: runDefinitionChecksSchema.optional(),
|
|
4332
4332
|
cases: z42.array(inputPathSchema).min(1).max(32).default(["gauge-evals/**/*.md"]),
|
|
4333
4333
|
inputs: z42.record(
|
|
4334
4334
|
z42.string().regex(/^[a-z0-9]+(?:-[a-z0-9]+)*$/).max(64),
|
|
@@ -7665,6 +7665,7 @@ description: Create and review Gauge eval prompts, model selections, and judging
|
|
|
7665
7665
|
- Keep repository revisions, credentials, connections, runtime setup, and budgets in supported configuration or fixtures where possible. Inspect \`gauge evals create --help\` and \`gauge evals edit --help\` for current controls; do not assume proposed limits are enforced.
|
|
7666
7666
|
- Saved evals use \`gauge evals create --file <path> --criteria-file <path>\` (criteria JSON: [{"name":"...","rubric":"..."}]). For committed Markdown cases, preview with \`gauge evals plan\` before \`gauge evals run\`; committed \`gauge-evals/**/*.md\` cases are discovered automatically and \`--files\` overrides discovery with paths/globs. Root \`gauge.json\` can attach existing GitHub Actions artifacts and committed files; Gauge consumes them without building. Website inputs route a canonical HTTPS origin to an existing public preview; sources are GitHub Deployments or an existing provider PR comment with exact author, readiness pattern, and candidate SHA. Comments without commit identity and private or path-prefixed previews are unsupported.
|
|
7667
7667
|
- Native command inputs use \`type: "binary"\`, \`platform: "linux-x64"\`, selected \`paths\`, and a \`commands\` map from command names to explicit relative ELF executable files. CI must upload already unpacked files. Gauge restores executable permissions and exposes declared commands on PATH; compatible system libraries are required, nested archives and arbitrary installers are unsupported.
|
|
7668
|
+
- Automatic GitHub PR checks support root \`gauge.json\` \`checks.paths\` as a suite gate and per-case frontmatter \`config.checks.paths\` as a narrower filter. Both accept positive repository-relative globs. Cases without filters run when the suite gate passes; changes to \`gauge.json\` select all cases and edits to a case select that case. Explicit CLI launches and GitHub reruns ignore both filters. Failed/unfinished previous checks, changed target branches, and unavailable diffs run conservatively. \`config.inputs\` selects inputs, not triggers.
|
|
7668
7669
|
- Pilot before scaling; confirm before spending credits. Creating definitions does not authorize launching runs. Report sample counts and distinguish incorrect behavior, missing evidence, environment failures, and budget exhaustion.
|
|
7669
7670
|
- Before committing setup changes, run \`gauge evals verify -o json\` for an offline working-tree checklist. Only attempted static checks are scored yes, no, or unknown; unknown means a specific declaration could not be resolved. Informational findings (such as zero cases) are separate. JSON includes a version, static status (passed/failed/incomplete), summary counts, global/per-input checks with stable IDs, facts, info, and excluded capabilities in notChecked. Exit 1 means a definite problem; exit 0 can still contain unknowns. Live delivery, account settings, and sandbox execution are outside this command's scope. It requires no login or PR and launches no sessions.
|
|
7670
7671
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@withgauge/cli",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.16.0",
|
|
4
4
|
"description": "Gauge customer CLI — Agent Preference, evals, runs, and organization settings over the Gauge API",
|
|
5
5
|
"private": false,
|
|
6
6
|
"author": "Gauge <support@withgauge.com> (https://withgauge.com)",
|