@withgauge/cli 0.29.0 → 0.31.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -566,13 +566,82 @@ JSON is still the only output. Automatic PR checks continue to use configured
566
566
  sources; disable those checks in Gauge's GitHub settings if CI owns the trigger.
567
567
  Save `results.json` as a CI artifact even when the check fails.
568
568
 
569
+ ### Self-contained product experiments (hosted)
570
+
571
+ A self-contained experiment supplies its product revision, input declarations,
572
+ and cases in one JSON file. It needs no checkout or `gauge.json`. Gauge resolves
573
+ the product through your organization's GitHub App connection; grant the App
574
+ access to that repository, including when it is public.
575
+
576
+ ```sh
577
+ gauge run --example --format experiment > experiment.json
578
+ # Edit the product, inputs, task, and criteria, then preview:
579
+ gauge run --hosted --org <org> --apply experiment.json --dry-run -o json > preview.json
580
+ # Save the resolved commit so a branch moving cannot change the launch:
581
+ jq '.experiment' preview.json > pinned-experiment.json
582
+ gauge run --hosted --org <org> --apply pinned-experiment.json --yes -o json
583
+ ```
584
+
585
+ For example:
586
+
587
+ ```json
588
+ {
589
+ "version": 2,
590
+ "product": { "repository": "example/product", "revision": "main" },
591
+ "inputs": { "docs": { "type": "files", "paths": ["docs/**"] } },
592
+ "cases": [{
593
+ "key": "first-request",
594
+ "prompt": "Use the supplied documentation to write a minimal API client.",
595
+ "criteria": [{ "name": "Working example", "rubric": "The client follows the documented API contract." }],
596
+ "config": { "agents": [{ "agent": "CODEX_CLI" }], "inputs": ["docs"] }
597
+ }]
598
+ }
599
+ ```
600
+
601
+ `product.repository` accepts `owner/repository` or an HTTPS GitHub repository URL;
602
+ `revision` accepts a branch, tag, or full commit SHA. Each invocation resolves it
603
+ once. The preview includes the immutable revision in `experiment`, suitable for
604
+ another `--apply`. Reusing the original branch-based file resolves the branch anew.
605
+ Use `gauge run --schema --format experiment` for the complete authoring schema.
606
+
607
+ After the agent's first turn, a simulated user answers its questions and decides
608
+ when the work is finished. By default that user expects working software. To
609
+ describe a different user, set `config.persona` on a case to a `name` and a
610
+ `description` of who the user is, what they want, and when they consider the
611
+ work done, for example "done once the agent reports the command it would run;
612
+ does not want a project created". A case uses either an inline `persona` or a
613
+ saved `profileId`, not both. Inline personas require hosted Gauge.
614
+
615
+ Input declarations use the same vocabulary as `gauge.json`: files, skills,
616
+ npm packages, binaries, web fixtures, and website previews. All product inputs
617
+ come from the selected repository and commit, using the declared Git, Actions,
618
+ or deployment source. An npm input selects an existing packed `.tgz` from Git or
619
+ an Actions artifact; Gauge does not build a package directory for you.
620
+
621
+ The tested agent's consumer workspace is separate. Omit `cases[].config.repoUrl`
622
+ to use the default workspace, or set it explicitly for a consumer project.
623
+ The product repository does not become the consumer workspace. Cases select
624
+ named inputs through `config.inputs`; omitting the selection includes all inputs
625
+ declared in this specification. An explicit empty `inputs` object is allowed.
626
+
627
+ This form never discovers cases or inputs from the working directory and does
628
+ not upload local files. `--input` overrides, `--cli-rules`, and local execution
629
+ are unavailable for this form. Existing case specifications keep their normal
630
+ workspace behavior. A preview validates the plan and reports source readiness;
631
+ it may read source metadata and manifests, but does not prepare input artifacts
632
+ or start sessions. Pinning a commit
633
+ does not freeze a future CI artifact or deployment: preparation records the
634
+ actual artifacts and receipts when the request runs. Launches use the ordinary
635
+ organization permissions, execution limits, and credits.
636
+
569
637
  ### Antigravity (hosted)
570
638
 
571
639
  Antigravity is available to organizations enrolled in Antigravity access. Run
572
640
  `gauge models list --hosted` to find the available model identifiers, then select
573
- one with `--agent antigravity:<model>` when creating or launching a saved eval or
574
- preference prompt. Case files and JSON specs use `agent: ANTIGRAVITY` with the
575
- model identifier in `models`.
641
+ one with `--agent antigravity:<model>` when creating or launching a saved eval,
642
+ in `gauge preference tracking set`, or when launching a preference prompt.
643
+ Case files and JSON specs use `agent: ANTIGRAVITY` with the model identifier in
644
+ `models`.
576
645
 
577
646
  Launch with `--hosted`, for example:
578
647
 
@@ -639,8 +708,8 @@ do not test discovery of the canonical docs. Static verification validates
639
708
  configuration; it does not prove backend support, delivery, or an eval pass.
640
709
 
641
710
  Editor schemas are published at
642
- `https://agents.withgauge.com/schemas/gauge/v2.10.0/gauge.json` and
643
- `https://agents.withgauge.com/schemas/gauge/v2.10.0/case.json`. Associate them in editor
711
+ `https://agents.withgauge.com/schemas/gauge/v2.11.0/gauge.json` and
712
+ `https://agents.withgauge.com/schemas/gauge/v2.11.0/case.json`. Associate them in editor
644
713
  settings; `gauge.json` does not accept a `$schema` field. These release URLs are
645
714
  immutable. Cases default to `gauge-cases/`, or use explicit `cases` paths.
646
715
 
@@ -678,21 +747,22 @@ prompt against a second repository or roster, copy the eval in the app and edit
678
747
  the copy — one eval is always one configuration.
679
748
 
680
749
  A **preference prompt** is a reusable question. The following hosted example
681
- uses scenarios and recurring scheduling. Locally, save a prompt and launch it
750
+ uses scenarios and tracking on the org schedule. Locally, save a prompt and launch it
682
751
  on demand with `gauge run --preference <id> --local`. It owns one set of run
683
- settings — roster, resources, cadence, samples — and any number of
752
+ settings — resources and samples — and any number of
684
753
  **scenarios**, each one repository × persona cell. Every scenario runs with
685
754
  the prompt's settings, so one prompt measures the same question across several
686
- repositories and personas at once:
755
+ repositories and personas at once. Hosted prompts run on the org's tracked
756
+ models (see Hosted schedules); local prompts keep their own `--agent` roster:
687
757
 
688
758
  ```sh
689
759
  gauge preference create --name "CLI deployment preference" \
690
760
  --kind head-to-head --brand-a acme --brand-b globex \
691
761
  --text "Which CLI should I use to deploy?" \
692
- --agent claude-code --cadence daily \
762
+ --tracked \
693
763
  --repo https://github.com/acme/api # first scenario
694
764
 
695
- gauge preference edit <promptId> --agent codex --samples 3 --cadence weekly
765
+ gauge preference edit <promptId> --samples 3
696
766
  gauge preference scenarios list <promptId>
697
767
  gauge preference scenarios add <promptId> --persona <profileId>
698
768
  gauge preference scenarios add <promptId> --repo https://github.com/acme/web --persona <profileId>
@@ -795,18 +865,36 @@ Launching runs consumes credits normally.
795
865
 
796
866
  ## Hosted schedules
797
867
 
798
- `--cadence none` is "manual": the measurement stops coming due and runs only
799
- when you launch it. There is no separate pause or enabled flag, in the CLI or
800
- the app. `--cadence daily|weekly|monthly` puts it back on a rhythm, and
801
- `gauge preference get <id>` reads a manual prompt back as `cadence: manual`.
868
+ An eval has its own cadence. `--cadence none` is "manual": the eval stops
869
+ coming due and runs only when you launch it. `--cadence daily|weekly|monthly`
870
+ puts it back on a rhythm.
871
+
872
+ Preference prompts have no cadence or models of their own. A new prompt is
873
+ **not tracked**: it runs only when you launch it. Track a hosted prompt to put
874
+ it on the org's one preference schedule, with every other tracked prompt, on
875
+ the org's tracked models. Rankings count only these scheduled runs, so each
876
+ point compares the same set of prompts on the same models.
877
+
878
+ ```sh
879
+ gauge preference tracking # schedule, models, tracked count
880
+ gauge preference tracking set --cadence weekly --day monday
881
+ gauge preference tracking set --agent claude-code:<model> --agent opencode:<model>
882
+ gauge preference tracking set --no-agents # stop scheduled runs
883
+ gauge preference track <promptId> [<promptId>...] # run on the org schedule
884
+ gauge preference untrack <promptId> [<promptId>...] # run only on demand
885
+ gauge preference list --tracked
886
+ ```
887
+
888
+ `--agent` on `tracking set` replaces the whole list; `gauge models list --hosted`
889
+ shows the model identifiers. `create|edit --tracked` / `--no-tracked` also work.
802
890
 
803
891
  `gauge run --eval <id>` and `gauge run --preference <id>` are the separate
804
892
  immediate launches. They use the stored configuration and ask before spending
805
893
  (`--yes` skips the prompt for scripts).
806
894
  `gauge run --preference <id> --scenario <scenarioId>` (repeatable) runs a subset;
807
- omit it and every scenario runs. On a recurring cadence, an eval launch or a
808
- full preference launch moves the next scheduled run to one period from now. A
809
- scenario subset leaves the schedule unchanged.
895
+ omit it and every scenario runs. On a recurring cadence, an eval launch moves
896
+ the next scheduled run to one period from now. A preference launch leaves the
897
+ org schedule unchanged and does not count toward rankings.
810
898
 
811
899
  ## Issues (hosted)
812
900
 
@@ -1,2 +1,2 @@
1
- import{runLocal}from"./chunk-4SZPSHNF.js";import"./chunk-ZZK6V3ZF.js";import"./chunk-QV7MOO3M.js";import"./chunk-QJ2UAX2V.js";import"./chunk-OIGVDK3F.js";import"./chunk-PSBF4TEK.js";import{randomBytes}from"crypto";import{readFile,rename,rm,writeFile}from"fs/promises";import{dirname,join}from"path";var requestPath=process.argv[2];var directory=dirname(requestPath);var requestLoaded=false;try{const request=JSON.parse(await readFile(requestPath,"utf8"));if(!request||typeof request.batchId!=="string"||typeof request.gaugeDir!=="string"||!request.definition)throw new Error("Invalid batch request");requestLoaded=true;await rm(requestPath);await runLocal(request.definition,[],request.gaugeDir,()=>{},{batchId:request.batchId,credentialSelections:request.credentialSelections,measurement:request.measurement})}catch(error){process.exitCode=1;const detail=requestLoaded?String(error):"Could not read a valid background batch request. Start a new batch.";console.error(detail);const statusPath=join(directory,"status.json");const previous=await readFile(statusPath,"utf8").then(raw=>JSON.parse(raw)).catch(()=>({}));const finishedSessions=previous.finishedSessions??0;const status={...previous,state:"failed",detail,finishedSessions,queuedSessions:0,skippedSessions:Math.max(0,(previous.expectedSessions??0)-finishedSessions),supervisorPid:process.pid};const temporary=`${statusPath}.${randomBytes(6).toString("hex")}.tmp`;await writeFile(temporary,`${JSON.stringify(status,null,2)}
1
+ import{runLocal}from"./chunk-EUIOGLV7.js";import"./chunk-ZZK6V3ZF.js";import"./chunk-QV7MOO3M.js";import"./chunk-QJ2UAX2V.js";import"./chunk-OIGVDK3F.js";import"./chunk-PSBF4TEK.js";import{randomBytes}from"crypto";import{readFile,rename,rm,writeFile}from"fs/promises";import{dirname,join}from"path";var requestPath=process.argv[2];var directory=dirname(requestPath);var requestLoaded=false;try{const request=JSON.parse(await readFile(requestPath,"utf8"));if(!request||typeof request.batchId!=="string"||typeof request.gaugeDir!=="string"||!request.definition)throw new Error("Invalid batch request");requestLoaded=true;await rm(requestPath);await runLocal(request.definition,[],request.gaugeDir,()=>{},{batchId:request.batchId,credentialSelections:request.credentialSelections,measurement:request.measurement})}catch(error){process.exitCode=1;const detail=requestLoaded?String(error):"Could not read a valid background batch request. Start a new batch.";console.error(detail);const statusPath=join(directory,"status.json");const previous=await readFile(statusPath,"utf8").then(raw=>JSON.parse(raw)).catch(()=>({}));const finishedSessions=previous.finishedSessions??0;const status={...previous,state:"failed",detail,finishedSessions,queuedSessions:0,skippedSessions:Math.max(0,(previous.expectedSessions??0)-finishedSessions),supervisorPid:process.pid};const temporary=`${statusPath}.${randomBytes(6).toString("hex")}.tmp`;await writeFile(temporary,`${JSON.stringify(status,null,2)}
2
2
  `,{mode:384});await rename(temporary,statusPath);await writeFile(join(directory,"error.txt"),detail,{mode:384})}
@@ -1,4 +1,4 @@
1
- import{CredentialError,batchDirectory,cancellationRequested,finishSession,getSessionStatus,openLocalStore,resolveSessionCredential,startSession,stopSession}from"./chunk-ZZK6V3ZF.js";import{CliError}from"./chunk-QJ2UAX2V.js";import{configDir}from"./chunk-OIGVDK3F.js";import{spawn as spawn2}from"child_process";import{randomBytes as randomBytes2}from"crypto";import{copyFile,mkdir as mkdir2,writeFile,open,readFile,rename}from"fs/promises";import{existsSync}from"fs";import{join as join3}from"path";import{setTimeout as delay}from"timers/promises";var LOCAL_MCP_HINT="Gauge exposes no local MCP configuration for registered servers. Hosted sessions can discover and call their tools; supplied files cannot test the live server. If hosted testing is acceptable, start with `gauge onboard`, select a workspace and MCP version, configure any required connection, then preview with --hosted. Login and static verification do not establish server reachability or access. Hosted sessions use credits; previews contact Gauge and may upload inputs. Do not issue hosted commands for a local-only request. See `gauge instructions hillclimb`.";function assertLocalCapabilities(config,prefix=""){if(config.lifecycle!=null)throw new CliError("usage",`${prefix}lifecycle requires hosted Gauge.`,{hint:"Setup and teardown scripts run in hosted sessions. Use --hosted or remove lifecycle."});for(const field of["profileId","marketId","skillRefs","mcpRefs","connectionIds"]){const value=config[field];if(Array.isArray(value)?value.length>0:Boolean(value))throw new CliError("usage",`${prefix}${field} requires hosted Gauge.`,{hint:field==="mcpRefs"?LOCAL_MCP_HINT:"Use --hosted for personas and hosted assets, or select local files and Skills through named inputs in gauge.json."})}if(config.addons&&Object.keys(config.addons).length>0)throw new CliError("usage",`${prefix}addons requires hosted Gauge.`,{hint:"Managed browser addons are unavailable locally. Use --hosted or remove the addon."})}import{fileURLToPath}from"url";import{mkdirSync,readFileSync,renameSync,writeFileSync}from"fs";import{randomBytes}from"crypto";import{homedir}from"os";import{join}from"path";import{z}from"zod";var LOCAL_TARGETS={CLAUDE_CODE:{kind:"claude-code",model:"claude-sonnet-5-5"},CODEX_CLI:{kind:"codex",model:"gpt-6-astra"},PI:{kind:"pi",model:"openai-codex/gpt-6-astra"},OPENCODE:{kind:"opencode",model:"openai/gpt-6-astra"}};var agents=["CLAUDE_CODE","CODEX_CLI","PI","OPENCODE"];var variable=z.string().regex(/^[A-Za-z_][A-Za-z0-9_]*$/);var profileSchema=z.object({agent:z.enum(agents),provider:z.enum(["anthropic","openai","openai-codex","fireworks","openrouter"]),auth:z.enum(["api_key","access_token","setup_token"]),env:variable.optional()}).strict();var storeSchema=z.object({version:z.literal(1),profiles:z.record(z.string(),profileSchema),defaults:z.record(z.string(),z.string())}).strict();var API_KEY_ENV={anthropic:"ANTHROPIC_API_KEY",openai:"OPENAI_API_KEY",fireworks:"FIREWORKS_API_KEY",openrouter:"OPENROUTER_API_KEY"};var profilePath=()=>join(configDir(),"local-profiles.json");function readProfiles(){let raw;try{raw=readFileSync(profilePath(),"utf8")}catch(error){if(error.code==="ENOENT")return{version:1,profiles:{},defaults:{}};throw error}try{const parsed=storeSchema.safeParse(JSON.parse(raw));if(parsed.success)return parsed.data}catch{}throw new CliError("usage","Invalid local credential profile file.",{hint:`Repair ${profilePath()}.`})}function writeProfiles(store){mkdirSync(configDir(),{recursive:true,mode:448});const temporary=`${profilePath()}.${randomBytes(6).toString("hex")}.tmp`;writeFileSync(temporary,`${JSON.stringify(storeSchema.parse(store),null,2)}
1
+ import{CredentialError,batchDirectory,cancellationRequested,finishSession,getSessionStatus,openLocalStore,resolveSessionCredential,startSession,stopSession}from"./chunk-ZZK6V3ZF.js";import{CliError}from"./chunk-QJ2UAX2V.js";import{configDir}from"./chunk-OIGVDK3F.js";import{spawn as spawn2}from"child_process";import{randomBytes as randomBytes2}from"crypto";import{copyFile,mkdir as mkdir2,writeFile,open,readFile,rename}from"fs/promises";import{existsSync}from"fs";import{join as join3}from"path";import{setTimeout as delay}from"timers/promises";var LOCAL_MCP_HINT="Gauge exposes no local MCP configuration for registered servers. Hosted sessions can discover and call their tools; supplied files cannot test the live server. If hosted testing is acceptable, start with `gauge onboard`, select a workspace and MCP version, configure any required connection, then preview with --hosted. Login and static verification do not establish server reachability or access. Hosted sessions use credits; previews contact Gauge and may upload inputs. Do not issue hosted commands for a local-only request. See `gauge instructions hillclimb`.";function assertLocalCapabilities(config,prefix=""){if(config.lifecycle!=null)throw new CliError("usage",`${prefix}lifecycle requires hosted Gauge.`,{hint:"Setup and teardown scripts run in hosted sessions. Use --hosted or remove lifecycle."});for(const field of["profileId","persona","marketId","skillRefs","mcpRefs","connectionIds"]){const value=config[field];if(Array.isArray(value)?value.length>0:Boolean(value))throw new CliError("usage",`${prefix}${field} requires hosted Gauge.`,{hint:field==="mcpRefs"?LOCAL_MCP_HINT:"Use --hosted for personas and hosted assets, or select local files and Skills through named inputs in gauge.json."})}if(config.addons&&Object.keys(config.addons).length>0)throw new CliError("usage",`${prefix}addons requires hosted Gauge.`,{hint:"Managed browser addons are unavailable locally. Use --hosted or remove the addon."})}import{fileURLToPath}from"url";import{mkdirSync,readFileSync,renameSync,writeFileSync}from"fs";import{randomBytes}from"crypto";import{homedir}from"os";import{join}from"path";import{z}from"zod";var LOCAL_TARGETS={CLAUDE_CODE:{kind:"claude-code",model:"claude-sonnet-5-5"},CODEX_CLI:{kind:"codex",model:"gpt-6-astra"},PI:{kind:"pi",model:"openai-codex/gpt-6-astra"},OPENCODE:{kind:"opencode",model:"openai/gpt-6-astra"}};var agents=["CLAUDE_CODE","CODEX_CLI","PI","OPENCODE"];var variable=z.string().regex(/^[A-Za-z_][A-Za-z0-9_]*$/);var profileSchema=z.object({agent:z.enum(agents),provider:z.enum(["anthropic","openai","openai-codex","fireworks","openrouter"]),auth:z.enum(["api_key","access_token","setup_token"]),env:variable.optional()}).strict();var storeSchema=z.object({version:z.literal(1),profiles:z.record(z.string(),profileSchema),defaults:z.record(z.string(),z.string())}).strict();var API_KEY_ENV={anthropic:"ANTHROPIC_API_KEY",openai:"OPENAI_API_KEY",fireworks:"FIREWORKS_API_KEY",openrouter:"OPENROUTER_API_KEY"};var profilePath=()=>join(configDir(),"local-profiles.json");function readProfiles(){let raw;try{raw=readFileSync(profilePath(),"utf8")}catch(error){if(error.code==="ENOENT")return{version:1,profiles:{},defaults:{}};throw error}try{const parsed=storeSchema.safeParse(JSON.parse(raw));if(parsed.success)return parsed.data}catch{}throw new CliError("usage","Invalid local credential profile file.",{hint:`Repair ${profilePath()}.`})}function writeProfiles(store){mkdirSync(configDir(),{recursive:true,mode:448});const temporary=`${profilePath()}.${randomBytes(6).toString("hex")}.tmp`;writeFileSync(temporary,`${JSON.stringify(storeSchema.parse(store),null,2)}
2
2
  `,{mode:384});renameSync(temporary,profilePath())}function validateProfile(value){const parsed=profileSchema.safeParse(value);if(!parsed.success)throw new CliError("usage","Invalid profile. Supply a supported --agent, --provider and --auth, and an environment variable name with --env.");const p=parsed.data;const expectedProvider=p.agent==="CLAUDE_CODE"?"anthropic":p.agent==="CODEX_CLI"?"openai":void 0;if(expectedProvider&&p.provider!==expectedProvider)throw new CliError("usage",`${p.agent} requires provider ${expectedProvider}.`);if(p.auth==="access_token"){const provider=p.agent==="CLAUDE_CODE"?"anthropic":p.agent==="PI"?"openai-codex":"openai";if(p.provider!==provider||p.env)throw new CliError("usage",`Native login for ${p.agent} requires provider ${provider} and no --env.`)}else if(p.auth==="setup_token"){if(p.agent!=="CLAUDE_CODE")throw new CliError("usage","Setup tokens are supported only for CLAUDE_CODE.");p.env??="CLAUDE_CODE_OAUTH_TOKEN"}else{if(!API_KEY_ENV[p.provider])throw new CliError("usage",`API-key auth is unsupported for ${p.provider}.`);p.env??=API_KEY_ENV[p.provider]}return p}function setProfile(name,value,replace=false){if(!/^[a-zA-Z0-9][a-zA-Z0-9_.-]{0,63}$/.test(name)||["__proto__","constructor","prototype"].includes(name))throw new CliError("usage","Profile names must be 1\u201364 letters, digits, dots, underscores or hyphens.");const p=validateProfile(value);const store=readProfiles();const previous=Object.hasOwn(store.profiles,name)?store.profiles[name]:void 0;if(previous&&!replace)throw new CliError("usage",`Profile ${name} already exists. Pass --replace to update it.`);if(previous&&previous.agent!==p.agent&&store.defaults[previous.agent]===name)delete store.defaults[previous.agent];store.profiles[name]=p;writeProfiles(store);return store}function requireProfile(store,name){if(!Object.hasOwn(store.profiles,name))throw new CliError("not_found",`Local credential profile ${name} does not exist.`,{hint:"Run gauge local profiles list."});return validateProfile(store.profiles[name])}function credentialKey(agent,model){return`${agent}:${model}`}function providerFor(agent,model){if(agent==="CLAUDE_CODE")return"anthropic";if(agent==="CODEX_CLI")return"openai";const slash=model.indexOf("/");if(slash<1||slash===model.length-1)throw new CliError("usage",`${agent} model ${model} must include provider/model.`);return model.slice(0,slash)}function resolveLocalProfile(agent,model,options={},store=void 0,env=process.env){const key=credentialKey(agent,model);if(options.credentialSelections){const selected=options.credentialSelections[key];if(!selected)throw new CliError("usage",`This batch has no saved login settings for ${agent} ${model}.`,{hint:"Check `gauge local profiles list`, then start a new batch with the profile you want to use."});validateCompatibility(selected,agent,model);return selected}store??=readProfiles();const provider=providerFor(agent,model);const name=options.profile??store.defaults[agent]??null;let p;if(name)p=requireProfile(store,name);else{const apiEnv=API_KEY_ENV[provider];const nativeProvider=agent==="CLAUDE_CODE"?"anthropic":agent==="PI"?"openai-codex":"openai";const auth=apiEnv&&env[apiEnv]?.trim()||provider!==nativeProvider?"api_key":agent==="CLAUDE_CODE"&&env.CLAUDE_CODE_OAUTH_TOKEN?.trim()?"setup_token":"access_token";p=validateProfile({agent,provider,auth})}validateCompatibility(p,agent,model);const nativeDirectory=p.auth==="access_token"?p.agent==="CLAUDE_CODE"?env.CLAUDE_CONFIG_DIR||void 0:env.CODEX_HOME||join(homedir(),".codex"):void 0;return{...p,name,source:p.env?`env:${p.env}`:p.agent==="CLAUDE_CODE"?"native:claude":"native:codex",...nativeDirectory?{nativeDirectory}:{}}}function validateCompatibility(p,agent,model){validateProfile(pickProfile(p));if(p.agent!==agent||p.provider!==providerFor(agent,model))throw new CliError("usage",`Credential profile for ${p.agent}/${p.provider} is incompatible with ${agent} ${model}.`,{hint:"Select a compatible --profile or change this harness's default with gauge local profiles use."})}function pickProfile(p){return{agent:p.agent,provider:p.provider,auth:p.auth,...p.env?{env:p.env}:{}}}function profileCredentialSources(profile,env=process.env){const mapped={...env};if(profile.env){if(!env[profile.env]?.trim())throw new CredentialError(`Credential environment variable ${profile.env} is not set.`,`Export ${profile.env} before starting the batch, or update the profile's --env reference.`);const canonical=profile.auth==="setup_token"?"CLAUDE_CODE_OAUTH_TOKEN":API_KEY_ENV[profile.provider];mapped[canonical]=env[profile.env]}if(profile.nativeDirectory)mapped[profile.agent==="CLAUDE_CODE"?"CLAUDE_CONFIG_DIR":"CODEX_HOME"]=profile.nativeDirectory;return{env:mapped,homeDir:homedir(),now:new Date}}function readinessSpec(agent,model,profile){return{image:"unused",agent:{kind:LOCAL_TARGETS[agent].kind,model},auth:{mode:profile.auth},workspace:null,resources:[],prompt:{text:""},limits:{turnTimeoutS:1800,responseTimeoutS:60,maxTurns:1}}}async function inspectLocalCredential(agent,model,options={},store){let profile;try{profile=resolveLocalProfile(agent,model,options,store);const credential=await resolveSessionCredential(readinessSpec(agent,model,profile),profileCredentialSources(profile));return{agent,model,profile,ready:true,...credential.expiresAt?{expiresAt:credential.expiresAt.toISOString()}:{},verification:"local"}}catch(error){const problem=error instanceof CredentialError?`${error.message} Fix: ${error.remedy}`:error instanceof CliError?error.message:"Could not read the credential source. Repair or renew the native login and retry.";return{agent,model,profile,ready:false,problem,verification:"local"}}}import{execFileSync,spawn}from"child_process";var runtime_manifest_default={version:"0.1.2",repository:"ghcr.io/gauge-sh/gauge-runtime",digest:"sha256:4056830a1efcb0116fb70e25a18576e26ad7222a87683d6f03387eb88f596c32",contract:"1",platforms:["linux/amd64","linux/arm64"],revision:"abb55dbfa9ceaa160c7a9685a0fd2b4e68a259aa"};var gateway_manifest_default={version:"0.1.0",repository:"ghcr.io/gauge-sh/gauge-gateway",digest:"sha256:ab5489cfe65feb717a79d7be6bd76be8bf915f3e44a5e832cee313772e590e1c",contract:"gateway-1",platforms:["linux/amd64","linux/arm64"],revision:"cdb3a7985666fbb6287be5b2b041aff327c874d7"};var RUNTIME_MANIFEST=runtime_manifest_default;var RUNTIME_IMAGE=runtime_manifest_default.digest?`${runtime_manifest_default.repository}@${runtime_manifest_default.digest}`:`${runtime_manifest_default.repository}:${runtime_manifest_default.version}`;var GATEWAY_IMAGE=gateway_manifest_default.digest?`${gateway_manifest_default.repository}@${gateway_manifest_default.digest}`:`${gateway_manifest_default.repository}:${gateway_manifest_default.version}`;var dockerRead=args=>execFileSync("docker",args,{encoding:"utf8",stdio:["ignore","pipe","pipe"],timeout:15e3});function inspectLocalRuntime(image=RUNTIME_IMAGE,docker=dockerRead,manifest=RUNTIME_MANIFEST){const result={reference:image,expectedContract:manifest.contract,supportedPlatforms:manifest.platforms,ready:false};const fail=(code,message,hint)=>({...result,problem:{code,message,hint}});let server;try{server=JSON.parse(docker(["version","--format","{{json .Server}}"]));if(!server?.Os||!server.Arch)throw new Error("No Docker server")}catch{return fail("docker_unavailable","Local runs need a running Docker daemon.","Start Docker Desktop (using Linux containers) or the Docker daemon, then retry.")}result.dockerPlatform=`${server.Os}/${server.Arch}`;if(!manifest.platforms.includes(result.dockerPlatform))return fail("platform_unsupported",`Docker ${server.Os}/${server.Arch} is not a qualified runtime platform.`,`Supported native Docker platforms: ${manifest.platforms.join(", ")}. Emulation is not qualified.`);let details;try{[details]=JSON.parse(docker(["image","inspect",image]))}catch{return fail("image_missing",`Local runs need the ${image} Docker image.`,image===RUNTIME_IMAGE?"Run: gauge local runtime pull":`Pull or build the selected runtime image: ${image}`)}if(!details?.Id||!details.Os||!details.Architecture)return fail("image_invalid",`Docker returned incomplete image metadata for ${image}.`,"Check docker image inspect for the selected image.");result.imageId=details.Id;result.repoDigest=details.RepoDigests?.find(digest=>digest.startsWith(`${manifest.repository}@`))??details.RepoDigests?.[0];result.contract=details.Config?.Labels?.["sh.gauge.contract"];result.platform=`${details.Os}/${details.Architecture}`;if(result.contract!==manifest.contract)return fail("contract_unsupported",`Runtime contract ${result.contract??"(missing)"} is unsupported; expected ${manifest.contract}.`,"Use a runtime derived from the supported Gauge image and preserve its contract.");if(!manifest.platforms.includes(result.platform)||server.Os!==details.Os||server.Arch!==details.Architecture)return fail("platform_unsupported",`Runtime ${result.platform} is not supported on Docker ${server.Os}/${server.Arch}.`,`Supported native Docker platforms: ${manifest.platforms.join(", ")}. Emulation is not qualified.`);result.ready=true;return result}function requireLocalRuntime(image=RUNTIME_IMAGE,manifest){const result=inspectLocalRuntime(image,void 0,manifest);if(!result.ready){const problem=result.problem;throw new CliError(problem.code==="image_missing"||problem.code==="docker_unavailable"?"not_found":"usage",problem.message,{hint:problem.hint})}return result}var inspectLocalGateway=(image=GATEWAY_IMAGE,docker)=>inspectLocalRuntime(image,docker,gateway_manifest_default);var requireLocalGateway=(image=GATEWAY_IMAGE)=>requireLocalRuntime(image,gateway_manifest_default);var pullLocalGateway=()=>pullLocalRuntime(GATEWAY_IMAGE,gateway_manifest_default);async function pullLocalRuntime(image=RUNTIME_IMAGE,manifest=RUNTIME_MANIFEST){const before=inspectLocalRuntime(image,void 0,manifest);if(!before.dockerPlatform||!manifest.platforms.includes(before.dockerPlatform)){const problem=before.problem;throw new CliError(problem.code==="docker_unavailable"?"not_found":"usage",problem.message,{hint:problem.hint})}const platform=before.dockerPlatform;await new Promise((resolve,reject)=>{const child=spawn("docker",["pull","--platform",platform,image],{stdio:["ignore","pipe","pipe"]});child.stdout.pipe(process.stderr);child.stderr.pipe(process.stderr);child.once("error",()=>reject(new CliError("not_found","Docker could not start.",{hint:"Install and start Docker, then retry."})));child.once("exit",code=>code===0?resolve():reject(new CliError("network",`Could not pull ${image}.`,{hint:"Check the network and public registry availability, then retry gauge local runtime pull."})))});return requireLocalRuntime(image,manifest)}import{execFile}from"child_process";import{randomUUID}from"crypto";import{mkdtemp,mkdir,rm}from"fs/promises";import{tmpdir}from"os";import{join as join2}from"path";import{promisify}from"util";var exec=promisify(execFile);var tools=["sh","bash","git","sha256sum","mkdir","cp","tar","chmod","touch","node","npm","jq","curl"];var harnesses={"claude-code":"claude",codex:"codex",pi:"pi",opencode:"opencode"};var marker="GAUGE_RUNTIME_COMPATIBLE";async function checkLocalRuntimeCompatibility(imageId,agents2){if(!process.getuid||!process.getgid)throw new CliError("usage","Local runtime execution requires a Unix host UID/GID.",{hint:"Run Gauge on macOS or Linux with local Docker running Linux containers, or inside WSL on Windows."});const directory=await mkdtemp(join2(tmpdir(),"gauge-runtime-check-"));const name=`gauge-runtime-check-${randomUUID()}`;try{if(/[,"\n]/.test(directory))throw new CliError("usage","The temporary directory cannot be mounted by Docker.",{hint:"Set TMPDIR to a path without commas, quotes or newlines, then retry."});await mkdir(join2(directory,"workspace"),{mode:448});const commands=[...new Set(agents2.map(agent=>harnesses[agent]))];const script=["set -eu",...tools.map(tool=>`command -v ${tool} >/dev/null || { echo 'missing tool: ${tool}' >&2; exit 41; }`),"mkdir -p /gauge/session/home /gauge/session/workspace || { echo 'session directory is not writable' >&2; exit 42; }","touch /gauge/session/workspace/probe /tmp/gauge-runtime-probe || { echo 'session directory or /tmp is not writable' >&2; exit 42; }","mkdir -p /gauge/session/npm/bin","printf '#!/bin/sh\\nexit 0\\n' > /gauge/session/npm/bin/gauge-runtime-path-probe","chmod 755 /gauge/session/npm/bin/gauge-runtime-path-probe",...["sh -c","sh -lc","bash -lc"].map(shell=>`${shell} gauge-runtime-path-probe >/dev/null || { echo 'environment-installed tools are missing from PATH in ${shell}; rebuild from the current Gauge runtime or preserve /gauge/session/npm/bin in the system login profile and image PATH' >&2; exit 44; }`),"cd /gauge/session/workspace","git init --quiet && git -c user.name=gauge -c user.email=gauge@localhost commit --quiet --allow-empty -m probe","tar --create --gzip --file /tmp/gauge-probe.tar.gz probe","tar --extract --gzip --file /tmp/gauge-probe.tar.gz --directory /gauge/session/workspace --no-same-owner --no-same-permissions","sha256sum probe | sha256sum -c --quiet -",...commands.map(command=>`${command} --version >/dev/null || { echo 'missing or broken harness: ${command}' >&2; exit 43; }`),`echo ${marker}`].join("\n");let stdout;try{({stdout}=await exec("docker",["run","--rm","--pull","never","--name",name,"--network","none","--user",`${process.getuid()}:${process.getgid()}`,"--env","HOME=/gauge/session/home","--env","CLAUDE_CONFIG_DIR=/gauge/session/claude","--env","CODEX_HOME=/gauge/session/codex","--env","PI_CODING_AGENT_DIR=/tmp/pi-agent","--env","XDG_DATA_HOME=/gauge/session/opencode/data","--env","XDG_STATE_HOME=/gauge/session/opencode/state","--mount",`type=bind,source=${join2(directory,"workspace")},target=/gauge/session`,imageId,"sh","-c",script],{timeout:6e4,maxBuffer:1024*1024}))}catch(error){const stderr=String(error.stderr??"").trim();const detail=stderr?` ${stderr.slice(-2e3)}`:" The probe failed or timed out.";throw new CliError("usage",`Runtime ${imageId} failed its compatibility check.${detail}`,{hint:"Derive from the supported Gauge runtime; preserve required tools, harnesses and the session npm bin in ordinary/login-shell PATH, permit the host UID/GID to write /gauge/session and /tmp, and have ENTRYPOINT execute the supplied command."})}if(!stdout.split(/\r?\n/).includes(marker))throw new CliError("usage",`Runtime ${imageId} did not execute Gauge's compatibility command.`,{hint:'Remove the custom ENTRYPOINT or make it finish with exec "$@". Gauge supplies sh -c and must receive its output.'})}finally{await exec("docker",["rm","--force",name],{timeout:1e4}).catch(()=>{});await rm(directory,{recursive:true,force:true})}}import{createHash}from"crypto";import{createReadStream}from"fs";import{list}from"tar";import{z as z3}from"zod";import{z as z2}from"zod";var websiteUrlSchema=z2.url().refine(raw=>{const u=new URL(raw);return u.protocol==="https:"&&!u.username&&!u.password&&!u.port&&!u.search&&!u.hash},"Expected an HTTPS documentation URL without credentials, port, query or fragment");var websiteUseSchema=z2.object({type:z2.literal("website"),url:websiteUrlSchema.refine(url=>new URL(url).pathname==="/","Website inputs require an origin without a path")}).strict();var deploymentSourceSchema=z2.object({type:z2.literal("github-deployment"),environment:z2.string().min(1).max(128)}).strict();var commentSourceSchema=z2.object({type:z2.literal("github-pr-comment"),author:z2.string().min(1).max(100).regex(/^[a-zA-Z0-9-]+(?:\[bot\])?$/),bodyIncludes:z2.string().min(1).max(256).describe("Literal marker identifying the provider's preview comment."),pattern:z2.string().min(1).max(2048).refine(pattern=>{try{new RegExp(pattern,"gm");return pattern.includes("(?<url>")&&pattern.includes("(?<sha>")}catch{return false}},"Use a valid regex with named url and sha captures; match only a ready preview").describe("JavaScript regex (gm flags) matching readiness and capturing url and sha (7\u201340 hex characters). Execution is time limited."),failureIncludes:z2.string().min(1).max(256).optional().describe("Literal failure marker in the selected comment; failure never falls back to older comments.")}).strict();var websiteSourceSchema=z2.discriminatedUnion("type",[deploymentSourceSchema,commentSourceSchema]);function withinWebsiteScope(raw,base){const u=new URL(raw),scope=new URL(base);const prefix=scope.pathname.replace(/\/$/,"");return u.origin===scope.origin&&(u.pathname===prefix||u.pathname.startsWith(`${prefix}/`))}var MAX_WEB_FIXTURE_BYTES=1e6;function normalizeWebFixtureUrl(raw){const url=new URL(raw);url.hash="";return url.href}var searchResultSchema=z3.object({title:z3.string().min(1),url:z3.url().transform(normalizeWebFixtureUrl),snippet:z3.string()});var searchFixtureSchema=z3.object({query:z3.string().trim().min(1),results:z3.array(searchResultSchema)});var pageFixtureSchema=z3.object({url:z3.url().transform(normalizeWebFixtureUrl),status:z3.int().min(100).max(599).default(200),contentType:z3.string().trim().min(1).default("text/html"),body:z3.string()});var originProxySchema=z3.object({origin:websiteUrlSchema.refine(raw=>new URL(raw).pathname==="/","Expected an origin").transform(raw=>new URL(raw).origin),upstream:websiteUrlSchema.refine(raw=>new URL(raw).pathname==="/","Expected an origin").transform(raw=>new URL(raw).origin)}).strict();var webFixtureSchema=z3.object({version:z3.literal(1),origins:z3.array(originProxySchema).max(32).optional(),scopes:z3.array(websiteUrlSchema.transform(normalizeWebFixtureUrl)).max(32).optional(),searches:z3.array(searchFixtureSchema).default([]),pages:z3.array(pageFixtureSchema).default([])}).superRefine((fixture,ctx)=>{if(fixture.scopes?.some(scope=>!fixture.pages.some(page=>new URL(page.url).origin===new URL(scope).origin)))ctx.addIssue({code:"custom",path:["scopes"],message:"Each scope must have a captured page on its origin"});const origins=new Set;for(const route of fixture.origins??[]){if(origins.has(route.origin)||route.origin===route.upstream||fixture.pages.some(page=>new URL(page.url).origin===route.origin)||fixture.scopes?.some(scope=>new URL(scope).origin===route.origin))ctx.addIssue({code:"custom",message:"Origin proxy overlaps another fixture route"});origins.add(route.origin)}const queries=new Set;fixture.searches.forEach((entry,index)=>{if(queries.has(entry.query)){ctx.addIssue({code:"custom",path:["searches",index,"query"],message:`duplicate normalized query: ${entry.query}`})}queries.add(entry.query)});const urls=new Set;fixture.pages.forEach((entry,index)=>{if(urls.has(entry.url)){ctx.addIssue({code:"custom",path:["pages",index,"url"],message:`duplicate normalized URL: ${entry.url}`})}urls.add(entry.url)});const bytes=new TextEncoder().encode(JSON.stringify(fixture)).byteLength;if(bytes>MAX_WEB_FIXTURE_BYTES){ctx.addIssue({code:"custom",message:`serialized fixture is ${bytes} bytes; maximum is ${MAX_WEB_FIXTURE_BYTES}`})}});var webFixtureInputSchema=webFixtureSchema.safeExtend({searches:z3.array(z3.never()).max(0).default([]),pages:z3.array(pageFixtureSchema.safeExtend({url:z3.url().refine(raw=>{const url=new URL(raw);return url.protocol==="https:"&&!url.port&&!url.username&&!url.password},"Expected an HTTPS URL without credentials or a custom port").transform(normalizeWebFixtureUrl),status:z3.int().min(200).max(599).default(200),contentType:z3.string().trim().min(1).regex(/^[\x20-\x7e]+$/).default("text/html"),body:z3.string().refine(body=>!/[\uD800-\uDBFF](?![\uDC00-\uDFFF])|(?<![\uD800-\uDBFF])[\uDC00-\uDFFF]/.test(body),"Body must be valid Unicode")}).strict().refine(page=>![204,205,304].includes(page.status)||page.body==="","This HTTP status requires an empty body")).default([])}).strict().refine(fixture=>fixture.pages.length>0||(fixture.origins?.length??0)>0,"Web fixture must contain pages or origin mappings");function parseWebFixtureInput(text){if(new TextEncoder().encode(text).byteLength>MAX_WEB_FIXTURE_BYTES)throw new Error(`Web fixture exceeds ${MAX_WEB_FIXTURE_BYTES} bytes`);return webFixtureInputSchema.parse(JSON.parse(text))}function mergeWebFixtures(fixtures){if(!fixtures.length)return void 0;const scopes=[];for(const[index,fixture]of fixtures.entries()){for(const scope of fixture.scopes??[]){if(scopes.some(other=>withinWebsiteScope(scope,other)||withinWebsiteScope(other,scope)))throw new Error("Web fixture scopes overlap in this case");if(fixtures.some((other,otherIndex)=>otherIndex!==index&&other.pages.some(page=>withinWebsiteScope(page.url,scope))))throw new Error("Web fixture scope overlaps another input's pages");scopes.push(scope)}}return webFixtureSchema.parse({version:1,scopes,origins:fixtures.flatMap(f=>f.origins??[]),searches:fixtures.flatMap(f=>f.searches),pages:fixtures.flatMap(f=>f.pages)})}async function readWebFixtureArchive(path,member,digest){const hash=createHash("sha256");for await(const chunk of createReadStream(path))hash.update(chunk);if(`sha256:${hash.digest("hex")}`!==digest)throw new Error("Web fixture archive digest mismatch");const chunks=[];let found=0;let bytes=0;let problem;await list({file:path,strict:true,onReadEntry(entry){if(entry.path.replace(/^\.\//,"")!==member||entry.type!=="File"||entry.size>MAX_WEB_FIXTURE_BYTES){problem=new Error("Web fixture archive must contain exactly its declared JSON file (at most 1 MB)");entry.resume();return}found++;entry.on("data",chunk=>{bytes+=chunk.length;if(bytes>MAX_WEB_FIXTURE_BYTES)problem=new Error("Web fixture exceeds 1 MB");else chunks.push(chunk)})}});if(problem)throw problem;if(found!==1)throw new Error("Web fixture archive must contain exactly one JSON file");return parseWebFixtureInput(Buffer.concat(chunks).toString("utf8"))}async function freezeWebFixtures(definition,archives){const inputs=new Map;for(const resource of definition.resources){if(resource.use.type!=="web-fixture")continue;if(resource.source.type!=="upload")throw new Error(`Web fixture ${resource.name} requires a local file`);const digest=resource.source.sha256;const archive=archives.find(a=>a.sha256===digest);if(!archive)throw new Error(`Missing web fixture archive: ${resource.name}`);inputs.set(resource.name,await readWebFixtureArchive(archive.path,resource.use.path,archive.sha256))}const cases={};for(const item of definition.cases){const fixture=mergeWebFixtures((item.config.inputs??[...inputs.keys()]).flatMap(name=>inputs.get(name)??[]));if(fixture)cases[item.path]={fixture,digest:`sha256:${createHash("sha256").update(JSON.stringify(fixture)).digest("hex")}`}}return cases}var CONCURRENCY=4;var POLL_MS=1e3;var LOCAL_AGENTS=LOCAL_TARGETS;function analysisNext(batchId){return["Next: analyze these runs. Run this command and follow it:"," gauge instructions session-analysis --local",`Analyze only batch ${batchId}.`].join("\n")}async function planLocal(definition,options={}){const{queue}=plan(definition);const warnings=[];const runtime=inspectLocalRuntime(options.runtimeImage);const gateway=definition.resources.some(r=>r.use.type==="web-fixture")?inspectLocalGateway(options.gatewayImage):void 0;if(gateway?.problem)warnings.push(`${gateway.problem.message} ${gateway.problem.hint}`);if(runtime.problem)warnings.push(`${runtime.problem.message} ${runtime.problem.hint}`);const credentials=await credentialReadiness(queue,options);for(const target of credentials)if(!target.ready)warnings.push(`${target.agent} ${target.model}: ${target.problem}`);return{mode:"local",runtime,gateway,runtimeProbe:"not_run",expectedSessions:queue.length,cases:definition.cases.map(item=>{const sessions=new Map;for(const job of queue.filter(job2=>job2.item===item)){const key=`${job.agent}\0${job.model}`;const entry=sessions.get(key)??{agent:job.agent,model:job.model,samples:0};entry.samples++;sessions.set(key,entry)}return{path:item.path,criteria:item.criteria?.length??0,sessions:[...sessions.values()]}}),inputs:definition.resources.map(r=>({name:r.name,use:r.use.type})),warnings,credentials}}function plan(definition){for(const item of definition.cases){assertLocalCapabilities(item.config,`${item.path}: config.`);if(item.config.repoRef&&!item.config.repoUrl)throw new CliError("usage",`${item.path}: config.repoRef requires config.repoUrl.`);for(const name of item.config.inputs??[])if(!definition.resources.some(resource=>resource.name===name))throw new CliError("usage",`${item.path}: input ${name} is missing from this launch.`)}const resources=new Map(definition.resources.filter(resource=>resource.use.type!=="web-fixture").map(resource=>[resource.name,toResourceRef(resource)]));const queue=definition.cases.flatMap(item=>item.config.agents.flatMap(entry=>{const agent=LOCAL_AGENTS[entry.agent];if(!agent)throw new CliError("usage",`${item.path}: agent ${entry.agent} cannot run locally.`,{hint:"Use CLAUDE_CODE, CODEX_CLI, PI, or OPENCODE, or pass --hosted."});const models=entry.models?.length?entry.models:[agent.model];return models.flatMap(model=>Array.from({length:item.config.sampleCount??1},(_,sample)=>({item,agent:entry.agent,model,sample:sample+1})))}));return{resources,queue}}async function prepareLocal(definition,uploads,gaugeDir,options={}){const{queue}=plan(definition);const webFixtures=await freezeWebFixtures(definition,uploads);const gateway=Object.keys(webFixtures).length?requireLocalGateway(options.gatewayImage):void 0;await freezeCredentials(queue,options);const runtime=requireLocalRuntime(options.runtimeImage);await checkLocalRuntimeCompatibility(runtime.imageId,[...new Set(queue.map(job=>LOCAL_AGENTS[job.agent].kind))]);const blobsDirectory=join3(gaugeDir,"blobs");await mkdir2(blobsDirectory,{recursive:true});for(const upload of uploads)await copyFile(upload.path,join3(blobsDirectory,upload.sha256.slice(7)));const batchId=`b_${randomBytes2(6).toString("hex")}`;const directory=batchDirectory(gaugeDir,batchId);await mkdir2(directory,{recursive:true,mode:448});await writeJson(join3(directory,"runtime.json"),runtime);await writeJson(join3(directory,"gateway.json"),{gateway,cases:webFixtures});await writeJson(join3(directory,"status.json"),{batchId,runtime,state:"queued",expectedSessions:plan(definition).queue.length});const store=openLocalStore(gaugeDir);try{store.recordLaunch({id:batchId,measurementId:options.measurement?.id,measurementRevision:options.measurement?.revision,configuration:definition,provenance:definition.source??{}})}finally{store.close()}return batchId}async function detachLocal(definition,uploads,gaugeDir,options={}){const credentialSelections=await freezeCredentials(plan(definition).queue,options);const batchId=await prepareLocal(definition,uploads,gaugeDir,{...options,credentialSelections});const directory=batchDirectory(gaugeDir,batchId);await writeFile(join3(directory,"request.json"),JSON.stringify({definition,gaugeDir,batchId,credentialSelections,measurement:options.measurement}),{mode:384});const log=await open(join3(directory,"supervisor.log"),"a",384);try{const bundledWorker=fileURLToPath(new URL("./batchWorker.js",import.meta.url));const workerArgs=existsSync(bundledWorker)?[bundledWorker]:[...process.execArgv,fileURLToPath(new URL("../batchWorker.ts",import.meta.url))];const child=spawn2(process.execPath,[...workerArgs,join3(directory,"request.json")],{detached:true,stdio:["ignore",log.fd,log.fd]});await new Promise((resolve,reject)=>{child.once("spawn",resolve);child.once("error",reject)});let exited=false;child.once("exit",()=>{exited=true});const deadline=Date.now()+1e4;while(true){const status=JSON.parse(await readFile(join3(directory,"status.json"),"utf8"));if(status.state!=="queued")break;if(exited||Date.now()>=deadline){await writeFile(join3(directory,"cancel"),"",{mode:384});throw new CliError("network",`Background supervisor did not become ready. See ${join3(directory,"supervisor.log")}`)}await delay(50)}child.unref()}finally{await log.close()}return{batchId,status:`gauge run status --batch ${batchId} --local`,wait:`gauge run wait --batch ${batchId} --local`,stop:`gauge run cancel --batch ${batchId} --local`}}async function runLocal(definition,uploads,gaugeDir,onRun=()=>{},options={}){const{resources,queue}=plan(definition);const credentialSelections=await freezeCredentials(queue,options);const batchId=options.batchId??await prepareLocal(definition,uploads,gaugeDir,{...options,credentialSelections});const directory=batchDirectory(gaugeDir,batchId);const runtime=JSON.parse(await readFile(join3(directory,"runtime.json"),"utf8"));const gatewayPlan=JSON.parse(await readFile(join3(directory,"gateway.json"),"utf8"));const expectedSessions=queue.length;let interrupted=false;const signal=()=>{interrupted=true;process.exitCode=130};process.on("SIGINT",signal);process.on("SIGTERM",signal);const canceled=()=>interrupted||cancellationRequested(directory);const runs=[];let progress=Promise.resolve();const save=state=>writeJson(join3(directory,"status.json"),{batchId,runtime,state,expectedSessions,finishedSessions:runs.length,queuedSessions:state==="running"?queue.length:0,skippedSessions:state==="running"?0:queue.length,supervisorPid:process.pid});try{await save("running");options.onStart?.(batchId);const worker=async()=>{while(!canceled()){const job=queue.shift();if(!job)break;const run=await runJob(job,{batchId,gaugeDir,blobsDirectory:join3(gaugeDir,"blobs"),resources,credentialSelections,runtime,gateway:gatewayPlan.gateway&&gatewayPlan.cases[job.item.path]?{image:gatewayPlan.gateway.imageId,...gatewayPlan.cases[job.item.path]}:void 0,canceled,measurement:options.measurement,provenance:definition.source??{}});runs.push(run);onRun(run);progress=progress.then(()=>save("running"));await progress}};const settled=await Promise.allSettled(Array.from({length:CONCURRENCY},worker));const failure=settled.find(s=>s.status==="rejected");await save(failure?"failed":canceled()?"canceled":"completed");if(failure?.status==="rejected")throw failure.reason;return{batchId,runs,next:analysisNext(batchId),canceled:canceled()}}finally{process.off("SIGINT",signal);process.off("SIGTERM",signal)}}function toResourceRef(resource){if(resource.source.type!=="upload")throw new CliError("usage",`Input ${resource.name} has no local file; local runs need every input in the working tree`);const base={name:resource.name,digest:resource.source.sha256,filename:`${resource.name}.tar.gz`,kind:"archive"};const use=resource.use;if(use.type==="files")return{...base,use:{type:"files"}};if(use.type==="skill"&&use.root===".")return{...base,use:{type:"skill"}};if(use.type==="npm-package")return{...base,use:{type:"npm-package",package:use.package,install:use.install??"workspace"}};throw new CliError("usage",`Input ${resource.name}: ${use.type==="skill"?"supply the Skill as a directory":`${use.type} inputs run on hosted Gauge only`}`)}async function credentialReadiness(queue,options){const store=options.credentialSelections?{version:1,profiles:{},defaults:{}}:readProfiles();const targets=new Map(queue.map(job=>[credentialKey(job.agent,job.model),job]));return Promise.all([...targets.values()].map(job=>inspectLocalCredential(job.agent,job.model,options,store)))}async function freezeCredentials(queue,options){const readiness=await credentialReadiness(queue,options);const failures=readiness.filter(target=>!target.ready);if(failures.length)throw new CliError("usage",failures.map(target=>`${target.agent} ${target.model}: ${target.problem}`).join("\n"),{hint:"Inspect credentials with gauge local profiles inspect <profile>, or preview every target with gauge run --local --dry-run -o json."});return Object.fromEntries(readiness.map(target=>[credentialKey(target.agent,target.model),target.profile]))}function sessionSpec(job,resources,profile,runtimeImage=RUNTIME_IMAGE){const names=job.item.config.inputs??[...resources.keys()];const selected=names.flatMap(name=>resources.get(name)??[]);const files=selected.filter(r=>r.use?.type==="files");const note=files.length?`
3
3
 
4
4
  Prepared inputs: